November 2011 AO3 Deploy and Reactions: an Apology from the Board

The Board would like to issue a public apology to our Support and AD&T staff and volunteers, our members, and the users of the AO3.

It is the Board’s responsibility — including the responsibility of the position I, Hele Braunstein, hold as Board liaison to both AD&T and Support — to keep an eye on any organizational issues that would affect both the quality of the product put forward to our beta install and the amount of work and responsibility of our volunteers. It is, in addition, our particular responsibility to ensure that they are not blamed and held accountable in public for what are, at their heart, organizational failings.

It was the Board’s responsibility to act on these failings, and avoid their having the impact that they, regrettably, have had.

As apologies are nothing if they are not followed by action, we can promise to take measures to avoid this happening in the future. The AD&T committee will work to figure out where we went wrong and what measures to take and guidelines to set to avoid it happening in the future. They will work with their current Board liaison, the Board as a whole, and their future Board liaison.

I especially apologize for being only able to set things in motion, as I’ll not be returning to Board next term. However, I promise to be available to the people working on it next term as well, if needed.

Finally, and most importantly, we want to thank each and every volunteer and staff member who has worked on this, both before and after the deploy. Your work was not at fault, and it’s deeply appreciated. Without you, and your efforts, OTW and its many projects would not exist.

Edited to add: This post was made in response to a number of conversations that were, and are, happening both inside and outside the OTW in reaction to the AO3 deploy; we find ourselves in the position of offering another apology as a result, as this post, in practice, proved to be opaque to the larger OTW community, which includes diverse interests in all our projects. To clarify, there were issues with the latest deploy to the AO3 — the deploy was somewhat hurried, limiting the ability of our testers, coders, support team, and other staff to do their jobs, and making the process harder than necessary, both for our volunteers as well as for our users.

As a result, many volunteers were put in a position where they felt personally liable for larger organizational or group-level errors; this is why we, as a Board, felt it was necessary to acknowledge their distress and to respect their very hard work. They addressed many of the problems and corrected them within 48 hours — we are grateful for their efforts, and wish to apologize for the necessity. (edits made 14 November 2011 5:00 UTC)

8 thoughts to “November 2011 AO3 Deploy and Reactions: an Apology from the Board”

  1. You know, bugs happen. I’ve been a release engineer for 18 years and I’ve rarely seen a release go out the door without them, even with full complements of professional QA on staff. One recent release has us looking at percentage of affected customers and being grateful every day if it stays below 8%. And this is a multi-billion dollar enterprise software company.

    So I don’t know that anyone can promise that it won’t happen in future. 🙂 Because that’s just not possible in software development. And if that’s what customers’ expectations are, well, that’s pretty unreasonable, frankly, especially since the OTW’s customers aren’t required to pay for the product.

    Just my .02. Thank you for your statement about the hard work the volunteers do, because I think that got lost in all the hoopferaw. Some fantastic features and changes were implemented that benefited me directly, and I’m so very grateful to them for all their devotion and labor and design and dedication to detail.

    1. I don’t mean to promise no bugs at all! There were more issues with the latest deploy to the AO3 than bugs — the documentation was somewhat lacking, and the deploy was hurried, limiting the ability of our testing, support, and other staff to do their jobs, and making the process harder than necessary, both for our volunteers as well as for our users.

      As a result, many volunteers were put in a position where they felt personally liable for larger organizational or group-level errors; this is why we, as a Board, felt it was necessary to acknowledge their distress and to respect their very hard work. They addressed many of the problems and corrected them within 48 hours – we are grateful for their efforts, and wish to apologize for the necessity.

  2. Really, feeling personally liable for bugs in a non-profit open source project? Isn’t it all just for fun? Relax.
    Ruby core developers are probably having fun after fixing a bug in their OpenSSL implementation which caused generated RSA keys not to really encrypt anything (exp = 1). Not writing apologies 😉

    I can accept tons of messed up deploys and hours of downtime in services I pay for. I’m sure I’m not the only one.
    You’re non-profit. Relax.

    Sure, there are demanding users. You know what? Screw them.

    1. The Board is responsible, not for the bugs, directly, but for the organizational issues that impede the volunteers from fixing them in as stress-free manner as possible, and that makes them feel they’re personally to blame if things go wrong.

      As we’re a by-fans-for-fans project, our users are our coders, developers, testers, and fund-raisers, both current and future, and while we appreciate the sentiment of support for the developer side (and we need it!), we’re actually all the same people – our developers are users, our users are developers.

  3. The company I work for pays $1600 per month for our web based online billing system which I really wish worked half as efficiently as AO3. I don’t mean this as a criticism of the folks we work with. They do a pretty good job of fixing problems as they come up and are very responsive to me, the crankiest and most expressively anxious of their customers.

Comments are closed.