This training session was led by Kylie, testing lead for Accessibility, Design, & Technology (AD&T), and covered introductory-level testing for the Organization for Transformative Works’ Archive of Our Own software. If you are interested in volunteering as a tester for the OTW, please contact the Volunteers & Recruiting committee–we’d be happy to have you!
Accessibility, Design, & Technology is the guiding body that coordinates software design and development on behalf of the Organization for Transformative Works.
This transcript has been edited for clarity.
Kylie Hi, thanks for coming, I’m Kylie the ADT test lead. I help organise testing (and testing parties), organise training of testers, and also do some actual testing. Today’s session will be a beginner’s guide to testing that will cover what testing is, how we do it for the archive and include some examples of some bugs/issues, and how you might start testing them.
Kylie Testing basically involves testing the archive to make sure it does what it’s supposed to do. This could take the form of testing new features (like subscriptions, prompt memes etc), testing that fixes put in by the lovely coders have worked, or sometimes investigating an issue for Support that has been reported by a user. There is also general testing, where you can play around with things on our test archive and see how things are working, but the the first three are the priority at the moment.
Kylie The general testing/coding cycle is this: Coders code (fixes, new features) -> the code is put on our test server (more on that in a second) -> and then we test it -> if any fixes are needed the coders makes their changes -> and then we test them again so if they’re working fine -> if everything is fine, we (Systems or Elz) deploy to the archive with shiny new code.
Kylie We’ve been trying a monthly release schedule this year, so that means that approx 2 weeks of every month there are issues on the test server for testers to test. That depends on how many issues we have – it can vary quite a bit – and if we need an emergency deploy for any reason (it happens! Although not often). For example, at the moment we just finished a deploy so there are no issues to test. But there will be very soon. *g*
Kylie So that can mean that it is a bit of an up and down thing – but testers can still do things when there are not issues on the test server – and sometimes the Coders need help testing things before they’re deployed to the test server. 🙂 I’d also like to get into having some tester training sessions so people can get experience on different areas of the archive – because it’s growing *g* and we don’t expect everyone to know everything about how things work – especially things like tag wrangling and running challenges.
Kylie So all you need is some experience using the archive (or really websites in general) and enthusiasm for findings out how thing work, and why they might be not working. 😉
Trainee 1 How much time is it good to have to do this?
Kylie Any amount of time you can give. 🙂 From helping with a couple of small issues (which might take half an hour) so helping with lots of issues (which might take longer).
AD&T staffer 1 when we’re right before a deploy, even popping by and testing one fix to mark it as “done!” is a big help.
Kylie Some issues do take longer than others, but you can do some of it and leave a note and others can pick up. Especially if it’s a front end fix and it needs to be tested on more than one browser. Just testing with one combination is a help.
Kylie The test server is a copy of the archive with any new code we’ve deployed to test. It doesn’t have exactly the same data as live, and what you do on it won’t affect the archive, but you do have to be a little careful.
Feel free to have a look around. But bearing in mind it has been used for testing, so there may be some odd works about. Mostly labelled ‘testing this lalala’ or the like. :p
The other big tool for testing is Google Code. Google Code is where we keep a record of all the issues – for coders and testers – for new features and things we’re fixing.
AD&T staffer 2 The main thing to note is that if people have subscribed to you, when you post a work on Test it will notify your subscribers.
Trainee 1 how do you know if anyone has subscribed to you?
AD&T staffer 2 There’s no way to know atm. But we do have a special testing account, so you can use that and be sure you’re not spamming real people 😀
Kylie The issue list is on Google Code. In Google Code, there are a variety of statuses – some mostly for coding, some for testing – the issues testers mostly need to deal with are DeployedToTest. Or DTTs. 🙂 What testers mostly want to look at is the DTTs – so you can filter by them. DeployedToTest issues [note: if the link returns all open issues, enter “status=DeployedToTest” in the search box to pull up DTTs]
When an issue has been tested and it works it gets changed to VerifedOnTest – or VOT.
AD&T co-chair I have an issue to test with PDF downloads. When a users downloads a PDF version of a fic, if the author edits the work and the user re-downloads the PDF, the edits aren’t showing up in the downloads. I’ve tested this and gotten some inconsistent results, so more data would be good 🙂
So somebody could post a work and edit it, and different people could download it and take a look.
Kylie Here is the issue to test.
At the top is a description – which should outline the issue. And then people have comments below that. This is all done though Google – so a Google account is very useful to add comments.
Kylie It’s also really helpful to look at the labels on the side of the issue box. Which can give you some useful information. Like for example if it’s a BackEnd issue (like this one) or FrontEnd (which is mostly to do with display and how things look). If it’s a BackEnd issues we only need to test it in one browser – as it will affect them all the same – if it’s FrontEnd then we need to test it in a variety of browsers because they can vary how they display things (and how – hi IE7). We test all FrontEnd issues in Firefox, IE7,8, Opera, Chrome and Safari.
Trainee 2 I posted a fic in the test archive!
AD&T staffer 2 I downloaded your exciting fic!
Kylie With testing, attention to detail is important, so even if it’s a small thing that you notice, make a note. Or even if you note something unrelated.
Trainee 2 I posted an edit to the fic! hopefully, the editing is obvious. It now includes the words ‘edited’
Trainee 2 I downloaded the fic as a PDF again and it worked for me (the edits showed up).
AD&T staffer 2 Worked for me also
Trainee 3 I tried posting to the test archive, downloading a pdf, editing the fic, and then downloading again. When I downloaded the second time (in a new Chrome window), the pdf didn’t capture the edits. When I downloaded again (in an incognito Chrome window), the pdf *did* capture the edits. So … I dunno?
AD&T co-chair Can you try the download in a different browser?
Trainee 2 I used Firefox 5.0
Trainee 3 Maybe it’s a cache thing?
AD&T staffer 2 One thing to check is whether you previewed when posting the second time
AD&T co-chair I think it’s browser-side caching, yeah
Kylie It’s browser dependent. Weird.
Trainee 3 this was fun!
AD&T staffer 1 “huh, this is weird” <-- perfect summary of a tester's job AD&T staffer 2 Edits worked in FF 5 and Safari 5.0.5
Kylie Oh Chrome. I can check it in that.
Trainee 3 What happens next, after testing something to see if it works or doesn’t — is the next step to leave a comment in google code?
Kylie Yes. That is the next step.
AD&T staffer 2 So at this point you would add a comment to the issue saying how you tested (e.g. the different browsers it did and didn’t work in, in this case).
Trainee 2 should we? to take advantage that we did test it?
Kylie Yeah, leave as much detail as you can.
Trainee 1 so what do you want people to do to volunteer?
Kylie Contact the Volunteers Committee to be added to the mailing list and I will be in touch. 🙂 We’ll have new issues very soon and I’m definitely going to try and get testers together if I can. It’s nice when people are testing together.
And that concludes our excellent session on how to test for the Archive of Our Own!