Months ago, I’d promised a blog post on the kind of documentation I create / use when testing. Since then, I’ve started working with a different company, which brings with it a reasonable possibility that the kind of documentation I create and use might change, to adapt to the needs of my new employer, its clients and how they each work. However, I’ve been reading a lot about session-based test management recently (with a view to try out the more structured exploratory testing approach and the session reports that come with it), which prompted me to pick up the topic again.
Since it’s taken me a while to get round to writing this post, I won’t spend too much time on it. However, I’d like to share this because I haven’t seen any other templates like it (other than what Paul describes) and because I want the origins to still be fresh in my mind when writing. This blog is a way for me to reflect, learn and grow, and so I like the idea of reading this back at some point in the future and seeing if / how things might have changed.
A Brief Word on Other Templates
When I first started testing less than two years ago at my previous company, I was told I’d be responsible for all test documentation. I was given the test case template below that one of the developers had come up with. I think I actually added the Excluded and Variables sections.
I did not enjoy using this. Here are just some of the problems I came across very early on: extremely time consuming to create, difficult to maintain, and most importantly – never actually used. For all the time it took me to create and update documents like this, I never actually used them for anything. They were documents for documents’ sake, and they were taking up valuable testing time.
I Googled around for other templates; to try and find out what other testers were doing. I felt that testing at the company was at too much of a developing stage to invest in any formal test management tool and, to be honest, the look of those didn’t appeal to me either. Sadly, it seemed that all anyone else was using (as far as I could see at the time) were Word documents similar to the one I’d be given, or Excel spreadsheets. On reflection, perhaps I was Googling for the wrong thing. I was actually introduced to this template for “test scripts”. I then spent some time Googling the difference between test cases and test scripts, and then those were the templates for which I was searching. I learned about SBTM much later (at least it seemed much later, within the grand two year total).
So, thinking that perhaps I just needed to make a few adjustments and try a little harder, I took inspiration from some templates online and came up with this Excel template.
Admittedly, I liked the layout of this one better, but it had all the same problems.
By this point, I’d been the sole tester in the business for a few months and I was more confident in my feeling that these documents just weren’t useful, at least for me. With the blessing of my CEO, who I worked with closely, I decided to scrap these test case documents altogether and come up with something completely of my own making; something I would actually use and get value from.
And so, HISToW was created: How It’s Supposed To Work.
Some of you might have noticed the word choice used in the opening sentence: “the documentation I create / use when testing”.
I deliberately avoided phrases such as, “test cases” or “test documentation” or “how I document testing”. Although (as far as I can remember) I was yet to discover the great debate of pre-written test cases vs exploratory testing, I knew very early on that written test cases weren’t for me. They were more counter-productive than helpful, and they were so restrictive! The first few times I tried to use test case documents in my testing, I noticed that I was more pre-occupied with trying to follow the exact steps detailed than focussing on the actual software in test.
And what if I saw something that looked like it might be a bug but wasn’t in the documentation? Was I supposed to ignore it? Was I “allowed” to deviate from the scripted steps to investigate? It was awful!
The strange thing, too, was that I wrote all those test documents, so I could add in the investigation steps if I wanted to. But that somehow felt like cheating. I was bound by my own creation…
The new HISToW documents weren’t to be anything like that. They were designed to be useful reference guides that I could use or not; that developers could refer to or not; that the product owner side of my role could check or not. They contained all the useful knowledge I’d gained from testing so far, including how it (the feature) is supposed to work, the accepted issues that still existed and weren’t to be raised as new issues, and any cases / pitfalls to watch out for.
The new HISToW documents served as great, succinct and combined summaries of requirements, learnings, use cases, scenarios and issues. They can actually be used, and not adhered to. And they are much easier to update, since they’re not granularly prescriptive, and are much shorter than test scripts or test suites. You can skim to the parts you’re really interested in without the worry that you’ve missed some important detail buried in step after step.
They concentrate much more on what we know about the product, allowing testers to use their own intelligence and creativity to test however they see fit, as opposed to restricting testers to only the cases documented, in only the documented variations.
It might also be worth noting that HISToWs are intended to be created or updated after testing and not before it. They’re designed to show what you’ve learnt from testing and act as a reference guide, not dictate what you “should” or shouldn’t be looking at. However, so long as you have sufficient information to start documenting how something is supposed to work and what you want to remember to test, I suppose there’s no reason why you couldn’t also use this to aid test planning.
Obviously I’m biased, but I really like the HISToW format / template. As a word play / way of remembering, you could even call it a HISToWry (as in “history”), since it’s based on a history of testing, discovery and learning, and it also tells a story of what that product or feature is supposed to do and how it might be used. But maybe I’m just grasping at straws trying to come up with a cool name =]
I’d love for you to try this out and let me know your thoughts on it. You can access the HISToW template here. Have you come up with any other templates for documentation in testing that you haven’t seen elsewhere? Let me know in the comments.