Skip to main content
Stop Sign

Stop Telling Testers to Prevent Issues

Reading Time: 4 minutes

 

 

TL; DR: Words matter.  By casually asserting that testers prevent issues, we are contributing to the continued misconception of what testers do, and damaging the “tester brand” by setting ourselves up for failure.

 

 

Stop telling testers to prevent issues

 

In the last two days, I’ve seen two separate pieces suggesting that testers should prevent issues.

 

In a job description I was sent:

 

And in an article about shifting from catching bugs to preventing bugs, that I’d like to assume was well-inteded.

 

I strongly believe that testers should be involved in the software development lifecycle as soon as possible – testing the ideas, assumptions and requirements before any code is even written.  However, asking questions and raising concerns early is not the same as preventing issues; it’s foreseeing them.  We try to predict issues by applying our knowledge, experience, intelligence, analytical skills, and instincts (AKA: Spidey senses).

 

 

Do you really think testers can prevent issues?

 

If I make a recommendation not to go ahead with a particular feature or implementation (giving reasons why) and the product owner / stakeholder / developer decides to go ahead anyway, only for it to cause issues / bugs, I have not failed to prevent issues.

 

I was never in a position to prevent them in the first place, as it was not up to me to decide what should be implemented, or how.  I’ve done my job in predicting what issues might arise and giving the relevant parties the necessary information to make their decision(s).

 

Being a tester does not give me the ability to control the actions of my colleagues any more than living in Scotland gives me the ability to control the actions of my neighbours.  In the same vein, weather experts can predict rain and advise you carry an umbrella, but they can’t stop the rain from falling, or force you to carry an umbrella.

 

To me, shifting left isn’t about testers preventing issues; it’s about consulting with testers earlier so we can provide valuable insight and predict issues before they arise.  Even this isn’t a complete safe-guard against finding bugs before or after release to production, or else there would be no need to test the actual software.

 

 

What do you actually mean to say?

 

By casually asserting that testers prevent issues, even if that’s not what we really mean, we are contributing to the continued misconception of what testers do, and damaging the “tester brand” by setting ourselves up for failure when we inevitably do not prevent issues.

 

By talking about ourselves in this way, we’re making it okay for others to talk about us this way.

 

Am I just being picky?  I don’t think so, but of course I wouldn’t.  Let’s look at this from a different angle…

 

In the case of someone asking, “Can I borrow your pen?” vs “May I borrow your pen?” you could say it was obvious that it’s technically possible and assume that they were actually asking for permission, even if they specifically used “can”.

 

But what about a requirement stating that the number of active users should be “≥ 0” vs “> 0”.  Would it be okay to assume that there should never be 0 active users and disregard what has actually been requested?  What if this meant that customers could never close their account, and would always be billed for at least 1 user?  That could be a pretty serious bug, all because you assumed to know what was “really” meant.

 

So, when is it okay to assume you know what someone “really” means, and when should you pay attention to the specific words they’ve used?  Perhaps more importantly, when is it okay to expect people to know you that you didn’t really mean what you said?  I’d argue never.

 

If we want people to understand what we’re really trying to say, then we must consistently endeavour to use words that most closely reflect what we really mean, as difficult as this can be.

 

Using the right words could be especially helpful to those prone to taking things literally.  Whether that’s because they’re speaking in a language that is not their native language, they have a condition which makes it difficult for them to detect things like sarcasm, or they practice active listening in an effort to apply less of their own bias to your message.  Can you really blame someone for misunderstanding you because they listened to what you actually said, instead of what they expected you to say?

 

 

Where else do words matter?

 

Words matter in a lot of contexts, but the other one I want to highlight is in discrimination.  In racism, sexism, homophobia…  I think it’s important to acknowledge that the words we use matter even more so here.

 

It always disappoints me when a good person casually uses a derogatory term.  Even if no ill will is intended, hearing a good person use harmful words subconsciously makes people think that it must be okay to say, because this kind person is doing it.

 

Ultimately, this is how we end up with whole communities of people who genuinely think it’s okay to talk about entire groups of people in a derogatory manner.  We all contribute to that.

 

 

I’d really love to know your thoughts on this.  Please share them in the comments.

28 thoughts on “Stop Telling Testers to Prevent Issues

    1. Hey,

      So sorry for the late reply – I’d totally forgotten about your comment notification until I logged in just now.

      Thanks for sharing the article and a different perspective. It seems quite similar to the one Amit shared in one of the first comments, so I’d still say Larry helped bring a potential issue to light, but unless the decision of how to proceed was with him, he didn’t prevent it =]

      He could, however, have been part of the team who prevented the issue.

      Think of it from another angle – if a tester proudly proclaimed, “I prevented that issue,” would the developer who actually altered the code to fix the issue have the right to be a bit annoyed, because they are the one who actually fixed it? And if there was something to fix, then surely its existence wasn’t prevented. Therefore, there’s also a case for us to be more specific in what counts as prevention – preventing from existing or preventing from going to production? Previous comments have touched upon that already but, as above, there is usually always someone other than the tester who actually makes a decision on how to proceed.

      What do you think?

      Thanks,

      Cassandra

  1. Great post!

    With regard to prediction vs. prevention, you might take a look at some of John Allspaw’s talks on risk and recovery in Web Operations.

    John talks about Mean Time To Detect issues (MTTD) versus Mean Time Between Failures (MTBF) and also about the time it takes to fix issues if they are detected. There’s a strong case that if you can detect issues fast and fix them fast then you actually experience fewer minutes of downtime on an annual basis than you would if you put all your effort into stopping errors from happening.

    It’s impossible to predict the future. So prevention and prediction are both troublesome practices to leverage for addressing systemic risk. Detection on the other hand is always valuable since errors *will* always crop up in prod no matter how much time is put into prevention / prediction / suppression of errors.

    And fixing errors is always useful.

    So: find and fix over predict and prevent? This is the devops model of risk that I have been espousing for 7 years now, and it works for Etsy and it works for a host of other software organizations too!

    Great post, glad to see all the dialog in the comments! Keep the posts coming!!

    1. Hi Noah,

      Thanks a lot for your feedback and thoughts =]

      I’d never heard of MTTD and MTBF before. I’m really interested in this idea of less downtime from fixing fast over all efforts on prevention. Do you really mean all efforts, as in if no one tried to find bugs that actually exist and only predicted ones that might? It would be odd to me to only predict, without ever trying to find what actually exists, now.

      Would also be interested to learn what that’s based on, if it’s just theory or if there’s data. Is that covered in the talks or any blogs?

      In an ideal world, I’d love to fix everything fast =] Unfortunately, it seems that in reality, new shiny things often take precedence over fixing the last new thing, which is a little less shiny now from being all scuffed up. I wonder how the scales tip when we assume things can’t be fixed very quickly?

      Yes, the comments are great! Love seeing the healthy debate going on, and being exposed to new ideas.

      Thanks for the encouragement!

      Cassandra

  2. Great post. Further to what’s been said, EVERYONE on the team is responsible for preventing issues (though I prefer to call them problems). Testers are included in this, but the statement you’re objecting to has the effect of making it specifically the Testers’ problem, and relieves others of their responsibility. That sets up a conflict. It’s also just bonkers. As Michael says, we can HELP prevent them, as can others.
    I would say that we have skills that make us especially good at helping predict them, since many of the same questions we might ask of actual software can be asked of the idea of that software before a line of code is cut. But it cannot be somehow made a tester’s sole responsibility.
    Also some problems arise from reuse of old code or from very specific code combinations or conditions and cannot realistically be foreseen and hence prevented.
    However, even more dangerous, IMO, is the unstated companion idea that “therefore we shouldn’t really need to test. And any bugs that get through are the testers’ fault for not having prevented them”. I know nobody is actually saying that (please tell me nobody is saying that!!!). But it’s all of a piece with the move to have teams where there are no testers, only coders and to downplay the need to test. People working on toy software can get away with quite a lot, and if the PO doesn’t want much testing that’s on them. But for real software (calculating your pension or mortgage payments, anyone?) you will have to test, and test thoroughly.
    Prevention is, of course, good. But it’s not perfect and it’s not enough. That’s why we test

    1. Hi Paul,

      Thanks, glad you like it!

      That’s interesting, why do you prefer the term “problem” to “issue”?

      You make a lot of good points. Sometimes things happen that no one expected, like freak storms, to go back to the weather analogy.

      Yes, the trend of somehow absorbing the skilled practice of testing in to some generic “engineer” role concerns me. For many reasons.

      It’s hard enough to tell someone their baby is ugly. How could you look at your own baby and call it ugly?! Or turn to your partner and say your step-child is ugly. It’s likely going to be a conflict of interest and a distraction, in my opinion. All I need to do is look at big companies like Facebook who’ve adopted this practice to see what happens when you do…

      Agree, prevention is not enough and we need to test if we really want “quality” software, however we define that.

      Thanks,

      Cassandra

    2. Cassandra, I don’t like “issue” in this context as an issue is just something that flows from something else, irrespective of its value or desirability. So all of the features of a product are issues from the development process. But they are not (we hope) problems.
      I consider the word Issue (meaning problem) the same way I consider Resource (meaning people)

    3. Hey,

      So I guess that’s kind of like “issue” in the sense of a magazine issue?

      I’ve never thought of it that way before. Probably because “issue”, to me, has negative connotations.

      Perhaps I also see “issue” as the investigation part before assigning “bug” status… Is what we’ve found actually a problem, or does it stop at being a point of discussion?

      You’ve given me something else to think about…

      Thanks,

      Cassandra

    4. Cassandra… for what it’s worth, in the Rapid Software Testing namespace we talk about a bug as any problem that threatens the value of the product. We talk about an issue as any problem that threatens the value of the testing. So, for us, it’s possible that testers can prevent (or at least deal with) some issues on our own—through better self-management of the testing effort, for example). Preventing bugs is something that we can’t do on our own, but we can certainly help those who do.

    5. Hi Michael,

      That’s an interesting distinction I haven’t come across before.

      With new ideas like this (new to me), I like to go away and think of examples that would fit in with the perspective I’ve just been introduced to.

      In particular, “any problem that threatens the value of testing” isn’t something I can really say I’ve specifically considered before. Thanks for brining it to my attention =]

      Thanks,

      Cassandra

  3. Hi Cassandra,

    This is a great blog and a difficult message to sell at times. Both within, and external of, the test community.
    Amit raised an interesting point but I don’t think it is counter to your message. By turning on the light he didn’t prevent you stubbing your toe. You control your body and actions so Amit didn’t prevent the accident. Why he did do, by turning on the light, was to give you more information by which you could make choices. You could ignore the feedback and still stub your toe. I do like that Amits scenario, at least in my mind, links nicely with the Bolton/Back idea that Testers shine a light on dark places.

    Cheers
    Paul

    1. Hi Paul,

      Thanks for your comment.

      I agree – the same principle can be applied to that scenario, and arguably any similar one that involves another party making a decision or taking action (or not!).

      Yes, it’s a nice idea and image that testers can shine light on things. And what an image it is for teams to sit in the darkness without them =]

      Thanks,

      Cassandra

    2. Hi Paul,
      There’s one thing I don’t like in “testers don’t prevent problems” – it’s too easy to misinterpret as a waver to pass on the responsibility. For me, a tester is first and foremost part of a delivery team, and share the team’s responsibility to fixing problems. Not many are capable of doing the mental switch that Michael is presenting here of “when I do X I wear another hat”, and so are more likely to use this statement as an excuse (e.g.: “preventing problems is not a tester’s work, so what do you want from me?”).
      The difference between actually preventing problems and helping to prevent problems is not important enough for me to risk boxing testers in a limited zone of inaction.

    3. Hi Amit,

      That’s an interesting point. I agree that some struggle to switch contents / wear different hats / play different positions.

      From what you’ve said, I’m not sure which mindset you’d rather a tester have (assuming they are limited to one) – “I prevent issues” or “I identify issues”?

      In other words, would you rather a tester only try to prevent issues or only try to find the ones that exist?

      Cassandra

    4. Well, if I have to choose one, I think I’ll go with the more active “prevent” and broaden the definition of prevention to accommodate that – as Michael points out, one definition of prevention does imply having the final call on something. If I have to choose, I would define “prevention” to be anything that causes less issues in production (and again, I don’t consider a bug that was the PM decided to release unfixed to be an issue). If I can show that having a tester in the team is causing less issues to reach production, that would suffice for me to say that the tester is preventing issues.

      The reason I prefer “prevent” over “inform” is that I don’t really care a great deal about role definitions – it’s a great tool to drive learning and specialization, but as part of the delivery team, I expect everyone to ship with as few issues as possible, and if there’s someone who thinks their work ends once they “inform” someone, I think they should change their focus.

    5. Ah, so you’re counting anything before shipping to production as “preventing”? For the purpose of this discussion, I see it as hypothesis only – if you can actually test the software (pre or post production) and say, “this is the bug I found and this is how to recreate it,” then that issue already exists and you didn’t prevent it from existing, although you might, of course, prevent it from being in the shipped product.

      I’m not saying your interpretation is wrong, I just have a different one.

      To me, you can’t really say you’ve predicted an issue if the first time you’ve acknowledged it is when you’re staring right at it. Have you ever seen Mean Girls? Haha there’s a scene in it that goes something like this…

      Karen: I can always tell when it’s about to rain.
      Cady: That’s amazing!
      Karen: Well, I can tell when it’s already raining…

      That seems like the perfect illustration of the difference between prediction (you can’t prevent without first predicting) and identifying, to me.

      I noticed that you compared “prevent” with “inform”. My question was about “prevent” vs “identify”, which maybe does more to imply my interpretation as above, maybe not.

      But if we’re specifically talking “prevent” vs “inform”, I’d ask what other action you might want the tester to take? It sounds like we’ve pretty much agreed that whether to address the issue is more up to the PO or PM. Is this where we start considering “bug advocacy”?

      Such interesting discussion, by the way! Thanks for helping me consider other points and angles.

      Thanks,

      Cassandra

    6. Hi, Always…

      If you think it’s too easy to misinterpret the (true) statement “testers don’t prevent problems”, there’s an antidote: don’t just say it; explain what you mean by it. Discuss it. Have a conversation with the people you think me be at risk of misinterpreting. Get feedback from them, so that you can be sure you’ve been clear. Test them.

      Not many are capable of doing the mental switch that Michael is presenting here

      That’s demonstrably false; all people do it all the time. They may not be conscious of it in the moment. For instance: I’m a tester and a writer and a speaker and a testing teacher and a dad and a banjo player and a mandolin player and a husband and a father and a cyclist. People take on roles temporarily, or emphasize one over the other in a given moment.

      More here: http://www.developsense.com/blog/2015/06/on-a-role/

    7. Hi Cassandra,
      I’m not sure I follow you exactly on the prediction part, so I’ll go back to prevention – Probably the most accurate concept I can think of (right now) is “business value”. Any issue is causing harm to the team’s ability to provide business value. An issue in production is impacting things such as product reputation, number of support calls to our call center and stuff like that. An issue caught before release is hindering the speed in which a team can deliver value. So, as a tester (and as a delivery team), the ideal goal is to identify and address issues where they have the smallest impact on the business value (and usually, the earliest you deal with an issue, the cheaper it is). So, if I have to choose but one goal for testers – eliminating issues as soon as possible is preferred to only identifying them.

      Also, consider this – you don’t have to identify a problem to avert it. If I read the requirements with the developer and we discuss how we understand them – A whole horde of issues is prevented (because we know where we don’t agree on interpretation of the requirements and take steps to clarify them before coding is done). If I suggest to the developer a simpler design than what that developer initially planned – I help averting issues that would have risen due to complex design.
      Whether or not it would be precise to say that I prevented an issue is not important for me. It is important that I invest my efforts in order to prevent issues.

      What can I do to prevent issues?
      Well, You’ve mentioned bug advocacy, that’s one thing. I can also advocate for simpler design & more maintainable code. I can provide a suite of automated checks for the developer to validate the new functionality, I can go and fix a bug myself, I can add information that the developer was missing (“oh, Sally have worked on that code not long ago, did you speak with her?” or “keep in mind that we have a legacy process that parses the output you’re changing, be careful not to break it”). Sometimes, I can prevent issues by lending myself as a rubber duck (https://en.wikipedia.org/wiki/Rubber_duck_debugging ). In some of those activities Michael will claim (rightfully) that I don’t wear the tester’s hat while doing them. For me, being a tester requires wearing other hats quite often (I’ve heard Jesse Alford call this “patching the holes”, and I quite like this notion. https://www.youtube.com/watch?v=Nf9skPMaqYk&t=5m0s ).

      Michael – I am hanging to one word you mentioned: *consciously*. A person switching roles unconsciously is more likely to label things as “not my job”. If I get the opportunity to have a deep discussion with such a person, I will spend some time on asserting mutual understanding and shared definitions, and so will be able to say that preventing bugs is not a testing activity and that I expect the person who is tester in job-title to fill here and there some other roles that do prevent bugs. However, in the context of a blog post (which most readers I know skim quickly through), or shouting around from the top of a soap-box, I know that I don’t have the luxury of a deep discussion, so I would rather be inaccurate and driver the correct behavior, than to be accurate and misunderstood.

      One thing to consider is that “tester” is an overloaded term here: it is a role that can be assumed or dropped, but it is also self-identification. If, during a session where I’m writing requirements, you would ask me “what is your role?” I will answer “I’m a tester”. Testers who assume roles unconsciously will give a similar answer.

    8. Amit: A person switching roles unconsciously is more likely to label things as “not my job”.

      Maybe. Also a person not switching roles consciously, and a person not switching roles unconsciously.

      However, in the context of a blog post (which most readers I know skim quickly through), or shouting around from the top of a soap-box, I know that I don’t have the luxury of a deep discussion, so I would rather be inaccurate and driver the correct behavior, than to be accurate and misunderstood.

      What if being inaccurate drives the incorrect behaviour? That seems at least an equally likely outcome to me. Why not invite a discussion, as Cassandra did? (You and I had a deep discussion on this very issue on my blog almost a year ago, as I recall.)

      One thing to consider is that “tester” is an overloaded term here: it is a role that can be assumed or dropped, but it is also self-identification. If, during a session where I’m writing requirements, you would ask me “what is your role?” I will answer “I’m a tester”. Testers who assume roles unconsciously will give a similar answer.

      So, let’s talk about that answer, if there’s any doubt about what it means from one moment to the next. As we give an answer, let’s consider the context of the question, the motivation for asking it, the problems that we might introduce when we answer too quickly. In other words: let’s be conscious.

      Speaking of blog posts and confusion about roles, I wrote this post specifically to address that issue. (http://www.developsense.com/blog/2015/06/on-a-role/)

  4. I admire you for having written this piece. Alas, having reported a bug in the way some people speak, I fear that you will not prevent some of them from speaking as they do.

    But I’m with you all the way. Testers don’t prevent bugs, or defects. What we can prevent, to some degree, is unawareness; oblivion; surprises. That’s our contribution to the process: by raising awareness of problems and risks, we *help* prevent bugs. And that’s fine. But as you say, testing on its own does not prevent the bug.

    When we DO prevent problems we have, however temporarily, left the testing role, like a goalie leaving the crease and running downfield to act as an attacker. That’s okay too, as long as the testing role is covered; as long as some attentive defender is covering the net soon enough.

    http://www.developsense.com/blog/2016/05/testers-dont-prevent-problems/

    Cheers,

    —Michael B.

    1. Hi Michael,

      Thank you, that’s very kind.

      No, I don’t think some will change either but I hope, at least, that reading this post might make those people at least think about the words they use. And that if they continue to speak in the same way, at least that’s a conscious choice they make.

      Another great analogy (I do love them!) that really helps to visualise the roles in a team and the importance they have. I like it =]

      Thanks,

      Cassandra

  5. I recently spoke on this topic at Software Test Professional conference – Spring 2017: http://www.stpcon.com/sessions/a-workflow-that-works/. I also guest-blogged on this topic back in May 2016: https://mysoftwarequality.wordpress.com/2016/05/06/testers-dodont-help-preventdetect-problems/.

    I wholeheartedly agree that “words matter”.

    And so, in my opinion, there is no answer to the *general*, “casual” question “Do testers prevent issues?”. And no response to the *general*, “casual” statement “Testers prevent issues.” Trying to answer or respond to these *general*, “casual” questions/statements might lead to misconceptions and damage.

    Instead, I think one should consider a number of *specific* factors (such as “Which perspective are we considering?”, “How is ‘issue’ and ‘tester’ defined’?”, and “Based on the helping verb used, what might the question/statement actually be intended to mean?”) in a *specific* context in order to properly determine an appropriate answer or response to a *specific* question/statement.

    1. Hey,

      Thanks for your comment.

      I think I understand what you’re saying. If I’ve picked it up correctly, it’s about fully understanding the question before you try to form an answer?

      If that’s the case, then it seems that we agree on the principle, but that perhaps we’re looking at it from slightly different angles. This is interesting, as it seems to highlight exactly what I think you’re talking about – perspective.

      I suppose the perspective I was coming from was more about expectation than general statements. “You will do this” vs “you guys technically do this, right?” One is casual in the sense of hoping for no particular consequence (and can therefore be taken more lightly, in my opinion), whereas the other is almost accusative and makes me want to say, “wait a minute, let’s talk about this!”

      I guess that’s my round about way of saying that context matters, as well as words. However, I still think it makes things far more straight forward to consistently practice using the words which most closely reflect your meaning. That way, you constantly try to reduce misunderstanding and make it far easier to choose the right words for you when it matters most.

      Hope that makes sense, and would be interested in any more thoughts you have on that.

      Thanks,

      Cassandra

  6. But I like preventing issues.
    When you enter a dark room and stub your toe, it’s painful. If I lit the room as you entered, would you say that I prevented your toe from being stubbed ? I would. Yes, you can still hit the table’s leg if you’re a bit careless or distracted, but the chances of this happening are lower – in the long run, me being there to light up the room will result in less time of sore toes. I may not be able to prevent every issue, and I sometimes need to work through (and with) other people to eliminate some of the issues, but the correlation between me (or any other decent tester) being in a team and less issues reaching production is not coincidental.

    Also, if we are attentive to using the right words, I really liked that you use “prevent issues”, and not (what I hear too often) “prevent bugs \ defects”. If a tester found a bug, and the PM decided to release the software with the bug unfixed, the software still has a bug, but this bug is no longer an issue – it’s a risk we’ve decided consciously to take (and yes, I say “we” even if it’s the product manager personal decision that I don’t agree with, since by working there I accept the approach that gives the business people the right to overrule my recommendations).

    1. Hi Amit,

      Thanks for your comment. I like your analogy and I see your point, totally agreeing that the correlation between having a good tester and less issues is absolutely not coincidental. I’m sad to say I know people who think it is coincidental, despite having data to the contrary.

      However, I also agree with your comment around “we”. If we, as a team, can share the decision to leave a bug unfixed, why can’t we also share the responsibility of preventing issues as a team? Instead of implying this is just the responsibility of testers (and only pointing the finger at them if issues still arise), wouldn’t it be more reasonable to act as a team earlier on too?

      I think the idea of team responsibility can be a separate issue too. I’ve witnessed mistakes by developers being absorbed by the team, whilst mistakes by other team members are left solely with that person. That doesn’t seem right to me. But as I say, I think that’s another, related, issue.

      Thanks,

      Cassandra

    2. Oh, wow, I completely missed the jump I did there. You are 100% right – It’s the same “we” that prevents issues, and the concept of team responsibility is so embedded in my thought process (I’m lucky to work in a team that is great on this aspect) that I didn’t even think on implying responsibility and finger-pointing.
      I think that I’m most comfortable with saying that as a tester, my contribution to the team is (also) addressing issues. Other team members do this too, but since the tester role is more oriented to risk mitigation, it makes sense that I focus a bit more than the others on issues.

    3. You’re very lucky indeed! Yes, I’m comfortable with that too and it does make sense for team members to have a slightly different focus. For me, that’s what a cross-functional team is, rather than having everyone share the exact same split of focus.

      Thanks again for commenting, this was fun =]

      Cassandra

Leave a Reply to dsynadinos Cancel reply