I’ve been to many testing conferences and workshops in my testing career (maybe you have, too), and I’ve noticed an interesting pattern. Speakers talk about all kinds of methods, techniques, approaches, technologies, tools, heuristics, mindsets, skill sets and even theories that will help you find bugs, but rarely what to do when you succeed in finding one!
I mean, is it a forgone conclusion that all you need to do is file a report about what you did and what you saw?
Well, wait a second…
You can file that bug, but are you ready for the questions that may come back to you? Are you willing to wager your reputation on the details you put in there?
It’s not hard to celebrate and quickly throw in a bug report, especially when time is short. But I’ve seen times where it’s worth more time – sometimes just a minute or two – to think before filing. And these minutes of investigation may increase your credibility.
For example:
Is it isolated to your machine?
Does it happen every time?
Can you make it happen a different way?
Does it happen on another environment?
Is this bug a fault or just a surface failure of a fault?
What tools might help?
Have you seen this before?
How could this be fooling you into something that may be by design?
Is any part of the system changing underneath your testing?
Are there any dependencies to the feature that might be failing?
Follow-up tests: “what if I try X?”
What are my assumptions?
Are my oracles well-defined?
Would looking at the code help?
What if you re-ran the tests and changed one of the test factors?
The point is now that you have the bug in your sights, is it really trapped or is there more you can do to unearth it?
When you’re driving on the interstate, you may have noticed some grooved lines in the shoulder pavement. These are meant to cause your car to vibrate if you stray off the road, to alert you to a bad problem that may happen if you continue to veer off course. It’s called the “rumble strip.” In testing, the notion of the rumble strip is there, too. It’s behavior that stimulates your notion of a problem. Treat this alert not like the bug it seems, but as an alert to a much bigger bug on the other side of that guard rail. Sure, you can file your account of the behavior you see initially, but try continuing off the cliff the rumble strip is trying to protect you from. It’s just software, but you may find yourself being able to tell a better story of a more spectacular crash.
——————-
Jon –
This is in response to your blog about “bug investigation.” While I am not a software tester or developer myself, this whole notion of diagnosis versus treatment is very common in my line of work as a psychotherapist. Some of the greatest diagnosticians can’t treat to save their souls, and some of the best clinicians havea hard time nailing down an actual diagnosis.
I have often thought that the whole idea behind this is the focus of what one is aiming to see. If one is able to look for bugs, then one can find them. If one is able to fix the bug, usually the “bug seeks him/her” and not the other way around. Similar to the concept of when the student is ready, the teacher will appear.
Best regards –
Kathleen B. Shannon
