Here are the notes I took after Mark Hanf, a detective from Seattle CSI came to speak to (QASIGwww.qasig.org) on 1/18/07:
- “We are asked to go to different locations”; parallel: think about testing on different computer platforms.
- “Look up, not just straight ahead”; parallel: change your perspective when thinking of software tests to run.
- “Look in the garbage; we go into toilets quite a bit”; parallel: software bugs could reside in places we don’t associate with normally having problems.
- “Proper documentation with photos”; parallel: we often document our tests and report our finding with screenshots.
- “Can’t be afraid of heights”; parallel: can’t be afraid of testing on new platforms.
- “Sometimes you have to match the bullet even though the crime is ‘solved'”; parallel: even though you have found the bug, there may be another cause.
- “Crime scenes might have CS gas residue”; parallel: “we may be digging in an area that complicates our ability to find bugs.
- Tools: reflective UV imaging screen, forensic stepping plates, sifting screens; parallel: we have special tools as well (inControl, log file tracing, LoadRunner).
- “We must gather, document, and demonstrate in court that we did everything possible”; parallel: software projects have “bug juries” that we are often called in to testify in front of to make our case.
- “We study different disciplines: entomology, odontology, etc.”; parallel: we also study different domains… cognitive psychology for usability, brain physiology, and Crime Scene Investigation!
- “Everybody’s interested in coming in and going right to the dead body”; parallel: We go right for the features that attract us or that are easy to test.
- “Detectives should cut their own path to an outside crime scene”; parallel: there is more than one way to reproduce or find a software bug.
- “You get to the scene, are briefed in an initial walkthrough”; parallel: we have client kick-off meetings that tell us what to focus on and where bugs may likely be hiding.
- “Footwear impressions and fingerprints are there whether we see them or not”; parallel: same is true for software defects… they are almost always hidden.
- “Sometimes you’re concerned about the floor, but can’t deal with it then and there”; parallel: bugs mask other bugs… we’re concerned about one feature but may not have time to test it right then.
- “Take photos with scale and without scale”; parallel: when filing a bug, think about its impact not just to the user but on other programs on the system.
- “Juries expect a lot more, so in some cases, we have to entertain (re: animation) as well as inform”; parallel: sometimes filing a bug is not enough, we have to be an advocate for what we find.
- “Defense attorneys could discount elements of our case, so we have to be thorough and careful”; parallel: same is true when we deal with programmers we have to anticipate scrutiny.
- Photogrammetry‚ a series of digital photographs in succession; parallel: we have mouse click and keystroke recording tools to document the repro of bugs.
- Talked about how a boyfriend/girlfriend got into a fight and then violence happened; parallel: we develop user stories and scenarios to test for bug pathologies in software.
- “We can’t say this is what happened, but we can give a logical range of possibilities”; parallel: we’re not always sure what the fault is, but we can suggest possibilities.
- Projectiles go through glass and leave different signatures; parallel: same is true for bugs… programs leave different signatures on how they use memory or install files.
- “We have to do presumptive tests sometimes (like the bullet through rubber)”; parallel: we also have to check our basic perceptions to make sure that a bug is really what we think it is.
- “We take elimination fingerprints to rule out different suspects”; parallel: we do follow-up tests or peripheral tests to rule out other causes.
- “Keep an open mind… don’t make your evidence fit your theory”; parallel: be mindful of your biases… don’t be fooled into thinking that this is a bug you’ve seen before.
- “There is a high cost for processing evidence, homicides get priority”; parallel: there is a cost to doing tests… high risk features that lead to crash, hang or data loss get priority.
- “Temporal evidence: fingerprints can last a long time… might have been there from months before”; parallel: this is the Primacy Bias… a bug might have started weeks ago and shown itself now.
- Harris vs United States, 1947: Only human failure to find it, study and understand it, can diminish its value; parallel: exactly the same for software testing.
- “Two heads are better than one; parallel: paired testing, “fresh eyes find bugs.”
- Staff: Team Lead, Sketch preparer, Photographer, Recorder, Specialists; parallel: Team Lead, Recording Tools, Subject Matter Experts.
