Have you ever done acceptance testing?

In a previous post , I mentioned my role on the Microsoft Patternsand Practices team as they produce a guidance book on acceptance testing. We’re right in the thick of production and one of the ideas I’m helping them flush out is the notion of what acceptance might be in as many contexts as possible.

If you’ve done acceptance testing, either on the Delivery side or the Customer side, we’d like your help.

This is a link to a survey that will help us expand and focus our ideas.

Forward the survey link to as many people as you’d like. The results will be read by me and three others on the team. If you have ideas for questions we could have asked about acceptance testing, but didn’t, email me.

Thanks!

Be Joe Schmoe

By Nat Burnett

When a person heads to Best Buy and purchases a brand new software application they take it home and set it on the kitchen table, make some lunch, and then carefully open the box.They then spend 4 hours reading the documentation, making sure they know how to operate it most efficiently and that they have the necessary Operating System, RAM, CPU, video card, etc.

Okay, okay you got me. This never happens!

Most likely they would rush home, install the software on their computer and start fiddling with it right away, no matter how much or how little they knew about the application.Most people learn due to action, instead of exercising action as a result of learning.While some people may take the time to read the manual first or as they go along, many, if not the majority of people, will use exploration as their learning tool and only when exploration leads them to a dead end will they resort to reading the manual.

These types of users are becoming more prevalent in the technological world as more and more people expect applications to be intuitive and accommodating for the inexperienced user.

This is why exploratory testing is so important.Strive to explore your application because you can guarantee the users will do the same. Scripted tests may be suitable to exercise scenarios that involve people who follow the guidelines of the manual and spend those 4 hours before firing up the application.

Questions like, “Following this route, can I get from one end of an application to the other?” are questions easily answered by a scripted test.

On the other hand, exploratory testing caters to those users who like to dig in and learn as they go.Exploratory test may fabricate a question such as, “How can I get there and what might happen along the way?”

The great thing about exploratory testing is that while it can not be entirely scripted, it can be recorded and reused for both tracking reasons as well as overall testing efficiency (Session Based Test Management is designed to record, organize, and track exploratory testing, See Jon Bach’swhitepaper on the subject in our Articles and Whitepapers section).

Follow up on Bug Investigation

Earlier in this space, I wrote an entry about Bug Investigation. But there is an important component I did not even begin to talk about – bug advocacy.

Bug advocacy means making stakeholders want to fix the bugs you find by finding effective ways to communicate. It might mean overcoming objections, motivating stakeholders with “salesmanship,” and researching new context about what you found.

“The best tester isn’t the one who finds the most bugs or embarrasses the most programmers,” says Dr. Cem Kaner, Professor of Software Engineering at the Florida Institute of Technology. “The best tester is the one who gets the right bugs fixed.”

Cem is the principal author of a course titled “Black Box Software Testing” and has given me permission to post the following links to his course material involving bug advocacy:

https://www.testingeducation.org/BBST/slides/BugAdvocacy2008.pdf

Videos:

https://www.testingeducation.org/BBST/videos/BugAdvocacy2008A.wmv

https://www.testingeducation.org/BBST/videos/BugAdvocacy2008B.wmv

https://www.testingeducation.org/BBST/videos/BugAdvocacy2008C.wmv

https://www.testingeducation.org/BBST/videos/BugAdvocacy2008D.wmv

https://www.testingeducation.org/BBST/videos/BugAdvocacy2008E.wmv

https://www.testingeducation.org/BBST/videos/BugAdvocacy2008F.wmv

I am a Software Test Engineer. I am a Software Tester. I am a Software User.

By Nat Burnett

Only the creativity of the tester limits what or who the tester can be.

Any person performing exploratory testing should be all three of these people at once. Depending on the application you are testing you may have very many or just a few shoes to fill. However, even with something as simple as a calculator you could have hundreds of different personal profiles. Determining who you are while testing and why is very important.

Keeping simplicity in mind, let’s use the basic calculator again. A tester of this tool may benefit from creating a scenario in which a woman is adding up some bills for the past month.

Of course there are certain things that will happen. Most bills have decimals, so she will definitely be using the decimal button. She might want to add them all up and subtract one bill from the total that she has paid already. You can be as creative as you want with this.

I like to create a scenario for the user, with a particular environment and often a current state of mind. For example, in my scenario the woman is in a rush to leave for work but has to add up her bills before leaving. She has experience with the Windows calculating tool and has been comfortable with using it in the past. Depending on relevancy to testing the application, you could add that she attends church every Sunday and has a dog named Spot, but for the calculator this might be a bit much.

When you create a scenario think about what things might be affected by the environment or current situation. She is in a rush, so things such as button size, readability, placement, and overall functionality of the calculator will have an impact on your testing.

Her prior use of the Windows calculator is important technical experience that will influence her use and expectations of our new tool. For example, are the buttons placed in a standard format?

In order to immerse yourself you might have to do some test setup.

If the tester is not familiar with the Windows calculator, he or she must become familiar with it in order to carryout the scenario. Now testing essentially becomes role-playing. You are a tester who has engineered a test around a specific software user.

With this scenario you might notice things you would not normally notice. For example, that the (+) is on the completely opposite side of the keypad, or that the whole key map is different.

And why stop there? You might wonder about other models of calculator so you pull out a TI-82 from your Business Calculus class in college. You may see differences and similarities, you will gain knowledge and be able to draw reasonable conclusions as to why different calculators were designed a certain way.

With this knowledge you have the ability to ask more questions and further reciprocate upon that which you have gained through the act of test and engineering that test as well.

My name is Nat Burnett and I am a Software Test Engineer

That title has a decent ring to it don’t you think? Perhaps the word “Engineer” reminds you of your early childhood playing with K’nex, Legos, or setting up that model train set that eventually snaked its way through your entire basement. Those are the things I thought of when I was given the title and as a novice tester I did not see myself as having anything in common with an engineer. I wondered why I was given the title, “I’m just a tester… a QA guy…” I pondered. Over the next few months I thought about this quite a bit.

More and more I began to realize my roles as a tester.I have a friend who is a civil engineer. For his senior project he examined the I-5 corridor from Tacoma to Everett coming up with hundreds of projected problems and issues that need to be addressed in the next 10 years. That was very impressive to me, and as I tested applications I realized why I was given the title of “Engineer.” My friend developed ways to survey the I-5 corridor in order to rate its current condition. He found problem areas and pointed them out, and often proposed what he thought would be the best fix. In the same way, I develop testing strategies to test an application’s integrity and usability. I find problem areas and point them out and am often involved in deciding the best way to fix a problem. Staying with this analogy, both I-5 and software provide a way for people to get from point A to B. My duty as a Test Engineer is to make sure people get through the software without running into slow moving traffic, road blocks, traffic accidents, or closures. In addition to this, my friend often had to think about the impact work on the freeway would have on side roads. I too must test the paths less traveled due to the possibility of weak links causing problems with the main application. In addition, there is no one way to get from Everett to Seattle, just as there is no one way to get through an application. All routes must be tested and given priority based on what has happened in the past, what may happen in the future, and other environmental or situational factors.

Different people may take different routes to work due to road conditions, weather, or because of what someone told them about a particular route the day before. Likewise, people use software in different ways because of training (or lack thereof), the task at hand, or the way in which they feel it was designed to be used. To further explain this, imagine giving a very simple application to ten people and explaining to them what it does. Each person will explore different areas for different reasons and each person will find how the application works best for their needs. This is why it is important as a Test Engineer to think as a consumer.

Scroll to top
Bitnami