November QASIG: Service Virtualization in Action: How Alaska Airlines tests for snow storms in July

#ServiceVirtualization | November 8 | QASIG-Seattle

Fall in Seattle

“Service Virtualization in Action: How Alaska Airlines tests for snow storms in July” is a QASIG-Seattle talk being presented by Ryan Papineau, Automated Testing Engineer at Alaska Airlines during the upcoming #QASIG #SeattleTech #Meetup at #Quardev | reserve your seat

Why did Alaska Airlines receive J.D. Powers’ “Highest in Customer Satisfaction” recognition for 8 years straight, plus the “#1 on-time major North American Carrier” award for the last 5 years? A large part of the credit belongs to their software testing team’s proactive approach to disrupting the traditional software testing process.

The team uses advanced test automation in concert with service virtualization to rigorously test their complex flight operations management application, which interacts with myriad dependent systems (fuel, passenger, crew, cargo, baggage, aircraft, and more). The result: operations that run smoothly—even if they encounter a snowstorm in July.

Attend this session to get a first-hand account of how Alaska Airlines leverages service virtualization to address common testing challenges and to learn Alaska Airlines’ best practices for managing the complexity of multiple dependent systems for testing.

About Our Speaker | Ryan uses systems engineering, cross-team collaboration, along with data analytics to provide complex test environments that behave like production.

Join us for this and other upcoming QASIG-Seattle events by reserving your seat at www.qasig.orgEvents are held at QUARDEV 2707 NE Blakeley Street in Seattle. Doors open at 6:00 for networking and nosh, presentation begins at 6:30 – all are welcome.

Join the Conversation

About #QASIG Seattle

QASIG seeks to engage members of the high-tech community in discussions on the evolving challenges of software quality assurance and look at ways to advance the leading methodologies and best practices.

Challenges Ahead

As software development continues to evolve, the way in which we approach quality assurance and software testing must keep pace or become obsolete.

Consumers demand software that is available, reliable, and secure. It is important for us to continually re-evaluate the QA and testing processes we are employing and why we are using them.

We hope to explore these and many other questions facing the software development, quality assurance, and software testing industries in upcoming QASIG meetings and invite you to take part in the dialog.

Join us for this and other upcoming events by reserving your seat at

Baby on Board

An honest admission:

I became a father this past February and I *still* don’t know what to do when I see a “Baby on Board” sticker on the car in front of me.

Seriously. What does that driver want me to do? NOT carjack them? NOT honk if they cut me off, else it wake their sleeping baby?

Now when I drive and baby Charlotte’s in the back seat, I *still* don’t know what people want me to do when I see their sign. But I did think of an interesting reason to have one.

One day while I was driving, Charlotte dropped her pacifier and shrieked uncontrollably. As I fussed and impulsively reached back to find it while driving, it occurred to me that I should pull over because I was beginning to drive a bit erratically. In other words, there was an internal system in my car that other drivers couldn’t see.

What would a sign have done? Maybe it’s more to connote risk. “I may start driving like an idiot because I am subject to the whims of a child capable of reaching 150 decibels at any second…”

If only software came with such signs… but then I guess I’d be out of a job and wouldn’t be able to afford one of those nifty little “Baby on Board” signs…

Expert Game

Twelve years after I started my testing career, I still feel like a student.

That’s why I felt so honored last week in being asked to give the Commencement Address for ITT’s spring quarter graduation ceremony.

One of their career advisors had seen me give a guest lecture to a software development class and thought I’d be a good fit, and I knew right away what I wanted to say to them.

I mean, I imagined a few rows of graduates in cap and gown, tassles ready to be turned, ready to party. What could I possibly say to them that had any hope of getting their attention?

What came to mind was something that helped me when I first got started.

I decided to tell them this:

“In programming, there’s a concept known as an If / Then statement. If condition A is met, THEN do instruction B. The code is compiled, it runs, it looks for the condition, and executes some kind of algorithm when it finds the right condition.

But in my career as a test manager, Ive learned that humans are programs, too. We are only as good as our programming. We get tired, nervous, moody when certain conditions are met.

So I offer this snippet of code to add to your existing programming in case those conditions occur. This is that If-Then statement:


  • you are able to get important things done
  • you are seen learning things on your own
  • you are seen trying to do things even though you don’t know how
  • you don’t hide your ignorance, but also don’t rest on it
  • you share freely the things that you know
  • you honor what other people know
  • you know more often than not how to find out what you don’t know
  • you know how to ask for help
  • you offer help to people ON THEIR OWN TERMS


  • no one will care whether you succeed by learning or succeed by already knowing
  • no one will care if you mess up occasionally because they assume you learn from it
  • no one will care if you forget (or don’t know) any given fact or method at any given time

… and you will be treated as if you’re smart and useful, even though everyone knows you have a lot to learn

This is a formula that has been true for me, a journalism major, author’s son, and autobiographer philosopher who is now considered an internationally recognized expert in exploratory software testing.

Baby Oracles

On February 16, I became a father.

This is related to testing in more ways than I can count, but let me give you my favorite.

Ever since I announced my wife was expecting, all kinds of parents have given me all kinds of advice, and now it’s time to fulfill the promise I made to most of them — to report how right they were.

In testing, “rightness” is determined by oracles — methods or principles used to recognize a problem. Without oracles, testing isn’t testing, it’s “touring.”

Oracle #1: Most full-term babies weigh between 5 and 9 pounds.

This oracle is probably why the medical staff gave an audible “whoa” when little Charlotte weighed in at 11 pounds. It’s also why I said “whoa” when another baby delivered that day weighed in at 13 pounds.

Oracle #2: Babies eat every two to three hours.

Yep, and here’s an interesting systems-thinking corollary. Charlotte, being a bit bigger than most newborns has a stomach that holds more milk, which means she’s satisifed for a longer time, which means she sleeps longer, which means *I* sleep longer.

Oracle #3: Babies are sometimes jaundiced when they are born.

True for Charlotte, but the other oracle here when a nurse said that her “Billy Rubin” level needed to be watched over the next couple of days. I didn’t know who Billy Rubin was, but suspected he was either a doctor or a scientist who had something to do with the discovery of jaundice. Now, some of you are laughing at this because you have an oracle that tells you that “Billy Rubin” is actually not a person but “bilirubin” — an artifact of the blood that the liver needs to process. I found this out by doing an internet search — a common course of action for all concerned parents who are hear a doctor’s concern and need to know more. Google is the world’s greatest source of oracles. One of my favorite Google hits was a reference to the novel, Silence of the Lambs. Hannibal Lector, trying to be clever, led the kidnapped girl’s mother, a senator, on a wild goose chase by saying the killer was Billy Rubin. Clarisse later figured out it was false oracle.

Mythbusters is my favorite show on TV right now because they take urban legends that have become “fact” by so many people who trust them (e.g. “drinking pop rocks and soda will make your stomach explode”) and they stage experiments to prove or disprove them.

So when you’re testing, think about how it is that you know what you know. Are you sure?

What is rapid test planning?

Rapid test planning is a process of cultivating test ideas within minutes, not days. Its opposed to rigorous test planning which strives to find every possible test. The trade-off is time vs. thoroughness because often were given tight constraints for testing. Management wants test results quickly and we can either freeze, get angry, or see what we can do.

I’ve experienced paralysis and anger many times in my 12 years of testing, but the more I stay in this business, the more I find myself seeing what I can do. These days, I consider it a challenge when a client says you have three days to test.

Enter rapid test planning. For me, it involves one or two exploratory testing “sessions” that result in a list of test ideas, issues, and risks. Exploratory testing is test design and execution happening at the same time, sometimes in the same second, as the tester gathers and analyzes data. The tester decides whats important, and the emphasis is on learning about the product.

A session could be 30 minutes or an hour or three hours, but the mission is the same what can we test?

At Quardev, clients usually don’t give us a lot of time to test, so our promise to them is to focus more on delivering information than creating written test cases. Thats why I often use rapid test planning.

Think of an American-style football game. You have 4 quarters, or roughly one hour of clock time to beat the other team by designing plays that advance the ball down 100 yeards of turf while the other team tries to stop you. Plays can involve throwing the ball or running with the ball until you reach the end of the field for a score. When the other team has the ball, it’s your turn to stop them from scoring, so each team has plans for offense and defense.

Let’s say my football team is the software were testing. Lets also pretend that I am a coach being interviewed. The sports reporter asks me a spontaneous question during the off-season to catch me off-guard to get a good sound bite for his story:

If you were playing the Seahawks today, what would be your game plan?

In answering rapidly to give them a great quote, I might think in terms of the following 5 elements:

  • Structure: The team’s composition.
  • Function: The team’s capabilities.
  • Data: What the team might “process” – i.e. what we do with the ball.
  • Platform: What the team depends upon.
  • Operations: How Ill use my teams skills.

Using a mnemonic like SFDPO is considered a heuristic a fallible method for solving a problem. Its not meant to be thorough or foolproof, just enough to start you on the road quickly to a useful solution. Using this particular heuristic (remembered as SFDPO or San Francisco Depot), I might say something like:

Ive got a deep bench (STRUCTURE), so Im confident that our past performance on special teams against the Seahawks has been great (FUNCTION) when we pass the ball instead of run it (DATA). Given my star players arent that healthy right now (PLATFORM), Id start them in the third quarter, especially if that wind picks up out there (OPERATIONS).

This sound bite gives you an idea of how a coach might inventory their ideas on-the-fly. As coach, I might be wrong about the plan since its just a gut check, so Ill likely adapt it once the game starts and get more information.

For more heuristics like SFDPO to use in rapid test planning, click here.


Ok, so heres how I do it:

  • The gauntlet is thrown down: Test this! You have 3 days.
  • I ask questions to get more context for my mission or information to enhance my plan
  • I start the session (setting the table for dinner, in effect), asking for anything I can get:
    • the software
    • specs or product docs
    • project docs
    • marketing literature
    • names of knowledgeable project staff who can help me later
    • a projector and a PC with a web connection (to do on-the-fly research)
  • I think of things that trigger other ideas
  • I end the session and go over it with as many stakeholders as possible
  • go to step 2


Declare that the plan is good enough for now to get testers engaged on their own exploratory testing sessions to find bugs with minimal supervision. Click here for ideas on how to tell if your plan is good enough.

For more on the mechanics of exploration and analysis, see this paper.

Other good sources:

  1. My brother James Bach at his testing consulting company Satisfice, most famous for offering the Rapid Software Testing course
  2. Michael Bolton, principal consultant at DevelopSense, who also gives the Rapid Software Testing course.
  3. Robert Sabourin, principal consultant at Amibug.
  4. James Lyndsay, principal consultant at Workroom Productions.
Scroll to top
Close Bitnami banner