Case Study: New Software Release Regression Testing

Overview:

Our client is a longstanding provider of label printing software with more than 30 years of experience providing services to the world’s largest companies. Their flagship software works to create, automate, and manage labels, barcodes and RFID tags and is available in more than 150 countries.

The project involved testing services for the latest version of the software covering functional, database, and security aspects. This client works to improve safety, efficiency, and compliance and is depended upon by numerous businesses of all sizes to optimize customers labeling and printing processes. To ensure a smooth and functional transition into a new version of the software, Quardev was tasked with running regression and exploratory tests to help prepare for the software’s release.

Challenge

The key challenge involved in the project was the large number of tests to be executed in a short amount of time. With a 60-day timeframe, the client supplied roughly 2500 regression test cases for existing features to be completed. In some test cases test steps were missing, making it more complex to learn a new system and understand the intent of the test at the same time. Additionally, the project involved using a sophisticated tool with many features, making it more challenging to learn than most. The project included collaborating with remote teams located in Bellevue and Taiwan, causing some challenges in communication and scheduling.

Solution

To counteract the short time frame, several skilled test engineers were assembled, and a team lead chosen. Team members were quickly brought up to speed and the tests were divided and executed. To overcome the complex tool, the team began having internal learning sessions where there would be a presentation and demo of different functional areas. In this way, team members were able to learn a space and share knowledge with others on the team who may not work in that area but have a dependency on that system.

The project involved regression test execution, taking about two months. When the team encountered test cases that were missing steps or had steps that were out of date and needed to be updated, a corresponding Quality Assurance (QA) JIRA ticket would be opened. The team added steps for some of the test cases that had none as well as suggestions for steps that needed to be modified or where steps were missing. These updated steps were written so that they could be approved and updated to the existing test case with a simple copy and paste - reducing the amount of time the client needed to spend on the effort. The defects and QA issues were logged in JIRA as well as a weekly status report provided to project management.

The final part of the project was to provide a list of recommendations to the client to help improve current QA methods. Additionally, if a test was missing steps or the expected results, the team would add a relevant detail in the collaboration tool to enable testing to continue and provide the detail needed to execute the test. Finally, Quardev recommended setting up a Final Verification Test (FVT) regression pass for after the initial pass had been completed.

Technical Implementation

Due to the nature of this project, implementing new technology was not a factor as the goal was the execution of existing tests. In order to spin up a clean environment from which to run the tests, the VirtualBox tool was used.

Results

The project was completed on time and well within budget. The client was happy with the work performed as well as well as the extensive reporting and metrics provided. This had been the second time Quardev worked with this company and following this experience the client noted they would absolutely engage in Quardev services in the future.