The Lifecycle of Agile Testing

Over the past few years, we have moved our approach of development from the Waterfall model to the Agile Methodology. The Agile model provides an iterative and incremental building approach for software development that helps teams deliver value to the customer faster and provide scope for quicker changes to the software rather than building the software once for all and delivering it to the customer with less scope and adaptability towards change. The testing practice followed in the Agile methodology starts at the start of the project and includes a continuous testing approach all throughout the iterations until the delivery to the end customer, unlike the Waterfall model, where it is a single phase.

In this article, I will be referring more towards the lifecycle of testing involved into Agile. Please Note the article would be inferencing words from the Agile world. Since I may not be defining them here, hence an explanation of those have been provided via source links.

When to Begin?

As part of an Agile team, before the kickstart of any testing activities, it's important to understand the release plan and planned user stories across the sprints in a release. A storyteller (a document enlisting the features or modules to be implemented in the upcoming releases) will be provided by the Product Owner which would be compiled into multiple epics and users stories. All user stories are distributed among sprints based on their priority, business needs and team capacity by the product owner. Post awareness on the release plan and features, the first building block of testing activity starts with a test plan.

Test Plan

A test plan is created for each release and needs to be updated after every release. The main purpose of creating the test plan before the start of a release is to outline the scope, test approach, entry and exit criteria for the release. The exit criteria becomes the acceptance criteria for the release to be pushed to UAT and approved by the Product Owner.

For details on how to create a test plan, refer to the Test Plan.

Sprint Planning

The next phase of testing activity starts in sprint planning. The team sits together to discuss the efforts involved across the planned user stories in a sprint and any dependencies involved in terms of development or testing. The team coordinate together to achieve the required velocity of the sprint. Efforts across a user story are defined via story points, which are the efforts involved in both development and testing. Therefore, during sprint planning estimation across user stories for the testing efforts are provided. On how to perform test estimation please refer to this article about test estimation. Please note story points across user stories always follow Fibonacci series [1,2,3,5,8].

Post sprint planning, work allocation across the sprint is designed, wherein the testers involved in the project are assigned user stories in the sprint for test case creation and execution

Implementation and Execution

The next testing activity involved in Agile is the creation of test cases across the assigned user stories. Post creation of user stories, peer review is performed. This activity provides a two-way value wherein the other QA gets awareness across the feature and helps to provide inputs for better test coverage. All test cases created in the test management tool must be linked across the stories. This helps to ensure all stories have the required test cases across them. For a better quality of deliverables, developers can view the test cases across the story and ensure on an overall basis that the code handles the scenarios enlisted. This helps to ensure a smaller cycle and lesser bugs across the story. Basically a quick delivery. Please note, as a good plan rule, all test cases should be created when the stories are under development. Once the stories are ready for testing, only execution across the story should be the left out task. This helps speed up the Agile process. Creating test cases after the stories are ready for testing just adds on the time for the closure of the story.

Post creation of test cases, once stories are ready for testing, the test cases created should be executed across it. Another good practice while execution is to ensure the assigned tester who created the test cases should not be executing them. The reason behind it is that the person would just execute the test cases and won't explore the further area via ad-hoc testing, since according to him all test cases across the features have been written and he could not think more out of it. But a person who is executing the test cases may come across more pointers and may perform ad-hoc testing, also. In an Agile environment, ad-hoc testing plays an important role due to the shrinking testing cycles available. During execution, all testing types should be implemented be it functional, UI, usability, cross-browser, etc. You can refer article on how to ace your execution skills for it.

All bugs encountered across stories should be logged in the bug management tool and linked with user stories as well as test cases. A triage meeting is conducted to propose a plan for the bugs logged and their fix. Post the bugs are fixed, they are retested across the stories and closed.

Sprint Closure Activities

Once all the bugs linked to the stories and test cases are retested and closed, the user stories are also closed and are made ready for the Product Owner to be accepted. A sprint review is performed demonstrating the deliverables of the sprint. If any story within a sprint does not gets closed due to outstanding bugs or development work pending, those get overflow to the next sprint. It is vital to understand a story is considered a deliverable only after testing activities are closed and not just with mere development work completion. A defect analysis report will be created after every sprint.  This report can also be created once the release is over rather than across each sprint end. A sprint retrospective meeting would be conducted to discuss what went good, bad and what should be done to ensure better productivity of the next sprint.

Regression

After all the sprints in a release are complete, regression testing will be performed. A regression suite would be created from the test cases of the features planned in the release along with the impacted test cases of the previous releases. The regression suite would be approved by the Product Owner. Based on the approval, a regression estimate would be prepared before kick-starting regression that would outline the time required to complete off regression.

Daily status emails would be shared with the stakeholders highlighting the execution status and the number of bugs encountered. Depending upon the number of bugs and complexity of business involved, the regression may be divided into multiple cycles. Retesting across the bugs would also be performed in those cycles. Post regression completion a sign-off report would be created and shared with the stakeholders. If required a cumulative defect analysis report for all sprints will be also shared along with it.

Release Activities

After a release closure, the build is deployed to the UAT environment and is made ready for a few users to perform user acceptance testing. After deploying the release to UAT, a smoke is performed by the alpha testers and after the smoke success, the release is given to the beta testers who test the build before signing off from their end to be shipped to production. Around a week time may be provided for beta testers to complete testing. Once testing is completed, the build is been deployed to production and is made ready for end users to be tested after performing a round of smoke by the alpha testers. Any bugs encountered during UAT or production release would be logged and provided quick fixes if possible or a point release would be planned for the issues raised.

During this time other members of the testing team would be performing root cause analysis across the bugs raised in UAT or production.

Getting Ready for The Next Release

After the release, the team would prepare themselves for the next release, which would involve going through the storytellers encapsulating the features planned up for the release. Any business clarifications or issues would be raised and cleared during this phase.

The below image highlights the testing cycle and deliverables followed in the Agile environment: