Opportunities for Growth: Continuous Delivery and Continuous Deployment for Testers

Development practices are constantly changing, and as testers, we must embrace change. One of the changes that we can experience is the move from monthly or quarterly releases to continuous delivery or continuous deployment. In addition, this move to continuous delivery or deployment offers testers the chance to learn new skills.

A project that makes monthly or quarterly releases has a familiar rhythm, and the team builds toward the release date. The testers must test all the cards and conduct manual regression testing. Every test in the manual scripts needs to be executed, and possibly a report needs to be made on the results of these tests. Once the release has been made, it is possible that there will be bugs that were in the release that need to be fixed. The testers will also need to start running the same manual regression tests on the next release and report on them again. Testing for monthly or quarterly releases is a repetitive process. This process has been compared to the Greek myth of Sisyphus, who had to roll a stone to the top of a hill, and then when the stone rolled to the bottom of the hill, he had to roll it to the top of the hill again.

Continuous delivery can be defined as “when all developers are working in small batches on the trunk, ….and when the trunk is kept in a releasable state, and when we can release at the push of a button”. A team that I worked with moved from making monthly releases to continuous delivery. The team kept the main branch in a state in which it could be deployed if needed, and the team made weekly releases. Continuous deployment can be defined as when, in addition to the practices that support continuous delivery, “we are deploying good builds into production on a regular basis through self-service (being deployed by Dev or by Ops). A team practicing continuous deployment deploys to production each time code is merged to the main branch.

Teams that practice continuous delivery or continuous deployment use small batch sizes. That means that the “batch” of code that is deployed is small. “The theoretical lower limit for batch size is single piece flow, where each unit is performed one at a time”; this is what happens in a continuous deployment where each merge to the main branch is deployed to production.

Teams that practice continuous delivery and continuous deployment try to create a flow of work at a sustainable pace and so should “enable and inject learning into daily work.” On the other hand, teams that make monthly releases build towards the release and so can not create this constant flow of work at a sustainable pace.

When a team moves from monthly releases to continuous delivery or continuous deployment, a change that occurs is that there is no release candidate. Testing is not done on a release candidate; instead, it is done on feature branches that have been taken from the main branch, and when they are merged back into the main branch, they must be ready for release to production. The main branch is kept in a state where it can be released to production. For the tester, this means that the testing on a feature branch has a different pattern than that on a release candidate.

When testing on a feature branch, you need to be confident that the new feature or fix in the feature branches does what it is supposed to do and that it has not caused any regression. When testing a monthly release, you can have time to execute manual regression tests, but if the main branch is to be kept in a state where it can be deployed to production, this is not possible. Regression testing needs to be automated if you are using continuous delivery or continuous deployment.

Regression testing on monthly releases often includes running a large number of manual tests; however, if your team is using continuous delivery or continuous deployment, regression testing is usually automated via continuous integration. Continuous integration (CI) “means that every single time somebody commits any change” to the code, the change is integrated into the codebase. This entails running automated tests before and after merging the code into the main branch. This provides an opportunity for the tester to learn how to understand CI. The tester must understand CI, including what tests run as part of CI. There will always be gaps in the tests that run on CI. If the tester knows what the gaps in CI are, they can work out how to automate tests to fill them and execute manual tests to cover the gaps when this is required. Making mind maps of the tests that are automated can help identify gaps in the automated tests.

Testers can also get involved in automating regression tests themselves, and in this way, testers can help to prevent bugs rather than find them. There are great free resources such as Test Automation University, LambdaTest Certifications, and Exercism which can help testers gain the skills they require to automate tests. There are also lots of resources to learn how to use javascript to aid testing.

That regression testing is automated creates time for the tester that they can spend on exploratory testing. Exploratory testing is a powerful way to uncover issues, and so it will help the projects that the tester is working on. Having extra time to do exploratory testing will also help the tester develop their exploratory testing skills.

Projects that use continuous delivery and continuous deployment also tend to have a microservices architecture. Microservices are services that have independent testing and deployment, and each service is simple. The tester has the opportunity to learn about the microservices, and ways to do this include talking to the developers, studying any architecture diagrams that exist, reading the readme files for each service in GitHub, and attending the developers’ meetings. 

Testers can build their relationships with developers by helping them test their code. In addition, the tester can share their knowledge of testing techniques, such as boundary value analysis, with developers, as this will help them test and produce better quality software.

Release processes for monthly releases can involve a certain amount of pain for testers. Sometimes we are asked to take the responsibility of making the release decision even though the tester is often a junior member of the team; other times, testers have to take part in large committee meetings of stakeholders that decide if the software can be released. Release processes for continuous deployment and continuous delivery should be automated; this means that making releases does not put pressure on the tester. The tester’s input to the release in continuous delivery and continuous deployment is their testing; this means that we can focus on our testing and learn new testing skills.

Development teams are not autonomous; they are open systems whose work affects other teams and is affected by other teams. This is systems thinking, and systems thinking contributed to continuous delivery and continuous deployment. Testers can learn to use systems thinking to enhance their testing and support their team. This can help the tester think beyond their role to understand what other systems are affected by their team’s work and what systems affect their team’s work.

One of the lessons of systems thinking is that everyone shares responsibility, so no one person should be blamed when something goes wrong. This view should also be at the heart of every agile and lean development team that implements continuous delivery or continuous deployment. This is something that testers should learn from and take to heart. When there is a failure, we need to learn from it and not blame someone.

Teams that make monthly releases can find that after each release, there is a flurry of activity fixing regression bugs that were deployed in the release. This does not happen with teams that practice continuous delivery or continuous deployment. Releases will be deployed multiple times a day using continuous deployment, and software will be deployed regularly with continuous delivery. These regular releases give the development team an opportunity to recover from bugs and incidents quickly, as fixes can be deployed quickly. Once a fix has been deployed, the tester can offer to take the lead in doing root cause analysis to find the root cause of the bug or incident. The tester can learn to use the Five Whys and Ishikawa Diagrams to do root cause analysis.

Testers can also support the work of their teams by producing metrics that help the team measure their quality improvement. DORA metrics were identified by the Accelerate State of DevOps survey report by DORA. These metrics are designed to help teams that practice continuous delivery and continuous deployment find areas for improvement and know how they are doing. These are different from the metrics for monthly releases as they are not about how many bugs have been put into production but are about how quickly a team recovers from an error.

Continuous delivery and continuous deployment offer testers opportunities to grow and learn new skills, so testers should embrace the opportunities when their teams move to continuous delivery and continuous deployment.