What Is a CI/CD Pipeline?

A series of steps that include all the stages from the outset of the CI/CD process and is responsible for creating automated and seamless software delivery is called a CI/CD pipeline workflow. With a CI/CD pipeline, a software release artifact can move and progress through the pipeline right from the code check-in stage through the test, build, deploy, and production stages. This concept is powerful because once a pipeline has been specified, parts or all of it can be automated, speeding the process and reducing errors. In other words, a CI/CD pipeline makes it easier for enterprises to deliver software multiple times a day automatically.

DevOps engineers often get confused with the CI/CD pipeline by automation of individual stages in CI/CD. Though different tools may automate each complicated stage in CI/CD, the whole software supply chain of CI/CD may still be broken because of manual intervention in between. But let us first understand various stages in a CI/CD process and why a CI/CD pipeline is essential for your organization to deliver code at speed and scale.

CI/CD Stages: Understanding People, Process and Technology

Enterprise application development teams typically consist of developers, testers/QA engineers, Operation engineers, and SRE (Site Reliability Engineers) or IT Operations teams. They work together closely to deliver quality software into customers’ hands. CI/CD is a combination of two separate processes: Continuous Integration and Continuous Deployment. The major steps in each are listed below.

Continuous Integration

Continuous integration (CI) is the process where software is built and initial tests are completed. Continuous deployment (CD) is the process of combining code with infrastructure, ensuring that all tests are completed and policies followed, and then deploying code into the intended environment. Of course, many companies have their process, but the major steps are below.

CI: Code-Commit 

• People:
  Developers and Engineers, Database Administrator (DBA), Infrastructure Team.
• Technology:
  GitHub, Gitlab, SVM, BitBucket.
• Process: 

A code commit stage is otherwise known as version control. A commit is an operation that sends the latest changes written by a developer to the repository. Every version of the code written by a developer is stored indefinitely. After a discussion and review of the changes with collaborators, developers will write the code and commit once the software requirements, feature enhancements, bug fixes, or change requests are completed. The repository where the edits & commit changes are managed is called Source Code Management (SCM tool).. After the developer commits the code (code Push Request), the code changes are merged into the base code branch stored at the central repository like GitHub. 

CI: Static Code Analysis

• People: Developers and Engineers, Database Administrator (DBA), Infrastructure team, Testers.
• Technology: GitHub, Gitlab, SVM, BitBucket.
• Process:

Once the developer has written a code and pushes it to the repository, the system gets triggered automatically to start the next process of code analysis. Imagine a step where the code committed gets to build directly and it failed during build or deployment. This becomes a slow and costly process in terms of resource utilization, both machine, and man. The code must be checked for static policies. SAST (Static Application Security Test): SAST is a white-box testing method to examine the code from inside using SAST tools like SonarQube, Veracode, Appscan, etc., to find software flaws, vulnerabilities, and weaknesses. (such as SQL injection etc.). This is a fast check process where the code is checked for any syntactic errors. Though this stage lacks the feature to check for runtime errors, which is performed at a later stage.

Placing an additional policy checking into an automated pipeline can dramatically reduce the number of errors found later in the process.

CI: Build

• People: Developers and Engineers.
• Technology: Jenkins, Bamboo CI, Circle CI, Travis CI, Maven, Azure DevOps.
• Process:

The Continuous Integration process’s goal is to take the regular code commits and continuously build binary artifacts. The continuous integration process helps to find bugs more quickly by checking if the new module that is added plays well with the existing modules. This helps reduce the time to verify a new code change.  The build tools help in compiling and creating executable files or packages (.exe,.dll, .jar, etc.) depending on the programming language used to write the source code. During the build, the SQL scripts are also generated and then tested along with infrastructure configuration files. In a nutshell, the build stage is where your applications are compiled. Other sub-activities that are a part of the Build process are Artifactory Storage, Build Verification, and Unit Tests.

Build Verification Test (BVT)/Smoke Tests and Unit Tests: 

Smoke testing is performed immediately after the build is created. BVT checks if all the modules are integrated properly and if the program’s critical functionalities are working fine. The purpose is to reject a badly broken application so that the QA team does not waste time installing and testing the software application.

Post these checks a UT (unit test) is added to the pipeline to further reduce failures at the production. Unit Testing tests individual units or components of a code written by the developer to validate if they perform as expected.

Artifactory Storage:

Once a build is prepared, the packages are stored in a centralized location or database called Artifactory or Repository tool. There can be many builds getting generated per day, and keeping track of all builds can be difficult. Hence, as soon as the build is generated and verified, it is sent to the repository for storage. Repository tools such as Jfrog Artifactory are used to store binary files such as .rar, .war, .exe, Msi, etc. From here, testers can manually pick, deploy an artifact in a test environment to test.

CI: Test Stages

• People: Testers, QA Engineers.
• Technologies: Selenium, Appium, Jmeter, SOAP UI, Tarantula.
• Process: 

Post a build process a series of automated tests validate the code veracity. This stage helps errors from reaching the production. Depending on the size of the build this check can last from seconds to hours. For large organizations where codes are committed and built from multiple teams, these checks are run in a parallel environment to save precious time and notify developers of bugs as early as possible.

These automated tests are set up by testers (or known as QA engineers) who have set up test cases and scenarios based on user stories. They perform regression analysis, stress tests to check deviations from the expected output. Activities that are involved in testing are Sanity tests, Integration tests, Stress tests. This is a much-advanced level of testing that happens. Here we find issues that were probably unknown to the developer developing the code.

Integration Tests:

Integration tests are performed using tools like Cucumber, Selenium, etc., where individual application modules are combined and tested as a group while evaluating the compliance with specified functional requirements.  After an integration test, someone needs to approve that the set of updates in that group should be moved to the next stage, which is typically performance testing. This verification process can be cumbersome, but it is an important part of the overall process. There are some emerging solutions to the verification process.

Load and Stress Testing:

Load balancing and stress testing are also performed using automated testing tools like Selenium, JMeter, etc., to check if an application is stable and performing well when exposed to high traffic. This test is typically not run on every single update, as the full stress testing is long-running. When major new capabilities are to be released, multiple updates are grouped and full performance testing is completed. In cases where a single update is being moved to the next stage, the pipeline may include canary testing as an alternative.

Continuous Deployment: Bake and Deploy

• People: Infrastructure Engineer, Site Reliability Engineer (SRE), Operation Engineer.
• Technology: Spinnaker, Argo CD, Tekton CD.
• Process:

After the testing stage is completed the codes that have cleared the criterion are ready to be deployed into the servers where they were meant to integrate with the main application. Before getting deployed into the production they will be deployed to test/ staging or a beta environment that is internally used by the product team. Before the builds are moved to these environments the builds have to pass through two substages named Bake and Deploy. Both of these stages are native to Spinnaker.

CD: Bake

Baking refers to creating an immutable image instance from the source code with the current configuration at the production. These configurations may be things like database changes and other infrastructure updates. Spinnaker can either trigger Jenkins to perform this task and some organizations prefer to use Packer.

CD: Deploy

Spinnaker will automatically pass the baked image to the deploy stage. This is where the server group is set to be deployed to a cluster. A functionally identical process is carried out during the deployment stage similar to the testing processes described above. Deployments are first moved to test, stage, and then finally to prod environments post approvals and checks. This whole process is handled by tools like Spinnaker.

CD: Verification

 This is also a key place for teams to optimize the overall CI/CD process. Because so much testing has happened now, failures should be rare. However, any failure at this point needs to be resolved as quickly as possible so the impact on the end customer is minimized. Teams should consider automating this portion of the process as well.

Deploying to prod is carried out using deployment strategies like Blue-Green, Canary Analysis, Rolling Update, etc. During the deployment stage, the running application is monitored to validate whether the current deployment is right or it needs to be rolled back.

CD: Monitoring

• People: SRE, Ops Team.
• Technology: Zabbix, Nagios, Prometheus, Elastic Search, Splunk, Appdynamics, Tivoli.
• Process:

To make a software release failsafe and robust, tracking the release’s health in a production environment is essential. Application monitoring tools will trace the performance metrics like CPU utilization and latency of releases. Log analyzers will scan torrents of logs produced by underlying middleware and OS to identify behavior and track the source of problems. In case of any issue in the production, stakeholders are notified to ensure the production environment’s safety and reliability. Moreover, the monitoring stage helps businesses gather intelligence about how their new software changes contribute to revenue and helps the infrastructure team track the system behavior trend and do capacity planning.

Continuous Deployment: Feedback and Collaboration Tool

• People: SRE, Ops, and Maintenance Team.
• Technology: JIRA, ServiceNow, Slack, Email, Hipchat.
• Process:

The goal of the DevOps team is to release faster and continuously and then continually reduce errors and performance issues. This is done through frequent feedback to developers, project managers about the new version’s quality and performance through slack or email and by raising tickets promptly in ITSM tools. Usually, feedback systems are a part of the entire software delivery process; so any change in the delivery is frequently logged into the system so that the delivery team can act on it.

Summary

An enterprise must evaluate a holistic continuous delivery solution that can automate or facilitate automation of these stages, described above.