Jenkins X Step-by-Step Tutorial to Continuous Deployment with Kubernetes

At TestProject we strive to use the most up-to-date best practices, and as part of an upgrade for some components and workflows in our infrastructure, we are partially shifting to Jenkins X. There is a lack of useful material available on Jenkins X serverless setup, so as part of our belief in sharing and giving back to the community we’ve decided to create a full-blown step by step tutorial on about it!

Jenkins X serverless and Kubernetes continuous integration solves the following problems:

1. Maintaining dedicated servers for various components involved in CI/CD, such as: Build servers, testing environments, orchestrating servers, etc.
2. Automatic deployment of concurrent on-demand build and testing environments, allowing testers and developers to release and test dedicated features in parallel.
3. Naturally integrates with microservice development and release flows.

In this step by step tutorial, we are going to build a CI/CD solution for a modern cloud application based on microservices with Jenkins X.

Jenkins X is a CI/CD solution for modern cloud applications on Kubernetes. It helps us create great pipelines for our projects and implement a full CI and CD.

Continuous integration (CI) is a software development methodology for automating the integration of code changes from multiple contributors into a single software project. Continuous deployment (CD) is a software development methodology, with this methodology software is automatically, rapidly and safely deployed into the production environment.

You may also enjoy: All You Need to Know About Jenkins X

Jenkins X automates the management of the environments and the promotion of new versions of applications between environments via GitOps.

GitOps is a software development methodology for Kubernetes cluster management and application delivery. When following this methodology, Git is the single source of truth for both the infrastructure-as-code and the application code. All changes to the desired state are Git commits.

Jenkins X automatically spins up preview environments for our pull requests so we can get fast feedback before changes are merged to master.

Let’s Get Started! 

For the purpose of this tutorial, we’ve created a simple demo project with a microservice design that will be deployed on Kubernetes. Our demo project consists of 3 main services: A frontend application and 2 backend services.

Our demo project displays the current version of the backend services and its own. This will be helpful to illustrate and test the CI/CD pipeline that we are building later.

Our demo project services are written in Python and packaged into a Docker container based on Alpine image. The image below illustrates our demo project:

By the end of this tutorial, we will have a working CI/CD solution that will enable us to develop, test and deliver our application in a fast and reliable way.

Jenkins X Serverless Architecture

Before we start the hands-on part of the tutorial, we need to cover the main components of Jenkins X serverless infrastructure. The image below illustrates these components:

Let’s drill down a bit more:

• Kubernetes – The core of our cloud application development, where we build, test and deploy our applications.
• Docker image registry – That’s where we store our Docker images after we build them. A small note on that: Jenkins X uses your cloud provider’s Docker registry by default, but it can be configured to use some other private registry.
• Git – That’s where we save our code – Application source code, system configuration and system state.
• Chart Museum – Where we store our Helm charts.
• Testing Framework – Where we test our application changes, we use TestProject (it’s free).
• Kaniko – Builds our Docker containers.
• Skaffold – Deploys our Docker containers on Kubernetes.
• Draft – Creates our Helm charts for deploying our applications.
• Tekton – That what runs our pipelines.
• Jenkins X – Orchestrates the CI/CD process.

Install & Setup Jenkins X Serverless

Now we will install Jenkins X and use Google Cloud as our cloud provider, but the installation process is very similar to other clouds.

Prerequisites

1. Google cloud account (if you don’t already have one, you can create one here).
2. Github account (if you don’t already have one, you can create one here).

Install Jenkins X on Google Kubernetes Engine (GKE)

1. Log into your Google cloud account.
2. Go to your console (console button at the right upper corner).
3. Go to your cloud shell ([>_] button on the upper blue bar right side).
4. Run the following commands to install jx:

mkdir -p ~/.jx/bin

curl -L "https://github.com/jenkins-x/jx/releases/download/$(curl --silent "https://github.com/jenkins-x/jx/releases/latest" | sed 's#.*tag/\(.*\)\".*#\1#')/jx-linux-amd64.tar.gz" | tar xzv -C ~/.jx/bin

export PATH=$PATH:~/.jx/bin

echo 'export PATH=$PATH:~/.jx/bin' >> ~/.bashrc

5. Run the following command to:

• Deploy a fresh Kubernetes cluster configured for Jenkins X.
• Deploy serverless Jenkins X components:

xxxxxxxxxx

jx create cluster gke --skip-login --git-public

6. Now you will be asked a series of questions that will configure the installation:

1. What project to associate your cluster with. Choose My First Project if you are a new google cloud user, or else choose the one you want.
2. Where to deploy your cluster, geographical region. Choose the one closest to you.
3. Machine type, if you’re not experienced just go with the default value.
4. Number of nodes to deploy, same as above you can go with the default values.
5. The kind of Jenkins X deployment you want, choose Serverless Jenkins X with Tekton Pipelines.
6. You will be asked if you want to create a load balancer, agree to that, or else you will not be able to work.
7. You will be asked if you want to use nip.io as your magic DNS. Agree to that.
8. Don’t forget the installation will create several repositories in your Github account and will ask for admin permissions, we strongly advise to create a fresh account for the purpose of the tutorial. You will be asked all sorts of questions about your Github account, answer accordingly, don’t use your email as the account name, this can make problems in later stages, use the nickname you chose for the account.
9. The installation will start and when it is done, you will be presented with some useful links and the password for your Jenkins X related services, don’t forget to save it somewhere for future use.

That’s it! You are ready to start.

Import the Existing Project into Jenkins X

In this part, we will learn how to import an existing project into Jenkins X. 

Different Environments Created by Default by Jenkins X

Before importing your project to Jenkins X, we need to understand what are the different environments created by default by Jenkins X. By default, Jenkins X creates the following environments:

1. jx environment – In this environment, we perform the building and unit testing.
2. Staging environment – In this environment, we perform the integration testing.
3. Production environment – Our final production release.
4. Preview environments – Environments for testing purposes.

Step 1 – Create a New Repository 

The first thing you will need to do is fork the repository with the source code of your project, and then clone it to your development environment.

In this chapter of the tutorial, we will use the Backend1 demo project as an example. You should fork the main tutorial repository and run the following commands in order to clone it to your development environment:

GIT_USER=[...]

PROJECT_NAME=[...]

git clone https://github.com/$GIT_USER/$PROJECT_NAME

cd $PROJECT_NAME/backend1

Now you have a new repository to work with, but you still don’t have all the needed files for Jenkins X to work with it. 

Step 2 – Add Necessary Files for Jenkins X

To add necessary files for Jenkins X, the first thing you need to do is import the project. The command below will import the project. Before running this command, verify that you are inside the home directory of the project we’ve just cloned to the development environment:

xxxxxxxxxx

jx import --batch-mode

As part of the default import process, Jenkins X will go over the code and choose the right default build pack for the project based on the programming language. Our project was developed in Python so it will be a Python build pack.

You can view the contents of the build pack by looking at:

xxxxxxxxxx

/.jx/draft/packs/github.com/jenkins-x-buildpacks/jenkins-x-kubernetes/python

To verify the import progress, run the command below: 

xxxxxxxxxx

jx get activities --filter $PROJECT_NAME

When the import is successfully finished, you will see that all the files that are in the build pack now appear in the project repository, the difference is that some of them were templates before and now they are legit files.

Wait, What Happened Here? 

Let’s go over what happens behind the curtains of the import process:

1. Jenkins X chooses the right build pack and pushed the new files to the project’s repository.
2. Jenkins X creates a Jenkins project.
3. Jenkins X creates a webhook to trigger builds whenever there is a code change in the project’s repository.
4. Jenkins X uses skaffold.yaml to build the container image.
5. Jenkins X uses the helm charts to create a build and pushes it to the chart-museum.
6. Jenkins X commits changes to the staging environment repository.
7. It adds the new service version as a dependency on the environment.
8. It adds an ingress for the new service.
9. Jenkins X applies the changes to the Kubernetes cluster, the new service is deployed by Kubernetes in the staging environment.
10. Jenkins X provides us with a URL to the new service.
11. We can interact with our service via HTTP.

Impressive, huh? 

The cool thing about this process is that all the changes are tracked in Git and can be viewed by going to:

In order to verify whether the service was deployed correctly to the staging environment, you can view the service by the URL that will be printed in the console at the end of the import process. For example, in our demo project Backend1, the following JSON should appear in your browser after opening the provided URL:

xxxxxxxxxx

{"name": "backend1", "version": "1"}

You can see that you get the name and version of the service, but what about unit testing? This stage is not included in the default build pack. To solve this, you will need a custom build pack for Jenkins X. 

Create Custom Build Packs for Jenkins X

In this part, we will create a custom build pack for Jenkins X and include unit testing as part of it.

Build packs define how to build, test and deliver your project each time a developer makes changes to the source code of your project. Jenkins X provides basic build packs, but in real life, you need to create a custom build pack for your project.

It is recommended to keep your build packs in a central repository and share them between the developers in your company.

In order to create a custom build pack for the tutorial project, the first thing you need to do is fork the official repository with community build packs. Then, clone the repository to your development environment using the following commands:

xxxxxxxxxx

GIT_USER=[...]

git clone https://github.com/$GIT_USER/jenkins-x-kubernetes

cd jenkins-x-kubernetes

It’s advised to add an upstream repository to receive future updates from the official repository, in order to stay up-to-date. Use the following command to do so: 

xxxxxxxxxx

git remote add upstream https://github.com/jenkins-x-buildpacks/jenkins-x-kubernetes

Next, in order to create a custom build pack, we need to create a branch of the repository. Use the following commands: 

xxxxxxxxxx

git checkout -b tutorial-build-pack master

git push -u origin tutorial-build-pack

Selecting the Source for Build Packs in Jenkins X

By default, the source of build packs is in the following folder:

xxxxxxxxxx

/.jx/draft/packs/github.com/jenkins-x-buildpacks/jenkins-x-kubernetes

To change the source to point to your newly created repository, run the following commands: 

xxxxxxxxxx

jx edit buildpack \

-u https://github.com/$GIT_USER/jenkins-x-kubernetes \

-r master \

-b

From now on, when you import a new project, Jenkins X will use the build packs from your newly created repository.

By default Jenkins X provides build packs for the popular development languages. As a base for our custom build pack, we will use the Python build pack since our project is written in Python.

Copy the Python build pack folder to a folder called tutorial-build-pack. To do this run the following command:

xxxxxxxxxx

cp -R packs/python packs/tutorial-build-pack

Now let’s look at the files inside the new build pack folder we’ve just created:

• Dockerfile – Contains the build of the docker image.
• charts – Contains Helm charts and Kubernetes files that describe how to deploy the project to staging and production environments.
• pipeline.yaml – Contains the steps for building, testing and delivering the project.
• preview – Contains Helm charts and Kubernetes files that describe how to deploy the project to a preview environment.
• skaffold.yaml – Contains the steps for building and storing the docker image.

Adding Unit Test Support as Part of A Custom Build Pack in Jenkins X

In order to implement unit testing in the custom build pack, you need to extend the basic build pack. To extend a build pack you simply need to modify the pipeline.yaml file that is located in the home directory of the build pack.

Let’s say you want to add a script that runs some unit tests before building the image. For the sake of example, you will run an echo command “Running unit tests!”. But in reality, you should run a command or script that will run your tests.

Now add a new step to the pipeline.yaml to execute the new script before we build our service. Use the following commands to do so:

xxxxxxxxxx

…

release:

  build:

    preSteps:

    - sh: echo “Running unit tests!”

…

The new file should look like this: https://github.com/testproject-io/jenkinsx-tutorial/blob/master/build-pack-pipeline.yaml

In order for the changes to take effect, we need to commit our changes to the build pack. However, the newly created build pack is not associated with any project.

For the purpose of the tutorial, if you have already forked and cloned the main tutorial repository, then skip the next step and simply enter the Backend2 folder inside the repository folder.

If you have not already cloned the main tutorial repository, then run the following commands:

xxxxxxxxxx

GIT_USER=[...]

PROJECT_NAME=[...]

git clone https://github.com/$GIT_USER/$PROJECT_NAME

cd $PROJECT_NAME/backend2

Now we have a new repository to work with. To import the new project with the build pack you’ve created, run the following command: 

xxxxxxxxxx

jx import --pack tutorial-build-pack --batch-mode

In order to check if the build was correctly executed with our new unit test, run the following command: 

xxxxxxxxxx

jx get build logs

Congrats! Your unit tests are now being executed!

Integration Testing in Jenkins X

In this part, we will learn how to implement integration testing for the services that we deploy with Jenkins X.

In order to understand where to implement the integration tests, you need to understand their purpose – to verify that all the services in our production environment will communicate with each other in the correct way.

In Jenkins X you can have as many environments as you want, but now we will focus on the staging environment since this environment simulates the next version that is going to be released to the production environment. Hence, if the services communicate in the correct way in the staging environment, then we can assume that if they will be transferred to the production environment they will work seamlessly.

If you’ve followed the tutorial from the start, you should have a staging environment that includes: Two working services, Backend1 and Backend2. These services have no interaction with each other so there is no actual integration testing we can perform to verify that the services are communicating correctly.

To make the integration testing example meaningful, we will add a frontend service that interacts with both of the backend services.

For the purpose of the tutorial, if you have already forked and cloned the main tutorial repository, then skip the next step and simply enter the Frontend folder inside the repository folder.

If you have not already cloned the main tutorial repository, then run the following commands:

xxxxxxxxxx

GIT_USER=[...]

PROJECT_NAME=[...]

git clone https://github.com/$GIT_USER/$PROJECT_NAME

cd $PROJECT_NAME/frontend

jx import --batch-mode

When the import is successfully finished, verify that the 3 services are up and running in your staging environment.

To verify that the 3 services are available, you can run the following command:

xxxxxxxxxx

jx get applications

How to Execute Integration Tests with TestProject via API

TestProject is a community-powered test automation platform that is completely free. It is built on top of Selenium and Appium, supports all major operating systems, and enables every member of your software team to collaborate and test Web, Android and iOS apps, effortlessly. TestProject is the perfect fit for integration testing within a CI/CD pipeline, since TestProject exposes a RESTful API which allows R&D teams to schedule and trigger automation, get status and retrieve testing results. In addition, TestProject Agents can be deployed with ease on various platforms (both local and remote), including a dockerized version of the Agent that can be deployed serverless, which allows for parallel distributed testing on-demand. 

Let’s get started!

1. Create a TestProject account here, it’s completely FREE!
2. Download and register the TestProject Agent.
3. Create your integration tests. In this example we will use recorded tests, but you can also use TestProject to develop Selenium and Appium coded tests in C# or Java as well.
4. Generate your API key with TestProject to access TestProject Platform from your CI/CD tools.
5. Execute a test automation job in TestProject using their API.
6. For example, you can use the following Python script to execute your TestProject job from your CI/CD tool: https://github.com/testproject-io/jenkinsx-tutorial/blob/master/trigger-job.py

Now let’s implement our integration tests! The first thing you need to do is to clone the staging environment repository to your development environment. Use the following commands:

xxxxxxxxxx

CLUSTER_NAME=[...]

GIT_USER=[...]

git clone https://github.com/$GIT_USER/environment-$CLUSTER_NAME-staging

cd environment-$CLUSTER_NAME-staging

After cloning the repository to your development environment, inside the home directory folder we can find a file called jenkins-x.yaml. This file describes the steps that represent the pipeline for the deployment of services to the staging environment.

In order to execute the integration tests after the build, we need to add the following commands to this file:

xxxxxxxxxx

…

pipelines:

  release:

    postBuild:

      steps:

      - sh: echo “Executing integration tests!”

      - sh: python trigger-job.py

The new file should look like this: https://github.com/testproject-io/jenkinsx-tutorial/blob/master/staging-pipeline.yml

Now that we’ve added the commands that will execute the integration tests, we can push the changes to Git.

Congratulations, the integration tests will be executed every time anything is deployed to the staging environment! 

You can verify this by looking at the build logs: 

xxxxxxxxxx

CLUSTER_NAME=[...]

GIT_USER=[...]

jx get build logs $GIT_USER/environment-$CLUSTER_NAME-staging/master

Happy jx’ing and good luck!

Further Reading

Jenkins: X or 2.0?

Hands-On CI/CD for Microservices With Jenkins X