Jenkins Cluster Setup With Docker Containers

* For impatient readers, skip to Setting a Jenkins Cluster Manually.

* For the automated solution, skip to I Want it Automated!

Jenkins is one of the mostly used Continuous Integration tools for build automation due to its capacity in managing a great number of nodes, called slaves, with executors for a wide range of tasks related to project build and deploy. Each executor runs Jenkins jobs, alleviating Jenkins server from running all the tasks.

Jenkins' capacity of running jobs should not be constrained by the hardware where it runs, or by the OS, or even by the locality where the jobs need to run. In this way, Jenkins slaves can do these jobs in other machines, in other networks (as long these machines can communicate by HTTP), in other operating systems, such as Windows or Solaris, and in other more powerful hardware. For instance, we can compile an application in C++ for Solaris 9 and several other Java 8 applications using Jenkins. Since there is no Java 8 distribution for Solaris 9, setting a slave was crucial to undertake this activity.

In spite of having a native way to configure Jenkins slaves, it is something error-prone and tedious when done several times. Therefore, we will use a Jenkins installation with Jenkins Swarm plugin, which allows the slaves to form a cluster by auto-discovering the master and self-registering them in it. We will also use Docker and Jelastic cloud for this demo due to the simplicity to customize Docker images and to operate Jelastic control panel and create environments in it.

Docker is a paravirtualization solution for Linux to run distributed applications. Due to its success, Docker became a de facto standard in the software industry, having several images made by the community and stored in its registry. For this reason, several cloud vendors implement Docker solutions in their products, such as Jelastic. In this demo, we'll use some Docker images from Jenkins Swarm and Jenkins Swarm slaves.

Jelastic is a cloud PaaS that uses containers, and it's possible to run Docker containers in it. In this article, we'll start a Jenkins master and two Jenkins slaves using the Jelastic control panel. For people who don't want to do all the work by hand, either due to the repetition or because it's tedious, there will be a demonstration below detailing how to create a Jenkins environment with slaves by importing a JSON file in the control panel.

Docker Images

For the demo in this article, we're going to use some (custom) Docker images. Firstly, we'll use the Docker image jelastic/jenkins-swarm as a customization of the official Jenkins Docker image with the installed Jenkins Swarm plugin. In addition, we're going to use the Docker image jelastic/jenkins-swarm-slave, which is a modified version of the image csanchez/jenkins-swarm-slave from Carlos Sanchez, by hard-coding the parameters passed to the swarm-client in jenkins-slave.sh script, and setting start and stop functions to be called by running the script with start or stop as arguments.

Setting a Jenkins cluster manually

Setting a Jenkins cluster manually is by no means a laborious task, as the Jelastic console is very practical for setting up environments. We can setup a Jenkins cluster following these simple steps:

1. Click the New Environment button in the top left of Jelastic control panel and open the Docker tab.

2. After that, click the Select Container button and search for jelastic/jenkins-swarm.

3. Select the image, click the Extra button and select the image jelastic/jenkins-swarm-slave.

4. Link the Jenkins slave containers to the master. To link the containers, click Links in the configuration section, then click Add and set the alias to jenkins. After that, click Save and Apply.

5. For this demo, you can use just one node of each Docker image and the default number of cloudlets. Now, set the name of your environment and click the Create button.

Several minutes later, your Jenkins cluster will be configured and ready to use. You can access it by clicking the Open in browser button within the Jenkins swarm container node row.

I Want it Automated!

There are enough steps in the previous section that can be automated. In the Jelastic platform, you can import environments definitions from JSON files. To do so, let's follow the three steps below:

1. Click the down arrow of the New environment button and choose Import.

2. After that, go to the URL tab and add the following URL and click Import https://raw.githubusercontent.com/jelastic-jps/jelastic-jenkins-cluster/master/jenkins-cluster.json.

3. Specify the environment name and click the Install button. It is effortless!

4. After loading the JSON file, we can check the status of the nodes by clicking on Open in Browser button.

Now, it's the right time to use Jenkins with slaves, configured without obscure command lines, arcane configuration procedures, or error-prone setups. Jelastic offers a set of different ways to implement this, whether you need to do it manually or automatically.

GitHub repositories:

https://github.com/jelastic-jps/jenkins-swarm

https://github.com/jelastic-jps/jenkins-swarm-slave-docker

https://github.com/carlossg/jenkins-swarm-slave-docker

https://github.com/jelastic-jps/jelastic-jenkins-cluster

Try to create Jenkins environments based on Docker containers and see how easy it is! Just register for free at any of our partners' platforms within the Jelastic Cloud Union.

Do you have any questions? Feel free to leave comments below.