Speed Up Development with Docker Compose

Assume a new developer or test engineer is added to your team. You develop an application with obviously some kind of database and you want them to get up to speed as soon as possible. You could ask them to install the application and database themselves or you could support them with it, but this would cause a lot of effort. What if you handed them over a simple YAML file which would get them up to speed in a few minutes? In this post, we will explore some of the capabilities of Docker Compose in order to accomplish this.

What Is Docker Compose?

Docker Compose allows you to define and run multi-container Docker applications. All you need to do is to define a single YAML file where you specify how the applications need to run and how they are connected to each other. A prerequisite, of course, is that you have a Docker image or Docker file for the applications. The official documentation of Docker Compose can be found here.

Sample Project

As a sample project, we will use a Spring Boot application which contains a basic Spring WebFlux CRUD application. The application makes use of MongoDB as a database. I based this on the sample project used in some previous posts I have written. The application consists of the following functionality:

• Create a show;
• Retrieve a list of shows;
• Retrieve a single show;
• Retrieve events of the show, the events are created automatically. We will not use this functionality in this post.

The source code can be found at GitHub.

Prerequisites and Installation

The following tooling is used for building and running the application:

• Ubuntu 18.04
• Maven 3.5.2
• JDK 11
• Docker 18.06

If you use Maven 3.3.9, then you will run into the following error when you run maven clean install -DskipTests (I did not list the complete stacktrace because it is quite long).

[WARNING] Error injecting: org.apache.maven.artifact.installer.DefaultArtifactInstaller
com.google.inject.ProvisionException: Unable to provision, see the following errors:

1) Error injecting: private org.eclipse.aether.spi.log.Logger org.apache.maven.repository.internal.DefaultVersionRangeResolver.logger
...
Caused by: java.lang.IllegalArgumentException: Can not set org.eclipse.aether.spi.log.Logger field org.apache.maven.repository.internal.DefaultVersionRangeResolver.logger to org.eclipse.aether.internal.impl.slf4j.Slf4jLoggerFactory
...

The problem seems to be fixed when you use Maven 3.5.2 or higher.

If you have Docker installed, you also have Docker Compose installed. You can check this with the following command, which will print the version installed:

$ sudo docker-compose -v

The YAML File Explained

First, let us take a look at the Dockerfile for our application. We take the openjdk:11-jdk as our base Docker image and then add our generated jar file to it. The jar file path is not hard coded, but a Docker build argument. In the pom file we make use of the dockerfile-maven-plugin of Spotify in order to build the Docker image.

FROM openjdk:11-jdk
VOLUME /tmp
ARG JAR_FILE
ADD ${JAR_FILE} app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]

In the root of our repository, we have added a docker-compose.yml file. This file will contain the configuration how to build and run our multi-container Docker application. In the next sections, the contents of the docker-compose.yml will be explained in detail.

Specify the Version

To start with, you have to specify which version of Docker Compose you want to use. We will use the last available version, namely version 3.

version: '3'

Specify the Services

The services section is an important one, because in this section you will specify which Docker containers you want to build or run. In our case, we will define two services. Namely, an app service for our application and a db service for MongoDB.

services:
  app:

  db:

The App Service

Let’s define the app service:

app:
  image: mydeveloperplanet/mydockercomposeplanet:${APP_IMAGE_VERSION}
  build: 
    context: .
    args:
      JAR_FILE: target/mydockercomposeplanet-${APP_BUILD_VERSION}.jar
  ports:
  - 8080:8080
  environment:
  - MONGO_URI=mongodb://mymongodb/dockercompose
  depends_on:
  - db
  networks:
  - mynetwork

At line 2 we define the Docker image to use. Assume that our build server creates the Docker image and pushes it to a Docker repository. Our test engineer is now able to use the Docker image with Docker Compose. In other words, if the Docker image exists in a Docker repository we have access to, Docker Compose will use this Docker image. Also notice the usage of the environment variable APP_IMAGE_VERSION. A way to set the environment variables for Docker Compose, is to provide a .env file in the directory where Docker Compose will be executed. For simplicity, I also committed the .env file in the root of our repository. In a real build environment, you can set these variables per environment (development, test, acceptance, production). We specify the following .env file:

APP_IMAGE_VERSION=0.0.1-SNAPSHOT
APP_BUILD_VERSION=0.0.1-SNAPSHOT

Lines 3-6 define what has to be done when the Docker image cannot be found. In that case, we set the context to our current directory where also our Dockerfile is located. Docker Compose will use the generated jar file from our local build to build the Docker image. Because our Dockerfileneeds the JAR_FILE argument, we also specify the value of it at line 6 as a build argument.

Lines 7-8 define the port mapping. The first port contains the external Docker container port which is mapped to the second port which contains the port our application is running at inside the Docker container.

Lines 9-10 is another way to define environment variables. Important to notice is that these values will overwrite values set in the .env file. Our application needs to know the database connection URL and this will be set with the MONGO_URI environment variable. The MONGO_URI environment variable is used in the file src/main/resources/application.properties:

spring.data.mongodb.uri=${MONGO_URI}

Lines 11-12 specify that our application is dependent on the existence of another service, namely our database 

Lines 13-14 specify a network connection between our two running Docker containers. Otherwise our application will not be able to connect to the MonogDB database.

The DB Service

Let’s define the db service:

db:
  image: mongo:4.0.4
  volumes:
  - mongodb:/data/db
  networks:
    mynetwork:
      aliases:
      - "mymongodb"

At line 2 we define the Docker image to use. In this case, we only define a Docker image to use, because we are not going to build MongoDB but only want to use an official Docker image.

Lines 3-4 define the volume we want to use to store our data. The volume will be a mapping between a location on our host which is called mongodband a location inside the Docker container where MongoDB stores its data (/data/db). If we don’t specify a volume, data will be lost when the Docker container is restarted.

Lines 5-8 specify the network. We also defined an alias for the network and used it in the database connection URL.

Specify the Network

As mentioned before, we need a network connection between our Docker containers which we name mynetwork. Also a network driver needs to be specified. We specify the bridge network driver which is the default when running on a single host.

networks:
  mynetwork:
    driver: bridge

Specify the Volume

The last thing to do is to specify the volume. When specified like below, a directory will be used where Docker Compose will have sufficient access rights to.

volumes:
  mongodb:

Run Docker Compose

Now that we have all the configuration in place, it is time to build and run our application. First, we need to build our application in order to generate our jar file. This will also generate a Docker image, which will be used when running Docker Compose. This works because the repository name and tag of our generated Docker image from our Maven build is identical to the one we use in the docker-compose.yml file.

$ sudo mvn clean install -DskipTests

Next thing to do is to start our multi-container application with Docker Compose. With the -p argument it is possible to give your application a unique name. By default, the name of the directory where the command is issued will be used as a project name. It is also possible to add the -dargument to run Docker Compose silently.

$ sudo docker-compose -p myapp-v0.0.1-snapshot up
Creating network "myapp-v001-snapshot_mynetwork" with driver "bridge"
Creating volume "myapp-v001-snapshot_mongodb" with default driver
Pulling db (mongo:4.0.4)...
4.0.4: Pulling from library/mongo
...
Digest: sha256:0823cc2000223420f88b20d5e19e6bc252fa328c30d8261070e4645b02183c6a
Status: Downloaded newer image for mongo:4.0.4
Creating myapp-v001-snapshot_db_1 ... done
Creating myapp-v001-snapshot_app_1 ... done
Attaching to myapp-v001-snapshot_db_1, myapp-v001-snapshot_app_1
...

When we take a look at the console output, we can clearly see that the network is created, the volume is created, the MongoDB image is pulled for our db service, the db service container myapp-v001-snapshot_db_1 and appservice container myapp-v001-snapshot_app_1 are created and finally, they are attached to each other. After this, both containers are started. The output of starting the containers is represented by the 3 dots.

So, we have a running application, but let’s see if it also works. Invoke the URL the retrieve the shows http://localhost:8080/shows. This returns an empty list. We can add a show by using curl:

$ curl -i -X POST -H 'Content-Type: application/json' -d '{"title": "title1"}' http://localhost:8080/shows
HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Content-Length: 50
{"id":"5c03a690a7b11b0001dd6b7a","title":"title1"}

When we now invoke the URL to invoke the shows, our output is:

[{"id":"5c03a690a7b11b0001dd6b7a","title":"title1"}]

We can now shut down the Docker container, remove the container and start the Docker container again without losing our data, as long as you don’t remove the volume on your host of course.

Other Docker Compose Commands

Make a Source Change

What happens when we make a change in one of our sources? To verify this, we change the URL to retrieve the shows into http://localhost:8080/getshows. Change line 24 in file src/main/java/com/mydeveloperplanet/WebConfig.java into:

.andRoute(RequestPredicates.GET("/getshows"), showHandler::all)

Run the Maven build again and execute docker-compose up. We are now able to retrieve the shows by means of the new URL. This only works because our Maven build has created the Docker image with the same repository and tag name. If your Maven build does not create the Docker image for you, the change would not have been available. When you change one of your sources or your Dockerfile, you need to rebuild the image with the command (where app is the service you want to build):

sudo docker-compose build app

It must be clear that in any case you need to run the Maven build to recreate your jar file, otherwise the change will not be available either.

Use Multiple Compose Files

By default, Docker Compose uses the file docker-compose.yml to build and run the containers. It is also possible to use multiple compose files. Assume that for testing purposes you want your application to be available on port 8081. We can define this in a new compose:

version: '3'

services:
  app:
    ports:
    - 8081:8080

We only specify the extra configuration. Docker Compose will add this configuration to our docker-compose.yml file. Notice that it is an addition and not a replacement. Our application will still be available on port 8080. This way, you can have a base configuration and for special purposes add extra configuration in additional compose files. We specify the compose files to be used with the -f argument.

$ sudo docker-compose -p myapp-v0.0.1-snapshot -f docker-compose.yml -f docker-compose-test.yml up

Remove Containers Created with Up

If you want to stop and remove containers and the network created with the up command, you can do so with the down command. With the optional argument --volumes you can also remove the volumes and with the optional argument --rmi all you can remove the Docker images.

$ sudo docker-compose -p myapp-v0.0.1-snapshot down --rmi all --volumes
Stopping myapp-v001-snapshot_app_1 ... done
Stopping myapp-v001-snapshot_db_1 ... done
Removing myapp-v001-snapshot_app_1 ... done
Removing myapp-v001-snapshot_db_1 ... done
Removing network myapp-v001-snapshot_mynetwork
Removing volume myapp-v001-snapshot_mongodb
Removing image mongo:4.0.4
Removing image mydeveloperplanet/mydockercomposeplanet:0.0.1-SNAPSHOT

Summary

In this post we discussed some of the capabilities of Docker Compose. With the help of a compose file, it is possible to give developers and test engineers a working environment for your application in just a few minutes.