Running Parallel Tests in Docker

Sometimes when you are running your tests on your CI environment, you want to run tests in parallel. This parallelism is programmed in build tools such as Maven or Gradle or by using Jenkins plugin. 

If you are using Docker as a testing tool for providing external dependencies to the application (for example databases, mail servers, FTP servers) you might find a big problem, and it is that in Docker Host, when running tests in parallel, all of them are going to try to start a container with the same name, so when you start the second test (in parallel) you will get a failure regarding a conflict container name because of Docker Host trying to start two containers with the same name or having the same binding port in two containers.

At this point, you can do two things:

• You can have one Docker Host for each parallel test.
• You can reuse the same Docker Host and use Arquillian Cube Star Operator.

Arquillian Cube is an Arquillian extension that can be used to manage Docker containers in your tests.

To use Arquillian Cube, you need a Docker daemon running on a computer (it can be local or not), but it will probably be local.

Arquillian Cube offers three different ways to define container(s):

• Defining a docker-compose file.
• Defining a Container Object.
• Using Container Object DSL.

In this example, I am going to show you how to use docker-compose and Container Object DSL. Star Operator lets you indicate to Arquillian Cube that you want to generate cube names randomly, and can adapt links as well. In this way when you execute your tests in parallel there will be no conflicts because of names or binding ports.

Let’s see an example:

plugins {
    id "io.spring.dependency-management" version "1.0.2.RELEASE"
}


apply plugin: 'java'

repositories {
    mavenCentral()
    jcenter()
}

dependencyManagement {
    imports {
        mavenBom 'org.jboss.arquillian:arquillian-bom:1.1.13.Final'
    }
}

dependencies {

    testCompile 'junit:junit:4.12'
    testCompile 'org.jboss.arquillian.junit:arquillian-junit-standalone'
    testCompile 'org.arquillian.cube:arquillian-cube-docker:1.3.2'
}

test {
    maxParallelForks = 2
    testLogging.showStandardStreams = true
}

#src/test/docker/docker-compose.yml

redis*:
  image: redis:3.0.7
  ports:
    - "6379"

@RunWith(Arquillian.class)
public class TestOne {

    @HostPort(containerName = "redis*", value = 6379)
    private int portBinding;


    @Test
    public void should_print_port_binding() throws InterruptedException {
        System.out.println(TestOne.class.getCanonicalName() + " - " + portBinding);
        TimeUnit.SECONDS.sleep(4);
    }

}

You can see in the docker-compose.yml file makes an important change from a typical docker-compose file, and it is that the name ends up with a star (*) operator [ redis*]. This is how you are instructing Arquillian Cube that this name should be generated dynamically for each execution. Then there are three tests (here I only showed the first one) that all look the same. Basically, it prints to console the binding port to connect to the server. Finally, there is a build.gradle file, which executes two tests in parallel, so if you run the tests in Gradle (  ./gradlew test ) you'll see that two tests are executed at the same time and when it finishes one of them, the remaining test is executed. Then, if you inspect the output, you'll see this next output:

org.superbiz.parallel.runner.TestOne STANDARD_OUT
    CubeDockerConfiguration:
      serverUri = tcp://192.168.99.100:2376
      machineName = dev
      certPath = /Users/alex/.docker/machine/machines/dev
      tlsVerify = true
      dockerServerIp = 192.168.99.100
      definitionFormat = COMPOSE
      clean = false
      removeVolumes = true
      dockerContainers = containers:
      redis_9efae4a8-fcb5-4f9e-9b1d-ab591a5c4d5a:
        alwaysPull: false
        image: redis:3.0.7
        killContainer: false
        manual: false
        portBindings: !!set {56697->6379/tcp: null}
        readonlyRootfs: false
        removeVolumes: true
    networks: {}



org.superbiz.parallel.runner.TestThree STANDARD_OUT
    CubeDockerConfiguration:
      serverUri = tcp://192.168.99.100:2376
      machineName = dev
      certPath = /Users/alex/.docker/machine/machines/dev
      tlsVerify = true
      dockerServerIp = 192.168.99.100
      definitionFormat = COMPOSE
      clean = false
      removeVolumes = true
      dockerContainers = containers:
      redis_88ff4b81-80cc-43b3-8bbe-8638dd731d8e:
        alwaysPull: false
        image: redis:3.0.7
        killContainer: false
        manual: false
        portBindings: !!set {56261->6379/tcp: null}
        readonlyRootfs: false
        removeVolumes: true
    networks: {}

    //......

org.superbiz.parallel.runner.TestThree > should_print_port_binding STANDARD_OUT
    org.superbiz.parallel.runner.TestOne - 56261

org.superbiz.parallel.runner.TestOne > should_print_port_binding STANDARD_OUT
    org.superbiz.parallel.runner.TestOne - 56697

org.superbiz.parallel.runner.TestTwo > should_print_port_binding STANDARD_OUT
    org.superbiz.parallel.runner.TestOne - 56697

As you can see in the log, the container name is not redis nor redis*, but redis followed by a UUID. Also, you can see that when the output is printed, the binding port is different in each case.

If you don't want to use the docker-compose approach, you can also define the container programmatically by using Container Object DSL, which also supports star operators.  In this case, the example looks like this:

@ClassRule
public static ContainerDslRule redisStar = 
  new ContainerDslRule("redis:3.2.6", "redis*")
  .withPortBinding(6379);

The approach is the same, but using Container Objects (you need Arquillian Cube 1.4.0 to run it with Container Objects). Notice that thanks to this feature, you can run the tests with any degree of parallel execution, since Arquillian Cube takes care of naming or port binding issues. Notice that in the case of linking between containers, you still need to use the star operator, and it will be resolved at runtime.

To read more about star operator just check here.

Source code: https://github.com/lordofthejars/parallel-docker.