Testing Spring Boot Integration With Vault and Postgres Using Testcontainers Framework

I have already written many articles where I was using Docker containers for running some third-party solutions integrated with my sample applications. Building integration tests for such applications may not be an easy task without Docker containers. Especially if our application integrates with databases, message brokers, or some other popular tools. If you are planning to build such integration tests, you should definitely take a look on Testcontainers.

Testcontainers is a Java library that supports JUnit tests, providing a fast and lightweight way for running instances of common databases, Selenium web browsers, or anything else that can run in a Docker container. It provides modules for the most popular relational and NoSQL databases like Postgres, MySQL, Cassandra, or Neo4j. It also allows running popular products like Elasticsearch, Kafka, Nginx, or HashiCorp's Vault. Today, I'm going to show you a more advanced sample of JUnit tests that use Testcontainers to check out an integration between Spring Boot/Spring Cloud application, Postgres database, and Vault.

For the purposes of that example, we will use the case described in one of my previous articles Secure Spring Cloud Microservices with Vault and Nomad. Let us recall that use case.

I described there how to use a very interesting Vault feature called secret engines for generating database user credentials dynamically. I used Spring Cloud Vault module in my Spring Boot application to automatically integrate with that feature of Vault. The implemented mechanism is pretty easy. The application calls Vault secret engine before it tries to connect to Postgres database on startup. Vault is integrated with Postgres via secret engine, and that's why it creates a user with sufficient privileges on Postgres. Then, generated credentials are automatically injected into auto-configured Spring Boot properties used for connecting with database spring.datasource.username and spring.datasource.password. The following picture illustrates the described solution:

Ok, we know how it works, now the question is how to automatically test it. With Testcontainers, it is possible with just a few lines of code.

1. Building the Application

Let's begin with a short intro to the application code; it is very simple. Here's the list of dependencies required for building an application that exposes REST API and integrates with Postgres and Vault.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-vault-config</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-vault-config-databases</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <version>42.2.5</version>
</dependency>

The application connects to Postgres, enables integration with Vault via Spring Cloud Vault, and automatically creates/updates tables on startup.

spring:
  application:
    name: callme-service
  cloud:
    vault:
      uri: http://192.168.99.100:8200
      token: ${VAULT_TOKEN}
      postgresql:
        enabled: true
        role: default
        backend: database
  datasource:
    url: jdbc:postgresql://192.168.99.100:5432/postgres
  jpa.hibernate.ddl-auto: update

It exposes the single endpoint. The following method is responsible for handling incoming requests. It just inserts a record to the database and returns a response with the app name, version, and ID of the inserted record.

@RestController
@RequestMapping("/callme")
public class CallmeController {

    private static final Logger LOGGER = LoggerFactory.getLogger(CallmeController.class);

    @Autowired
    Optional<BuildProperties> buildProperties;
    @Autowired
    CallmeRepository repository;

    @GetMapping("/message/{message}")
    public String ping(@PathVariable("message") String message) {
        Callme c = repository.save(new Callme(message, new Date()));
        if (buildProperties.isPresent()) {
            BuildProperties infoProperties = buildProperties.get();
            LOGGER.info("Ping: name={}, version={}", infoProperties.getName(), infoProperties.getVersion());
            return infoProperties.getName() + ":" + infoProperties.getVersion() + ":" + c.getId();
        } else {
            return "callme-service:"  + c.getId();
        }
    }

}

2. Enabling Testcontainers

To enable Testcontainers for our project, we need to include some dependencies to our Maven pom.xml. We have dedicated modules for Postgres and Vault. We also include a Spring Boot Test dependency because we would like to test the whole Spring Boot app.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>vault</artifactId>
    <version>1.10.5</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>testcontainers</artifactId>
    <version>1.10.5</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>postgresql</artifactId>
    <version>1.10.5</version>
    <scope>test</scope>
</dependency>

3. Running Vault Test Container

Testcontainers framework supports JUnit 4/JUnit 5 and Spock. The Vault container can be started before tests if it is annotated with @Rule or @ClassRule. By default, it uses version 0.7, but we can override it with the newest version, which is 1.0.2. We also may set a root token, which is then required by Spring Cloud Vault for integration with Vault.

@ClassRule
public static VaultContainer vaultContainer = new VaultContainer<>("vault:1.0.2")
    .withVaultToken("123456")
    .withVaultPort(8200);

That root token can be overridden before starting the JUnit test on the test class.

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = {
    "spring.cloud.vault.token=123456"
})
public class CallmeTest { ... }

4. Running Postgres Test Container

As an alternative to @ClassRule, we can manually start the container in a @BeforeClass or @Before method in the test. With this approach, you will also have to stop it manually in @AfterClass or @After method. We start Postgres container manually because by default, it is exposed on a dynamically generated port, which needs to be set for Spring Boot application before starting the test. The listen port is returned by method getFirstMappedPort invoked on PostgreSQLContainer.

private static PostgreSQLContainer postgresContainer = new PostgreSQLContainer()
    .withDatabaseName("postgres")
    .withUsername("postgres")
    .withPassword("postgres123");

@BeforeClass
public static void init() throws IOException, InterruptedException {
    postgresContainer.start();
    int port = postgresContainer.getFirstMappedPort();
    System.setProperty("spring.datasource.url", String.format("jdbc:postgresql://192.168.99.100:%d/postgres", postgresContainer.getFirstMappedPort()));
    // ...
}

@AfterClass
public static void shutdown() {
    postgresContainer.stop();
}

5. Integrating Vault and Postgres Containers

Once we have successfully started both Vault and Postgres containers, we need to integrate them via Vault secret engine. First, we need to enable the database secret engine Vault. After that, we must configure the connection to Postgres. The last step is to configure a role. A role is a logical name that maps to a policy used to generate those credentials. All these actions may be performed using Vault commands. You can launch a command on the Vault container using execInContainer method. Vault configuration commands should be executed just after Postgres container startup.

@BeforeClass
public static void init() throws IOException, InterruptedException {
    postgresContainer.start();
    int port = postgresContainer.getFirstMappedPort();
    System.setProperty("spring.datasource.url", String.format("jdbc:postgresql://192.168.99.100:%d/postgres", postgresContainer.getFirstMappedPort()));
    vaultContainer.execInContainer("vault", "secrets", "enable", "database");
    String url = String.format("connection_url=postgresql://{{username}}:{{password}}@192.168.99.100:%d?sslmode=disable", port);
    vaultContainer.execInContainer("vault", "write", "database/config/postgres", "plugin_name=postgresql-database-plugin", "allowed_roles=default", url, "username=postgres", "password=postgres123");
    vaultContainer.execInContainer("vault", "write", "database/roles/default", "db_name=postgres",
        "creation_statements=CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';GRANT SELECT, UPDATE, INSERT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";GRANT USAGE,  SELECT ON ALL SEQUENCES IN SCHEMA public TO \"{{name}}\";",
        "default_ttl=1h", "max_ttl=24h");
}

6. Running Application Tests

Finally, we may run application tests. We just call the single endpoint exposed by the app using TestRestTemplate, and verify the output.

@Autowired
TestRestTemplate template;

@Test
public void test() {
    String res = template.getForObject("/callme/message/{message}", String.class, "Test");
    Assert.assertNotNull(res);
    Assert.assertTrue(res.endsWith("1"));
}

If you are interested in what exactly happens during the test, you can set a breakpoint inside the test method and execute the docker ps command manually.