Testing Microservices: Tools and Frameworks

• Teams coordination - With many independent teams managing their own microservices, it becomes very challenging to coordinate the overall process of software development and testing
• Complexity - There are many microservices that communicate with each other. We need to ensure that every one of them is working properly and is resistant to slow responses or failures from other microservices
• Performance - Since there are many independent services, it is important to test the whole architecture under traffic close to production.

Let's discuss some interesting frameworks helping that may help you in testing microservice-based architecture.

Components Tests With Hoverfly

Hoverfly's simulation mode may be especially useful for building component tests. During component tests, we verify the whole microservice without communication over a network with other microservices or external datastores. The following picture shows how such a test is performed for our sample microservice.

Hoverfly provides simple DSL for creating simulations, and a JUnit integration for using it within JUnit tests. It may be orchestrated via JUnit's @Rule. We are simulating two services and then overriding Ribbon properties to resolve the address of these services by client name. We should also disable communication with Eureka discovery by disabling registration after application boot or fetching a list of services for the Ribbon client. Hoverfly simulates responses for PUT and GET methods exposed with the passenger-management and driver-management microservices. The Controller is the main component that implements business logic in our application. It stores data using an in-memory repository component and communicates with other microservices through @FeignClient interfaces. By testing the three methods implemented by the controller, we are testing the whole business logic implemented in the trip-management service.

@SpringBootTest(properties = {
 "eureka.client.enabled=false",
 "ribbon.eureka.enable=false",
 "passenger-management.ribbon.listOfServers=passenger-management",
 "driver-management.ribbon.listOfServers=driver-management"
})
@RunWith(SpringRunner.class)
@AutoConfigureMockMvc
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class TripComponentTests {

 ObjectMapper mapper = new ObjectMapper();

 @Autowired
 MockMvc mockMvc;

 @ClassRule
 public static HoverflyRule rule = HoverflyRule.inSimulationMode(SimulationSource.dsl(
  HoverflyDsl.service("passenger-management:80")
  .get(HoverflyMatchers.startsWith("/passengers/login/"))
  .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Walker'}")))
  .put(HoverflyMatchers.startsWith("/passengers")).anyBody()
  .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Walker'}"))),
  HoverflyDsl.service("driver-management:80")
  .get(HoverflyMatchers.startsWith("/drivers/"))
  .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'David Smith','currentLocationX': 15,'currentLocationY':25}")))
  .put(HoverflyMatchers.startsWith("/drivers")).anyBody()
  .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'David Smith','currentLocationX': 15,'currentLocationY':25}")))
 )).printSimulationData();

 @Test
 public void test1CreateNewTrip() throws Exception {
  TripInput ti = new TripInput("test", 10, 20, "walker");
  mockMvc.perform(MockMvcRequestBuilders.post("/trips")
    .contentType(MediaType.APPLICATION_JSON_UTF8)
    .content(mapper.writeValueAsString(ti)))
   .andExpect(MockMvcResultMatchers.status().isOk())
   .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
   .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("NEW")))
   .andExpect(MockMvcResultMatchers.jsonPath("$.driverId", Matchers.any(Integer.class)));
 }

 @Test
 public void test2CancelTrip() throws Exception {
  mockMvc.perform(MockMvcRequestBuilders.put("/trips/cancel/1")
    .contentType(MediaType.APPLICATION_JSON_UTF8)
    .content(mapper.writeValueAsString(new Trip())))
   .andExpect(MockMvcResultMatchers.status().isOk())
   .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
   .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("IN_PROGRESS")))
   .andExpect(MockMvcResultMatchers.jsonPath("$.driverId", Matchers.any(Integer.class)));
 }

 @Test
 public void test3PayTrip() throws Exception {
  mockMvc.perform(MockMvcRequestBuilders.put("/trips/payment/1")
    .contentType(MediaType.APPLICATION_JSON_UTF8)
    .content(mapper.writeValueAsString(new Trip())))
   .andExpect(MockMvcResultMatchers.status().isOk())
   .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
   .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("PAYED")));
 }

}

The tests above verify only positive scenarios. What about testing unexpected behavior, like network delays or server errors? With Hoverfly, we can easily simulate such a behavior and define some negative scenarios. In the following fragment of code, I have defined three scenarios. In the first of them, the target service has been delayed for two seconds. In order to simulate timeout on the client side, I had to change the default readTimeout for Ribbon load balancer and disable the Hystrix circuit breaker for the Feign client. The second test simulates an HTTP 500 response status from the passenger-management service. The last scenario assumes an empty response from the method responsible for searching the nearest driver.

@SpringBootTest(properties = {
 "eureka.client.enabled=false",
 "ribbon.eureka.enable=false",
 "passenger-management.ribbon.listOfServers=passenger-management",
 "driver-management.ribbon.listOfServers=driver-management",
 "feign.hystrix.enabled=false",
 "ribbon.ReadTimeout=500"
})
@RunWith(SpringRunner.class)
@AutoConfigureMockMvc
public class TripNegativeComponentTests {

 private ObjectMapper mapper = new ObjectMapper();
 @Autowired
 private MockMvc mockMvc;

 @ClassRule
 public static HoverflyRule rule = HoverflyRule.inSimulationMode(SimulationSource.dsl(
  HoverflyDsl.service("passenger-management:80")
  .get("/passengers/login/test1")
  .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Smith'}")).withDelay(2000, TimeUnit.MILLISECONDS))
  .get("/passengers/login/test2")
  .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Smith'}")))
  .get("/passengers/login/test3")
  .willReturn(ResponseCreators.serverError()),
  HoverflyDsl.service("driver-management:80")
  .get(HoverflyMatchers.startsWith("/drivers/"))
  .willReturn(ResponseCreators.success().body("{}"))
 ));

 @Test
 public void testCreateTripWithTimeout() throws Exception {
  mockMvc.perform(MockMvcRequestBuilders.post("/trips").contentType(MediaType.APPLICATION_JSON).content(mapper.writeValueAsString(new TripInput("test", 15, 25, "test1"))))
   .andExpect(MockMvcResultMatchers.status().isOk())
   .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.nullValue()))
   .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("REJECTED")));
 }

 @Test
 public void testCreateTripWithError() throws Exception {
  mockMvc.perform(MockMvcRequestBuilders.post("/trips").contentType(MediaType.APPLICATION_JSON).content(mapper.writeValueAsString(new TripInput("test", 15, 25, "test3"))))
   .andExpect(MockMvcResultMatchers.status().isOk())
   .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.nullValue()))
   .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("REJECTED")));
 }

 @Test
 public void testCreateTripWithNoDrivers() throws Exception {
  mockMvc.perform(MockMvcRequestBuilders.post("/trips").contentType(MediaType.APPLICATION_JSON).content(mapper.writeValueAsString(new TripInput("test", 15, 25, "test2"))))
   .andExpect(MockMvcResultMatchers.status().isOk())
   .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.nullValue()))
   .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("REJECTED")));
 }

}

All the timeouts and errors in communication with external microservices are handled by the bean annotated with @ControllerAdvice. In such cases, the trip-management microservice should not return a server error response, but a 200 OK with a JSON response containing a field status equal to REJECTED.

@ControllerAdvice
public class TripControllerErrorHandler extends ResponseEntityExceptionHandler {

    @ExceptionHandler({RetryableException.class, FeignException.class})
    protected ResponseEntity handleFeignTimeout(RuntimeException ex, WebRequest request) {
        Trip trip = new Trip();
        trip.setStatus(TripStatus.REJECTED);
        return handleExceptionInternal(ex, trip, null, HttpStatus.OK, request);
    }

}

Contract Tests With Pact

The next type of test strategy usually implemented for microservices-based architecture is consumer-driven contract testing. In fact, there are some tools especially dedicated for this type of tests. One of them is Pact. Contract testing is a way to ensure that services can communicate with each other without implementing integration tests. A contract is signed between two sides of communication: consumer and provider. Pact assumes that contract code is generated and published on the consumer side, and then verified by the provider.

Pact provides a tool that can store and share the contracts between consumers and providers. It is called Pact Broker. It exposes a simple RESTful API for publishing and retrieving pacts, and an embedded web dashboard for navigating the API. We can easily run Pact Broker on the local machine using its Docker image.

We will begin by running Pact Broker. Pact Broker requires a running instance of PostgreSQL, so first, we have to launch it using the Docker image, and then link our broker container with that container.

docker run -d --name postgres -p 5432:5432 -e POSTGRES_USER=oauth -e POSTGRES_PASSWORD=oauth123 -e POSTGRES_DB=oauth postgres
docker run -d --name pact-broker --link postgres:postgres -e PACT_BROKER_DATABASE_USERNAME=oauth -e PACT_BROKER_DATABASE_PASSWORD=oauth123 -e PACT_BROKER_DATABASE_HOST=postgres -e PACT_BROKER_DATABASE_NAME=oauth -p 9080:80 dius/pact-broker

The next step is to implement contract tests on the consumer side. We will use the JVM implementation of the Pact library for that. It provides the PactProviderRuleMk2 object responsible for creating stubs of the provider service. We should annotate it with JUnit's @Rule. Ribbon will forward all requests to passenger-management to the stub address, in that case, localhost:8180. Pact JVM supports annotations and provides DSL for building test scenarios. The test method responsible for generating contract data should be annotated with @Pact. It is important to set the state and provider fields because the generated contract will be verified on the provider side using these names. Generated pacts are verified inside the same test class by the methods annotated with @PactVerification. Field fragment points to the name of the method responsible for generating pact inside the same test class. The contract is tested using PassengerManagementClient@FeignClient.

@RunWith(SpringRunner.class)
@SpringBootTest(properties = {
 "driver-management.ribbon.listOfServers=localhost:8190",
 "passenger-management.ribbon.listOfServers=localhost:8180",
 "ribbon.eureka.enabled=false",
 "eureka.client.enabled=false",
 "ribbon.ReadTimeout=5000"
})
public class PassengerManagementContractTests {

 @Rule
 public PactProviderRuleMk2 stubProvider = new PactProviderRuleMk2("passengerManagementProvider", "localhost", 8180, this);
 @Autowired
 private PassengerManagementClient passengerManagementClient;

 @Pact(state = "get-passenger", provider = "passengerManagementProvider", consumer = "passengerManagementClient")
 public RequestResponsePact callGetPassenger(PactDslWithProvider builder) {
  DslPart body = new PactDslJsonBody().integerType("id").stringType("name").numberType("balance").close();
  return builder.given("get-passenger").uponReceiving("test-get-passenger")
   .path("/passengers/login/test").method("GET").willRespondWith().status(200).body(body).toPact();
 }

 @Pact(state = "update-passenger", provider = "passengerManagementProvider", consumer = "passengerManagementClient")
 public RequestResponsePact callUpdatePassenger(PactDslWithProvider builder) {
  return builder.given("update-passenger").uponReceiving("test-update-passenger")
   .path("/passengers").method("PUT").bodyWithSingleQuotes("{'id':1,'amount':1000}", "application/json").willRespondWith().status(200)
   .bodyWithSingleQuotes("{'id':1,'name':'Adam Smith','balance':5000}", "application/json").toPact();
 }

 @Test
 @PactVerification(fragment = "callGetPassenger")
 public void verifyGetPassengerPact() {
  Passenger passenger = passengerManagementClient.getPassenger("test");
  Assert.assertNotNull(passenger);
  Assert.assertNotNull(passenger.getId());
 }

 @Test
 @PactVerification(fragment = "callUpdatePassenger")
 public void verifyUpdatePassengerPact() {
  Passenger passenger = passengerManagementClient.updatePassenger(new PassengerInput(1 L, 1000));
  Assert.assertNotNull(passenger);
  Assert.assertNotNull(passenger.getId());
 }

}

Just running the tests is not enough. We also have to publish pacts generated during tests to Pact Broker. In order to achieve this, we have to include the following Maven plugin in our pom.xml and  execute the command mvn clean install pact:publish.

<plugin>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-maven_2.12</artifactId>
<version>3.5.21</version>
<configuration>
<pactBrokerUrl>http://192.168.99.100:9080</pactBrokerUrl>
</configuration>
</plugin>

Pact provides support for Spring on the provider side. Thanks to that, we may use MockMvc controllers or inject properties from application.yml into the test class. Here's the dependency declaration that has to be included in our pom.xml

<dependency>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-spring_2.12</artifactId>
<version>3.5.21</version>
<scope>test</scope>
</dependency>

Now the contract is being verified on the provider side. We need to pass the provider name inside the @Provider annotation and name of states for every verification test inside @State. These values have been in the tests on the consumer side inside @Pact annotation (fields state and provider).

@RunWith(SpringRestPactRunner.class)
@Provider("passengerManagementProvider")
@PactBroker
public class PassengerControllerContractTests {

 @InjectMocks
 private PassengerController controller = new PassengerController();
 @Mock
 private PassengerRepository repository;
 @TestTarget
 public final MockMvcTarget target = new MockMvcTarget();

 @Before
 public void before() {
  MockitoAnnotations.initMocks(this);
  target.setControllers(controller);
 }

 @State("get-passenger")
 public void testGetPassenger() {
  target.setRunTimes(3);
  Mockito.when(repository.findByLogin(Mockito.anyString()))
   .thenReturn(new Passenger(1 L, "Adam Smith", "test", 4000))
   .thenReturn(new Passenger(3 L, "Tom Hamilton", "hamilton", 400000))
   .thenReturn(new Passenger(5 L, "John Scott", "scott", 222));
 }

 @State("update-passenger")
 public void testUpdatePassenger() {
  target.setRunTimes(1);
  Passenger passenger = new Passenger(1 L, "Adam Smith", "test", 4000);
  Mockito.when(repository.findById(1 L)).thenReturn(passenger);
  Mockito.when(repository.update(Mockito.any(Passenger.class)))
   .thenReturn(new Passenger(1 L, "Adam Smith", "test", 5000));
 }
}

The Pact Broker host and port are injected from the application.yml file.

pactbroker:
  host: "192.168.99.100"
  port: "8090"

Performance Tests With Gatling

An important step of testing microservices before deploying them to production is performance testing. An interesting tool in this area is Gatling. It is a highly capable load testing tool written in Scala. That means that we also have to use Scala DSL in order to build test scenarios. Let's begin by adding the required library to the pom.xml file.

<dependency>
<groupId>io.gatling.highcharts</groupId>
<artifactId>gatling-charts-highcharts</artifactId>
<version>2.3.1</version>
</dependency>

Now we may proceed to the test. In the scenario above, we are testing two endpoints exposed by trip-management: POST /trips and PUT /trips/payment/${tripId}. In fact, this scenario verifies the whole functionality of our sample system, where we are setting up a trip and then pay for it after the finish.

Every test class using Gatling needs to extend the Simulation class. We are defining our scenario using the scenario method and then setting its name. We may define multiple executions in a single scenario. After every execution of the POST /trips method test, it saves the generated id returned by the service. Then, it inserts that id into the URL used for calling the method PUT /trips/payment/${tripId}. Every single test expects a response with a 200 OK status.

Gatling provides two interesting features worth mentioning. You can see how they are used in the following performance test. The first of them is feeder. It is used for polling records and injecting their content into the test. The feed rPassengers selects one of five defined logins randomly. The final test result may be verified using the Assertions API. It is responsible for verifying whether global statistics like response time or the number of failed requests match expectations for a whole simulation. In the scenario below, the criterium is max response time, which needs to be lower than 100 milliseconds.

class CreateAndPayTripPerformanceTest extends Simulation {

 val rPassengers = Iterator.continually(Map("passenger" -> List("walker", "smith", "hamilton", "scott", "holmes").lift(Random.nextInt(5)).get))

 val scn = scenario("CreateAndPayTrip").feed(rPassengers).repeat(100, "n") {
  exec(http("CreateTrip-API")
   .post("http://localhost:8090/trips")
   .header("Content-Type", "application/json")
   .body(StringBody(""
    "{"
    destination ":"
    test$ {
     n
    }
    ","
    locationX ":${n},"
    locationY ":${n},"
    username ":"
    $ {
     passenger
    }
    "}"
    ""))
   .check(status.is(200), jsonPath("$.id").saveAs("tripId"))
  ).exec(http("PayTrip-API")
   .put("http://localhost:8090/trips/payment/${tripId}")
   .header("Content-Type", "application/json")
   .check(status.is(200))
  )
 }

 setUp(scn.inject(atOnceUsers(20))).maxDuration(FiniteDuration.apply(5, TimeUnit.MINUTES))
  .assertions(global.responseTime.max.lt(100))

}

In order to run Gatling performance tests, you need to include the following Maven plugin in your pom.xml. You may run a single scenario or  multiple scenarios. After including the plugin, you only need to execute the command mvn clean gatling:test.

<plugin>
<groupId>io.gatling</groupId>
<artifactId>gatling-maven-plugin</artifactId>
<version>2.2.4</version>
<configuration>
<simulationClass>pl.piomin.performance.tests.CreateAndPayTripPerformanceTest</simulationClass>
</configuration>
</plugin>

Here are some diagrams illustrating the result of performance tests for our microservice. Because maximum response time is greater than set in the assertion (100ms), the test has failed.

And...

Summary

The right selection of tools is not the most important element of microservices testing, however, the right tools can help you face the key challenges related to it. Hoverfly allows us to create full component tests that verify if your microservice is able to handle delays or errors from downstream services. Pact helps you to organize the team by sharing and verifying contracts between independently developed microservices. Finally, Gatling can help you implement load tests for selected scenarios in order to verify the end-to-end performance of your system.

The source code used as a demo for this article is available on GitHub.