Microservices With Spring Boot, Spring Cloud Gateway and Consul Cluster

The Spring Cloud Consul project provides integration for Consul and Spring Boot applications through auto-configuration. By using the well-known Spring Framework annotation style, we may enable and configure common patterns within microservice-based environments.

You may also like: Microservices With Spring Boot, Part 1 — Getting Started

These patterns include service discovery using Consul agent, distributed configuration using Consul key/value store, distributed events with Spring Cloud Bus, and Consul Events. The project also supports a client-side load balancer based on Netflix's Ribbon and an API gateway based on Spring Cloud Gateway.
In this article I will cover the following topics:

• Integrating Spring Boot application with Consul discovery.
• Integrating Spring Cloud Gateway with Consul discovery.
• Using Consul KV for distributing configuration across Spring Boot applications.
• Running Consul in a clustered mode.
• Defining virtual zones for microservices and gateway.

Microservices Architecture

Let's proceed to the example system. It consists of four independent microservices. Some of them may call endpoints exposed by the others. The application source code is available on GitHub here: https://github.com/piomin/sample-spring-cloud-consul.git.

In the current example, we will try to develop a simple order system where customers may buy products. If a customer decides to confirm a selected list of products to buy, the POST request is sent to the order-service. It is processed by the Order prepare(@RequestBody Order order) method inside the REST controller.

This method is responsible for order preparation. First, it calculates the final price, considering the price of each product from the list, customer order history, and their category in the system by calling the proper API method from the customer-service.

Then, it verifies if the customer's account balance is enough to execute the order by calling the account-service, and finally, it returns the calculated price. If the customer confirms the action, the PUT /{id} method is called. The request is processed by the method Order accept(@PathVariable Long id) inside the REST controller.

It changes the order status and withdraws money from the customer's account. The system architecture is broken down into the individual microservices hidden behind API gateway as shown here:

The description created above should give you a big picture of our example system. However, the business logic plays a supporting role, technically we have four Spring Boot applications using Consul discovery and KV store communicating with each other through REST APIs. The whole system is hidden for the external client behind the API gateway built on top of the Spring Cloud Gateway. Let's proceed to the implementation.

1. Building Microservices

Let's begin with dependencies. We use the currently newest stable version of Spring Boot - 2.2.0.RELEASE together with Spring Cloud Release Train Hoxton.RC1. The minimal set of required dependencies is to have Spring Web, Actuator (optionally) and Spring Cloud Consul (discovery + config).

<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.0.RELEASE</version>
</parent>

<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Hoxton.RC1</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-consul-all</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
</dependencies>

When running the application we will use dynamic listen to port number generation feature by setting the property server.port to 0. Because we will run more than an instance of every service we also need to override the default value of spring.cloud.consul.discovery.instance-id which is based on the port number that is not applicable when it is set to 0. Here's our application.yml file for account-service.

spring:  
  cloud:
    consul:
      discovery:
        instance-id: "${spring.cloud.client.hostname}:${spring.application.name}:${random.int[1,999999]}"

server:
  port: 0

The configuration is deployed on Consul, which means we are only having bootstrap.yml file on the classpath. If you have both Spring Cloud Consul Discovery and Config dependencies distributed configuration is enabled by default. You only have to override the address of the Consul server if required.

spring:  
  application:
    name: account-service
  cloud:
    consul:
      host: 192.168.99.100
      port: 8500

In the current version of Spring Cloud we don't have to enable anything, so just need to declare the main class:

@SpringBootApplication
public class AccountApplication {

public static void main(String[] args) {
SpringApplication.run(AccountApplication.class);
}
}

2. Running Consul Cluster using Docker

In this section how to set up the local environment similar to the production model. Therefore, we would like to have a scalable, production-grade service discovery infrastructure, consisting of some nodes working together inside the cluster.

Consul provides support for clustering based on a gossip protocol used for communication between members and a Raft consensus protocol for a leadership election. I wouldn't like to go into the details of that process, but some basics about Consul architecture should be clarified.
The first important element is the Consul agent.

An agent is a long-running daemon on every member of the Consul cluster. It may be run in either client or server mode. All agents are responsible for running checks and keeping services registered, in different nodes and sync, globally. Our main goal in this section is to set up and configure the Consul cluster using its Docker image. First, we will start the container, which acts as a leader of the cluster.

There is only one difference in the currently used Docker command than for the standalone Consul server. We have set the environment variable CONSUL_BIND_INTERFACE=eth0 to change the network address of the cluster agent from 127.0.0.1 the one available for the other member containers. My Consul server is now running at the internal address 172.17.0.2.

To check out what your address is (it should be the same) you may run the command docker logs consul. The appropriate information is logged just after the container startup. Here's the command that starts the first Consul node:

$ docker run -d --name consul-1 -p 8500:8500 -e CONSUL_BIND_INTERFACE=eth0 consul

Knowledge of that address is very important since now we have to pass it to every member container startup command as a cluster join parameter. We also bind it to all interfaces by setting 0.0.0.0 as a client address. Now, we may easily expose the client agent API outside the container using the -p parameter:

$ docker run -d --name consul-2 -e CONSUL_BIND_INTERFACE=eth0 
-p 8501:8500 consul agent -dev -join=172.17.0.2
$ docker run -d --name consul-3 -e CONSUL_BIND_INTERFACE=eth0 
-p 8502:8500 consul agent -dev -join=172.17.0.2

After running two containers with Consul agent, you may check out the full list of cluster members by executing the following command on the leader's container:

We can always get the same information using the Consul Web Console.

We may easily change the default Consul node address for the Spring Boot application by changing configuration properties. Spring Cloud allows you to define only a single host address and port number of Consul agent. It is worth to note that in normal production mode with multiple machines you would install only Consul agent on every machine, which is connected with a cluster of Consul servers.

spring:
  application:
    name: customer-service
  cloud:
    consul:
      host: 192.168.99.100
      port: 8501

3. Inter-Service Communication

An inter-service communication is performed using OpenFeign declarative REST client. We can also include Spring Cloud Sleuth dependency for propagating correlationId between subsequent calls.

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>

The OpenFeign client is auto-integrated with service discovery. To use it we need to declare an interface with required methods for communication. The interface has to be annotated with @FeignClient that points to service using its discovery name.

@FeignClient(name = "account-service")
public interface AccountClient {

@GetMapping("/customer/{customerId}")
List<Account> findByCustomer(@PathVariable("customerId") Long customerId);

}

Finally, OpenFeign client needs to be enabled for the whole application.

@SpringBootApplication
@EnableFeignClients
public class CustomerApplication {

public static void main(String[] args) {
SpringApplication.run(CustomerApplication.class, args);
}

}

4. Enable Zone Affinity Mechanism

When using Spring Cloud Discovery we may take advantage of zones affinity mechanism. If your microservices has been deployed to multiple zones, you may prefer that those services communicate with other services within the same zone before trying to access them in another zone.

The same rule applies to the API gateway that prefers communication with microservices within the same zone as a gateway. For testing purposes we run two instances of every microservice distributed across two zones: zone1 and zone2. The same with gateway-service. The current architecture of our system looks as shown below.
  The whole mechanism is enabled through the configuration. We need to set the default zone name for our microservice using property spring.cloud.consul.discovery.instanceZone. I defined two profiles for each application, that may be set during startup with --spring.profiles.active command-line argument.

---
spring:
  profiles: zone1
  cloud:
    consul:
      discovery:
        instanceZone: zone1

---
spring:
  profiles: zone2
  cloud:
    consul:
      discovery:
        instanceZone: zone2

Spring Cloud provides a zone affinity mechanism based on Consul tags. If you set spring.cloud.consul.discovery.instanceZone property, Spring Cloud Consul tags a registered instance of service with zone metadata. The name of that tag may be overridden with spring.cloud.consul.discovery.defaultZoneMetadataName property.

Assuming you have run two instances of each microservice divided into two zones using the command, for example java -jar --spring.profiles.active=zone1 target/order-service-1.1.jar, you should see the following list of registered services on your Consul instance.

Here's a more detailed view of is Nodes section that prints all tags and listens to a port number for every instance of microservice.

We can also display all running instances of a single service. In the following picture, you can see instances of account-service.

5. Building API Gateway With Spring Cloud

Since now, we have successfully run all the microservices in two instances distributed across two different zones. Because they are all listening on dynamically generated ports we need an API gateway that is exposed to the static port to an external client. Here's the list of dependencies used for building gateway-service:

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-consul-all</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Because we would like to run the gateway on the static port the configuration of Maven profiles is slightly larger than for microservices. We also don't need to register gateway in Consul discovery, because it is not accessed internally. We will run two instances of the gateway, first available under port 8080 in zone1, and second available under port 9080 in zone2. It is not registered in Consul discovery we have to manually set value for zone tag.

---
spring:
  profiles: zone1
  cloud:
    consul:
      discovery:
        instanceZone: zone1
        register: false
        registerHealthCheck: false
        tags: zone=zone1
server:  
  port: ${PORT:8080}

---
spring:
  profiles: zone2
  cloud:
    consul:
      discovery:
        instanceZone: zone2
        register: false
        registerHealthCheck: false
        tags: zone=zone2
server:  
  port: ${PORT:9080}

To enable integration with Consul discovery we need to set the property spring.cloud.gateway.discovery.locator.enabled to true. To expose service under a custom path, we should define Path predicate and RewritePath filter for each service. In that case, account-service is available under address, customer-service, etc. under http://localhost:8080/customer/ , etc.

spring:
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
        - id: account-service
          uri: lb://account-service
          predicates:
            - Path=/account/**
          filters:
            - RewritePath=/account/(?<path>.*), /$\{path}
        - id: customer-service
          uri: lb://customer-service
          predicates:
            - Path=/customer/**
          filters:
            - RewritePath=/customer/(?<path>.*), /$\{path}
        - id: order-service
          uri: lb://order-service
          predicates:
            - Path=/order/**
          filters:
            - RewritePath=/order/(?<path>.*), /$\{path}
        - id: product-service
          uri: lb://product-service
          predicates:
            - Path=/product/**
          filters:
            - RewritePath=/product/(?<path>.*), /$\{path}

Now you can be sure that each request incoming to gateway-service started in zone1 would be forwarded to in the first place to microservice also started in zone1. And the same for zone2.

6. Distributed Configuration

Consul Config is automatically enabled for the application just after including dependency spring-cloud-starter-consul-config. Of course, it is included together with spring-cloud-starter-consul-all also. The configuration is stored in the /config folder by default. We can create the configuration per all applications or just for a single application in a dedicated folder.

Assuming we have four microservices and API gateway deployed in two zones we would have to define ten configuration folders. We have different options for storing application properties, but I chose the YAML format. YAML must be set in the appropriate data key in consul. So the Consul folders structure for all our sample applications looks as shown below.

config/account-service,zone1/data
config/account-service,zone2/data
config/customer-service,zone1/data
config/customer-service,zone2/data
config/order-service,zone1/data
config/order-service,zone2/data
config/product-service,zone1/data
config/product-service,zone2/data
config/gateway-service,zone1/data
config/gateway-service,zone2/data

Here's the typical configuration for one our sample microservice running in zone1 zone.

spring:  
  cloud:
    consul:
      discovery:
        instanceId: "${spring.cloud.client.hostname}:${spring.application.name}
          ":${random.int[1,999999]}"
        instanceZone: zone1
server.port: 0

And the same configuration created on Consul for account-service with active zone1 profile.

In case we use Consul Config for our application the only file that should be available on the classpath is bootstrap.yml. Except for overriding Consul IP address or port if required we have to set the format of configuration properties to YAML. Here's bootstrap.yml file for account-service.

spring:  
  application:
    name: account-service
  cloud:
    consul:
      host: 192.168.99.100
      port: 8500
        config:
          format: YAML

Summary

In this article, I show you how to run microservices using Spring Consul Discovery and Config in the local environment similar to production. The applications use zone affinity mechanism for inter-service communication and integrate with a cluster of Consul nodes.

The configuration may be stored on the classpath or externalized to be stored on Consul with division into different active profiles. I think that Consul is a future of Spring Cloud microservices in the post-Netflix era, so it is worth to know Spring Cloud Consul better.

Further Reading

Spring Boot Microservices: Building a Microservices Applications Using Spring Boot

Quick Guide to Microservices With Spring Boot 2.0, Eureka, and Spring Cloud

Microservices Architecture With Spring Boot and Spring Cloud