The Future of Spring Cloud Microservices After Netflix Era

If somebody asked you about Spring Cloud, the first thing that would come to your mind would probably be Netflix OSS support. Support for tools like Eureka, Zuul, or Ribbon is provided not only by Spring, but also by some other popular frameworks used for building microservices architecture like Apache Camel, Vert.x, or Micronaut. Currently, Spring Cloud Netflix is the most popular project that's a part of Spring Cloud. It has around 3.2k stars on GitHub, while the second best has around 1.4k. Therefore, it is quite surprising that Pivotal has announced that most of Spring Cloud Netflix modules are entering maintenance mode. You can read more about in the post published on the Spring blog by Spencer Gibb.

Ok, let's give a short summary of the changes. Starting from the Spring Cloud Greenwich release Train, Netflix OSS, Archaius, Hystrix, Ribbon, and Zuul are entering maintenance mode. This means that there won't be any new features added to these modules, and the Spring Cloud team will perform only some bug fixes and fix security issues. The maintenance mode does not include the Eureka module, which is still supported.

The explanation of these changes is pretty easy. Especially for two of them. Currently, Ribbon and Hystrix are not actively developed by Netflix, although they are still deployed at scale. Additionally, Hystrix has been already superseded by the new solution for telemetry called Atlas. The situation with Zuul is not so obvious. Netflix announced the open sourcing of Zuul 2 on May 2018. The new version of the Zuul gateway is built on top of the Netty server, and includes some improvements and new features. You can read more about them on the Netflix blog. Despite this decision being made by the Netflix cloud team, the Spring Cloud team has abandoned development of the Zuul module. I can only guess that it was caused by the earlier decision of starting a new module inside the Spring Cloud family dedicated specifically to being an API gateway in Spring Cloud Gateway's microservices-based architecture.

The last piece of that puzzle is Eureka — a discovery server. It is still in development, but the situation is also interesting here. I will describe that in the next part of this article.

All this news has inspired me to take a look at the current situation of Spring Cloud and discuss some potential changes to come in the future. As an author of the Mastering Spring Cloud book I'm trying to follow an evolution of that project to stay current. It's also worth mentioning that we are have microservices inside my organization — of course built on top of Spring Boot and Spring Cloud using modules like Eureka, Zuul, and Ribbon. In this article, I would like to discuss some popular microservices patterns like service discovery, distributed configuration, client-side load balancing, and API gateways.

Service Discovery

Eureka is the only important Spring Cloud Netflix module that has not been moved to the maintenance mode. However, I would not say that it is actively developed. The last commit in the repository maintained by Netflix is from January 11. Some time ago, they have started working on Eureka 2, but it seems this work has been abandoned or they just have postponed open sourcing the newest version.

Here, you can find an interesting comment about it: "The 2.x branch is currently frozen as we have had some internal changes w.r.t. to eureka2, and do not have any time lines for open sourcing of the new changes."So, we have two possibilities. Maybe, Netflix will decide to open source those internal changes as a version 2 of the Eureka server. It is worth remembering that Eureka is a battle proven solution used at scale by Netflix directly, and probably by many other organizations through Spring Cloud.

The second option is to choose another discovery server. Currently, Spring Cloud supports discovery based on various tools: ZooKeeper, Consul, Alibaba Nacos, and Kubernetes. In fact, Kubernetes is based on etcd. Support for etcd is also being developed by Spring Cloud, but it is still in the incubation stage, and it is not known if it will be ever promoted to the official release train. In my opinion, there's one leader amongst these solutions — HashiCorp's Consul.

Consul is now described as a service mesh solution providing a full featured control plane with service discovery, configuration, and segmentation functionality. It can be used as a discovery server or a key/value store in your microservices-based architecture. The integration with Consul is implemented by Spring Cloud Consul project. To enable Consul client for your application you just need to include the following dependency to your Maven pom.xml:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-consul-discovery</artifactId>
</dependency>

By default, Spring tries to connect with Consul on the address localhost:8500. If you need to override this address you should set the appropriate properties inside application.yml:

spring:  
  cloud:
    consul:
      host: 192.168.99.100
      port: 8500

You can easily test this solution with local instance of Consul started as the Docker container:

$ docker run -d --name consul -p 8500:8500 consul

As you can see, Consul discovery implementation with Spring Cloud is very easy — much like it is for Eureka. Consul has one undoubted advantage over Eureka — it is continuously maintained and developed by HashiCorp. Its popularity is growing fast. It is a part of biggest HashiCorp's ecosystem, which includes Vault, Nomad, and Terraform. In contrast to Eureka, Consul can be used not only for service discovery, but also as a configuration server in your microservices-based architecture.

Distributed Configuration

Netflix Archaius is an interesting solution for managing externalized configuration in microservices architecture. Although it offers some interesting features like dynamic and typed properties or support for dynamic data sources such as URLs, JDBC, or AWS DynamoDB, Spring Cloud has also decided to move it to maintenance mode. However, the popularity of Spring Cloud Archaius was limited, due to the existence of a similar project, fully created by the Pivotal team and community - Spring Cloud Config. Spring Cloud Config supports multiple source repositories including Git, JDBC, Vault, or simple files. You can find many examples of using this project for providing distributed configuration for your microservices in my previous posts. Today, I'm not going to talk about it. We will discuss an alternative solution which is also supported by Spring Cloud.

As I mentioned in the end of the previous section, Consul can also be used as a configuration server. If you use Eureka as a discovery server, using Spring Cloud Config as a configuration server is a natural choice, because Eureka simply does not provide such features. This is not the case if you decide to use Consul. Now it makes sense to choose between two solutions: Spring Cloud Consul Config and Spring Cloud Config. Of course, both of them have their advantages and disadvantages. For example, you can easily build a cluster with Consul nodes, while with Spring Cloud Config you must rely on external discovery.

Now, let's see how to use Spring Cloud Consul for managing external configuration in your application. To enable it on the application side you just need to include the following dependency to your Maven pom.xml:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-consul-config</artifactId>
</dependency>

The same as for service discovery, If you would like to override some default client settings you need to set properties spring.cloud.consul.*. However, such a configuration must provided inside bootstrap.yml.

spring:  
  application:
    name: callme-service
  cloud:
    consul:
      host: 192.168.99.100
      port: 8500

The name of the property source created on Consul should be the same as the application name provided in bootstrap.yml inside the config folder. You should create key server.port with a value of 0, to force Spring Boot to randomly generate a listening port number. If you need to set the application's default listening port, you should using the following configuration.

When enabling dynamic port number generation, you also need to override the application's instance id so that it is unique across a single machine. These features are required if you are running multiple instances of a single service on the same machine. We will do it for callme-service, so we need to override the the property spring.cloud.consul.discovery.instance-id with our value as shown below.

Then, you should see the following log on your application startup.

API Gateway

The successor of Spring Cloud Netflix Zuul is Spring Cloud Gateway. This project was started around two years ago, and is now the second most popular Spring Cloud project with 1.4k stars on GitHub. It provides an API Gateway built on top of the Spring Ecosystem, including: Spring 5, Spring Boot 2 and Project Reactor. It is running on Netty, and does not work with traditional servlet container like Tomcat or Jetty. It allows us to define routes, predicates, and filters.

API gateway, just like every Spring Cloud microservice, may be easily integrated with service discovery based on Consul. We just need to include the appropriate dependencies inside pom.xml. We will use the latest development version of Spring Cloud libraries — 2.2.0.BUILD-SNAPSHOT. Here's the list of required dependencies:

<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-consul-discovery</artifactId>
	<version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>
<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-consul-config</artifactId>
	<version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>
<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-gateway</artifactId>
	<version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>

The gateway configuration will also be served by Consul. Because, we have more configuration settings than for sample microservices, we will store it as a YAML file. To achieve this, we should create a YAML file available under the /config/gateway-service/data path on Consul's Key/Value. The configuration visible below enables service discovery integration and defines routes to the downstream services. Each route contains the name of the target service under which it is registered in service discovery, matching path and rewrite path used for call endpoint exposed by the downstream service. The following configuration is loaded on startup by our API gateway:

spring:
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
        - id: caller-service
          uri: lb://caller-service
          predicates:
            - Path=/caller/**
          filters:
            - RewritePath=/caller/(?.*), /$\{path}
        - id: callme-service
          uri: lb://callme-service
          predicates:
            - Path=/callme/**
          filters:
            - RewritePath=/callme/(?.*), /$\{path}

Here's the same configuration visible on Consul.

The last step is to force gateway-service to read configurations stored as YAML. To do that, we need to set the property spring.cloud.consul.config.format to YAML. Here's the full configuration provided inside bootstrap.yml.

spring:
  application:
    name: gateway-service
  cloud:
    consul:
      host: 192.168.99.100
      config:
        format: YAML

Client-Side Load Balancer

In version 2.2.0.BUILD-SNAPSHOT of Spring Cloud Commons Ribbon, it is still the main auto-configured load balancer for HTTP clients. Although the Spring Cloud team has announced that Spring Cloud Load Balancer will be the successor of Ribbon, we currently won't find much information about that project in the documentation or on the web. We may expect that the same is true for Netflix Ribbon, i.e. any configuration will be transparent for us, especially if we use discovery client. Currently, thespring-cloud-loadbalancer module is a part of Spring Cloud Commons project. You may include it directly to your application by declaring the following dependency in pom.xml:

<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-loadbalancer</artifactId>
	<version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>

For test purposes, it is worth it to exclude some Netflix modules included together with the spring-cloud-starter-consul-discovery starter. Now, we are sure that Ribbon is not used in the background as a load balancer. Here's the list of exclusions I set for my sample application:

<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-consul-discovery</artifactId>
	<version>2.2.0.BUILD-SNAPSHOT</version>
	<exclusions>
		<exclusion>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-netflix-core</artifactId>
		</exclusion>
		<exclusion>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-starter-netflix-archaius</artifactId>
		</exclusion>
		<exclusion>
			<groupId>com.netflix.ribbon</groupId>
			<artifactId>ribbon</artifactId>
		</exclusion>
		<exclusion>
			<groupId>com.netflix.ribbon</groupId>
			<artifactId>ribbon-core</artifactId>
		</exclusion>
		<exclusion>
			<groupId>com.netflix.ribbon</groupId>
			<artifactId>ribbon-httpclient</artifactId>
		</exclusion>
		<exclusion>
			<groupId>com.netflix.ribbon</groupId>
			<artifactId>ribbon-loadbalancer</artifactId>
		</exclusion>
	</exclusions>
</dependency>

Treat my example just as a playground. Certainly the targeted approach is going to be much easier. First, we should annotate our main or configuration class with @LoadBalancerClient. As always, the name of the client should be the same as the name of the target service registered in the registry. The annotation should also contain the class with client configurations.

@SpringBootApplication
@LoadBalancerClients({
 @LoadBalancerClient(name = "callme-service", configuration = ClientConfiguration.class)
})
public class CallerApplication {

 public static void main(String[] args) {
  SpringApplication.run(CallerApplication.class, args);
 }

 @Bean
 RestTemplate template() {
  return new RestTemplate();
 }

}

Here's our load balancer configuration class. It contains the declaration of a single @Bean. I have chosen RoundRobinLoadBalancer type.

public class ClientConfiguration {

 @Bean
 public RoundRobinLoadBalancer roundRobinContextLoadBalancer(LoadBalancerClientFactory clientFactory, Environment env) {
  String serviceId = clientFactory.getName(env);
  return new RoundRobinLoadBalancer(serviceId, clientFactory
   .getLazyProvider(serviceId, ServiceInstanceSupplier.class), -1);
 }

}

Finally, here's the implementation of caller-service controller. It uses LoadBalancerClientFactory directly to find list of available instances of callme-service. Then it selects a single instance, get its host and port, and sets in as an target URL.

@RestController
@RequestMapping("/caller")
public class CallerController {

 @Autowired
 Environment environment;
 @Autowired
 RestTemplate template;
 @Autowired
 LoadBalancerClientFactory clientFactory;

 @GetMapping
 public String call() {
  RoundRobinLoadBalancer lb = clientFactory.getInstance("callme-service", RoundRobinLoadBalancer.class);
  ServiceInstance instance = lb.choose().block().getServer();
  String url = "http://" + instance.getHost() + ":" + instance.getPort() + "/callme";
  String callmeResponse = template.getForObject(url, String.class);
  return "I'm Caller running on port " + environment.getProperty("local.server.port") +
   " calling-> " + callmeResponse;
 }

}

Summary

The following picture illustrates the architecture of the sample system. We have two instances of callme-service, a single instance of caller-service, which uses Spring Cloud Balancer to find the list of available instances of callme-service. The ports are generated dynamically. The API gateway is hiding the complexity of our system from external client. It is available on port 8080, and is forwarding requests to the downstream basing on request context path.

After starting, all the microservices you should be registered on your Consul node.

Now, you can try to endpoint exposed by caller-service through gateway: http://localhost:8080/caller. You should something like this:

The sample application source code is available on the GitHub in repository.