Spring Cloud Config Server Without RabbitMQ/Kafka

Background

Before we dive into the practical challenges while implementing Spring Cloud Config Server (SCCS), let's understand the high level of SCCS, what it is and what is required to set it up.

Spring Cloud Config Server

SCCS provides an HTTP resource-based API for external configuration (name-value pairs or equivalent YAML content). The server is embeddable in a Spring Boot application, by using the @EnableConfigServer annotation.

PCF does provide this Config Server as a marketplace Tile which is known as "Spring Cloud Service." It just needs the Git Repository as input where the configuration files reside.

More details of Spring Cloud Config Server tile is available in this link.

Spring Cloud Config Client

Spring Cloud Config includes the Config Client library for Spring applications to use to connect to a Spring Cloud Config configuration server and consume configuration values. When the Config Client starter (spring-cloud-starter-config) dependency is included, the application becomes a client of a Spring Cloud Config configuration server.

It just needs to bind the Client application with Spring Cloud config server service instance in PCF.

This whole setup will ensure if a configuration file is hosted in Git Repo, it will be loaded as environment variables of the Cloud Config Client application. Now, if any change is done in the Git Configuration file, it needs to be refreshed in the Client application.

Spring Boot Actuator adds a refresh endpoint to the application. This endpoint is mapped to /refresh, and a POST request to the refresh endpoint refreshes any beans which are annotated with @RefreshScope. You can thus use @RefreshScope to refresh properties which were initialized with values provided by the Config Server.

Issues and Solution Approaches

Issue 1:

This refresh mechanism works well if there is only one instance of the client application. However, if there are multiple instances of the application in PCF, hitting router url with /refresh endpoint doesn't give guarantee that it will hit all the instances and refresh them even if you hit them concurrently or parallel.

Solution:

To resolve this issue, Spring does provide a spring-cloud-starter-bus-amqp jar. This Jar internally uses RabbitMQ as a publish-subscribe message queue. When the /refresh endpoint is hit, it will put the message in MQ and that message will be read by other instances of the app which are subscribers of this queue and hit /refresh endpoint. This task is done by the jar put as a dependency.

Issue 2:

The above solution has a dependency on RabbitMQ. But what if RabbitMQ is not available on the PCF instance which you use? (RabbitMQ commercial support is not free.) 

Solution:

There is another streaming platform which works well with Spring cloud bus, Kafka. Adding spring-cloud-starter-bus-kafka as a dependency will connect to the Kafka and do the same publish-subscribe with other instances of the app.

Issue 3:

What if both RabbitMQ and Kafka are not available on PCF instance you are working on. Is it going to block you to use Spring Cloud Config Server?

Solution:

There is a manual process of refreshing each instance of the application running on PCF, in which you will need to grab the GUID of the application and then hit the/refresh endpoint for all the instances one by one.

 Step 1:  Login to the CF org/space.

 Step 2: run command cf app <app name> --guid    This will retrieve the guid of the application.

 Step 3: run command cf app <app name> This will retrieve number of the instances running for the application.

 Step 4: run command to refresh the each instance of the app : curl -X POST https://<app router url>/refresh -H "X-CF-APP-INSTANCE":"App guid:Instance Index"

    e.g. curl -X POST https://config-client.apps.pcf.com/refresh -H "X-CF-APP-INSTANCE":"3b23gs23-6fc9-32b9-824e-180ce43ga57a:0"

         curl -X POST https://config-client.apps.pcf.com/refresh -H "X-CF-APP-INSTANCE":"3b23gs23-6fc9-32b9-824e-180ce43ga57a:1"

This manual process can be automated and run as a shell script which will avoid manual intervention and avoid disruption.

The best approach is to have a CI/CD pipeline which will trigger the shell script and do all 4 steps. I have created a Bamboo pipeline which does this job.

The following inputs will be required for the application to run the script:

•  app_name  — This is PCF app name. It is mentioned in Step 3 screenshot.
•  app_url  — This is the route url of the application. It is mentioned in Step 3 screenshot.
•  org_name  — PCF org name.
•  space_name  — PCF org space name.
•  pcf_username  — user to login to your Org space.
•  pcf_password  — password of the user.

These are defined as Plan Variables. Input them as per your application and the pipeline can refresh all the instances of the app for you.

If you need the source code for the script, find it in my Github repository. 

That's all for this blog. Please put comments if you find any other alternative way to do this.