Configuring Java Apps With Kubernetes ConfigMaps and Helm

In this article, we’ll build Java microservices that use ConfigMaps, see how the ConfigMaps are used, change the config and roll out the updated config to the services. Kubernetes is all about orchestration so we’ll build several services and make config changes that apply across multiple services. You can code along with the article or clone the project from GitHub.

You may also enjoy Linode's Beginner's Guide to Kubernetes.

Pre-Requisites

We will need to containerize our microservices to run them in Kubernetes—we’ll use Docker (v18.03.0-ce) for this. We’ll use Minikube(v0.24.1) to sandbox with Kubernetes locally. Later (optionally) we’ll also use Helm (v2.8.2).

Building a Service

We want each service to clearly identify itself and what configuration it is in. For this we’ll build Spring Boot web applications that respond to HTTP requests, generating an initial project from the Spring Initializr with Web and Actuator (for health checks) dependencies:

We start by building Optimus Prime - we want services that we can reconfigure and clearly see when we’ve reconfigured them, so Transformers are a good illustration (using characters from writeups.org). The idea is to be able to respond to a web request with a page like:

We can be ready to build multiple Transformers by putting the optimus-prime directory from the Initializr inside the directory we’ll use for the Transformers project (I’ve called the parent directory ‘configmaps-transfomers’). Then import into your IDE (either the parent directory or just optimus for now).

To code up Optimus Prime, we need a Controller that can handle HTTP requests:

@RestController
public class Controller {

   private final String name="Optimus Prime";
   private final String allegiance="Autobot";
   @Value("${transformer.mode:disguised}")
   private String mode;

   @Value("${transformer.disguised:TRUCK}")
   private String disguised;

   @Value("${transformer.robot:ROBOT}")
   private String robot;

   @GetMapping
   public String transformer(){
       StringBuilder stringBuilder = new StringBuilder();
       stringBuilder.append("<h1>");
       stringBuilder.append("Name: ").append(name).append("<br/>");
       stringBuilder.append("Allegiance: ").append(allegiance).append("<br/>");
       stringBuilder.append("Mode: ").append(mode).append("<br/>");
       stringBuilder.append("</h1>");

       if("robot".equalsIgnoreCase(mode)){
           stringBuilder.append(robot);
       } else if("disguised".equalsIgnoreCase(mode)){
           stringBuilder.append(disguised);
       }

       return stringBuilder.toString();
   }
}

We hard-code the name and allegiance of Optimus as these can’t change. We get the mode from a configurable property (‘transformer.mode’) using Spring Boot’s @Value annotation and we default the value to ‘disguised’. We also use @Value to find out what ‘disguised’ and ‘robot’ modes represent for Optimus - we default these values to ‘TRUCK’ and ‘ROBOT’ but we can override with ASCII art from a properties file. We can make the two modes more visual by embedding ASCII art instead of ‘TRUCK’ and ‘ROBOT’. To do this we can use images from writesup.org and put them through ascii-art-generator using the HTML option and embed the resulting HTML DIVs in the application.properties file as the values for transformer.disguised and transformer.robot. 

We respond to HTTP requests using GetMapping and output everything we need to see which Transformer this is, what mode it is in and what the mode translates to for this Transformer. We wrap it all in an <h1> to make sure the text output is large enough.

We want to be able to build this project as a docker image. To do this we can modify the pom.xml file and add an extra plugin entry to the plugins section under build:

 <plugin>
  <groupId>io.fabric8</groupId>
  <artifactId>docker-maven-plugin</artifactId>
  <configuration>
   <images>
    <image>
     <name>transformers/${project.artifactId}</name>
     <alias>${project.artifactId}</alias>
     <build>
      <from>openjdk:alpine</from>
      <assembly>
       <descriptorRef>artifact</descriptorRef>
      </assembly>
      <cmd>java ${JAVA_OPTS} -jar maven/${project.artifactId}-${project.version}.jar</cmd>
     </build>
    </image>
   </images>
  </configuration>
  <executions>
   <execution>
    <goals>
     <goal>build</goal>
    </goals>
   </execution>
  </executions>
 </plugin>

Now we can build an image using ‘mvn clean install’ from the optimus-prime directory. And then run it using:

docker run -it -p 8080:8080 transformers/optimus-prime

Or for the robot mode, use:

docker run -it -p 8080:8080 -e TRANSFORMER_MODE=robot transformers/optimus-prime

Adding More Services

Now let’s create more services. First, let’s create a parent-child maven module structure so that optimus-prime is a child project from the parent. In the parent directory we create this pom.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <groupId>org.transformers</groupId>
 <artifactId>transformers-parent</artifactId>
 <version>0.0.1-SNAPSHOT</version>
 <packaging>pom</packaging>

 <parent>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-parent</artifactId>
   <version>2.0.0.RELEASE</version>
   <relativePath/> <!-- lookup parent from repository -->
 </parent>

 <name>Transformers</name>
 <description>ConfigMaps Intro</description>
 <modules>
   <module>optimus-prime</module>
   <module>megatron</module>
   <module>gears</module>
   <module>shockwave</module>
 </modules>
</project>

The modules section says that our optimus-prime project is a child. The parent section has been copied over from the optimus-prime project. This is inheriting a standard Spring Boot project configuration. Now the optimus-prime module can inherit that configuration from this new parent. So let’s go to the pom.xml in the optimus-prime module and replace the parent section with:

<parent>
<groupId>org.transformers</groupId>
<artifactId>transformers-parent</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>

Now if we do ‘mvn clean install’ from the parent directory we should see the project build and ‘docker images’ should show that the transformers/optimus-prime image has been built again.

Building the parent also builds the children so if we add more modules under this parent then we can build images for all of them with one command. To add more services we can just copy-paste the optimus-prime directory and change all of the optimus-specific references to whichever transformer we are adding and put an extra module entry in the modules section of the parent pom.xml. In the GitHub there’s Optimus Prime, Gears, Megatron and Shockwave.

Configuring Services With ConfigMaps

Now we have a maven project to build docker images. Next to deploy them. To get a sense of what Kubernetes ConfigMaps can do for us we’ll split the Autobots and Decepticons into separate deployments, each with their own config. First, let’s create an ‘autobots’ directory and create an autobots-config.yml file within it. It should contain:

apiVersion: v1
kind: ConfigMap
metadata:
 name: autobots-config
 namespace: default
data:
 transformer.mode: robot

So the autobots will be in ‘robot’ mode. Now let’s create an ‘optimus-prime.yml’ file, which should contain:

---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
 name: optimus-prime
 labels:
   serviceType: optimus-prime
spec:
 replicas: 2
 strategy:
   type: RollingUpdate
   rollingUpdate:
     maxUnavailable: 1
     maxSurge: 1
 template:
   metadata:
     name: optimus-prime
     labels:
       serviceType: optimus-prime
   spec:
     containers:
       - name: optimus-prime
         image: transformers/optimus-prime:latest
         imagePullPolicy: Never
         ports:
           - name: http
             containerPort: 8080
             protocol: TCP
         env:
         - name: JAVA_OPTS
           value: -Xmx64m -Xms64m
         - name: TRANSFORMER_MODE
           valueFrom:
             configMapKeyRef:
               name: autobots-config
               key: transformer.mode
         livenessProbe:
           httpGet:
             path: /actuator/health
             port: http
           initialDelaySeconds: 40
           timeoutSeconds: 1
           periodSeconds: 15
         readinessProbe:
           httpGet:
             path: /actuator/health
             port: http
           initialDelaySeconds: 40
           timeoutSeconds: 1
           periodSeconds: 15
---
apiVersion: v1
kind: Service
metadata:
 name: optimus-prime-entrypoint
 namespace: default
spec:
 selector:
   serviceType: optimus-prime
 ports:
   - port: 8080
     targetPort: 8080
     nodePort: 30080
 type: NodePort

The role of the ‘Deployment’ here is to create Pods that run containers built using the transformers/optimus-prime:latest image. These Pods are labeled as ‘optimus-prime’ Pods so that the Service can pick them up. The Service will be exposed to the world outside Kubernetes—with Minikube a request to port 30080 will go to the Service. (This bit of the Service would be a little different for real Kubernetes as we’d use LoadBalancer rather than NodePort and wouldn’t be restricted to the minikube port range.) The Service will handle it using Pods that are matched to the Service by label. The Deployment will create two 2 replicas and during an update, it will take one down, bring a new one up and then take the other down and replace it.

The transformer.mode property of our Spring Boot application is set using the TRANSFORMER_MODE environment variable. That is looked up from the config map called ‘autobots-config’ and we use the entry in that map called ‘transformer.mode’.

We can create further autobots by copy-pasting the optimus-prime.yml file and replacing all references to ‘optimus-prime’ with the transformer name and incrementing the nodePort value by 1. (We can later avoid this copy-pasting using Helm but for now we’ll copy-paste.) In GitHub, the example has Optimus Prime and Gears.

Once we’re done creating autobots, we can create a decepticons folder and create a decepticons-config similar to the autobots. We’ll set them to disguised so we can see the config is different. In GitHub, the example has Megatron and Shockwave.

Now, we start minikube: 

minikube start --memory 4000 --cpus 3 

Then, build images for Transformers - from the top-level directory (I called this ‘configmaps-transfomers’) with:

eval $(minikube docker-env) 
mvn clean install 

Deploy the Autobots (their ConfigMap has them as in robot mode):

kubectl create -f autobots --save-config

And look at them with: 

minikube service optimus-prime-entrypoint 
minikube service gears-entrypoint  

To deploy the Decepticons (their ConfigMap has them disguised):

kubectl create -f decepticons --save-config

And to see the first two decepticons do: 

minikube service megatron-entrypoint 
minikube service shockwave-entrypoint 

We now see this configuration (Autobots at the top, Decepticons at the bottom):

Changing ConfigMaps

We can see the ConfigMaps by doing ‘minikube dashboard’ and navigating to the ‘Config Maps’ section under ‘Config and Storage’. We can also edit them in place through this dashboard:

But if we change one of the values (e.g. change the Decepticons from disguised to robot) and refresh the affected service in the browser, we don’t see any change - at least in Kubernetes v1.9. (This area is under discussion for future versions - it is possible to do live loading when accessing the ConfigMap as a file but we’ll load as environment variables here.) 

The reason for this is that the Pod is set with the config values when it is first created. There are ways to make the app look for changes but here we’re going to try to do the updating through Kubernetes. We could recreate the Pods for a deployment by scaling it down to zero instances (replicas) and back up to 1. For example, we could (but won’t):

kubectl scale deployment/megatron --replicas=0; 
kubectl scale deployment/megatron --replicas=2;

This would be minimal downtime and we should see the update pretty quickly (note that a new browser window might be required due to browser caching - or use a private browsing mode). But it would be nice to have a true RollingUpdate.

In this case, we can’t initiate a rolling update by simply doing a ‘kubectl apply decepticons’ because we haven’t actually changed any of the Deployments so Kubernetes won’t treat them as changed. So what we can do is create a new version of the ConfigMaps and repoint the Deployments at those.

To do this we can create new ConfigMaps with ‘-v2’ on the end of their files and names and the modes of the transformers reversed. Then we update the references in the .yml files for each of the Transformers in the ‘autobots’ and ‘decepticons’ directories by replacing ‘autobots-config’ with ‘autobots-config-v2’ and ‘decepticons-config’ with ‘decepticons-config-v2’. Then do:

kubectl apply -f autobots --record
kubectl apply -f decepticons --record

Now Kubernetes will do a rolling upgrade and we see the roles reversed - the Autobots are disguised and the Decepticons are robots:

(We can see both of the rollouts for a Transformer with e.g. ‘kubectl rollout history deployment optimus-prime’)

But what if we want a rolling update and don’t want to create a new ConfigMap? This is a little more advanced. For this, we’ll take a helm-based approach which will both solve this problem and give us a more flexible deployment method.

Creating a Helm Chart

First, we’ll have to have Helm installed in order to create a Helm chart. Because all of the transformers are quite similar, we can create one chart to cover all of them and parameterize the differences.

We start by creating a directory in our project named ‘helm’ in which we run:

helm create transformer

This creates an initial Chart called ‘transformer’ following the default structure. We’ll now modify this Chart to be a Transformer Chart that defaults to optimus-prime but can be overridden to deploy any Transformer. If you’re unfamiliar with Helm charts then think of them as parameterized templates for Kubernetes deployment descriptors (have a look at the examples in the Helm docs as a primer).

The modification steps are:

1. Change description and name in Chart.yml to say this is a transformer.

2. The values.yml specifies default values passed into a Chart in each deploy. Change the entries for image.repository and image.tag in values.ymlto point us by default to transformer/optimus-prime:latest.

3. Add transformer.name (optimus-prime) and transformer.mode in values.yml 

4. Change defaults in values.yml for serviceType to nodePort and port to 30080 as we intend to use minikube (but note we could if we wanted still override these at deploy time with parameters)

5. Set the replicaCount in the values.yml to two so that they can be switched between during a rolling update

6. In deployment.yml change containerPort to 8080 as our spring boot apps run on 8080

7. Add a strategy section to deployment.yml configure that a RollingUpdate should be done progressively.

8. Configure the probes that check the Pod health to hit the actuator health check.

9. We need to specify some environment variables in the deployment so that each transformer Pod knows which mode to run in - disguised/robot(we also do some config to limit Java memory consumption).

This has everything we need to be able to deploy all four transformers and change their config without downtime. There’s a branch from which to run it in GitHub.

But it doesn’t strictly point to any shared ConfigMaps as it uses an environment variable instead. We could have the Transformer chart create a ConfigMap but don’t want to deploy a ConfigMap for every Transformer as we want Gears and Optimus Prime to share an Autobots ConfigMap, not to have one each. We could change the chart to point to a ConfigMap with the name parameterized and either fail or fall back to env vars if the parameter isn’t supplied.  But we might rather have a deployment process that always creates what it needs. So what we want is to take a step towards further parameterization and have a helm chart for the transformers that includes each transformer as a subchart.

Helm Chart With Subcharts and ConfigMaps

We create a new ‘transformers’ helm chart in the helm directory for which ‘transformer’ will be a subchart. We create a starter chart using:

helm create transformers

Again we change the description in the Chart.yaml - this time to say the chart is for Transformers.

The new parent chart won’t have a Deployment or Service - it’ll leave those to the subcharts to deal with. So under transformers/templates,we delete everything except _helpers.tpl.

Then we can cut and paste the transformer chart (whole dir) into the transformers/charts directory. The transformers parent chart will have the transformer child chart as a dependency. To do this we create a requirements.yaml file for the transformers parent chart and first add:

 - name: transformer
   version: 0.1.0
   alias: optimusprime
   condition: optimusprime.enabled
   tags:
     - autobots

This includes the transformer subchart within the transformers parent chart using the alias ‘optimusprime’. It also defines that we can exclude this subchart in a deployment by setting an optimusprime.enabled parameter or an autobots parameter to false (if both set optimusprime.enabled will override autobots as it’s more specific). We repeat this once for each transformer.

Next, we create configmaps for the autobots and decepticons. This is the one for the autobots:

{{ if (.Values.tags.autobots) and eq .Values.tags.autobots "true" }}
apiVersion: v1
kind: ConfigMap
metadata:
 name: {{ template "transformers.fullname" . }}-autobots-config
 labels:
   heritage: {{ .Release.Service }}
   release: {{ .Release.Name }}
   chart: {{ .Chart.Name }}-{{ .Chart.Version }}
   app: {{ template "transformers.name" . }}
data:
 transformer.mode: {{ .Values.autobots.mode }}
{{ end }}

It’s wrapped in an if block so that it’s only included if the autobots tag is true. The name in the metadata section is the name of the configmap. This name will be partially-dynamic. The ‘-autobots-config’ part will be static but the prefix on this will be the release name assigned when we perform a deploy with Helm. To accommodate this we have to change the deployment of the subchart to point to this ConfigMap by using:

- name: TRANSFORMER_MODE
 valueFrom:
   configMapKeyRef:
     name: "{{ .Release.Name }}-{{ .Values.transformer.allegiance }}-config"
     key: transformer.mode

We’re now recording the allegiance of each transformer so that we point to the correct ConfigMap. We can default this for optimus in the values.yaml of the subChart. We’ll also set it in the parent chart so for the parent chart we need to create a values.yaml and add:

optimusprime:
  image:
    repository: transformers/optimus-prime
  service:
    port: 30080
  transformer:
    allegiance: autobots
    name: optimusprime

This sets the defaults for optimus. We repeat for the other transformers, incrementing ports as we go.

We also add:

tags:
 autobots: true
 decepticons: true

This says that we do want both autobots and decepticons included by default. Then we set:

autobots:
  mode: robot
decepticons:
  mode: disguised

These values are used in the ConfigMaps and will default the autobots to their robot form and the decepticons to their disguised form.

Now, from the Helm directory, we can deploy with:

helm install --name=transformers1 ./transformers/ 

And view with: 

minikube service transformers1-optimusprime
minikube service transformers1-gears
minikube service transformers1-megatron
minikube service transformers1-shockwave

Transform with: 

helm upgrade transformers1 --recreate-pods --set autobots.mode=disguised,decepticons.mode=robot ./transformers/ 

Note this takes a little while but if we refresh the browser (preferably with private browsing - or run the four minikube service commands again) in theory we should see it reflected without downtime (though that’s not guaranteed at the time of writing due to https://github.com/kubernetes/helm/issues/1702 and https://github.com/kubernetes/kubernetes/issues/55667).

And delete with:

helm del --purge transformers1 

Notice that everything created in the install is prefixed with ‘transformers1’. If we do another install with another name (e.g. ‘transformers2’) then that will result in another set of Kubernetes objects each prefixed with ‘transformers’. So we can reconfigure (by setting the parameters), upgrade, rollback and delete ‘transformers1’ and ‘transformers2’ separately. This helm setup gives us a highly flexible way of deploying different sets of transformers in different configurations.