Running an Infinispan Cluster on Kubernetes

In this post, we look at how to deploy an Infinispan cluster on Kubernetes. Read on to find out how!

Aug. 22, 16 · Tutorial

Likes (1)

Comment

Save

4.0K Views

In the previous post, we looked how to run Infinispan on OpenShift. Today, our goal is exactly the same, but we'll focus on Kubernetes.

Running Infinispan on Kubernetes requires using proper discovery protocol. This blog post uses Kubernetes Ping, but it's also possible to use Gossip Router.

Our Goal

We'd like to build Infinispan cluster based on Kubernetes hosted locally (using Minikube). We will expose a service and route it to our local machine. Finally, we will use it to put data into the grid.

Spinning Up a Local Kubernetes Cluster

There are many ways to spin up a local Kubernetes cluster. One of my favorites is Minikube. At first, you will need the 'minikube' binary, which can be downloaded from GitHub releases page. I usually copy it into '/usr/bin', which makes it very convenient to use. The next step is to download 'kubectl' binary. I usually use Kubernetes GitHub releases page for this. The 'kubectl' binary is stored inside the release archive under 'kubernetes/platforms/<your_platform>/<your_architecture>/kubectl'. I'm using linux/amd64 because I'm running Fedora F23. I also copy the binary to '/usr/bin'.

We are ready to spin up Kubernetes:

$ minikube start                                                                                                                                                               1 ↵
Starting local Kubernetes cluster...
Kubernetes is available at https://192.168.99.100:8443.
Kubectl is now configured to use the cluster.

Deploying the Infinispan Cluster

This time, we'll focus on automation, so there will be no 'kubectl edit' commands. Below is the YAML file for creating all necessary components in Kubernetes cluster:

apiVersion: v1
items:
- apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    annotations:
    labels:
      run: infinispan-server
    name: infinispan-server
    namespace: default
  spec:
    replicas: 3
    selector:
      matchLabels:
        run: infinispan-server
    strategy:
      rollingUpdate:
        maxSurge: 1
        maxUnavailable: 1
      type: RollingUpdate
    template:
      metadata:
        creationTimestamp: null
        labels:
          run: infinispan-server
      spec:
        containers:
        - args:
          - cloud
          - -Djboss.default.jgroups.stack=kubernetes
          env:
          - name: OPENSHIFT_KUBE_PING_NAMESPACE
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: metadata.namespace
          image: jboss/infinispan-server
          imagePullPolicy: Always
          name: infinispan-server
          ports:
          - containerPort: 8080
            protocol: TCP
          - containerPort: 8181
            protocol: TCP
          - containerPort: 8888
            protocol: TCP
          - containerPort: 9990
            protocol: TCP
          - containerPort: 11211
            protocol: TCP
          - containerPort: 11222
            protocol: TCP
          resources: {}
          terminationMessagePath: /dev/termination-log
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        securityContext: {}
        terminationGracePeriodSeconds: 30
- apiVersion: v1
  kind: Service
  metadata:
    labels:
      run: infinispan-server
    name: infinispan-server
  spec:
    clusterIP: 10.0.0.218
    ports:
    - name: rest
      nodePort: 32348
      port: 8080
      protocol: TCP
      targetPort: 8080
    selector:
      run: infinispan-server
    sessionAffinity: None
    type: NodePort
  status:
    loadBalancer: {}
kind: List
metadata: {}

(lines 28-30): We added additional arguments to the bootstrap script.
(lines 31-36): We used Downward API for pass the current namespace to the Infinispan.
(lines 41-52): We defined all ports used by the Pod.
(lines 60-78): We created a service for port 8080 (the REST interface).
(line 76): We used NodePort service type which we will expose via Minikube in the next paragraph.

Save it somewhere on the disk and execute 'kubectl create' command:

$ kubectl create -f infinispan.yaml
deployment "infinispan-server" created
You have exposed your service on an external port on all nodes in your
cluster.  If you want to expose this service to the external internet, you may
need to set up firewall rules for the service port(s) (tcp:32348) to serve traffic.

See http://releases.k8s.io/release-1.3/docs/user-guide/services-firewalls.md for more details.
service "infinispan-server" created

Exposing the Service Port

One of the Minikube's limitations is that it can't use Ingress API and expose services to the outside world. Thankfully there's another way — use the Node Port service type. With this simple trick, we will be able to access the service using '<minikube_ip>:<node_port_number>'. The port number was specified in the YAML file (we could leave it blank and let Kubernetes assign random one).

The node port can easily be checked using the following command:

$ kubectl get service infinispan-server --output='jsonpath="{.spec.ports[0].NodePort}"'
"32348"%

In order to obtain the Kubernetes node IP, use the following command:

$ minikube ip
192.168.99.100

Testing the Setup

Testing is quite simple. The only thing to remember is to use the proper address — <minikube_ip>:<node_port>:

$ curl -X POST -H 'Content-type: text/plain' -d 'test' 192.168.99.100:32348/rest/default/test                                                                                  7 ↵
$ curl 192.168.99.100:32348/rest/default/test                                                
test%

Clean Up

Minikube has all-in-one command to do the clean up:

$ minikube delete
Deleting local Kubernetes cluster...
Machine deleted.

Conclusion

Kubernetes' setup is almost identical to OpenShift, but there are a couple of differences to keep in mind:

OpenShift's DeploymentConfiguration is similar Kubernetes Deployment with ReplicaSets.
OpenShift's Services work the same way as in Kubernetes.
OpenShift's Routes are similar to Kubernetes' Ingresses.

Happy scaling, and don't forget to check if Infinispan formed a cluster (hint — look into the previous post).

Related Refcard:

Kubernetes Essentials

Kubernetes cluster Infinispan

Published at DZone with permission of Sebastian Laskawiec. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

Trending