Overriding Docker ENTRYPOINT of a Base Image

Recently my DevOps team and I decided to bring all the dev tools of all engineering teams (backend, frontend, mobile, operations) in a Kubernetes cluster. We have many reasons to do so. One of them is that we need to have centralized management of all those tools that are configured and upgraded manually sometimes by our IT department and some other times by our DevOps department. Another important reason is that we need to scale Jenkins jobs, especially when many releases or automated procedures occur by many different teams. Last, but not least, is that we need a playground with real-life issues and problems to cope with before we use a K8s cluster in production.

There are several things that we need to do before moving our apps to Kubernetes, like Dockerizing components, applications, etc.

One of the last experiments was to migrate our HornetQ servers to ActiveMQ ones. You may find more details here. In order to move ActiveMQ servers to Kubernetes, we would like to create a custom Dockerfile with specific configuration files. Unfortunately, we do not have one set of configuration files but many. Therefore, we should invent a way to copy configuration files conditionally based on Docker runtime environment variables.

To be more precise, let’s see a basic Dockerfile that would copy the configuration files we need into some specific folders in the Docker image:

FROM vromero/activemq-artemis:latest

MAINTAINER Marios Karagiannopoulos <mkaragiannopoulos@zulutrade.com>

ENV MODE_PARAMS_FOLDER /var/lib/artemis/

RUN mkdir -p ${MODE_PARAMS_FOLDER}
ADD int ${MODE_PARAMS_FOLDER}/etc-override-int
ADD ext ${MODE_PARAMS_FOLDER}/etc-override-ext

We could build an image with etc-override-int configuration and another one with etc-override-ext. But why waste disk space with too many images? There is no reason to do it since we can create one image and select our configuration based on a runtime environment variable.

Problem

We have a problem here. The base image: “vromero/activemq-artemis:latest” has an ENTRYPOINT  that does not take into account our configuration files at all.

One idea is to override ENTRYPOINT  by adding your CMD  command following by an ENTRYPOINT  ["/usr/bin/env"] like:

# trick to override base image's ENTRYPOINT
ENTRYPOINT ["/usr/bin/env"]
CMD ["bash", "/sed_broker_files.sh /var/lib/artemis/etc/broker-05.xml"]

Nevertheless, if you want to use container’s runtime environment variables this is not going to work.

So, what I’ve been thinking of is to look at how the ENTRYPOINT of a base image is written and try to inject other scripts or modifying script code into it during the Docker image build time. The base image ENTRYPOINT script is located here. If you see carefully the variable we’re interested of is:

OVERRIDE_PATH=$BROKER_HOME/etc-override

Based on the selected configuration in runtime of the container, it should be etc-override-int  or  etc-override-ext .

A sed command could do the job during the build time of the Docker image. However, we also need to run some other commands before the startup of the application. We may put these commands in another script and inject the execution of this script inside the ENTRYPOINT  of the base image. Let’s see the final Dockerfile:

FROM vromero/activemq-artemis:latest

MAINTAINER Marios Karagiannopoulos <mkaragiannopoulos@zulutrade.com>

# trick to override artemis user when entering the container
USER root

ENV MODE_PARAMS_FOLDER /var/lib/artemis/

RUN mkdir -p ${MODE_PARAMS_FOLDER}
ADD int ${MODE_PARAMS_FOLDER}/etc-override-int
ADD ext ${MODE_PARAMS_FOLDER}/etc-override-ext

# trick to override base image's ENTRYPOINT
COPY activate_mode_param.sh /
RUN head -2 /docker-entrypoint.sh > /docker-entrypoint.sh.tmp
RUN echo "/activate_mode_param.sh" >> /docker-entrypoint.sh.tmp
RUN all_lines=`wc -l /docker-entrypoint.sh | cut -d' ' -f1` && \
  new_lines=`expr $all_lines - 3` && \
  tail -$new_lines /docker-entrypoint.sh >> /docker-entrypoint.sh.tmp && \
  mv /docker-entrypoint.sh.tmp /docker-entrypoint.sh
RUN sed -i "s/etc-override/etc-override-\${MODE_PARAM}/g" /docker-entrypoint.sh

RUN chmod 777 /docker-entrypoint.sh
RUN chown -R artemis:artemis ${MODE_PARAMS_FOLDER}

USER artemis

As you can see, we copy the activate_mode_param.sh   script under root folder and then we inject its call inside /docker-entrypoint.sh . Also, we change the etc-override presence to etc-override-${MODE_PARAM} where $MODE_PARAM  is a runtime environment. Take a look here:

docker build -t 10.0.8.171:5000/amq:latest -f Dockerfile .

docker run -it --name='amq-master-int-206' \
-v /opt/amq/sharedstore:/var/lib/artemis/data \
-e 'MODE_PARAM=int' \
-e 'AMQ_MASTER_IP=10.0.9.206' \
-e 'AMQ_MASTER_PORT=61616' \
-e 'AMQ_SLAVE_IP=10.0.9.206' \
-e 'AMQ_SLAVE_PORT=61617' \
-e 'ARTEMIS_PERF_JOURNAL=ALWAYS' \
-e 'ARTEMIS_USERNAME=admin' \
-e 'ARTEMIS_PASSWORD=admin' \
-e 'ARTEMIS_MIN_MEMORY=512M' \
-e 'ARTEMIS_MAX_MEMORY=1024M' \
-e 'ENABLE_JMX=true' \
-e 'JAVA_OPTS=-Dorg.apache.activemq.SERIALIZABLE_PACKAGES=*' \
-p 8161:8161 \
-p 61616:61616 \
-d 10.0.8.171:5000/amq:latest

Deploying to Kubernetes

After building the image with the command above, we need to push it to our local registry:

docker login 10.0.8.171:5000 --username zulu --password ********
docker push 10.0.8.171:5000/amq:latest

alias k='kubectl'
k create namespace jms
k -n jms apply -f regced.yaml
k -n jms apply -f deployment-amq-master-int-206.yaml
k -n jms apply -f svc-amq-master-int-206.yaml

With contents:

regced.yaml (secret file to read images from our private registry)

apiVersion: v1
kind: Secret
metadata:
  namespace: jms
  name: regcred
data:
  .dockerconfigjson: "HIDDEN_HASH"
type: kubernetes.io/dockerconfigjson

deployment-amq-master-int-206.yaml 

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: amq-master-int-206
  namespace: jms
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: amq-master-int-206
    spec:
      imagePullSecrets:
      - name: regcred
      containers:
      - name: amq-master-int-206
        image: 10.0.8.171:5000/amq:latest
        ports:
        - name: http
          containerPort: 8161
        - name: jnp
          containerPort: 61616
        env:
        - name: MODE_PARAM
          value: "int"
        - name: AMQ_MASTER_IP
          value: "10.0.8.170"
        - name: AMQ_MASTER_PORT
          value: "6116"
        - name: AMQ_SLAVE_IP
          value: "10.0.8.170"
        - name: AMQ_SLAVE_PORT
          value: "6117"
        - name: ARTEMIS_PERF_JOURNAL
          value: always
        - name: ARTEMIS_USERNAME
          value: "admin"
        - name: ARTEMIS_PASSWORD
          value: "admin"
        - name: ARTEMIS_MIN_MEMORY
          value: "512M"
        - name: ARTEMIS_MAX_MEMORY
          value: "1024M"
        - name: ENABLE_JMX
          value: "true"
        - name: JAVA_OPTS
          value: "-Dorg.apache.activemq.SERIALIZABLE_PACKAGES=*"
        volumeMounts:
          - name: nfs-amq
            mountPath: /var/lib/artemis/data
      volumes:
      - name: nfs-amq
        nfs:
          server: 10.0.8.64
          path: /volume1/Storage/YB/k8s/amq206

svc-amq-master-int-206.yaml 

apiVersion: v1
kind: Service
metadata:
  namespace: jms
  name: amq-master-int-206
spec:
  type: NodePort
  ports:
    - port: 8161
      name: http
      targetPort: 8161
      nodePort: 8161
    - port: 6116
      name: jnp
      targetPort: 61616
      nodePort: 6116
  selector:
    k8s-app: amq-master-int-206