Kubernetes Tutorial: Using Secrets in Your Application

Applications deployed to a Kubernetes cluster often need access to sensitive information such as credentials to access a database and authentication tokens to make authenticated API calls to services. Kubernetes allows you to specify such sensitive information cleanly in an object called a Secret. This avoids putting sensitive data in a Pod defintion or a Docker image. In this blog, we demonstrate how you can easily hookup Kubernetes Secrets to your pod using Shippable.

Creating a Kubernetes Secret

Secrets are defined in a yml file in a Secret object. A Secret object can specifiy multiple secrets in name-value pairs. Each secret has to be base64 encoded before specifying it in the yml.

Let's define an API token as a secret for a fake token 9ggc8k9d-5jhj-4eaa-97f8-82db7845c323.

1. Base 64 encode the token.

$ echo -n "9ggc8k9d-5jhj-4eaa-97f8-82db7845c323" | base64
OWdnYzhrOWQtNWpoai00ZWFhLTk3ZjgtODJkYjc4NDVjMzIz

2. Create the secrets yml called create-secret.yml.

apiVersion: v1
kind: Secret
metadata:
  name: auth-token-secret
type: Opaque
data:
  AUTH_TOKEN_VALUE: YWRtaW4=OWdnYzhrOWQtNWpoai00ZWFhLTk3ZjgtODJkYjc4NDVjMzIz

3. Create the secret in the kubernetes cluster using kubectl.

$ kubectl create -f secrets.yml
secret "auth-token" created

Accessing a Secret in Your Application

Once a Secret is created, the next step is to access it in your application running in the container. Secrets can be mounted as data volumes or be exposed as environment variables to be used by a container in a pod. In this blog, we will mount an authentication token Secret as a data volume and consume the secret to make API calls. We will also define the secret as an environment variable for demonstartion purposes. 

Changing Secrets Dynamically in Different Environments

A typical application is deployed to multiple environments such as Dev, Test, and Prod and you would want different values for the same secret in those environments as a good security best practice. Typically, organizations will create multiple secret files for each environment and hard-code the credentials in each file. 

Shippable offers a much cleaner approach that avoids the nightmare of creating and maintaining multiple secrets yml file for an application per environment. Instead of specifying the secret directly in a secrets yml file, we will only define a placeholder value in the yml file, thereby creating a flexible, reusable template. The actual secret key-value will be defined in a key-value integration. In doing so we separate data from configuration allowing flexible data binding. Thereafter, before creating the secret, we dynamically databind the template with the key-value pair defined in the integration and then create the secret. After reading this blog, you will be able to easily adapt this powerful and simple approach to all your Kubernetes environments. 

Scenario

Our scenario is a single container Node.js application which consumes an authentication token Kubernetes secret. The authentication token is a Shippable API token. Shippable API is exposed to all paid users of Shippable and steps to create a token can be found here. The secret is volume mounted and the application consumes the secret in an exposed shipciprojects API. This API invokes Shippable API to return all repositories enabled for CI by making a HTTPS GET request on the Shippable Projects API.  

We will build the scenario using the following steps using a Shippable workflow -

• Define the container packaged in the pod. The container runs a Node.js application that can be found in the sample repository.
• Create the Shippable authentication token secret in the cluster using a key-value pair integration data bound to the secrets template that exists in our sample repository.
• Define the secret volume. 
• Create the pod and the secret volume in the Kubernetes cluster.
• Create a Kubernetes load balancer/service for the application.
• Test the secret volume mount using an exposed route in the Node.js application that makes the Shippable API call.

Deployment Workflow

This is a pictorial representation of the workflow we're going to configure. The green boxes are jobs and the grey boxes are the input resources for the jobs. The workflow is defined across two configuration files: shippable.jobs.yml and shippable.resources.yml.

Resources (grey boxes)

• dks_app_img is a required image resource that represents the docker image of your application.
• dks_gitRepo is a required gitRepo resource which is a pointer to the git repository that contains your source code and config files.
• dks_cliConfig is a required cliConfig resource which is a pointer to the private key of your service account needed to initialize the cloud CLI.
• dks_secrets is a required integration resource which is a pointer to the key-value pair integration that stores the secrets.
• dks_secret_opts is a required dockerOptions resource where we define the secrets volume mount configuration.
• dks_kube_cluster is a required cluster resource that represents the Kubernetes cluster.
• dks_lb is an optional loadBalancer resource that defines the loadbalancer properties such as labels, port, cluster etc.

Jobs (green boxes)

• dks-app-def is a required manifest job that defines all the containers that run in the pod. This definition is versioned and each version is immutable.
• create_secret is a required runSh job that uses kubectl to create the secret in the cluster using the key-value pair integration data bound to the secrets template.
• dks-app-deploy is a required deploy job which builds the Deployment spec for our application and deploys it to the Kubernetes cluster.
• dks-provision-lb is a optional provision job used to create the load balancer for the Kubernetes cluster.

Prerequisites

• An existing Kubernetes cluster where you will deploy this sample application.
• Any Supported Docker registry with a repository for your application. We have used Docker hub as the Docker registry in this sample.
• A GitHub account where you will fork and run this sample.
• Sign in with GitHub to create a Shippable account

If you're not familiar with Shippable, it is also recommended that you read the Platform overview doc to understand the overall structure of Shippable's DevOps Assembly Lines platform.blo

Sample Project

The code for this example is in a GitHub repository called devops-recipes/deploy-kubernetes-secrets. You can fork the repository to try out this sample yourself or just follow instructions to add Shippable configuration files to your existing repository.

• The Node.js application source code and Dockerfile can be found here in the repository.
• This repository also has the Shippable configuration files to create the workflow. 

1. Define the Containers in a Pod

A. Create an account integration using your Shippable account for your Docker registry.

 Instructions to create an integration can be found here. Copy the friendly name of the integration, which we have set as drship_dockerhub. 

B. Define dks_app_img

dks_app_img is an image resource that represents the docker image of your application. In our example, we're using an image hosted on Docker Hub.

Add the following yml block to your shippable.resources.yml file.

resources:

  - name: dks_app_img
    type: image
    # replace drship_dockerhub with your docker hub integration name
    integration: drship_dockerhub
    pointer:
      # replace devopsrecipes/dks_node_sample_app with your docker registry
      # repository
      sourceName: devopsrecipes/dks_node_sample_app
    seed:
      # replace latest with your image tag
      versionName: "latest"

C. Define dks-app-def

dks-app-def is a manifest job that defines all the containers that run in the pod. This definition is versioned and each version is immutable.

Add the following yml block to your shippable.jobs.yml file.

jobs:

  - name:dks-app-def
    type: manifest
    steps:
      - IN: dks_app_img
      - TASK: managed

D. Commit config files and add them to your Shippable account.

Once you have these configuration files as described above, commit them to your repository. The shippable.jobs.yml and shippable.resources.yml can be committed to the same app repository, or to a separate repository.

The repository containing your jobs and resources ymls is called a Sync repository and represents your workflow configuration.

Follow these instructions to import your configuration files into your Shippable account.

2. Create the Secret

A. Add the subscription integration for your SCM account integration

By default, you will already have an account integration with whichever SCM provider you've used to log into Shippable. If your source code repository is on that SCM account, you should use that account integration as is. 

If your source code repository, however, is on another SCM account, create an account integration for it first by using one of the following supported SCM docs:

• GitHub
• Bitbucket
• GitLab
• GitHub Enterprise

Check if you have the SCM subscription integration for your account integration. If you do not have one, add the account integration to your Subscription by following the steps here: Adding an account integration to a Subscription. 

B. Add the Key-value integration using instructions found here.

• Name the integration drship_kube_secrets. If you change the name, change it also in the yml in Step C.
• Base64 encode your Shippable API token and specify it in a key called AUTH_TOKEN_VALUE.
• Ensure you give access to the organization that your repository exists in Subscription scopes.

C. Define resources needed to create the Secret

Add the following to your shippable.resources.yml file: 

resources:

  - name: dks_secrets
    type: integration
    integration: drship_kube_secrets

  - name: dks_gitRepo
    type: gitRepo
    # replace dr_github with your GitHub integration name
    integration: dr_github
    pointer:
      # replace with source code location (e.g. GitHub) where you cloned this
      # sample project.
      sourceName: devops-recipes/deploy-kubernetes-secrets
      branch: master

  - name: dks_kube_cliConfig
    type: cliConfig
    #replace with your Kubernetes integration name
    integration: drship_kube
    pointer:
      # replace us-central1-a with your availability zone
      region: us-central1-a

D. Define create_secret

Add the create_secret job to your shippable.jobs.yml file.

It is a runSh job that lets you run any shell script. The script databinds the dks_secrets integration to the create-secret.yml secret template file, that exists in the repository in the kubernetes-secrets directory, using the shipctl replace command. 

To access the repository, we have specified dks_gitRepo as an input. Also, to automatically inject the kubernetes configuration file into the job required by kubectl, we have specified dks_kube_cliConfig as an input. 

Since it needs to run after the pod definition job in the workflow, dks-app-def is specified as an input.

jobs:

  - name: create_secret
    type: runSh
    steps:
      - IN: dks_gitRepo
        # manually trigger the job and not on every commit to the repository
        switch: off
      - IN: dks_kube_cliConfig
      - IN: dks-app-def
      - IN: dks_secrets
      - TASK:
        - script: |
            pushd $(shipctl get_resource_state "dks_gitRepo")/kubernetes-secrets
 # Replace placeholders in the secret yml with environment variables
 # injected by the key-value pair integration
              shipctl replace ./create-secret.yml
              cat ./create-secret.yml

              # Delete secret if it exists and create the secret
              kubectl delete secret shipsecret 2>/dev/null || echo "secret does not exist"
              kubectl create -f ./create-secret.yml
            popd

E. Commit config files and add them to your Shippable account.

Once you have these configuration files as described above, commit them to your repository.

3. Define the Secret Volume

dks_secret_opts is a dockerOptions resource where we define the secret volume mount configuration. 

Add the following yml block to your shippable.resources.yml file and commit the file.

resources:

  - name: dks_secret_opts
    type: dockerOptions
    version:
      volumeMounts:
        - name: secret-volume
          mountPath: "/etc/secrets"
          readOnly: true
      pod:
        volumes:
          - name: secret-volume
            secret:
              secretName: auth-token-secret

4. Create a Load Balancer for the Application

This is an optional step and the configuration required to create the load balancer can be found in this document. The sample application also has the load balancer configuration.

5. Deploy the Pod 

A. Create an account integration for Kubernetes in your Shippable UI. 

Instructions to create an integration are here:

• Add an account integration.
• Kubernetes integration

Copy the friendly name of the integration. We have set it to drship_kube and use in the next step.

B. Define dks_kube_cluster

dks_kube_cluster is a cluster resource that represents the Kubernetes cluster.

Add the following yml block to your shippable.resources.yml file.

resources:

  - name: dks_kube_cluster
    type: cluster
    #replace with your Kubernetes integration name
    integration: drship_kube
    pointer:
      # replace devops-test-cluster with your google container engine cluster name
      sourceName: "devops-test-cluster"
      # replace us-central1-a with your availability zone
      region: us-central1-a

C. Create deployment job
dks-app-deploy is a deploy job which builds the Deployment spec for our application and deploys it to the Kubernetes cluster. Since it needs to run after the secret is created in the workflow, create_secret is specified as an input.

Add the following yml block to your shippable.jobs.yml file.

jobs:

  - name: dks-app-deploy
    type: deploy
    method: replace
    steps:
      - IN: create_secret
      - IN: dks_secret_opts
      - IN: dks-app-def
        switch: off
      - IN: dks_kube_cluster

D. Commit the shippable.resources.yml and shippable.jobs.yml file to your repository. 

Your pipeline should now look like this in the SPOG view.

5. Trigger Your Pipeline

Right click on dks-app-def in the SPOG and click on Build Job. This will trigger the entire pipeline.

If you have created the load balancer configuration, you will first need to right click on your load balance job dks_provision_lb and click Build Job to create the load balancer.

Screenshot of a run of the create_secret job:

Screenshot of a run of the dks-app-deploy job:

6. Testing the Secret  Volume

Screenshot of the load balancer created in Google Cloud, since the Kubernetes cluster that we used runs in Google cloud:

Screenshot of the shipciprojects API returning the projects enabled for CI:

Try the sample above to automate your deployment pipeline for your Kubernetes application using secrets.