Deploying Spinnaker on Docker Desktop

After preparation for leading an internal workshop on Spinnaker, I decided to share the tutorial for getting Spinnaker up and running on OSX using Docker Desktop:

• Halyard installation: locally on OSX
• Cloud Provider: Kubernetes (Manifest-based)
• Environment: Distributed installation on Kubernetes
• Persistent storage: S3 via Minio deployed on Kubernetes
• Deploy Spinnaker: 1.11.9 (Cobra Kai)

Install Halyard

Follow the steps below after opening Terminal.App.

# Download the installer script 
$ curl -O https://raw.githubusercontent.com/spinnaker/halyard/master/install/macos/InstallHalyard.sh

# Run the installer script
$ sudo bash ./InstallHalyard.sh -y --user $USER

# Check whether Halyard was installed properly
$ hal -v

Cloud Provider Kubernetes V2

The Spinnaker Kubernetes V2 provider fully supports manifest-based deployments, is actively maintained, and aligns with the Kubernetes deployment model.

Prepare Docker for Desktop

• Assure Docker for Desktop is installed
• Allocate the following resources via Preferences, and restart:
  • 4Gb of memory
  • 2Gb of swap space
  • 4 CPU cores
• Enable Kubernetes via Preferences. and wait until the service is active.

Next, install CLIs, target the context, and deploy helm into the local cluster

# Install kubectl on your OSX host 
$ brew install kubectl

# Assure we have targeted context 
$ kubectl config use-context docker-for-desktop

# Install helm on your OSX host and enable in Kubernetes cluster 
$ brew install kubernetes-helm 
$ helm init --wait

Configure Kubernetes Account and Options

Enable the Kubernetes provider in Halyard, and configure the account.

# Make sure that the provider is enabled 
$ hal config provider kubernetes enable

# Set context and account name 
$ CONTEXT=$(kubectl config current-context) 
$ ACCOUNT=my-k8s-v2-account

# Add account 
$ hal config provider kubernetes account add $ACCOUNT \ 
   --provider-version v2 \ 
   --context $CONTEXT

# Set config option 
$ hal config features edit --artifacts true

Distributed Installation on Kubernetes

Spinnaker is deployed to the local Kubernetes cluster, with each microservice deployed independently. Halyard creates a smaller, headless Spinnaker to update your Spinnaker and its microservices, ensuring zero downtime updates.

# Set the distributed environment option
$ hal config deploy edit --type distributed --account-name $ACCOUNT

External Storage

Spinnaker requires an external storage provider for persisting your application settings and configured pipelines — we will deploy Minio S3 in a separate context in Kubernetes.

Deploy Minio to Kubernetes

Minio is an S3-compatible object store that you can host yourself. This is the persistent storage solution I recommend when you don’t want to depend on a cloud provider to host your Spinnaker data.

# Using some randomly generated access and secret keys
$ MINIO_ACCESS_KEY=VHK5D359RAXHEXAMPLE
$ MINIO_SECRET_KEY=vE8qPsiWjluG2mZRD1WHZANzSYW8X8C3BExAmPLe

# Deploy Minio to the (default) namespace
$ helm install \
   --set persistence.size=8Gi \
   --set accessKey=$MINIO_ACCESS_KEY,secretKey=$MINIO_SECRET_KEY \
   --name my-s3-storage \
   --namespace storage \
   stable/minio

# Wait until the pods are in running state and READY 1/1 
$ kubectl get pods --namespace=storage -w

# To access Minio from your OSX host, run the commands below 
$ export POD_NAME=$(kubectl get pods --namespace storage \
   -l "release=my-s3-storage" \
   -o jsonpath="{.items[0].metadata.name}")

# Start the proxy in the background and keep running! 
$ nohup kubectl port-forward $POD_NAME 9001:9000 --namespace storage &

You can now reach the Minio Console via http://localhost:9001; set the following environment variable to continue the manual:

# Define the endpoint we like to use 
$ ENDPOINT=http://my-s3-storage.storage.svc.cluster.local:9000

Configure Halyard to Use Minio S3

Given that Minio doesn’t support versioning objects, we need to disable it in Spinnaker.

 # Set deployment name 
 $ DEPLOYMENT=default

# Create the directory (does not exist yet) 
$ mkdir -p ~/.hal/$DEPLOYMENT/profiles

# Set the option to ignore S3 versioning 
$ echo "spinnaker.s3.versioning: false" > \ 
   ~/.hal/$DEPLOYMENT/profiles/front50-local.yml 

Run the following commands (notice we are picking S3 as our storage type because Minio implements the S3 API):

# Configure S3 storage endpoint and creds for Halyard 
$ echo $MINIO_SECRET_KEY | hal config storage s3 edit \ 
   --endpoint $ENDPOINT \ 
   --access-key-id $MINIO_ACCESS_KEY \ 
   --secret-access-key # will be read on STDIN

# Enable S3 storage for Halyard 
$ hal config storage edit --type s3

Deploy Spinnaker

Now that we’ve enabled the Kubernetes (Docker for Desktop) cloud provider, picked the distributed deployment environment, and configured Minio S3 persistent storage, we’re ready to pick a version of Spinnaker, deploy it, and connect to it.

# Set Spinnaker version 
$ VERSION=1.11.9 
$ hal config version edit --version $VERSION

# Deploy 
$ hal deploy apply

# Wait until the pods are started 
$ kubectl get pods --namespace=spinnaker -w

# This command automatically forwards 
# ports 9000 (Deck UI) and 8084 (Gate API service)
$ hal deploy connect

You can now reach the Deck UI via http://localhost:9000.

Next steps

You can start working on deployment pipeline development now on your local machine.

Cleanup

Once you have finished playing around with Spinnaker, delete the created Spinnaker deployments and Minio dependencies.

# Purge any Spinnaker deployments first... 
$ hal deploy clean

# Wait until the pods are terminating 0/1 READY 
$ kubectl get pods --namespace=spinnaker -w

# Delete namespace Spinnaker 
$ kubectl delete namespace spinnaker

# Stop the Minio console proxy 
$ kill $(ps -ef | grep port-forward | grep my-s3-storage | awk '{ print $2 }')

# Delete Minio 
$ kubectl delete deployment my-s3-storage --namespace storage 
$ helm delete --purge my-s3-storage

# Wait until the pods are terminating 0/1 READY 
$ kubectl get pods --namespace=storage -w

# Delete namespace Storage 
$ kubectl delete namespace storage

If you also want to de-install Halyard:

# Kill the halyard process 
$ kill $(ps -ef | grep com.netflix.spinnaker.halyard.Main | awk '{ print $2 }')

# Now you can safely uninstall Halyard: 
$ sudo ~/.hal/uninstall.sh

Let us know your thoughts and any questions you might have by leaving a comment below.