How to Automate Certificate Issuance to Kubernetes Deployments Using Autocert
Using TLS everywhere is one of the Kubernetes team's recommendations for hardening cluster security and increasing resilience. In this tutorial, you'll learn how to automate TLS certificate issuance to Kubernetes deployments.
Join the DZone community and get the full member experience.
Join For FreeIn public web browsing, TLS authentication happens in one direction — only servers present their certificates. Serving public web pages without authenticating the client's identity is reasonable, but it isn't for Kubernetes. If other entities are going to be accessing sensitive data on services/clusters, then it makes sense that the identity of such entities should be validated too.
Using TLS everywhere is one of the Kubernetes team's recommendations for hardening cluster security and increasing resilience.
“TLS should be enabled for every component that supports it to prevent traffic sniffing, verify the identity of the server, and (for mutual TLS) verify the identity of the client.”
In public web browsing, TLS authentication happens in one direction — only servers present their certificates. Serving public web pages without authenticating the client's identity is reasonable, but it isn't for Kubernetes. If other entities are going to be accessing sensitive data on services/clusters, then it makes sense that the identity of such entities should be validated too. This is known as mutual TLS (mTLS).
What Is Mutual TLS?
Mutual TLS (mTLS), also known as two-way authentication, is a security process in which entities authenticate each other before communication occurs. This means that in order for a connection to be established, both the client and the server must exchange, verify, and trust each other's certificates. mTLS authentication is a best practice for improving Kubernetes security and ensuring that only authenticated entities communicate with your clusters.
Unlike VPNs and SDNs, implementing mTLS is pretty easy. There's just one hurdle: you need certificates issued by your own certificate authority (CA). Building and operating a CA, issuing certificates, and making sure they're renewed before they expire can be tricky. Autocert and Smallstep Certificate Manager make the whole process easy.
A Little Background on the Tools We’ll Be Using
Autocert
Autocert is an admission webhook that intercepts and patches pod creation requests with some YAML to inject an init container and sidecar that handle obtaining and renewing certificates, respectively. In other words, is an open-source Kubernetes add-on that automatically injects TLS certificates directly into your containers.
Autocert is useful for when you don’t trust the backing datastore where Kubernetes TLS Secrets are stored (often this is etcd
). Depending on how your datastore is configured, the data may or may not be encrypted. With Autocert, certificates are injected directly into containers, and private keys are never transmitted across the network and aren't stored in etcd
.
To obtain a certificate, you just need to tell autocert your workload's name, using the autocert.step.sm/name
pod annotation. Autocert will issue a cert to the pod, make it available in var/run/autocert.step.sm
, and keep it renewed. However, it requires a certificate authority to issue certificates. That's where Smallstep Certificate Manager comes in. This guide shows how to configure autocert to use Certificate Manager as the upstream CA.
Smallstep Certificate Manager
Smallstep Certificate Manager is an opinionated, extensible platform for DevSecOps public key infrastructure (PKI) that delivers highly available hosted certificate authorities. It also offers expiry notifications and alerts, a management dashboard, Active Revocation, and other features. With Smallstep Certificate Manager, you can easily issue private TLS and SSH certificates to all your workloads/developers.
Technically, Kubernetes doesn't come with a CA. It has integration points that allow you to use any CA. So, we’ll be using Smallstep Certificate Manager since it’s from the same team that built and maintains Autocert.
Before We Begin
Before we get into the main steps, you need to:
- Create a free account on Smallstep Certificate Manager
- Create a Certificate Authority in Certificate Manager that will act as your upstream CA
- Install step CLI. To interact with Certificate Manager via terminal, you will need the
step
CLI command on your local machine. It’s used for many common crypto operations. Check here for a list of all commands.
Setting Up Autocert
1. Bootstrap With Your CA
Bootstrapping with your Authority configures your workstation to trust the authority's root certificate. Run:
step ca bootstrap --ca-url [your CA URL] \\
--fingerprint [your CA fingerprint] \\
--install
2. Add a Provisioner
Provisioners are methods used to verify the legitimacy of certificate signing requests and attest to the identity of the requesting service or human. Autocert requires a JWK provisioner. The command below creates a default JWK provisioner called autocert
. Run:
step ca provisioner add autocert --create
You’ll be asked to provide a password to encrypt the provisioner private key.
3. Create ConfigMaps and a Secret for Autocert
In Kubernetes, create a namespace for autocert:
kubectl create ns step
Output:
namespace/step created
Use the same password you entered when creating the provisioner to create the Secret.
kubectl -n step create secret generic autocert-password --from-file=password=autocert-password.txt
Output:
secret/autocert-password created
Now create a ConfigMap
that includes your (bootstrapped) config dir:
kubectl -n step create configmap config --from-file $(step path)/config`
Output:
configmap/config created
We'll do the same thing for the certs
dir, which contains our CA's root certificate:
kubectl -n step create configmap certs --from-file $(step path)/certs
Output:
configmap/certs created
4. Deploy Autocert
Download the Autocert YAML config:
curl -O <https://raw.githubusercontent.com/smallstep/autocert/master/install/02-autocert.yaml>
Edit the caUrl
in the autocert-config
ConfigMap of the yaml file you just downloaded. Change it from https://ca.step.svc.cluster.local
to your Certificate Manager authority URL, e.g. https://autocert.areed.ca.smallstep.com
.
Then run:
kubectl apply -f <https://raw.githubusercontent.com/smallstep/autocert/master/install/03-rbac.yaml>
Output:
clusterrole.rbac.authorization.k8s.io/autocert-controller created
clusterrolebinding.rbac.authorization.k8s.io/autocert-controller created
Now let's deploy Autocert. Run:
kubectl apply -f 02-autocert.yaml
Output:
service/autocert created
configmap/autocert-config created
deployment.apps/autocert created
Next, let's deploy the admission webhook. You can see in this block that we include the root CA certificate (in base64) as part of the client configuration for Autocert.
cat <<EOF | kubectl apply -f -
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: autocert-webhook-config
labels: {app: autocert}
webhooks:
- name: autocert.step.sm
sideEffects: None
admissionReviewVersions: ["v1beta1"]
clientConfig:
service:
name: autocert
namespace: step
path: "/mutate"
caBundle: $(cat $(step path)/certs/root_ca.crt | base64 | tr -d '\\n')
rules:
- operations: ["CREATE"]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
namespaceSelector:
matchLabels:
autocert.step.sm: enabled
EOF
Output:
mutatingwebhookconfiguration.admissionregistration.k8s.io/autocert-webhook-config created
Autocert is now added to the cluster and configured. You can run this command to verify the autocert pods are marked ready.
kubectl -n step get deployment/autocert
Use Autocert to Enable mTLS Between Server and Client
Let’s create a test app that will use Autocert. It’s a “Hello World” web server that uses mutual TLS authentication.
Since this deployment is in the default namespace, label it to tell Autocert to issue and renew certificates for new pods with the autocert.step.sm/name
annotation:
kubectl label namespace default autocert.step.sm=enabled
Output:
namespace/default labeled
To test things out, we'll create a deployment with the autocert.step.sm/name
pod annotation. This example uses the name localhost
, since we will be testing from our workstation.
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: hello-mtls
name: hello-mtls
spec:
selector:
matchLabels:
app: hello-mtls
template:
metadata:
annotations:
autocert.step.sm/name: localhost
labels:
app: hello-mtls
spec:
containers:
- image: smallstep/hello-mtls-server-go:latest
name: hello-mtls
EOF
Output:
deployment.apps/hello-mtls created
For testing, forward localhost:8443
to port 443 on the pod.
kubectl port-forward deploy/hello-mtls 8443:443
Output:
Forwarding from 127.0.0.1:8443 -> 443
Forwarding from [::1]:8443 -> 443
Keep this running in the background during the next steps.
Now issue a client certificate signed by your CA. You’ll need this to authenticate to the “Hello mTLS” test server.
step ca certificate linda.ikechukwu@smallstep.com demo.crt demo.key
Finally, you should be able to verify it's all working:
curl --cacert $(step path)/certs/root_ca.crt \\
--cert demo.crt --key demo.key \\
<https://localhost:8443>
Output:
Hello linda.ikechukwu@smallstep.com
Conclusion
It can be challenging to secure Kubernetes while still managing the various moving parts of your cluster. Regardless of the network architecture and policies in place, automating security by design will always make your cluster’s workload safer and more reliable.
Autocert, alongside Smallstep Certificate Manager, makes it easy to automate certificate issuance into a Kubernetes deployment. All you need is a little bit of YAML and a working cluster to begin issuing Kubernetes TLS certificates to your microservices and thwarting nefarious actors.
Published at DZone with permission of Linda Ikechukwu. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments