Deploying Envoy as an API Gateway for Microservices

An API Gateway is a façade that sits between the consumers and producers of an API. Cross-cutting functionality such as authentication, monitoring, and traffic management is implemented in your API Gateway so that your services can remain unaware of these details. In addition, when multiple services are responsible for different APIs (e.g., in a microservices architecture), an API Gateway hides this abstraction detail from the consumer.

There are dozens of different options for API Gateways, depending on your requirements. The Amazon API Gateway is a hosted Gateway that runs in Amazon. Traefik, NGINX, Kong, or HAProxy are all open source options, with their own strengths and weaknesses.

And, of course, there's Envoy, which we've grown fond of at Datawire. Envoy is interesting because, in addition to providing the reverse proxy semantics you need to implement an API Gateway, it also supports the features you need for distributed architectures (in fact, the Istio project builds on Envoy to provide a full-blown services mesh).

So let's take a closer look at deploying Envoy as a full-fledged, self-service API gateway. If you've been following along with our Envoy experiments so far, you've seen that to get a working microservice-based application, we've had to:

1. deploy our services.
2. deploy Envoy.
3. deploy Envoy's SDS.
4. configure Envoy to use our SDS.
5. configure Envoy to relay requests for our services.

Of the five steps, only one has to do with our real application- the other four have to do with Envoy. So, we're going to show you how to get Envoy set up as an API Gateway. To simplify things, we're going to use Ambassador, an open source API Gateway built on Envoy. 

Setting Up

We're going to assume that your basic infrastructure is set up enough that you have a Kubernetes cluster running in your cloud environment of choice -- if you don't, Loom can help you get set up. For now, we assume that:

• You have kubectl correctly talking to a Kubernetes cluster running in EC2 or GKE.
  • This is probably obvious, but it's tough to work with a Kubernetes cluster if you can't talk to it with kubectl.
• You have docker installed and working.
  • Since we'll be building Docker images, we need a working docker command.
• You have credentials to push Docker images to either DockerHub or the Google Container Registry (gcr.io).

That last point is worth a little more discussion. To run something in Kubernetes, we have to be able to pull a Docker image from somewhere that the cluster can reach. When using Minikube, this is no problem, since Minikube runs its own Docker daemon: by definition, anything in the Minikube cluster can talk to that Docker daemon. However, things are different once GKE or EC2 come into play: they can't talk to a Docker daemon on your laptop without heroic measures, so you'll need to explicitly push images somewhere accessible.

Where, exactly? For our purposes today, it doesn't really matter- DockerHub, gcr.io, your private registry, whatever. Going into production, you might want to think about bandwidth costs- for example, on GKE, pulling images from gcr.io will never incur bandwidth charges, and that might well be an important factor.

The Application

We'll be using the same simple user service as before for our application, so once again we'll be using Python, Flask, and Postgres. Here's the really easy way to get all that running:

kubectl apply -f https://raw.githubusercontent.com/datawire/ambassador/master/demo-usersvc.yaml

This will spin up pods for the usersvc itself and for its backing Postgres instance. Once it's applied, you should see its pods and services:

$ kubectl get pods
NAME                        READY     STATUS    RESTARTS   AGE
postgres-1385931004-v3czl   1/1       Running   0          5s
usersvc-3828982842-gs94p    1/1       Running   0          5s

$ kubectl get services
NAME         CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
postgres     109.166.172.9  <none>        5432/TCP   10s
usersvc      109.169.19.62  <none>        80/TCP     10s

(As before, all the actual pod names and IPs will be different on your system.)

At this point, our usersvc is running, but it can't be reached from outside our cluster. To verify that it's running, we can get a shell on the usersvc pod:

usersvc_pod=$(kubectl get pods -l service=usersvc -o jsonpath='{.items[0].metadata.name}')
kubectl exec -it $usersvc_pod /bin/bash

and then do a health check with curl:

curl http://localhost/user/health

If that works, you'll see the usual usersvc health check result:

{
  "hostname": "usersvc-3828982842-czb53",
  "msg": "user health check OK",
  "ok": true,
  "resolvedname": "109.196.5.2"
}

Given that we can talk to the usersvc locally, it's time to set up access from the outside world- which is exactly what Ambassador is all about.

The Ambassador Service and TLS

First things first: are you going to speak TLS to Ambassador or not? It's possible to switch this later, but you'll likely need to muck about with your DNS and such to do it, so it's a pain.

Ambassador With TLS (recommended)

We recommend using TLS, speaking to Ambassador only over HTTPS. To do this, you need a TLS certificate, which means you'll need the DNS set up correctly. So start by creating the Ambassador's kubernetes service:

kubectl apply -f https://raw.githubusercontent.com/datawire/ambassador/master/ambassador-https.yaml

Be aware that repeating this command will wipe out any certificates you previously saved. On the other hand, repeating it probably means that you had to change hostnames anyway, so that's probably OK.

ambassador-https.yaml will create an L4 load balancer that will later be used to talk to Ambassador. By leaving the service intact as you mess with deployments, pods, etc., you should be able to stably associate a DNS name with the service, which will let you request the TLS certificate that you need.

Sadly, setting up your DNS and requesting a cert are a bit outside the scope of this document -- if you don't know how to do this, check with your local DNS administrator. (If you are the local DNS admin and are just hunting a CA recommendation, check out Let's Encrypt.)

Once you have the cert, you can publish certificates:

curl -O https://raw.githubusercontent.com/datawire/ambassador/master/scripts/push-cert
sh push-cert $FULLCHAIN_PATH $PRIVKEY_PATH

where

• $FULLCHAIN_PATH is the path to a single PEM file containing the certificate chain for your cert, including the certificate for your Ambassador and all relevant intermediate certs -- Let's Encrypt calls this fullchain.pem
• $PRIVKEY_PATH is the path to the corresponding private key -- Let's Encrypt calls this privkey.pem

push-cert will push the cert into Kubernetes secret storage, for Ambassador's later use.

Again: if you repeat the kubectl apply above, you will wipe out your certificates and you'll have to rerun push-cert.

Ambassador Without TLS

If you really, really, really want to, you can spin up Ambassador without TLS. We do not recommend this for any production use but you can do it:

kubectl apply -f https://raw.githubusercontent.com/datawire/ambassador/master/ambassador-http.yaml

will create a service to listen only for plaintext HTTP on port 80.

Be aware that executing this command will wipe out any certificates you previously saved. On the other hand, if you're disabling HTTPS, that's probably what you want to happen.

Starting Ambassador

Once its service is created, we can get Ambassador running in our fabric. Here's the easy way:

kubectl apply -f https://raw.githubusercontent.com/datawire/ambassador/master/ambassador.yaml

(If for some reason you really want to, you can apply ambassador-store.yaml, ambassador-sds.yaml, and then ambassador-rest.yaml instead, but that's not the easy way...)

Once that's done, you should see two pods and services for Ambassador's component parts:

$ kubectl get pods
NAME                                READY     STATUS    RESTARTS   AGE
ambassador-3176426918-13v2v         1/1       Running   0          3m
ambassador-store-2691475196-bzdn7   1/1       Running   0          3m
postgres-1385931004-v3czl           1/1       Running   0          5m
usersvc-3828982842-gs94p            1/1       Running   0          5m

$ kubectl get services
NAME               CLUSTER-IP        EXTERNAL-IP        PORT(S)    AGE
ambassador         109.168.235.177   a1128c0831f9e...   80/TCP     3m
ambassador-admin   109.166.226.158   <none>             8888/TCP   3m
ambassador-store   109.164.125.75    <none>             5432/TCP   3m
kubernetes         109.164.0.1       <none>             443/TCP    9m
postgres           109.166.172.9     <none>             5432/TCP   5m
usersvc            109.169.19.62     <none>             80/TCP     5m

Ambassador comprises two pods and two services:

• ambassador is the REST API to access microservices through Ambassador. It's accessible from outside the cluster.
• ambassador-admin is the REST API to configure Ambassador. It is only accessible from inside the cluster.
• ambassador-store is Ambassador's persistent storage. You won't be interacting directly from this.

At present, only one of each should be run.

Accessing the Admin Interface

You'll need the Ambassador admin interface to configure Ambassador. This isn't exported outside the cluster: you use kubectl port-forward to reach it (best do this in a new shell window):

POD=$(kubectl get pod -l service=ambassador -o jsonpath="{.items[0].metadata.name}")
kubectl port-forward "$POD" 8888

Once this is done, http://localhost:8888/ will reach the admin interface.

Setting AMBASSADORURL

In order to get access to your microservices through Ambassador, you'll need an external URL to Ambassador's service interface. We'll use $AMBASSADORURL as shorthand for the base URL of Ambassador.

Make sure that the ambassador service has an external IP address listed, then, if you're using TLS, you can set $AMBASSADORURL by hand with something like

export AMBASSADORURL=https://your-domain-name

where your-domain-name is the name you set up when you requested your certs.

Without TLS, if you have a domain name, great! do the above. If not, look at the LoadBalancer Ingress line of kubectl describe service ambassador and set $AMBASSADORURL based on that. (On Minikube, you'll need to use minikube service --url ambassador) .

In any case, do not include a trailing / in $AMBASSADORURL, or the examples in this document won't work.

Ambassador Health Check

Once all of the above is done, you should be able to do

curl http://localhost:8888/ambassador/health

to verify that Ambassador is running. If all is well, you'll see output like:

{
  "hostname": "ambassador-3176426918-13v2v",
  "msg": "ambassador health check OK",
  "ok": true,
  "resolvedname": "109.196.3.8",
  "version": "0.8.2"
}

and we're in business! If this doesn't work, the most likely scenario is that you're using TLS and the certificates aren't correctly set up -- use

kubectl get secret ambassador-certs

to check what certificates (if any) have been set up where Ambassador can see them, and use

kubectl delete deployment ambassador
kubectl apply -f https://raw.githubusercontent.com/datawire/ambassador/master/ambassador-rest.yaml

to reset everything after you've changed certificates.

Mappings, Resources, and Services

Once you get a clean health check, you can start setting up mappings. Ambassador maps resources named by URL prefixes to services running in Kubernetes. We can map URLs starting with /user/ to our usersvc with the following POST request:

curl -XPOST -H "Content-Type: application/json" \
      -d '{ "prefix": "/user/", "service": "usersvc" }' \
      http://localhost:8888/ambassador/mapping/usermapping

Once that's done, we can speak to the usersvc using Ambassador:

$ curl $AMBASSADORURL/user/health
{
  "hostname": "usersvc-3828982842-czb53",
  "msg": "user health check OK",
  "ok": true,
  "resolvedname": "109.196.5.2"
}

We can verify all the mappings that Ambassador knows about with

curl http://localhost:8888/ambassador/mappings

At present, of course, this will show us only the usermapping:

$ curl http://localhost:8888/ambassador/mappings
{
  "count": 1,
  "hostname": "ambassador-3176426918-13v2v",
  "ok": true,
  "resolvedname": "109.196.3.8",
  "services": [
    {
      "name": "usermapping",
      "prefix": "/user/",
      "rewrite": "/",
      "service": "usersvc"
    }
  ],
  "version": "0.8.2"
}

Finally, you can remove mappings with a DELETE request:

curl -XDELETE http://localhost:8888/ambassador/mapping/$mapping

But let's not delete the usermapping just yet!

Notes About Mappings

A few critical notes about mappings:

1. When creating a mapping, the service name must match the name of a service defined in Kubernetes.
2. When deleting a mapping, use the mapping name, not the URL prefix or service name.
3. Ambassador can take up to five seconds to update Envoy's configuration after a mapping change.

Ambassador Statistics

Ambassador also tracks various runtime statistics, which can be retrieved with

curl http://localhost:8888/ambassador/stats

This will return a JSON dictionary of statistics about resources that Ambassador presently has mapped. Most notably, the services dictionary lets you know basic health information about the services to which Ambassador is providing access:

• services.$service.healthy_members is the number of healthy back-end systems providing the service;
• services.$service.upstream_ok is the number of requests to the service that have succeeded; and
• services.$service.upstream_bad is the number of requests to the service that have failed.

Adding a Service

Of course, an API gateway isn't about only mapping a single service. Suppose we write a new service, now that Ambassador is up and running? Let's set up a service to keep track of the grues that lurk in the caverns our users might go exploring. We'll call that the gruesvc. The code (which you can find in the gruesvc repo on GitHub is very similar to the usersvc, and you can get it running with:

kubectl apply -f https://raw.githubusercontent.com/datawire/ambassador/master/demo-gruesvc.yaml

At this point, we can map /grue/ to the gruesvc with a single POST:

curl -XPOST -H "Content-Type: application/json" \
      -d '{ "prefix": "/grue/", "service": "gruesvc", "rewrite": "/grue/" }' \
      http://localhost:8888/ambassador/mapping/gruemapping

Note the new rewrite keyword: by default, Ambassador rewrites whatever prefix matches in a URL to a single /, effectively removing it. The gruesvc code, though, expects requests with a prefix of /grue/, so we tell Ambassador to preserve that. We could, of course, rewrite to anything else we wanted.

Once that's done, we can speak to the gruesvc using Ambassador:

$ curl $AMBASSADORURL/grue/health
{
  "hostname": "gruesvc-3828982842-czb53",
  "msg": "grue health check OK",
  "ok": true,
  "resolvedname": "109.196.5.2"
}

and Ambassador will show us both services if we ask if for its list of mappings:

$ curl http://localhost:8888/ambassador/mappings
{
  "count": 1,
  "hostname": "ambassador-3176426918-13v2v",
  "ok": true,
  "resolvedname": "109.196.3.8",
  "services": [
    {
      "name": "usermapping",
      "prefix": "/user/",
      "rewrite": "/",
      "service": "usersvc"
    },
    {
      "name": "gruemapping",
      "prefix": "/grue/",
      "rewrite": "/grue/",
      "service": "gruesvc"
    }
  ],
  "version": "0.8.2"
}

Note that that single POST to create the mapping is the only thing that a service author needs to do to get their service hooked into the world: one post, and a few seconds later you can reach your service from wherever. While this obviously means that you need apps to pay attention to security and authorization, it makes things very easy as you develop services.

How does Ambassador work?

Ambassador has to keep track of all of the services as well as handling the minutiae of network communications between everyone. All of this routing information needs to be propagated to all the Envoy instances. Luckily, Kubernetes and Envoy provide 90% of the basic functionality, so Ambassador just implements the last 10%.

On the routing side of the world, Kubernetes already has to keep track of where every instance of every service is running, and it already provides APIs to access this information. So all we need to take a crack at this is a simple way to associate URL prefixes with service names.

On the configuration side of the world, Envoy already supports the Service Discovery Service for dynamic configuration of where services can be found (as we saw last time). At this point, it also supports dynamic discovery of clusters and routes, and of course, it supports hot reloads, so we have multiple mechanisms for managing Envoy's configuration itself[^1].

[^1]: Ambassador currently uses the hot-reload capability, because Envoy only recently gained the ability to use a route-discovery service.