Supabase CDC Webhooks to Memphis Rest Gateway
A step-by-step tutorial on Supabase CDC Webhooks to Memphis Rest Gateway.
Join the DZone community and get the full member experience.
Join For FreeIntroduction
Supabase is an open-source Firebase alternative.
In databases, change data capture (or CDC) is a set of software design patterns used to determine and track the data that has changed so that action can be taken using the changed data or because of it.
In simple words: Each time a new event occurs on a specific table, such as INSERT / UPDATE / DELETE, an event will be created and, if configured — published to a destination so some service will be notified for it.
Flow
- Install Memphis over Kubernetes.
- Expose Memphis REST Gateway.
- Create a new Supabase table.
- Configure a webhook for the newly created table.
- Check the incoming CDC events in Memphis.
Step 1: Install Memphis Over Kubernetes
Memphis can be installed on any Kubernetes cluster above v1.20.
For this tutorial, I used Memphis Terraform to deploy a 3-node AWS EKS cluster with a Memphis cluster on top.
Output of AWS EKS creation:
Apply complete! Resources: 67 added, 0 changed, 0 destroyed.
Outputs:
cluster_endpoint = "https://CA2D8E9674B2025*******.gr7.eu-west-1.eks.amazonaws.com"
cluster_id = "memphis-cluster-ed****q7"
cluster_name = "memphis-cluster-ed****q7"
cluster_security_group_id = "sg-09a8f95********02"
region = "eu-west-1"
Output of Memphis cluster deployment:
To access Memphis using UI/CLI/SDK with service EXTERNAL-IP, use the following URLs:
Dashboard/CLI : http://a34d36344a12b48a7a9f1ef09z576023-1416400140.eu-west-1.elb.amazonaws.com:9000
Memphis broker: http://a34d36344a12b48a7a9f1ef09z576023-1416400140.eu-west-1.elb.amazonaws.com:6666 (Client Connections)
Bonus: Create a DNS A record for the address above to work with a real hostname easily.
Reach the Memphis dashboard and initiate the cluster.
The address is the one with port 9000.
Step 2: Expose Memphis Rest Gateway
By default, Memphis terraform does not deploy an LB to the HTTP proxy
pod.
To do it, we need to transform the memphis-http-proxy
service from ClusterIP
to LoadBalancer
by running the following:
kubectl patch svc memphis-http-proxy -n memphis --type='json' -p '[{"op":"replace","path":"/spec/type","value":"LoadBalancer"}]'
kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
memphis-cluster ClusterIP None <none> 9000/TCP,7770/TCP,6666/TCP,8222/TCP 104m
memphis-cluster-external LoadBalancer 172.20.251.246 a34d36224a00b48a7a9f1ac09c576023-1416400140.eu-west-1.elb.amazonaws.com 9000:32173/TCP,6666:32013/TCP,7770:31214/TCP 102m
memphis-http-proxy LoadBalancer 172.20.131.157 a5ce9a9c1006a2d7a64faaa6949dcfbb-1553258514.eu-west-1.elb.amazonaws.com 4444:30686/TCP 104m
mongo ClusterIP None <none> 27017/TCP 104m
Let’s save the newly created LB external IP of the HTTP proxy:
a5ce9a9c1006a2d7a64faaa6949dcfbb-1553258514.eu-west-1.elb.amazonaws.com
Step 3: Create a New Supabase Table
Step 4: Configure a Webhook for the Newly Created Table
First, we need to generate an “Application” type user on Memphis:
Please save both the username and connection token.
The result will be the JWT token for the Supabase webhook. Save it.
{"expires_in":36000000000,"jwt":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MDk4MTIzOTJ9.Xlxq56c0wsfOwMsFheUNPKrnECX4JX_vrz45su5JmVU"}
Back to Supabase. Go to Database -> Webhooks -> Create a webhook
Configure it as follows:
Published at DZone with permission of Yaniv Ben Hemo. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments