Build a Serverless URL Shortener With Go
Learn how to use DynamoDB Go SDK using a URL shortener sample application and integrate it with AWS Lambda and API Gateway to build a serverless solution.
Join the DZone community and get the full member experience.
Join For FreeThis blog post covers how to build a Serverless URL shortener application using Go. It leverages AWS Lambda for business logic, DynamoDB for persistence, and API Gateway to provide the HTTP endpoints to access and use the application. The sample application presented in this blog is a trimmed-down version of bit.ly
or other solutions you may have used or encountered.
It's structured as follows:
- I will start off with a quick introduction and dive into how to deploy and try the solution.
- After that, I will focus on the code itself. This will cover:
- The part which is used to write the infrastructure (using Go bindings for AWS CDK)
- Also, the core business logic which contains the Lambda function (using Lambda Go support) as well as the DynamoDB operations (using the DynamoDB Go SDK)
In this blog, you will learn:
- How to use the DynamoDB Go SDK (v2) to execute CRUD operations such as
PutItem
,GetItem
,UpdateItem
andDeleteItem
- How to use AWS CDK Go bindings to deploy a Serverless application to create and manage a
DynamoDB
table, Lambda functions, API Gateway, and other components as well.
Once you deploy the application, you will be able to create shortcodes for URLs using the endpoint exposed by the API Gateway and also access them.
# create short code for a URL (e.g. https://abhirockzz.github.io/)
curl -i -X POST -d 'https://abhirockzz.github.io/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL
# access URL via short code
curl -i $URL_SHORTENER_APP_URL/<short-code>
Let’s Get Started: Deploy the Serverless Application
Before you proceed, make sure you have the Go programming language (v1.16 or higher) and AWS CDK installed.
Clone the project and change it to the right directory:
git clone https://github.com/abhirockzz/serverless-url-shortener-golang
cd cdk
Start the Deployment
To start the deployment, all you will do is run a single command (cdk deploy
), and wait for a bit. You will see a (long) list of resources that will be created and will need to provide your confirmation to proceed.
Don't worry: in the next section, I will explain what's happening.
cdk deploy
# output
Bundling asset ServerlessURLShortenerStack1/create-url-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/access-url-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/update-url-status-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/delete-url-function/Code/Stage...
✨ Synthesis time: 10.28s
This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:
.......
Do you wish to deploy these changes (y/n)?
This will start creating the AWS resources required for our application.
If you want to see the AWS CloudFormation template which will be used behind the scenes, run
cdk synth
and check thecdk.out
folder.
You can keep track of the progress in the terminal or navigate to the AWS console: CloudFormation > Stacks > ServerlessURLShortenerStack
Once all the resources are created, you can try out the application. You should have:
- Four Lambda functions (and related resources)
- A DynamoDB table
- An API Gateway (as well as routes and integrations)
- A few others (like IAM roles, etc.)
Before you proceed, get the API Gateway endpoint that you will need to use. It's available in the stack output (in the terminal or the Outputs tab in the AWS CloudFormation console for your Stack):
Shorten Some URLs!
Start by generating shortcodes for a few URLs:
# export the API Gateway endpoint
export URL_SHORTENER_APP_URL=<replace with apigw endpoint above>
# for example:
export URL_SHORTENER_APP_URL=https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/
# invoke the endpoint to create short codes
curl -i -X POST -d 'https://abhirockzz.github.io/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL
curl -i -X POST -d 'https://dzone.com/users/456870/abhirockzz.html' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL
curl -i -X POST -d 'https://abhishek1987.medium.com/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL
To generate a short code, you need to pass the original URL in the payload body as part of an HTTP POST request (e.g., https://abhishek1987.medium.com/
).
'Content-Type: text/plain' is important. Otherwise, API Gateway will do
base64
encoding of your payload.
If all goes well, you should get a HTTP 201
along with the shortcode in the HTTP response (as a JSON payload).
HTTP/2 201
date: Fri, 15 Jul 2022 13:03:20 GMT
content-type: text/plain; charset=utf-8
content-length: 25
apigw-requestid: VTzPsgmSoAMESdA=
{"short_code":"1ee3ad1b"}
To confirm, check the DynamoDB table.
Notice an active attribute there? More on this soon.
Access the URL Using the Shortcode
With services like bit.ly, etc., you typically create short links for your URLs and share them with the world. We will do something similar. Now that you have the shortcode generated, you can share the link with others (it's not really a short link like bit.ly, but that's ok for now!), and once they access it, they would see the original URL.
The access link will have the following format:
<URL_SHORTENER_APP_URL>/<generated short code>
(e.g., https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b)
If you navigate to the link using a browser, you will be automatically redirected to the original URL that you had specified. To see what's going on, try the same with curl
:
curl -i $URL_SHORTENER_APP_URL/<short code>
# example
curl -i https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/0e1785b1
This is simply an HTTP GET
request. If all goes well, you should get an HTTP 302
response (StatusFound
) and the URL re-direction happens due to the Location HTTP header which contains the original URL.
HTTP/2 302
date: Fri, 15 Jul 2022 13:08:54 GMT
content-length: 0
location: https://abhirockzz.github.io/
apigw-requestid: VT0D1hNLIAMES8w=
How about using a shortcode that does not exist?
Set the Status
You can enable and disable the shortcodes. The original URL will only be accessible if the association is in an active state.
To disable a short code:
curl -i -X PUT -d '{"active": false}' -H 'Content-Type: application/json' $URL_SHORTENER_APP_URL/<short code>
# example
curl -i -X PUT -d '{"active": false}' -H 'Content-Type: application/json' https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b
This is an HTTP PUT
request with a JSON payload that specifies the status (false
, in this case, refers to disable) along with the shortcode, which is a path parameter to the API Gateway endpoint. If all works well, you should see an HTTP 204 (No Content) response:
HTTP/2 204
date: Fri, 15 Jul 2022 13:15:41 GMT
apigw-requestid: VT1Digy8IAMEVHw=
Check the DynamoDB record: the active attribute must have switched to false
.
As an exercise, try the following:
- Access the URL via the same short code now and check the response.
- Access an invalid short code, i.e., that does not exist.
- Enable a disabled URL (use
{"active": true}
).
Ok, so far we have covered all operations, except delete. Let's try that and wrap up the CRUD
!
Delete
curl -i -X DELETE $URL_SHORTENER_APP_URL/<short code>
# example
curl -i -X DELETE https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b
Nothing too surprising here. We use an HTTP DELETE
along with the shortcode. Just like in the case of an update, you should get a HTTP 204
response:
HTTP/2 204
date: Fri, 15 Jul 2022 13:23:36 GMT
apigw-requestid: VT2NzgjnIAMEVKA=
But this time, of course, the DynamoDB record should have been deleted. Confirm the same.
What happens when you try to delete a short code that does not exist?
Don’t Forget To Clean Up!
Once you're done, to delete all the services, simply use:
cdk destroy
Alright, now that you've actually seen "what" the application does, let's move on to the "how."
We will start with the AWS CDK code and explore how it does all the heavy lifting behind setting up the infrastructure for our serverless URL shortener service.
With AWS CDK, Infrastructure-IS-Code!
You can check out the code in this GitHub repo. I will walk you through the keys parts of the NewServerlessURLShortenerStack
function, which defines the workhorse of our CDK application.
I have omitted some of the code for brevity.
We start by creating a DynamoDB
table. A primary key is all that's required in order to do that: in this case, shortcode
(we don't have a range/sort key in this example).
dynamoDBTable := awsdynamodb.NewTable(stack, jsii.String("url-shortener-dynamodb-table"),
&awsdynamodb.TableProps{
PartitionKey: &awsdynamodb.Attribute{
Name: jsii.String(shortCodeDynamoDBAttributeName),
Type: awsdynamodb.AttributeType_STRING}})
Then, we create an API Gateway (HTTP API) with just one line of code!
urlShortenerAPI := awscdkapigatewayv2alpha.NewHttpApi(stack, jsii.String("url-shortner-http-api"), nil)
We move on to the first Lambda function that creates shortcodes for URLs. Notice that we use an experimental module awscdklambdagoalpha (here is the stable version at the time of writing). If your Go project is structured in a specific way (details here) and you specify its path using Entry
, it will automatically take care of building, packaging, and deploying your Lambda function! Not bad at all!
In addition to Local bundling (as used in this example), Docker-based builds are also supported.
createURLFunction := awscdklambdagoalpha.NewGoFunction(stack, jsii.String("create-url-function"),
&awscdklambdagoalpha.GoFunctionProps{
Runtime: awslambda.Runtime_GO_1_X(),
Environment: funcEnvVar,
Entry: jsii.String(createShortURLFunctionDirectory)})
dynamoDBTable.GrantWriteData(createURLFunction)
Finally, we add the last bit of plumbing by creating a Lambda-HTTP API integration (notice how the Lambda function variable createURLFunction
is referenced) and adding a route to the HTTP API we had created. This, in turn, refers to the Lambda integration.
createFunctionIntg := awscdkapigatewayv2integrationsalpha.NewHttpLambdaIntegration(jsii.String("create-function-integration"), createURLFunction, nil)
urlShortenerAPI.AddRoutes(&awscdkapigatewayv2alpha.AddRoutesOptions{
Path: jsii.String("/"),
Methods: &[]awscdkapigatewayv2alpha.HttpMethod{awscdkapigatewayv2alpha.HttpMethod_POST},
Integration: createFunctionIntg})
This was just for one function. We have three more remaining. The good part is that the template for all these is similar. For example:
- Create the function.
- Grant permission for DynamoDB.
- Wire it up with API Gateway (with the correct HTTP method i.e.
POST
,PUT
,DELETE
).
I will not repeat it over here. Feel free to grok through the rest of the code.
Now that you understand the magic behind the "one-click" infrastructure setup, let's move on to the core logic of the application.
URL Shortener Lambda Function and DynamoDB Logic
There are four different functions, all of which are in their respective folders, and all of them have a few things in common in the way they operate:
- They do some initial processing: process the payload, or extract the path parameter from the URL, etc.
- Invoke a common database layer to execute the CRUD functionality (more on this soon).
- Handle errors as appropriate and return response.
With that knowledge, it should be easy to follow the code.
As before, some parts of the code have been omitted for brevity.
Create Function
func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {
url := req.Body
shortCode, err := db.SaveURL(url)
if err != nil {//..handle error}
response := Response{ShortCode: shortCode}
respBytes, err := json.Marshal(response)
if err != nil {//..handle error}
return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusCreated, Body: string(respBytes)}, nil
}
This function starts by reading the payload of the HTTP
request body. This is a string
which has the URL for which the shortcode is being created. It invokes the database layer to try and save this record to DynamoDB and handles errors. Finally, it returns a JSON response with the shortcode.
Here is the function that actually interacts with DynamoDB to get the job done.
func SaveURL(longurl string) (string, error) {
shortCode := uuid.New().String()[:8]
item := make(map[string]types.AttributeValue)
item[longURLDynamoDBAttributeName] = &types.AttributeValueMemberS{Value: longurl}
item[shortCodeDynamoDBAttributeName] = &types.AttributeValueMemberS{Value: shortCode}
item[activeDynamoDBAttributeName] = &types.AttributeValueMemberBOOL{Value: true}
_, err := client.PutItem(context.Background(), &dynamodb.PutItemInput{
TableName: aws.String(table),
Item: item})
if err != nil {//..handle error}
return shortCode, nil
}
For the purposes of this sample app, the shortcode is created by generating a UUID
and trimming out the last 8 digits. It's easy to replace this with another technique: all that matters is that you generate a unique string that can work as a short code. Then, it is all about calling the PutItem API with the required data.
Access the URL
func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {
shortCode := req.PathParameters[pathParameterName]
longurl, err := db.GetLongURL(shortCode)
if err != nil {//..handle error}
return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusFound, Headers: map[string]string{locationHeader: longurl}}, nil
}
When someone accesses the short link (as demonstrated in the earlier section), the shortcode is passed in as a path parameter e.g. http://<api gw url>/<short code>
. The database layer is invoked to get the corresponding URL from DynamoDB
table (errors are handled as needed). Finally, the response is returned to the user wherein the status code is 302
and the URL is passed in the Location header. This is what re-directs you to the original URL when you enter the short code (in the browser).
Here is the DynamoDB call:
func GetLongURL(shortCode string) (string, error) {
op, err := client.GetItem(context.Background(), &dynamodb.GetItemInput{
TableName: aws.String(table),
Key: map[string]types.AttributeValue{
shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}}})
if err != nil {//..handle error}
if op.Item == nil {
return "", ErrUrlNotFound
}
activeAV := op.Item[activeDynamoDBAttributeName]
active := activeAV.(*types.AttributeValueMemberBOOL).Value
if !active {
return "", ErrUrlNotActive
}
longurlAV := op.Item[longURLDynamoDBAttributeName]
longurl := longurlAV.(*types.AttributeValueMemberS).Value
return longurl, nil
}
The first step is to use GetItem API to get the DynamoDB
record containing URL and status corresponding to the shortcode. If the item
object in the response is nil
, we can be sure that a record with that shortcode does not exist: we return a custom error, which can be helpful for our function which can then return an appropriate response to the caller of the API (e.g., a HTTP 404
). We also check the status (active or not) and return an error if active is set to false
. If all is well, the URL is returned to the caller.
Update Status
func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {
var payload Payload
reqBody := req.Body
err := json.Unmarshal([]byte(reqBody), &payload)
if err != nil {//..handle error}
shortCode := req.PathParameters[pathParameterName]
err = db.Update(shortCode, payload.Active)
if err != nil {//..handle error}
return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusNoContent}, nil
}
The first step is to marshal the HTTP request payload, which is a JSON (e.g., {"active": false}
), and then get the shortcode from the path parameter. The database layer is invoked to update the status and handle errors.
func Update(shortCode string, status bool) error {
update := expression.Set(expression.Name(activeDynamoDBAttributeName), expression.Value(status))
updateExpression, _ := expression.NewBuilder().WithUpdate(update).Build()
condition := expression.AttributeExists(expression.Name(shortCodeDynamoDBAttributeName))
conditionExpression, _ := expression.NewBuilder().WithCondition(condition).Build()
_, err := client.UpdateItem(context.Background(), &dynamodb.UpdateItemInput{
TableName: aws.String(table),
Key: map[string]types.AttributeValue{
shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}},
UpdateExpression: updateExpression.Update(),
ExpressionAttributeNames: updateExpression.Names(),
ExpressionAttributeValues: updateExpression.Values(),
ConditionExpression: conditionExpression.Condition(),
})
if err != nil && strings.Contains(err.Error(), "ConditionalCheckFailedException") {
return ErrUrlNotFound
}
return err
}
The UpdateItem API call takes care of changing the status. It's fairly simple except for all these expressions
that you need, especially if you're new to the concept. The first one (mandatory) is the update expression where you specify the attribute you need to set (active in this case) and its value. The second one makes sure that you are updating the status for a short code that actually exists in the table. This is important since otherwise, the UpdateItem
API call will insert a new item (we don't want that!). Instead of rolling out the expressions by hand, we use the expressions package.
Delete Shortcode
func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {
shortCode := req.PathParameters[pathParameterName]
err := db.Delete(shortCode)
if err != nil {//..handle error}
return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusNoContent}, nil
}
The delete handler is no different. After the shortcode to be deleted is extracted from the path parameter, the database layer is invoked to remove it from the DynamoDB table. The result returned to the user is either an HTTP 204
(on success) or the error.
func Delete(shortCode string) error {
condition := expression.AttributeExists(expression.Name(shortCodeDynamoDBAttributeName))
conditionExpression, _ := expression.NewBuilder().WithCondition(condition).Build()
_, err := client.DeleteItem(context.Background(), &dynamodb.DeleteItemInput{
TableName: aws.String(table),
Key: map[string]types.AttributeValue{
shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}},
ConditionExpression: conditionExpression.Condition(),
ExpressionAttributeNames: conditionExpression.Names(),
ExpressionAttributeValues: conditionExpression.Values()})
if err != nil && strings.Contains(err.Error(), "ConditionalCheckFailedException") {
return ErrUrlNotFound
}
return err
}
Just like UpdateItem
API, the DeleteItem API also takes in a condition expression. If there is no record in the DynamoDB table with the given short code, an error is returned. Otherwise, the record is deleted.
That completes the code walk-through!
Wrap Up
In this blog post, you learned how to use DynamoDB Go SDK using a URL shortener sample application. You also integrated it with AWS Lambda and API Gateway to build a serverless solution whose infrastructure was also defined using actual code (as opposed to yaml
, JSON, etc.), thanks to the Go support in AWS CDK.
Published at DZone with permission of Abhishek Gupta, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments