Build a Serverless Application for Image Label Detection

In this blog post, you will learn how to build a Serverless solution to process images using Amazon Rekognition, AWS Lambda and the Go programming language. Images uploaded to Amazon Simple Storage Service (S3) will trigger a Lambda function which will detect labels (using the AWS Go SDK) and persist the image label data to an Amazon DynamoDB table.

You will be using the Go programming language for the business logic (thanks to the AWS-lambda-go library) as well as the infrastructure component (Go bindings for AWS CDK) to deploy the solution.

The code is available on GitHub

Introduction

Amazon Rekognition is a service that lets you analyze images and videos in your applications. You can identify objects, people, text, scenes, and activities, and detect inappropriate content. You can also do facial analysis, face comparison, and face search for various use cases like user verification and public safety. Amazon Rekognition is built on deep learning technology that doesn't require machine learning expertise. It has an easy-to-use API that can analyze any image or video file in Amazon S3.

Common use cases for using Amazon Rekognition include:

• Making images and videos searchable - Discover objects and scenes that appear within them.
• Face-based user verification - Confirm user identities by comparing their live image with a reference image.
• Sentiment and demographic analysis - Interpret emotional expressions such as happy, sad, or surprise, and demographic information.
• Facial search - Search images, stored videos, and streaming videos for faces that match those stored in a container known as a face collection.
• Unsafe content detection - Detect adult and violent content in images and in stored videos and use the returned metadata to filter inappropriate content based on business needs. Text detection - Used for visual search, cataloguing, and identifying vehicles based on license plate numbers

Let's learn Amazon Rekognition with a hands-on tutorial.

Pre-Requisites

Before you proceed, make sure you have the following installed:

• Go programming language (v1.18 or higher)
• AWS CDK
• AWS CLI

Clone the project and change it to the right directory:

git clone https://github.com/abhirockzz/ai-ml-golang-rekognition-label-detection
cd ai-ml-golang-rekognition-label-detection

Use AWS CDK To Deploy the Solution

The AWS Cloud Development Kit (AWS CDK) is a framework that lets you define your cloud infrastructure as code in one of its supported programming and provision it through AWS CloudFormation.

To start the deployment, simply invoke cdk deploy and wait for a bit. You will see a list of resources that will be created and will need to provide your confirmation to proceed.

cd cdk
cdk deploy
# output
Bundling asset RekognitionLabelDetectionGolangStack/rekognition-function/Code/Stage...
✨  Synthesis time: 5.44
//.... omitted
Do you wish to deploy these changes (y/n)? y

Enter y to start creating the AWS resources required for the application.

If you want to see the AWS CloudFormation template which will be used behind the scenes, run cdk synth and check the cdk.out folder

You can keep track of the stack creation progress in the terminal or navigate to the AWS console: CloudFormation > Stacks > RekognitionLabelDetectionGolangStack.

Once the stack creation is complete, you should have:

• An S3 bucket - Source bucket to upload images.
• A Lambda function to extract image labels using Amazon Rekognition.
• A DyanmoDB table to store the label data for each image.
• .... along with a few other components (like IAM roles etc.)

You will also see the following output in the terminal (resource names will differ in your case). In this case, these are the names of the S3 buckets created by CDK:

✅  RekognitionLabelDetectionGolangStack

✨  Deployment time: 119.56s

Outputs:
RekognitionLabelDetectionGolangStack.abeldetectionoutputtablename = rekognitionlabeldetectio-labeldetectioninputbucke-v3vn9o06q3kb_labels_output
RekognitionLabelDetectionGolangStack.labeldetectioninputbucketname = rekognitionlabeldetectio-labeldetectioninputbucke-v3vn9o06q3kb
.....

You can now try out the end-to-end solution!

Extract Labels From the Image

To try the solution, you can either use an image of your own or use the sample files provided in the GitHub repository. I will be using the S3 CLI to upload the file, but you can use the AWS console as well.

export SOURCE_BUCKET=<enter source S3 bucket name - check the CDK output>

aws s3 cp ./car.png s3://$SOURCE_BUCKET

# verify that the file was uploaded
aws s3 ls s3://$SOURCE_BUCKET

This Lambda function will extract labels from the image and store them in a DynamoDB table.

Upload another file:

export SOURCE_BUCKET=<enter source S3 bucket name - check the CDK output>
aws s3 cp ./skate.png s3://$SOURCE_BUCKET

Check the DynamoDB table in the AWS console - you should see the label detection results for both images.

DynamoDB table is designed with the source file name as the partition key and the (detected) label name as the sort key. This allows for a couple of query patterns:

• You can get all the labels for a given image.
• You can query for the metadata (category and confidence) for a specific source image and its label.

You can also use the CLI to scan the table:

aws dynamodb scan --table-name <enter table name - check the CDK output>

Don’t Forget To Clean Up

Once you're done, to delete all the services, simply use:

cdk destroy
#output prompt (choose 'y' to continue)
Are you sure you want to delete: RekognitionLabelDetectionGolangStack (y/n)?

You were able to set up and try the complete solution. Before we wrap up, let's quickly walk through some of the important parts of the code to get a better understanding of what's going on behind the scenes.

Code Walkthrough

We will only focus on the important parts - some of the code has been omitted for brevity.

CDK

You can refer to the complete CDK code here

bucket := awss3.NewBucket(stack, jsii.String("label-detection-input-bucket"), &awss3.BucketProps{
        BlockPublicAccess: awss3.BlockPublicAccess_BLOCK_ALL(),
        RemovalPolicy:     awscdk.RemovalPolicy_DESTROY,
        AutoDeleteObjects: jsii.Bool(true),
    })

We start by creating the source S3 bucket.

 table := awsdynamodb.NewTable(stack, jsii.String("label-detection-output-table"),
        &awsdynamodb.TableProps{
            PartitionKey: &awsdynamodb.Attribute{
                Name: jsii.String("source_file"),
                Type: awsdynamodb.AttributeType_STRING},
            SortKey: &awsdynamodb.Attribute{
                Name: jsii.String("label_name"),
                Type: awsdynamodb.AttributeType_STRING},
            TableName: jsii.String(*bucket.BucketName() + "_labels_output"),
        })

Then, we create a DynamoDB table to store the label data for each image.

function := awscdklambdagoalpha.NewGoFunction(stack, jsii.String("rekognition-function"),
        &awscdklambdagoalpha.GoFunctionProps{
            Runtime:     awslambda.Runtime_GO_1_X(),
            Environment: &map[string]*string{"TABLE_NAME": table.TableName()},
            Entry:       jsii.String(functionDir),
        })

    table.GrantWriteData(function)
    function.Role().AddManagedPolicy(awsiam.ManagedPolicy_FromAwsManagedPolicyName(jsii.String("AmazonRekognitionReadOnlyAccess")))
    bucket.GrantRead(function, "*")

Next, we create the Lambda function, passing the DynamoDB table name as an environment variable to the function. We also grant the function access to the DynamoDB table and the S3 bucket. We also grant the function access to the AmazonRekognitionReadOnlyAccess managed policy.

function.AddEventSource(awslambdaeventsources.NewS3EventSource(sourceBucket, &awslambdaeventsources.S3EventSourceProps{
        Events: &[]awss3.EventType{awss3.EventType_OBJECT_CREATED},
    }))

We add an event source to the Lambda function to trigger it when a new file is uploaded to the source bucket.

    awscdk.NewCfnOutput(stack, jsii.String("label-detection-input-bucket-name"),
        &awscdk.CfnOutputProps{
            ExportName: jsii.String("label-detection-input-bucket-name"),
            Value:      bucket.BucketName()})

    awscdk.NewCfnOutput(stack, jsii.String("label-detection-output-table-name"),
        &awscdk.CfnOutputProps{
            ExportName: jsii.String("label-detection-output-table-name"),
            Value:      table.TableName()})

Finally, we export the bucket and DynamoDB table names as CloudFormation output.

Lambda Function

You can refer to the complete Lambda Function code here

func handler(ctx context.Context, s3Event events.S3Event) {
    for _, record := range s3Event.Records {

        sourceBucketName := record.S3.Bucket.Name
        fileName := record.S3.Object.Key

        err := labelDetection(sourceBucketName, fileName)
    }
}

The Lambda function is triggered when a new image is uploaded to the source bucket. The function iterates through the list of files and calls the labelDetection function for each image.

Let's go through it.

func labelDetection(sourceBucketName, fileName string) error {

    resp, err := rekognitionClient.DetectLabels(context.Background(), &rekognition.DetectLabelsInput{
        Image: &types.Image{
            S3Object: &types.S3Object{
                Bucket: aws.String(sourceBucketName),
                Name:   aws.String(fileName),
            },
        },
    })

    for _, label := range resp.Labels {
        item := make(map[string]ddbTypes.AttributeValue)

        item["source_file"] = &ddbTypes.AttributeValueMemberS{Value: fileName}
        item["label_name"] = &ddbTypes.AttributeValueMemberS{Value: *label.Name}
        item["label_category"] = &ddbTypes.AttributeValueMemberS{Value: *label.Categories[0].Name}
        item["label_confidence"] = &ddbTypes.AttributeValueMemberN{Value: fmt.Sprintf("%v", aws.ToFloat32(label.Confidence))}

        _, err := dynamodbClient.PutItem(context.Background(), &dynamodb.PutItemInput{
            TableName: aws.String(table),
            Item:      item,
        })
    }

    return nil
}

• The labelDetection function uses the DetectLabels API to detect labels in the image.
• The API returns a list of labels, each with a confidence score.
• The function iterates through the list of labels and stores the label name, category and confidence score in the DynamoDB table.

Conclusion and Next Steps

In this post, you saw how to create a serverless solution that detects image labels using Amazon Rekognition. The entire infrastructure life-cycle was automated using AWS CDK. All this was done using the Go programming language, which is well-supported in AWS Lambda and AWS CDK.

Here are a few things you can try out to extend this solution:

• Build a solution to analyze videos stored in an S3 bucket.
• Even better, try processing streaming video to detect faces, objects etc.

Happy building!