Dynamic Data Processing Using Serverless Java With Quarkus on AWS Lambda (Part 1)
This step-by-step tutorial explores how Quarkus enables Java developers to implement serverless functions on AWS Lambda to process dynamic data on AWS DynamoDB.
CORE
·
Join the DZone community and get the full member experience.
Join For FreeWith the growth of the application modernization demands, monolithic applications were refactored to cloud-native microservices and serverless functions with lighter, faster, and smaller application portfolios for the past years. This was not only about rewriting applications, but the backend data stores were also redesigned in terms of dynamic scalability, high performance, and flexibility for event-driven architecture. For example, traditional data structures in relational databases started to move forward to a new approach that enables to storage and retrieval of key-value and document data structures using NoSQL databases.
However, faster modernization presents more challenges for Java developers in terms of steep learning curves about new technologies adoption and retaining current skillsets with experience. For instance, Java developers need to rewrite all existing Java applications to Golang and JavaScript for new serverless functions and learn new APIs or SDKs to process dynamic data records by new modernized serverless applications.
This article will take you through a step-by-step tutorial to learn how Quarkus enables Java developers to implement serverless functions on AWS Lambda to process dynamic data on AWS DynamoDB. Quarkus enables developers not only to optimize Java applications for superfast startup time (e.g., milliseconds) and tiny memory footprints (e.g., less than 100 MB) for serverless applications, but developers can also use more than XX AWS extensions to deploy Java applications to AWS Lambda and access AWS DynamoDB directly without steep learning curves.
Creating a New Serverless Java Project Using Quarkus
We’ll use the Quarkus command to generate a new project with required files such as Maven Wrapper, Dockerfiles, configuration properties, and sample code. Find more information about the benefits of the Quarkus command (CLI) here.
Run the following Quarkus command in your working directory.
quarkus create piggybank --java=17You need to use the JDK 17 version since AWS Lambda currently supports JDK 17 as the latest version by default Java runtime (Corretto).
Let’s start Quarkus Live Coding, also known as quarkus dev mode, using the following command.
cd piggybank && quarkus dev modeDeveloping Business Logic for Piggybank
Now let's add a couple of Quarkus extensions to create a DynamoDB entity and relevant abstract services using the following Quarkus command in the Piggybank directory.
quarkus ext add amazon-dynamodb resteasy-reactive-jacksonThe output should look like this.
[SUCCESS] ✅ Platform io.quarkus.platform:quarkus-amazon-services-bom has been installed
[SUCCESS] ✅ Extension io.quarkiverse.amazonservices:quarkus-amazon-dynamodb has been installed
[SUCCESS] ✅ Extension io.quarkus:quarkus-resteasy-reactive-jackson has been installedCreating an Entity Class
You will create a new data model (entry.java) file to define Java attributes that map into the fields in DynamoDB. The Java class should look like the following code snippet (you can find the solution in the GitHub repository):
@RegisterForReflection
public class Entry {
public Long timestamp;
public String accountID;
...
public Entry() {}
public static Entry from(Map<String, AttributeValue> item) {
Entry entry = new Entry();
if (item != null && !item.isEmpty()) {
entry.setAccountID(item.get(AbstractService.ENTRY_ACCOUNTID_COL).s());
...
}
return entry;
}
...
}The @RegisterForReflectionannotation instructs Quarkus to keep the class and its members during the native compilation. Find more information here.
Creating an Abstract Service
Now you will create a new AbstractService.java file to consist of helper methods that prepare DynamoDB to request objects for reading and adding items to the table. The code snippet should look like this (find the solution in the GitHub repository):
public class AbstractService {
public String accountID;
...
public static final String ENTRY_ACCOUNTID_COL = "accountID";
...
public String getTableName() {
return "finance";
}
protected ScanRequest scanRequest() {
return ScanRequest.builder().tableName(getTableName())
.attributesToGet(ENTRY_ACCOUNTID_COL, ENTRY_DESCRIPTION_COL, ENTRY_AMOUNT_COL, ENTRY_BALANCE_COL, ENTRY_DATE_COL, ENTRY_TIMESTAMP, ENTRY_CATEGORY).build();
}
...
}Adding a Business Layer for REST APIs
Create a new EntryService.java file to extend the AbstractService class that will be the business layer of your application. This logic will store and retrieve the entry data from DynamoDB synchronously. The code snippet should look like this (solution in the GitHub repository):
@ApplicationScoped
public class EntryService extends AbstractService {
@Inject
DynamoDbClient dynamoDB;
public List<Entry> findAll() {
List<Entry> entries = dynamoDB.scanPaginator(scanRequest()).items().stream()
.map(Entry::from)
.collect(Collectors.toList());
entries.sort((e1, e2) -> e1.getDate().compareTo(e2.getDate()));
BigDecimal balance = new BigDecimal(0);
for (Entry entry : entries) {
balance = balance.add(entry.getAmount());
entry.setBalance(balance);
}
return entries;
}
...
}Creating REST APIs
Now you'll create a new EntryResource.java file to implement REST APIs to get and post the entry data from and to DynamoDB. The code snippet should look like the below (solution in the GitHub repository):
@Path("/entryResource")
public class EntryResource {
SimpleDateFormat piggyDateFormatter = new SimpleDateFormat("yyyy-MM-dd+HH:mm");
@Inject
EntryService eService;
@GET
@Path("/findAll")
public List<Entry> findAll() {
return eService.findAll();
}
...
}Verify the Business Services Locally
First, we need to install a local DynamoDB that the piggy bank services access. There’re a variety of ways to stand up a local DynamoDB such as downloading an executable .jar file, running a container image, and deploying by Apache Maven repository. Today, you will use the Docker compose to install and run DynamoDB locally. Find more information here.
Create the following docker-compose.yml file in your local environment.
version: '3.8'
services:
dynamodb-local:
command: "-jar DynamoDBLocal.jar -sharedDb -dbPath ./data"
image: "amazon/dynamodb-local:latest"
container_name: dynamodb-local
ports:
- "8000:8000"
volumes:
- "./docker/dynamodb:/home/dynamodblocal/data"
working_dir: /home/dynamodblocalThen, run the following command-line command.
docker-compose upThe output should look like this.
[+] Running 2/2
⠿ Network quarkus-piggybank_default Created 0.0s
⠿ Container dynamodb-local Created 0.1s
Attaching to dynamodb-local
dynamodb-local | Initializing DynamoDB Local with the following configuration:
dynamodb-local | Port: 8000
dynamodb-local | InMemory: false
dynamodb-local | DbPath: ./data
dynamodb-local | SharedDb: true
dynamodb-local | shouldDelayTransientStatuses: false
dynamodb-local | CorsParams: null
dynamodb-local | Creating an Entry Table Locally
Run the following AWS DynamoDB API command to create a new entry table in the running DynamoDB container.
aws dynamodb create-table --endpoint-url http://localhost:8000 --table-name finance --attribute-definitions AttributeName=accountID,AttributeType=S AttributeName=timestamp,AttributeType=N --key-schema AttributeName=timestamp,KeyType=HASH AttributeName=accountID,KeyType=RANGE --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 --table-class STANDARDAdding DynamoDB Clients Configurations
DynamoDB clients are configurable in the application.properties programmatically. You also need to add to the classpath a proper implementation of the sync client. By default, the extension uses the java.net.URLConnection HTTP client.
Open the pom.xml file and copy the following dependency right after the quarkus-amazon-dynamodb dependency.
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>url-connection-client</artifactId>
</dependency>Then, add the following key and value to the application.properties to specify your local DynamoDB's endpoint.
%dev.quarkus.dynamodb.endpoint-override=http://localhost:8000Starting Quarkus Live Coding
Now you should be ready to verify the Piggybank application using Quarkus Dev mode and local DynamoDB.
Run the Quarkus Dev mode using the following Quarkus command.
quarkus devThe output should end up this.
__ ____ __ _____ ___ __ ____ ______
--/ __ \/ / / / _ | / _ \/ //_/ / / / __/
-/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
[io.quarkus] (Quarkus Main Thread) Profile dev activated. Live Coding activated.quarkus xx.xx.xx.) s2023-04-30 21:14:49,824 INFO [io.quarkus] (Quarkus Main Thread) Installed features: [amazon-dynamodb, cdi, resteasy-reactive, resteasy-reactive-jackson, smallrye-context-propagation, vertx]
--
Tests paused
Press [r] to resume testing, [o] Toggle test output, [:] for the terminal, [h] for more options>Run the following curl command to insert several expense items into the piggybank account (entry table).
curl -X POST http://localhost:8080/entryResource -H 'Content-Type: application/json' -d '{"accountID": "Food", "description": "Shrimp", "amount": "-20", "balance": "0", "date": "2023-02-01"}'
curl -X POST http://localhost:8080/entryResource -H 'Content-Type: application/json' -d '{"accountID": "Car", "description": "Flat tires", "amount": "-200", "balance": "0", "date": "2023-03-01"}'
curl -X POST http://localhost:8080/entryResource -H 'Content-Type: application/json' -d '{"accountID": "Payslip", "description": "Income", "amount": "2000", "balance": "0", "date": "2023-04-01"}'
curl -X POST http://localhost:8080/entryResource -H 'Content-Type: application/json' -d '{"accountID": "Utilities", "description": "Gas", "amount": "-400", "balance": "0", "date": "2023-05-01"}'Verify the stored data using the following command.
curl http://localhost:8080/entryResource/findAllThe output should look like this.
[{"accountID":"Food","description":"Shrimp","amount":"-20","balance":"-30","date":"2023-02-01"},{"accountID":"Drink","description":"Wine","amount":"-10","balance":"-10","date":"2023-01-01"},{"accountID":"Payslip","description":"Income","amount":"2000","balance":"1770","date":"2023-04-01"},{"accountID":"Car","description":"Flat tires","amount":"-200","balance":"-230","date":"2023-03-01"},{"accountID":"Utilities","description":"Gas","amount":"-400","balance":"1370","date":"2023-05-01"}]You can also find a certain expense based on accountID. Run the following curl command again.
curl http://localhost:8080/entryResource/find/DrinkThe output should look like this.
{"accountID":"Drink","description":"Wine","amount":"-10","balance":"-10","date":"2023-01-01"}Conclusion
You learned how Quarkus enables developers to write serverless functions that connect NoSQL databases to process dynamic data. To stand up local development environments, you quickly ran the local DynamoDB image using the docker-compose command as well. Quarkus also provide various AWS extensions including amazon-dynamodb to access the AWS cloud services directly from your Java applications. Find more information here.
In the next article, you’ll learn how to create a serverless database using AWS DynamoDB and build and deploy your local serverless Java functions to AWS Lambda by enabling SnapStart.
Opinions expressed by DZone contributors are their own.
Comments