Designing High Performant Responsive Web Application With AWS Services and Finetuning for Performance

The advent of multiple asynchronous and event-driven technologies has influenced a paradigm shift in the traditional three-tier web architecture infusing changes in each layer of the architecture. Frontend web development has been supported by React JS, Micro Frontend jQuery, etc. The unique requirements to differentiate the user experience in the web application require significant architectural shifts in the controller, business, and backend layer and most importantly require an event-driven and asynchronous approach. In modern global user experience applications typically used by front desk or customer online applications, these architectural shifts are ubiquitous. This article provides strategic guidance for architectural decisions on choosing AWS services and further finetuning the performance of such responsive and performant web applications based on some real and practical production scale solution developments and deployment experience.

Solution Approach

Some of the key quality of service or nonfunctional requirements for these applications as follows drive the architecture decisions around choosing the right services in the architecture layers.

• High throughput
• Low latency
• Seamless Data Access
• Global Deployment
• Cost Efficiency
• Serverless

Based on the experiences of multiple such web application architecture approaches, we suggest the following opinionated architecture that has worked for the multifaceted requirements mentioned above.

Architecture Template for Key AWS Services Decisions

• Use Amazon CloudFront with Amazon S3 to host the web application.
• Use AWS AppSync to build the application APIs.
• Use AWS MSK to get the different events from event source applications. 

Architecture Overview:

In the architectural framework, an event-driven microservice is employed to construct the business capability. This microservice subscribes to events in a Kafka topic and subsequently exposes them to frontend channels via a WebSocket using the Publish/Subscribe model, facilitated through AWS AppSync.

The architecture comprises event-driven microservices implemented using Java Spring boot, and Kafka Stream API, deployed as a container in AWS Red Hat OpenShift (ROSA). The event-driven microservice is integrated with AWS AppSync.

Key Architectural Components

AWS Streaming Data Service:

Amazon Managed Streaming for Apache Kafka (Amazon MSK) is an AWS streaming data service designed to manage Apache Kafka infrastructure and operations. Amazon MSK excels in operating, maintaining, and scaling Apache Kafka clusters, offering robust security features, and providing built-in AWS integrations that expedite the development of streaming data applications. The key components of Amazon MSK include Broker Nodes, Zookeeper Nodes, Producers, Consumers, Topic Creators (Kafka Data Plane), and Cluster Operations via AWS Management Console, AWS CLI, and SDK APIs.

For microservice-style application development, we considered three Kafka options:

1. Producer and Consumer API

It provides a straightforward way to consume messages from Kafka topics.

2. Kafka Streams API (Recommended)

The Kafka Streams API is ideal for real-time data stream processing. This API enables data transformations, aggregations, and other operations on data streams, producing results for new Kafka topics. It is commonly used to construct stream processing applications and microservices, available through a Java library. Notable features include its ability to create highly scalable, elastic, fault-tolerant, and distributed applications for critical real-time use cases. The API capitalizes on key stream processing concepts like efficient application state management, rapid and efficient aggregations and joins, and proper handling of event-time vs. processing-time distinctions. 

3. Spring Cloud Streams

Spring Cloud Stream builds on top of Spring Boot and integrates with Apache Kafka to enable seamless communication between microservices through messaging. Spring Cloud Stream provides features like “Binder” for building event-driven microservices in a Spring Boot environment. Also, Spring Cloud Stream provides automatic serialization and deserialization of messages based on the content types, simplifying the handling of messages with different formats.

Connect API:

The Kafka Connect API is employed for integrating Kafka with external data sources and sinks. Connectors can be configured to stream data between databases, message queues, other data systems, and Kafka topics.

Kafka Streams Data Storage:

While Kafka Streams API effectively manages state, we recommend using AWS Managed storage services for data storage. These services ensure high availability, built-in fault tolerance, and scalability.

 Microservice and Agent Application UI:

The Microservice is subscribed to Kafka Topic. It ingests the desired message from the Kafka topic, applies business logic, and uses HTTP clients to connect to the GraphQL endpoint (https://) for invoking mutation APIs, while the AGENT APPLICATION UI uses subscriptions via the Real-Time endpoint (wss://). Integration with DynamoDB is achieved through DynamoDB resolvers, enabling GraphQL APIs to store data in existing Amazon DynamoDB tables.

 AWS AppSync:

AWS AppSync is a fully managed service that allows to deployment of Serverless GraphQL backends in the AWS cloud. The supported request types are queries (for getting data from the API), mutations (for changing data via the API), and subscriptions (long-lived connections for streaming data from the API). It auto-scales the GraphQL API execution engine based on request volumes, offering managed GraphQL subscriptions for real-time data updates over Websockets. AppSync provides real-time subscriptions and offline access, synchronizing updates when devices reconnect.

The AWS::AppSync::DataSource resource creates data sources for resolvers in AWS AppSync to connect to, such as Amazon DynamoDB.

Key Performance Tuning Techniques

AWS AppSync

The problem in the resolver has been identified after analyzing the logs generated by AWS AppSync, the data access time is significantly higher than expected thus impacting the API's overall performance. 

{
  "requestID": "12352289",
  "query execution time": "500ms",
  "resolver": "GetUserData",
  "DataSource": "DynamoDB",
  "dataAccessTime": "980ms",
  "cachet": "false",
  "error": "false",
  "message": "Resolver 'GetUserData' experienced high execution time",
  "additional details": "The resolver 'GetUserData' is fetching data from DynamoDB and took 980ms, which is significantly higher than expected. This may impact the overall performance of the API.",
  "request": {
    "query": "query GetUser($userID: ID) { user(userID: $userID) { name email }"
  },
  "variables": {
    "userID": "99345"
  }
}

The following measures have been taken to improve the performance of AWS AppSync

Steps To Improve Performance:

1. Caching Strategies: Implement caching strategies, including in-memory caching and integration with services like AWS DynamoDB or Amazon ElastiCache, to reduce the load on your GraphQL API and enhance response times. AWS AppSync provides server-side data caching that accelerates performance and reduces latency by storing data in a high-speed, in-memory cache
2. Batching and Collapsing: Leverage the built-in batch and defer features of AWS AppSync to combine multiple queries and mutations into a single request, reducing latency.
3. Dataloader Pattern: Implement the dataloader pattern to batch and cache database queries, reducing database load and enhancing response times.
4. Throttling and Request Limiting: Configure requests throttling and limiting to prevent overloading your AWS AppSync API, ensuring fair usage and resource management.
5. Compression: AWS AppSync offers the capability to compress API responses, enhancing the speed of loading and downloading payload content. This feature not only potentially lightens the load on your applications but also has the potential to lower data transfer costs.

Subscription Implementation:

The Web APPLICATION UI subscribes to real-time events via secure web sockets (wss://) and leverages two subscriptions—one for real-time events and the other for Init state sync up.

Use Cases:

• If the Web APPLICATION UI reconnects to AppSync within 5 minutes, it retains the session and syncs mutation events from the last 5 minutes.
• If the agent is away for more than 5 minutes, the browser disconnects, and upon reconnection, the Init state sync-up is initiated.

Subscription Library:

We propose using the Apollo WebSocket client library in Web APPLICATION UI for both low-level and high-level WebSocket abstractions.

Solution for AWS MSK Performance Optimization:

Monitor producer and consumer request quotas and adjust as needed to prevent throttling, which can impact cluster performance, causing reduced throughput, increased latency, error rates, inefficient resource utilization, contention, and operational challenges. Mitigate throttling by proper monitoring, capacity planning, request optimization, retry strategies, and awareness of AWS service quotas.

AWS AppSync Delta Sync Logging:

AWS AppSync supports Delta Sync Logging for mutations, allowing the recording of changes to versioned data sources, and optimizing incremental updates. The Delta Sync table is used to implement Delta Sync, facilitating the sync process with a single resolver and using two DynamoDB tables—Base and Delta—for partitioning sync queries.

Note: Amplify DataStore was ruled out due to security policy constraints enforced by the security team. Amplify DataStore offers a programming model for working with distributed data in offline and online scenarios.

The high-level design for AWS AppSync and Delta Sync logging is shown below.

Conclusion

While designing a solution for highly responsive, throughput, and low latency global web applications it is highly explicative to make the right architecture choices of selecting the AWS services that support the nonfunctional requirements. It is also evident that each of the AWS service designs and service implementations is finetuned for low latency, high throughput, and performance.