Performance Testing AWS Deployments

Using a multitude of components to test the infrastructure at every level, check out how this testing stack can monitor AWS performance.

Shreyas Chaudhari

Updated May. 22, 19 · Tutorial

Likes (7)

Comment

Save

13.1K Views

Below is the performance testing strategy used for implementation using JMeter, BlazeMeter and Selenium Webdriver.

Goals

Ninety-nine percent of the page-load times for the users accessing the applications should be less than or equal to two seconds.
Users are based out of continents — North America (New York), Europe (London), Asia (Singapore).
N concurrent users should be able to use the application without any page-load lags.

Image title

Components

Third Party Systems — Researchers gather the data.
SQL Server —
1. Researched Info Database
2. Identity Provider Database
Event Messenger — Component that picks up data from the Researched Info Database and puts it in the Event Broker.
Event Processor — Components that are listening to the Event Broker, who pick up the data from the Event Broker and push it into MongoDB and Elastic Search.
Middle Tier — Services that pull the data from MongoDB and the Elastic Search and send it to the Front End in the form of REST APIs.
Front End — Data entered by the Researchers to be displayed.

Architecture

Researchers enter the data in the Third Party Systems.
Data is thereby stored in SQL Database —Researched Database.
A dedicated table is created in the Researched Info Database mentioned in Step 2, wherein the data entered in the original schema of the Researched Info Database is stored in the format for Event Sourcing.
Event Messenger does the job of picking up the data from the dedicated table from Researched Info Database in Step 3 and pushes it to the Event Broker.
Event Processors pick up the data in the Event Broker and push it to MongoDB and Elastic Search.
On the other end, there is a service-oriented architecture, in which each of the services fetches the data from the Elastic Search and provides the data to the front end in the form of REST APIs.
The front-end is a single-page application that consumes the data sent by REST APIs and renders the page.
There is a dedicated Identity provider, which manages the identity of the user across all the applications the user has access to as a part of the subscription. The database that holds details related to the user identity is present in the SQL Database mentioned in Step 2.

Technology Stack

Event Messenger / Event Processor / Services — .NET Core
Event Store [Event Broker]
MongoDB
Elastic Search
Amazon Elastic Search Service
ReactJS [User Interface]
Amazon EC2
TeamCity

Test Environment Configuration

The configuration of the production-like environment that we used for running our performance tests was as shown in the table below. Details about each of the Instance types can be found here.

Image title

Performance Testing Tools

Performance Test Types

Load Test — It is conducted for validating the performance characteristics of the system when subjected to workload/load volume that is anticipated during the production load. Conducting this type of test before releasing the system to the market provides confidence and mitigates the risk of losing business due to performance issues.
Spike Test — It is conducted to find out the stability of the system when it is loaded in bursts of very small time, releasing the load quickly.
Endurance Test — It is conducted to find out whether the system is capable of handling the expected load without any deterioration of response time/throughput when run for a longer time. Conducting this type of test before releasing to the market provides confidence on the availability and stability of the system.
Page Load Test — It is conducted to compute the page load times from the end user experience—when the desired number of concurrent users generates a specific amount of load.

Strategy

Replicating the load for n concurrent users is the first task, especially in the scenarios wherein the number of concurrent users will be on the higher side. There are two approaches for achieving this:

Replicating the end-user behavior by having automated UI tests that perform operations on the application as the real world user would do. Using a browser automation library like Selenium Web Driver for writing UI automation, which in turn will mimic the end-user behavior, is an option. This can be a Selenium Grid setup, which will drive user flows on multiple machines.
The second approach is wherein load can be generated on the server for the n-1 concurrent users using REST APIs. Then have just one user to execute the UI test flows and thereby compute the page load times.

There may be pros and cons for taking up either of these approaches. However, we opted for the latter.

JMeter Test — REST APIs

Let’s take the example of all the REST API calls made when a specific page loads, as in the snapshot below.

The JMeter script for the corresponding page will be as in the snapshot below. It consists of all the REST APIs chained together when a specific page loads.

The JMX file in the snapshot above is one-to-one mapping for every API call made in the network tab for the corresponding page load. Multiple such JMX files were created, which consist of REST APIs chained when the specific pages loaded.

Selenium Web Driver — Page Load Test

The UI test would navigate to the specific page in the application; note the timestamp before clicking on the page, then click on the link, and then note the timestamp when the page load is completed. This script used to wait for the entire page to be loaded— left to right and top to bottom.

import org.openqa.selenium.By;
import org.openqa.selenium.TimeoutException;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.Test;

import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.TimeUnit;

public class ComputePageLoadTime {

private static WebDriver driver = null;
private static String fileSeparator = File.separator;
private static String resourcesPath = System.getProperty("user.dir") + fileSeparator + "src" + fileSeparator
+ "main" + fileSeparator + "resources" + fileSeparator;
private static String resultsDirectoryPath = System.getProperty("user.dir") + fileSeparator + "results"
+ fileSeparator;
private static String configPropertiesFileName = "config.properties";
private static String currentSessionResultsDirectoryPath = "";
private static String reportName = "";
private static Properties configProperties = null;
private static Properties testProperties = null;
private static String testEnv = "";

private void readProperties(String propertiesFileName) {

configProperties = new OrderedProperties();
InputStream inputStream = getClass().getClassLoader().getResourceAsStream(configPropertiesFileName);

try {

configProperties.load(inputStream);

} catch (IOException e) {

e.printStackTrace();

}

if (System.getProperty("env") != null) {

testEnv = System.getProperty("env");

}else {

testEnv = configProperties.get("env").toString();

}

testProperties = new OrderedProperties();
inputStream = getClass().getClassLoader().getResourceAsStream(propertiesFileName);

try {

testProperties.load(inputStream);

} catch (IOException e) {

e.printStackTrace();
}

reportName = testProperties.get("ReportName").toString();

}

private void configureDriver() {

String operatingSystem = System.getProperty("os.name").toLowerCase();
String chromeDriverPath = "";

if (operatingSystem.contains("win")) {

chromeDriverPath = configProperties.getProperty("chromedriverWindows");

}else {

System.exit(1);

}

System.setProperty("webdriver.chrome.driver", resourcesPath + chromeDriverPath);

driver = new ChromeDriver();
driver.manage().window().maximize();

}

private void createResultsDirectory() {

DateFormat dateFormat = new SimpleDateFormat("dd-MM-yyyy-HH-mm-ss");
Date date = new Date();

currentSessionResultsDirectoryPath = resultsDirectoryPath + reportName + "_" + dateFormat.format(date)
+ fileSeparator;

File file = new File(currentSessionResultsDirectoryPath);
file.setWritable(true);

try {

if (!Files.exists(Paths.get(resultsDirectoryPath)))

Files.createDirectory(Paths.get(resultsDirectoryPath));

Files.createDirectory(Paths.get(currentSessionResultsDirectoryPath));

} catch (IOException e) {

e.printStackTrace();

}

}

@BeforeSuite
public void setupTheAutomationEnvironment(inputTestDataPropertiesFileName) {

readProperties();

String inputTestDataPropertiesFileName = System.getProperty("testDataPropertiesFileName",
"testData.properties");

createResultsDirectory();

}

@Test
public void loadPagesAndComputeLoadTime() {

int numLoops = Integer.parseInt(System.getProperty("loops", "20"));
int pageLoadTimeout = Integer.parseInt(System.getProperty("pageLoadTimeout", "20"));
List<List<String>> allResultsData = new ArrayList<>();
List<List<String>> resultsData = new ArrayList<>();
List<String> headers = Arrays.asList("Test Case", "URL", "Load time (ms)");
String allResultsFilePath = currentSessionResultsDirectoryPath + "Results.csv";
String loginUrl = configProperties.getProperty("applicationUrl");
String userName = configProperties.getProperty("userNameStr");
String password = configProperties.getProperty("passwordStr");

resultsData.add(headers);

for (int i = 0; i < numLoops; i++) {

String outputFileName = "Run" + i + ".csv";
Enumeration<String> enums = (Enumeration<String>) testProperties.propertyNames();

while (enums.hasMoreElements()) {

String key = enums.nextElement();
String value = testProperties.getProperty(key);

if (value.contains("https")) {

configureDriver();

driver.get(loginUrl);
driver.findElement(By.xpath(configProperties.getProperty("userName"))).sendKeys(userName);
driver.findElement(By.xpath(configProperties.getProperty("password"))).sendKeys(password);
driver.findElement(By.xpath(configProperties.getProperty("rememberMeBtn"))).click();
driver.findElement(By.xpath(configProperties.getProperty("submitBtn"))).click();

String url = testProperties.getProperty(key).replace("env", testEnv);
long startTime = System.currentTimeMillis();
long endTime, totalLoadTime;
driver.get(url);
WebDriverWait wait = new WebDriverWait(driver, pageLoadTimeout);
String pageLoadElementWorkdayIconXpath = "//div[@id='workDayIcon']";
List<String> currentLoopResults = new ArrayList<>();

try {

wait.until(ExpectedConditions.or(ExpectedConditions
.visibilityOfElementLocated(By.xpath(pageLoadElementWorkdayIconXpath))));
endTime = System.currentTimeMillis();
totalLoadTime = endTime - startTime;
currentLoopResults = Arrays.asList(key, url, String.valueOf(totalLoadTime));

} catch (TimeoutException e) {

currentLoopResults = Arrays.asList(key, url, String.valueOf(-1));
resultsData.add(currentLoopResults);
allResultsData.add(currentLoopResults);
driver.close();
continue;

}

resultsData.add(currentLoopResults);
allResultsData.add(currentLoopResults);
driver.close();

}

}

String currentSessionResultFilePath = currentSessionResultsDirectoryPath + outputFileName;
CSVUtils.writeToCsv(currentSessionResultFilePath, resultsData);

}

CSVUtils.writeToCsvNormalized(allResultsFilePath, allResultsData);

}

@AfterSuite
public void teardownTheAutomationEnvironment() {

if (driver != null) {

driver.quit();

}

}

}

The overall performance test setup was as in the snapshot below:

The performance test setup consisted of the BlazeMeter REST API tests mimicking the behaviour of n-1 users. The users were equally distributed among London, New York, and Singapore. At the same time, there were AWS EC2 instances spawned in the same three locations. There was a Selenium Web Driver test executed on each of the EC2 instances, which would help to compute the page load times. The Selenium Web Driver tests would perform the following steps: