Introduction

Cloud computing is the term everyone is talking about these days. If we use an online service to send an email, edit documents, watch movies or TV, listen to music, play games or store pictures and other files, cloud computing is likely making it all possible behind the scenes.

Here are a few of the things that can be done with the cloud:

• Create new apps and services
• Store, back up and recover data
• Host websites and blogs
• Stream audio and video
• Deliver software on demand
• Analyze data for patterns and make predictions

The impact of the cloud on testing practices has grown with the cloud’s growing presence in the IT industry. Testing practices are now dealing with several aspects of the cloud simultaneously. The strength of cloud computing's security barrier for user-protection and corporate compliance is crucial, especially if the company will store sensitive information in the system.

Cloud Computing

Cloud computing is a form of Internet-based computing that provides shared computer processing resources and data to computers and other devices on demand. It is a model for enabling global access to a shared pool of configurable computing resources (e.g., computer networks, servers, storage, applications, and services), which can be rapidly provisioned with minimal management effort.

Whether you are running applications that share photos to millions of mobile users or you’re supporting the critical operations of your business, a cloud services platform provides rapid access to flexible and low-cost IT resources. With cloud computing, you don’t need to make large upfront investments in hardware and spend a lot of time on the heavy lifting of managing that hardware. Instead, you can provision exactly the right type and size of computing resources you need to power your newest bright idea or operate your IT department. You can access as many resources as you need, almost instantly, and only pay for what you use. 

Cloud Testing

Cloud testing is the process of testing the performance, scalability, and reliability of web applications in a cloud computing environment.

The focus areas for cloud testing are performance and security testing. For performance testing the critical factors which should be considered are:

• Addressing the loads that will be applied by other clients/customers of the cloud provider.
• Addressing the loads that will be applied against internet providers.

To build successful software, both performance and security should be migrated to the cloud. 

Conventional vs. Cloud-Based Load Testing

Conventional Load Testing

• Performance testing setup time is generally minutes, days, weeks, or more;

• It cannot be scaled as large as cloud-based load testing;

• Costs are higher in the long run, but may seem to be cheaper when cloud-based load testing is still in its initial set up phases;

• Not flexible;

• No cases of data security challenges.

Cloud-Based Load Testing

• Cuts down its testing setup time from weeks to minutes;

• Enables large-scale and valuable real-time online testing for internet-based software in clouds;

• Easily leverage scalable cloud computing infrastructure to test and evaluate system (SaaS) performance and scalability;

• Reduced costs since it utilizes the same computing resources in the cloud(s) for its testing purposes;

• More flexibility;

• Data security challenges.

Distinct Features of Cloud-Based Load Testing

On-Demand — Cloud-based testing facility is accessible whenever it is needed by clients.

Simplicity — Setting up of end-to-end testing in the cloud can be relatively simple, given that the required servers and images are available for use in the creation of an end-to-end environment. Configuration can be set with easiness too.

Real-time testing — Cloud-based testing environments permit testers to analyze application performance, identifying bottlenecks even when tests are running.

Simulation Tests — These help in reducing the complexity as well as separating the quality concerns, plus it can focus on and analyses system behavior under multiple various scenarios.

Scalability — A cloud-based testing facility must be able to handle several requests, as rooted by clients, and must be able to either scale up or down depending on the number of requests at a particular point in time.

Infrastructure Reliability — Cloud-based testing facility must have a high level of availability to satisfy 24/7 continuous testing.

Tools and Frameworks — Various tools and frameworks do exist at various layers and in different forms to enhance cloud-based testing, and tools aids in automation.

Platform and Environment diversity — Cloud-based testing must provide a diverse environment along with computing platform to handle different requests from different clients using different platforms.

Billing model — Cloud-based testing is charged according to the predefined cost models, and in the form of a pay-per-use model.

Improved test quality — Cloud-based testing mechanisms put in place standardized pre-configured software images and infrastructure that are capable of eradicating the incorrect configuration of software test environments

Improved test efficiency — Using cloud-based testing, the time to set up a performance test environment can be greatly reduced.

More realistic performance testing — Performance tests and other different tests are more realistic when compared to on-premise

Time to market — Normally it is time-consuming to create on-premise test environments, and this might cause project delays. However, with cloud-based testing, this is not the case, since there will be standardized testing environments which can be used depending on the type of applications.

Cloud Load Testing

As cloud computing continues to develop, one is hard-pressed to identify a class of enterprise software that is not delivered and consumed as a service. Performance and load-based application testing, important parts of application lifecycle management (ALM), can be counted among these cloud offerings. But cloud-based testing also changes the way the tests themselves are performed.

 One of the biggest challenges in ALM is performance. Performance will make or break whether or not someone is going to use your app. If you think about the type of apps you use — enterprise or personal apps — performance is the determining factor, so make sure that performance is there and that you're able to test appropriately for performance. Moving performance and load-based application testing to the cloud brings cloud benefits to performance testing by lowering capital and operational costs, and support for distributed development and testing teams. It also allows you to simulate load tests constituting millions of concurrent users coming from multiple geographical locations. 

Reasons to Consider a Cloud Model

Performance testing demands extensive capital investments in hardware. Cloud is a good choice for organizations that do not want to have a full dedicated investment in testing infrastructure, as it fulfills all test environment needs and requirements.

If cloud computing is utilized effectively, it can help in achieving enough test coverage while The usage of the right environment for testing will also help in controlling the costs by enabling defect detection earlier in the life cycle. With a cloud-based model, the testing team is provided with a test lab for the required timeframe. Organizations can shorten the provisioning time as the cloud enables provisioning of test servers on an on-demand basis.  

Load Testing Best Practices

The following best practices can help maximize the advantages and minimize the challenges of load testing with the cloud. 

Two-Stage Process

In the first stage of the process, conduct internal tests with a medium load to quickly identify and resolve preliminary performance issues. Then, increase the load incrementally, then proceed to the second stage, cloud-based load testing, for large scale tests that validate the entire delivery chain of the application. This approach enables teams to isolate problems. The source of any performance issue identified in the first stage is clearly within the firewall (because no other systems are involved in the test). With the two-stage process, one doesn't have to wait for the application to be deployed and accessible from the Internet to test it. This lowers costs. When you can test internally on the hardware you already have, you can reduce the amount of testing that you need to perform from the cloud and cut costs. 

Data Security

Encrypt the communication between your controller and load generators. This helps secure data sent to the load generators during the test (including account information) as well as the data that is retrieved (including error messages). Lastly, ensure that the load generators are secured with their firewalls to protect them from outside threats. To ensure complete data security, we can add an OTP check.

Using Different Cloud Providers

There are several advantages to using multiple cloud providers. First, it helps to test from more geographical regions, which provides more realistic results that capture the effects of various third-party servers and content delivery networks. Second, it is more scalable. For exceptionally large scale tests, one can engage multiple providers simultaneously to bypass limitations that a single provider may place on bandwidth or the number of machines in use. Third, detects potential network issues at the cloud provider level. Load testing solutions that are locked into a single provider limit the test engineer’s ability to conduct realistic, reliable, large-scale tests. 

Tune Load Generators

To ensure that the load generator machines in the cloud are capable of generating large loads, one must properly tune the system to support the creation of a high number of sockets and threads per process. Additionally, allocate an appropriate heap size for Java-based load generators.

Monitor the Servers

Once a performance bottleneck is identified, the information to track down its root cause is required. This information should be gathered during the test by monitoring each component of the infrastructure, including application servers, database servers, and so on. Specifically, you should monitor both the system (including the operating system, disks, and network) and the server software (including connection pools, threads, cache hits, and indexes. Linking all the information gathered during the tests with the tests themselves is much easier when the monitoring is integrated with the load testing tool. This enables us to correlate the response times and errors generated by load testing with the monitored data to track down the cause of problems quickly. 

Deciding Factors

• Application Under Test

• Communication Protocols

• Virtual Users

• Virtual Users locations

• Tester Skillset

• Existing test assets

• Script complexities

• Test Data Requirements

• Security

• Budget

• Timescales

• Which cloud to use

Case Study

The Indian railway system is physically distributed and managed at a regional level, and the information system of Indian Railway is a highly complex distributed mission-critical system, integrated into networks. There are 120,000 tickets generated every day in Indian Railway, each 3KB in size. The tracking period is normally 45 days, and the ticket needs to be stored online for one year. Therefore, 160GB data is generated per day and the amount of data in the ready-to-be-searched state is over 1 Terabyte. Also, 4-6 replicas of the data are required, and must be considered. A cloud bed in this system can offer key advantages including efficiency, scalability, reliability, and flexibility. 

Conclusion

Load testing in the cloud isn’t much different from traditional environments, but it does require additional planning and communication. Most of the risks associated with the cloud come from non-functional requirements not being met or often not even being articulated and therefore not being tested or supported. To avoid such risks a proper plan for load and security testing should be in place. Try to gather as much information as possible ahead of time, and work with the smallest amount of contacts at the service provider as possible. This helps ensure a more legitimate, normal environment. 

If one is unable to test any portion of the cloud, make sure to get configuration files, security audit policies, and previous load testing results from the provider. Read the Service Level Agreement (SLA) thoroughly and make sure it is covered in the case of a security breach. Security in the cloud is still a combination of careful planning and constant vigilance.  

Further Reading