Performance Testing of Mobile — How Is This Different?

Below is a quick start guide on performance testing a mobile application.

To begin, learn how your end-users access applications on their devices and then establish an expectation from performance testing. Once users’ needs are identified, create a strategy document to simulate realistic load conditions.

Almost all performance testing tools available in the industry support performance testing of Mobile apps — native, hybrid or app accessed via browser from the device. Most tools come with out-of-the-box features to conduct tests irrespective of the communication protocol (HTTP or HTTPs).

You may also like: Six Must-Know Performance Testing Criteria for Mobile Applications

As modern applications get complex and feature-rich, it is pivotal to build a clear strategy to overcome challenges based on the tool you choose.

Below are few prerequisites to help you develop a detailed test strategy —

• Type of app – native or hybrid; communication protocol.
• Security features – almost all communication via devices is secured using security encryption logic.
• Network conditions.
• Virtual users, real users; distributed the load.
• Device-level (client-side) performance testing.
• Server-side performance testing.
• Types of testing and Load conditions (workload modeling).
• Business-critical scenarios and test data.
• Availability of devices – physical and cloud-hosted.
• Application monitoring.
• Features that may need to be bypassed or disabled such as OTP/Captcha etc.
• Test data and Load test environment.
• Back-up/restore of assets.

Once a good test strategy is defined, the phases that would follow will be planning, development, execution, and Analysis.

Following are some of our recommend approaches for the above phases —

Scripting 

Choose a proxy recording feature provided by load testing tools. The steps are simple and straight forward. Set a proxy on the device and ensure both the tool and the device are on the same subnet. Once done, start capturing the traffic. If your application security features restrict you to capture the traffic, consider using packet capture tools from the play store or app store.

Export the network capture to other tools. You may use JMeter, LoadRunner, Fiddler, Blazemeter, Wireshark or any packet capture tool to record the HTTP calls from the client to the server. You may also record each for Android and IOS.

If the application uses SOA, REST or Jason calls, seek help from the application vendor or developers to assist you in understanding the various HTTP request formats (HTTP GET, POST, PUT) and parameters. These would be useful when dealing with applications that do not allow you to capture traffic using various tools, for security reasons.

Script Enhancements

In this phase, test using parameters instead of hardcoded values. Check the need for changing the header file (If you prefer to simulate load from both Android and iOS). Ensure all dynamic parameters are correlated and properly handled to ensure the script works for multiple users for various test data.

One of the key parameters is to simulate realistic network conditions such as 2G, 3G, 4G LTE, etc. Several tools have built-in features to simulate these conditions. Some tools allow you to make changes to the configuration file and so it’s critical to consider the right tool depending on your requirements.

Execution

In this phase, set-up a test environment using controller and agents to support the desired type of test and volume of users. If you are keen to simulate load from various geo-locations, set-up your load test agents to simulate similar behavior.

We recommend running a few sanity dry runs for a few users to ensure that there are no issues with the implementation.

Ensure your end-users or BA’s access the application during peak load conditions to help you monitor application behavior from an end user’s perspective. Request your team members to perform a few business transactions and record their experiences.

As most tools do not include rendering time or client-side processing in the response time, it’s a good practice to ask someone familiar with the application to perform a few transactions manually or run tests that work at the GUI layer.

We recommend involving all key stakeholders (BAs, dev, and infra) during the load test execution. Most issues or bottlenecks identified during execution are much easier to fix when all key members are involved during the formal load test runs.

If these processes are remotely coordinated, get into a conference call to ensure each layer of the application is proactively monitored.

You may integrate the APM — Application Performance Monitoring tool with a load testing tool to get a detailed breakdown of transaction response times. Though this would be a time-consuming process, it is an effective way to discover application issues, which could be fixed immediately.

Analysis and Reporting

Spend a considerable amount of time in this phase to determine how systems behaved before load test, during load tests and post-load tests. We recommend you to seek assistance from DBA’s or the concerned team to help you gauge the number of transactions that were performed during the load test. The data pattern would indicate any performance degradation.

We recommend you to create separate reports for your target audience and stakeholders. The reason being, while your business team may be interested in end-user experience or response time during a peak load, the IT infrastructure team would be more interested to know about the server utilization.

Create detailed reports for people who can help you assess the system and can work with you in fixing bottlenecks.

So Finally, How Is This Different?

If you are experienced in load testing of web-based applications, then load testing of a mobile application is no different. The major difference is only during script preparation. As you cannot install load testing tools on the device itself the proxy feature helps to record the HTTP traffic from devices and export to load testing tool.

The other notable differences are:

• Network conditions — Such as 2G/3G/4G. Do not run your tests in LAN if you are keen on understanding the end-user experience.
• Device for recording — Instead of the browser use the device itself, as the header file (user-agent) in each device are unique and servers now understand requests originated from different devices by reading these header files.
• Device-level performance — Record the device level performance manually during peak loads. The functional testing tools that can help record can be integrated with load testing tools. At the least measure response time, CPU, memory and battery utilization per each step during the business transaction.

Further Reading

How to Record Mobile Application With Apache JMeter