Client-Side vs. Cloud-Side Execution for Mobile Test Automation

One of the requirements of using Appium for mobile test automation is to start the Appium server in the background so that it can listen to the requests from the mobile app under test. And there are multiple ways that we can install and start the Appium server in the background.

While running Appium tests from a local machine, we have to launch the Appium server on a specific port and update the desired capabilities to point to the server running locally. But there should be a better way to handle these and avoid extra work on desired capabilities. Fortunately, cloud-based app testing platforms like Bitbar provide the ways to run Appium client-side as well as cloud-side. In this post, we will compare the client-side and cloud-side Appium and see both sides of the coin.

Client-Side vs. Cloud-Side Appium

With client-side Appium, we need to configure the desired capabilities in order to execute the test on the Bitbar Device Cloud from the local machine. However, the Appium server and the real devices are hosted in the cloud. You can read more about client-side Appium here. In the case of cloud-side Appium, we have to upload our Appium tests, and we don't need to configure the desired capabilities. In cloud-side Appium, we will get parallel testing and device selection from the device group. This is the diagram showing the critical difference between client-side and cloud-side Appium.

There is a great slide share document explaining the difference between the client-side and cloud-side Appium here.

Pros of Client-Side Appium

Either way, Appium execution will achieve your end goal of verifying your defined test steps. Let's first see some advantages of client-side Appium.

Local and Cloud Execution

One of the critical benefits of Appium Client-Side Execution is the ability to launch the tests from developers' local machines. This will allow for the quick testing of new features that are under development straight from the developer's machine. We can trigger Appium tests on real devices by updating the Appium server address from the local computer to the Bitbar hub. This will allow us to unleash the test run from any machine.

Ability to Configure the Devices of Choice

With client-side Appium, developers and testers can configure the devices of their own choice as client-side execution enables us to configure the desired capabilities and devices from the code.

Upload Local Test Results Onto the Cloud

With client-side Appium, once the execution of test suites is finished, you can get access to the video recording of the test directly from Bitbar. Additionally, you can upload .xml files to Bitbar and view test steps in a more readable way.

Cons of Client-Side Appium

In many ways, client-side Appium can help teams get started with Appium test automation in the early stages of a project. However, there are a few critical drawbacks of client-side Appium that might be overlooked and may have a substantially negative impact.

Flaky Tests

Client-side Appium allows developers to start the Appium server from their local machines, so test scripts need to establish the connection between the local Appium instance and the devices. As the tests are so dependent on the network connection with the local machine, they can become very flaky. Flaky tests can cause a loss of developer trust and waste time. 

Slow Tests

While using client-side Appium, there must be a connection between the local machine and the devices in the cloud. All the test files need to be transferred over the internet to the target devices for test execution, and when the tests are finished, the test results will also be sent back to the local machine through HTTP. This is really not an agile approach, as the transferring of test files and results is dependent on the speed of the internet connection.

When the internet speed suffers, the whole team might be waiting for the test results rather than getting instant feedback. The slowness, in general, affects the team productivity and wastes time.

Extra Work on Desired Capabilities

Desired capabilities are a set of keys and values sent to the Appium server and inform the server about the configuration of the automation session. When running Appium tests on the client side, you will need to configure those desired capabilities so that the test is ready for execution on Bitbar.

When you want to execute the Appium test on a specific device on the cloud or change the target device from Android to iOS, you'll need to edit some of the desired capabilities to get tests successfully executed every single time.

No Parallelism - One Device at a Time

Due to the nature of client-side Appium, one of the most significant disadvantages is teams can only execute tests on one device at a time unless you have the luxury of having multiple terminals/VMs for parallel connections to the devices in the cloud.

This might sound okay if you run Appium tests only to verify whether the new test works on any single device, but it's undoubtedly counterproductive when you want to run these tests on a large number of real devices with client-side Appium.

Pros of Cloud-Side Appium

As the default approach for test execution, though client-side Appium does have its merits, the drawbacks are quite significant. And there's a much better way, cloud-side Appium.

Zero Configuration

Unlike client-side Appium, with which we have to continually configure the desired capabilities of the target devices to enable test execution, there is no need for that when using cloud-side Appium. Once your test script is ready, just select the devices on which you want to run your tests on Bitbar. The rest is taken care of by the platform. This reduces the hassle of configurations and potential errors in the configuration.

Parallel Testing

With the cloud-side Appium, we can execute tests on hundreds of real devices in parallel, at the same time. All the configurations that are needed to run parallel tests will be handled by Bitbar under the hood. Appium servers get started under the hood and attached to selected target devices for parallel execution.

Faster and Stable Tests

As stated above, one of the leading causes of flaky tests with client-side Appium is that all the test files and results need to go through multiple layers of the network. That's not the case for cloud-side Appium, as all the test files are uploaded to Bitbar first and test execution happens on the server. It's considerably faster than the client-side execution approach to get more reliable test results.

Monitoring

With cloud-side Appium, the monitoring of the tests becomes painless, as we can see test execution in real-time and get screenshots of test results quickly.

Cons of Cloud-Side Appium

There are not many cons of cloud-side Appium. One possible disadvantage could be:

Hard to Integrate With Other Local Systems 

The only potential drawback for cloud-side Appium could be that if you have tests integrated with other local systems, it will become tough to trigger the test execution over the server side and trigger the following tasks on your local machines.

Which One to Use

It's clear that the cloud-side Appium approach is a way better approach than the client-side Appium approach. Not only can you enjoy the benefits of parallel device testing and alleviate the pain of maintaining scripts for executing tests, but it also dramatically improves team productivity and the trustworthiness of tests.