As you probably know, continuous testing is becoming a more and more important part of the software development life cycle. Since any change to the product needs to be verified prior to going to production, the idea behind continuous testing is to make the whole workflow from a single commit to publishing the new release 100% automated and, as a result, unattended.

When it comes to functional testing of web applications, the same rule should be applied. In addition to backend unit testing and performance testing, it makes sense to run a set of automated functional tests on the client side to ensure that everything is working fine, all elements are visible and operating correctly, rendering speed is acceptable, and so on. As of now, the most popular and powerful tool for browser automation is Selenium, a free and open-source browser automation framework.

We already covered the point of having Selenium automated tests as a part of the Continuous Delivery process in the How to Automate Testing Using Selenium WebDriver, Jenkins and Allure post. Today’s article is about running Selenium tests in “headless” mode, i.e., when no browser window is being displayed.

The importance of having “headless” tests is:

• The majority of Linux server deployments don’t have GUI at all.

• The same applies to Windows Core Servers.

• When it’s not possible to open the browser in the foreground (for example, when the build agent is being used for desktop applications testing that requires focus, or you would like to continue using your machine normally while Selenium tests are in progress).

Today, you will learn how you can launch Selenium tests when Jenkins is in “headless” mode.

Quick Solution

The fastest and the easiest way would be switching to “headless” Selenium WebDriver implementations such as:

• HtmlUnitDriver: wrapper for HtmlUnit, a Java-based browser implementation.

• PhantomJSDriver: wrapper for PhantomJS, so-called “headless WebKit,” a non-GUI browser engine. By the way, WebKit is under the hood of such browsers as Chrome (and derivatives), Safari (including mobile versions), Dolphin browser (Android mobiles), etc.

However, in some situations, switching to a headless browser might not be an option, like if the application is under test restrictions (for example, the application supports Internet Explorer browsers only), there is a requirement to test plugin content (Flash, Silverlight, Java), or there is a requirement to support certain technologies like ActiveX or authentication types like single sign-on).

So the remaining part of the article will cover running Selenium tests using real browsers in headless mode on Jenkins nodes.

Test Project

All of the following examples assume the following minimal test scenario:

• Use Selenium Firefox Driver and open as an instance of the Firefox browser.

• Open BlazeMeter main page.

• Print the global title.

• Close the browser.

The test class is simple: 

package com.blazemeter.junit.selenium;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
public class JMeterSeleniumDemoTest {
   @Test
   public void openBlazeMeterTest() {
       FirefoxDriver driver = new FirefoxDriver();
       driver.get("http://blazemeter.com");
       WebElement title = driver.findElement(By.xpath("//title"));
       System.out.println("********************************************************");
       System.out.println("*" + title.getAttribute("text") + "*");
       System.out.println("********************************************************");
       driver.quit();
   }
}

 This is what it looks like in action: 

Now, let’s see how this test can be executed in Jenkins on different operating systems.

Windows 

In general, running Selenium tests on Windows is not possible. The Windows operating system family always has GUI, and even Windows Core Server deployments are capable of running GUI applications without any extra configuration (you just need to get the GUI application somehow using the command-line, for example using Invoke-WebRequest PowerShell cmdlet). 

However, it is possible to run your test on behalf of a different user. This way, you can continue working while the test is being executed in the background without having to worry that your normal keyboard and mouse activity will break the test, and vice versa.

We recommend you install Jenkins master (or node) as a Windows service. This will kill two birds with one stone:

• You will get failover and resilience if the machine is restarted after installing updates, if there is a power outage or if the Jenkins process dies for some reason.

• The process will be run under a different user account (for instance, Local System) so the browser window won’t appear in the current session. 

Here is the demo of the Test Project execution on a Jenkins node running as a Windows service:

As you can see, no Firefox browser window is present, but the test execution is successful. The BlazeMeter page title (JMeter and Performance Testing for DevOps | BlazeMeter) is being printed to the build log.

N.B., in some cases (for example, if you need to use Internet Explorer browser or other applications requiring interacting with the Desktop), you may also need to set java.awt.headless Java system property to “false” for Jenkins master and/or slave processes. 

Linux

Now, let’s try running the same test on a Linux box:

As can be seen from the bottom of the build log, Jenkins failed to start the Firefox browser. Being a GUI application, Firefox requires a “display” to run at. 

Linux operating systems use display server to draw windows, receive input, and react with output. Any GUI-based program requires something called DISPLAY, a screen (or multiple screens) where output is rendered. If the Linux installation doesn’t assume GUI, you can use X virtual framebuffer to mock up the missing display. 

If you don’t have the X virtual framebuffer installed (i.e., if Xvfb returns the “command not found” message), then you will have to install it from your Linux distribution repositories.

The process of running X virtual framebuffer is as simple as:

Xvfb :N

...where N stands for the number of the display. In Linux, display numbers are zero-based and normally, the user is sitting on display 0, so you can start X virtual frame buffer on the display 0 as:

Xvfb :0 >& /dev/null &

The remainder of the command after 0 is to suppress Xvfb output. It prints quite a lot of noise to the command line, so if you add >& /dev/null & postfix, you will see only Xvfb process id (14169 in my case). 

The next step is to “tell” Jenkins to use this Display 0 for launching Firefox. On Linux systems, it can be done by setting DISPLAY environment variable so you can either refer your Linux distribution documentation to learn how to set it or do it on Jenkins level (for example, under the Manage Jenkins -> Configure System -> Global Properties -> Environment Variables section. 

Or, in case of individual build agents, the same settings live under Manage Jenkins -> Manage Nodes -> %Node Name% -> Node Properties -> Environment Variables.

To get to the configuration dialog, click on the gear icon next to the node name.

Once you have launched the X virtual framebuffer and set up the DISPLAY environment variable matching virtual display number, the build should succeed and you should see the BlazeMeter main page title in the build log.

MacOSX

As MacOSX is a UNIX-based operating system, the approach should be the same as for Linux. X window server implementation for MacOSX is XQuartz. However, you may have to recompile it to support Xvfb feature and the browser you’re intending to use so it would use X11 server instead of Aqua.

Selenium and JMeter Integration

If you want to combine your Selenium functional tests with JMeter load tests you, this can be done via WebDriver Sampler. I believe there is no need to say that BlazeMeter was, is, and will be 100% JMeter-compatible, including any plugins. If you don’t need the integration, just Selenium is also supported. Check out the Webdriver Test page for more details.