Selenium Grid 4 Tutorial For Distributed Testing

Selenium Grid has been an integral part of automation testing, as it lets you perform test case execution on different combinations of browsers, operating systems (or platforms), and machines. It also enables you to perform parallel execution to expedite the cross-browser testing process.

Selenium Grid 4, the successor to Selenium Grid 3, has been in the Alpha stage for the last couple of months. The developer community was very excited regarding Selenium 4 (Alpha), and the addition of several useful features such as Selenium 4 Relative Locator has helped accelerate activities related to Selenium test automation. Selenium 4 Alpha features an improved Selenium Grid design; the significant change is the introduction of ‘fully distributed mode.’ In this Selenium 4 tutorial, we take a deep dive into Selenium 4 and the vital architectural differences in comparison to Selenium 3.

Introduction to Selenium Grid 4

A Selenium Grid is primarily used for distributed test execution. It works on the hub-node architecture and those principles of grid design are carried forward in Selenium 4 Alpha.

Unlike the earlier versions of Selenium (i.e. version 3.x), the Selenium 4 Server jar contains everything (including dependencies) required to run a Grid. As per Selenium’s official documentation, Selenium Grid 4 is designed from scratch and does not share any codebase with the earlier versions.

How to Download Selenium Grid 4?

The latest version of Selenium 4 (Alpha) is 4.0.0-alpha-6 and the same can be downloaded from the official website of Selenium. For demonstration, we will be using Selenium Grid version 4.0.0-alpha-2, as it is more stable than the other versions. It offers the advantage of the latest technologies in order to facilitate scaling up, whilst still letting the users perform Selenium test automation.

It is recommended to download the jar file in the same location where the Selenium WebDriver for different browsers is present. This is because Selenium 4 Alpha has the ability to automatically detect the WebDrivers present on the node machine.

Selenium 4 Alpha can be run by executing the following command from the terminal:

java -jar selenium-server-4.0.0-alpha-2.jar

In my case, the jar file is downloaded in the same directory where Selenium WebDriver for Chrome, Firefox, and other browsers are present.

Salient Features of Selenium Grid 4

The release of Selenium 4 was delayed for quite some time and this built up a lot of excitement about the features offered for Selenium test automation. Without further ado, let us take a look at some of the most important features of Selenium 4 Alpha before moving ahead with the Selenium 4 tutorial:

Single Jar for Hub and Node

Until now, the process to set up Selenium Grid involved starting up the Hub and Node(s) separately. With Selenium 4, the process has become much smoother, as the hub and node are a part of the same jar file. Once the server is started, it acts both as Hub and Node.

Improved Architecture

From Selenium 2 onwards, the Hub consisted of three processes – Router, Session Map, and Distributor. Selenium 4 has a new architecture that supports four separate processes – Router, Session Map, Distributor, and Node.

• Router – Listens to the new session request.

• Distributor – Selects the appropriate node where the test should be executed.

• Session Map – Maps the session ID to the node.

• Node – Test machine where the test execution takes place.

In further sections of the Selenium 4 tutorial, we would take a detailed look at the architecture of Selenium 4 Alpha.

Different Grid Types

Until version 3 of the Selenium Grid, it could only be used as Hub and Node(s). Though this is still supported in Selenium 4, it now supports two more types of Grid. Here are the three grid types supported in Selenium 4:

• Standalone Mode

• Classical Grid (Hub and Node like earlier versions)

• Fully Distributed (Router, Distributor, Session, and Node)

Docker Support

This version also comes with Docker support. As of now, the Docker integration does not make use of Unix domain sockets, so you need to make sure that the daemon is listening on port 2375.

When compared to Selenium Grid 3, this version is easier to use on virtual machines. We will also discuss the ease of use in this Selenium 4 tutorial.

Architectural Overview of Selenium Grid 4

Before we take a detailed look into the architecture of Grid 4, we do a quick recap of Selenium Grid 3. It will help in getting a clearer picture of the architectural differences between Grid 4 and earlier versions of the Grid. Also helps us understand how it affects the process of Selenium test automation.

The Architecture of Selenium Grid 3

As I have mentioned in this section of the Selenium 4 tutorial, the Grid 3 version consists of two major blocks – Hub and Node, as shown in the graphic below.

As you can see, this version of Selenium Grid only consisted of a Hub and Nodes (where tests are executed) connecting to the Hub.

The Architecture of Selenium Grid 4

Earlier versions of the Grid only had two components – Hub and Node but that has changed with Selenium Grid 4. Now, Selenium Grid 4 consists of four processes – Router, Distributor, Session Map, and Node.

Here is the top-level architecture diagram of Selenium 4 Alpha (in a Fully Distributed Mode):

Here is the detailed list of steps performed as a part of the cross-browser testing on the fully distributed variant of Selenium Grid 4:

Step 1 – The first step is starting the Session map which is primarily responsible to map Session IDs to the corresponding Node on which the session is running.

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar sessions

When a new session is created, the combination of a Session ID and Node URI (Uniform Resource Identifier) is stored in the Sessions Map.

Step 2 – The next step is to start the distributor process. When there is a request from the Selenium client to create a Sessions request, the Distributor is responsible for assigning an appropriate node.

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar distributor --sessions http://localhost:5556

Step 3 – The Router is the one that is normally exposed to the web. The router listens to the new Session requests on http://localhost:4444.

Client requests are sent to the Router and depending on the request type, the appropriate path is selected. Incoming request for creating a new session is redirected to the Router process.

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar router --sessions http://localhost:5556 --distributor http://localhost:5553

All other types of requests are sent to the Node based on the request after querying the node URI from the Sessions map using the Session ID.

Step 4 – The Grid is of no use without the Node. Selenium WebDrivers of the respective browsers are placed in the same directory where the Grid 4 jar file is located.

The –detect-drivers option is used for automatically identifying the Selenium WebDrivers present in the system.

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar node --detect-drivers

When a node is created under the distributor, the details of the Node along with Node URI are updated in the Session Map. In the screenshot shown below, the Node URI is 5555.

Step 5 – The request by the Selenium WebDriver for starting a remote session is sent to the router.

xxxxxxxxxx

public void testSetUp() throws MalformedURLException {

     String Node = "http://localhost:4444";

        DesiredCapabilities caps = new DesiredCapabilities();

        caps.setBrowserName("chrome");

        /* The execution happens on the Selenium Grid with the address mentioned earlier */

        driver = new RemoteWebDriver(new URL(Node), caps);

    }

The command curl http://localhost:4444/status is used for checking the status of the invocation, along with checking whether the session is established or not.

Once the session is established under the node, the Session ID is updated in the Session Map for the corresponding node. As mentioned in Step (3), this helps the Router process in identifying the Node so that other ‘matching’ client requests can be directly sent to that node.

Step 6 – Session creation calls are diverted by the Router to the Distributor and all other types of requests are sent directly from the Router to the Node.

In the earlier versions of Selenium Grid, these processes occurred inside the Hub hence, the internals were not completely known to the developers and testers using that version of Selenium Grid. The renewed architecture of Selenium Grid 4 makes the process of debugging and troubleshooting easier which, in turn, makes Selenium test automation more seamless.

Now, we will get to the interesting part of the Selenium 4 tutorial i.e. running tests for Selenium test automation.

Running Tests on Selenium Grid 4

In this part of the Selenium 4 tutorial, we demonstrate different ways to run tests on Grid 4 for Selenium test automation. The test case used for demonstrating the usage is below:

1. Navigate to this URL: https://lambdatest.github.io/sample-todo-app/

2. Select the first two checkboxes.

3. Send ‘Yey, Let’s add it to list’ in the textbox with id – sampletodotext.

4. Select the Add Button and verify whether the text has been added or not.

Implementation

Further in this Selenium 4 tutorial, we will use the TestNG automation framework for performing Selenium test automation. You can refer to our earlier blogs in case you require a quick recap of the TestNG framework.

xxxxxxxxxx

package org.selenium4;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import org.openqa.selenium.remote.DesiredCapabilities;

import org.openqa.selenium.remote.RemoteWebDriver;

import org.testng.annotations.AfterClass;

import org.testng.annotations.BeforeClass;

import org.testng.annotations.Test;

import org.openqa.selenium.By;

import org.openqa.selenium.JavascriptExecutor;

import org.openqa.selenium.WebElement;

import java.net.MalformedURLException;

import java.net.URL;

import io.github.bonigarcia.wdm.WebDriverManager;

public class CrossBrowserTest {

   public WebDriver driver;

   String URL = "https://lambdatest.github.io/sample-todo-app/";

   String Node = "http://localhost:4444";

   boolean status = false;

   @BeforeClass

   public void testSetUp() throws MalformedURLException {

       DesiredCapabilities caps = new DesiredCapabilities();

       caps.setBrowserName("chrome");

       /* The execution happens on the Selenium Grid with the address mentioned earlier */

       driver = new RemoteWebDriver(new URL(Node), caps);

   }

   @Test

   public void test_ToDo_App() throws InterruptedException {

       driver.navigate().to(URL);

       driver.manage().window().maximize();

       try {

           /* Let's mark done first two items in the list. */

           driver.findElement(By.name("li1")).click();

           driver.findElement(By.name("li2")).click();

           /* Let's add an item in the list. */

           driver.findElement(By.id("sampletodotext")).sendKeys("Yey, Let's add it to list");

           driver.findElement(By.id("addbutton")).click();

           /* Let's check that the item we added is added in the list. */

           String enteredText = driver.findElement(By.xpath("/html/body/div/div/div/ul/li[6]/span")).getText();

           if (enteredText.equals("Yey, Let's add it to list")) {

               status = true;

           }

           System.out.println("Selenium Grid 4 Standalone Testing Is Complete");

       } catch (Exception e) {

           System.out.println(e.getMessage());

       }

   }

   @AfterClass

   public void tearDown() {

       if (driver != null) {

           driver.quit();

       }

   }

}

xxxxxxxxxx

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"

        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

   <modelVersion>4.0.0</modelVersion>

   <groupId>org.example</groupId>

   <artifactId>org.selenium4.SeleniumGrid4</artifactId>

   <version>1.0-SNAPSHOT</version>

   <properties>

       <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

   </properties>

   <dependencies>

       <!-- https://mvnrepository.com/artifact/com.github.lambdatest/lambdatest-tunnel-binary -->

       <dependency>

           <groupId>org.testng</groupId>

           <artifactId>testng</artifactId>

           <version>6.9.10</version>

           <scope>test</scope>

       </dependency>

       <dependency>

           <groupId>org.seleniumhq.selenium</groupId>

           <artifactId>selenium-java</artifactId>

           <version>4.0.0-alpha-6</version>

       </dependency>

       <dependency>

           <groupId>org.testng</groupId>

           <artifactId>testng</artifactId>

           <version>6.8.8</version>

       </dependency>

       <dependency>

           <groupId>org.seleniumhq.selenium</groupId>

           <artifactId>selenium-chrome-driver</artifactId>

           <version>4.0.0-alpha-6</version>

       </dependency>

       <dependency>

           <groupId>io.github.bonigarcia</groupId>

           <artifactId>webdrivermanager</artifactId>

           <version>RELEASE</version>

           <scope>test</scope>

       </dependency>

       <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-nop -->

       <dependency>

           <groupId>org.slf4j</groupId>

           <artifactId>slf4j-nop</artifactId>

           <version>1.7.28</version>

           <scope>test</scope>

       </dependency>

   </dependencies>

   <build>

       <plugins>

           <plugin>

               <artifactId>maven-compiler-plugin</artifactId>

               <version>3.0</version>

               <configuration>

                   <source>1.8</source>

                   <target>1.8</target>

               </configuration>

           </plugin>

           <plugin>

               <groupId>org.apache.maven.plugins</groupId>

               <artifactId>maven-surefire-plugin</artifactId>

               <version>2.12.4</version>

           </plugin>

       </plugins>

   </build>

</project>

We have used the IntelliJ IDE for development and test execution. You could also use the same project on the preferred IDE of your choice e.g. Eclipse for this Selenium 4 tutorial demonstration.

Code Walkthrough

Allow me to take you through the code used above in this Selenium 4 tutorial. The code under @BeforeClass annotation sets the browser capabilities for the Chrome browser. The RemoteWebDriver instance is created and the same is executed on Grid 4 (Hub Address – http://localhost:4444)

xxxxxxxxxx

@BeforeClass

    public void testSetUp() throws MalformedURLException {

        DesiredCapabilities caps = new DesiredCapabilities();

        caps.setBrowserName("chrome");

        /* The execution happens on the Selenium Grid with the address mentioned earlier */

        driver = new RemoteWebDriver(new URL(Node), caps);

The test case (test_ToDo_App()) is implemented under the @Test annotation. Standard Selenium methods (e.g. sendKeys, click, etc.) are applied on the required web elements.

xxxxxxxxxx

@Test

   public void test_ToDo_App() throws InterruptedException {

       driver.navigate().to(URL);

       driver.manage().window().maximize();

       try {

           /* Let's mark done first two items in the list. */

           driver.findElement(By.name("li1")).click();

           driver.findElement(By.name("li2")).click();

           /* Let's add an item in the list. */

           driver.findElement(By.id("sampletodotext")).sendKeys("Yey, Let's add it to list");

           driver.findElement(By.id("addbutton")).click();

We will execute the same test code on different Grid types – Standalone Selenium Grid, Classical Selenium Grid, and Fully Distributed Selenium Grid.

We know by now in this Selenium 4 tutorial that Grid 4 automatically detects the Selenium drivers you have in the system. It is recommended that either all the executables are in the PATH or in the folder where the Selenium jar file is present.

Here comes the most important part of the Selenium 4 tutorial which will be essential for successful Selenium test automation.

Standalone Selenium Grid

In the Standalone mode, the Selenium server is running everything in-process. It is invoked by executing the following command on the terminal:

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar standalone

The Grid automatically identifies that the WebDrivers for Chrome and Firefox are present on the system.

The server is listening on http://localhost:4444/ which is the same address used in the configuration of the Remote WebDriver. Once the test code is executed, the Chrome WebDriver is registered in the Grid.

Check the output of this command: curl http://localhost:4444/status to confirm that a session with Session ID 01c2215d66abfdaceb8d06c363bbb017 is created.

xxxxxxxxxx

............................

"sessions": [

         {

           "sessionId": "01c2215d66abfdaceb8d06c363bbb017",

           "stereotype": {

             "browserName": "chrome"

           },

           "currentCapabilities": {

             "acceptInsecureCerts": false,

             "browserName": "chrome",

             "browserVersion": "83.0.4103.116",

             "chrome": {

               "chromedriverVersion": "83.0.4103.39 (ccbf011cb2d2b19b506d844400483861342c20cd-refs\u002fbranch-heads\u002f4103@{#416})",

               "userDataDir": "C:\\Users\\Lenovo\\AppData\\Local\\Temp\\scoped_dir8060_2003886737"

             },

             "goog:chromeOptions": {

               "debuggerAddress": "localhost:64249"

             },

             "networkConnectionEnabled": false,

             "pageLoadStrategy": "normal",

             "platformName": "windows",

             "proxy": {

             },

             "se:options": {

               "cdp": "http:\u002f\u002flocalhost:64249"

             },

             "setWindowRect": true,

             "strictFileInteractability": false,

             "timeouts": {

               "implicit": 0,

               "pageLoad": 300000,

               "script": 30000

             },

             "unhandledPromptBehavior": "dismiss and notify",

             "webauthn:virtualAuthenticators": true

           }

         }

       ]

............................

............................

I hope this Selenium 4 tutorial has helped you understand how to create a session with Grid 4 for Selenium test automation.

‘Hub and Node’ Selenium Grid

This is the classical way of using the Grid for Selenium test automation that consists of two major components – Hub and ‘N’ Nodes. If the Hub and Nodes are running on the same machine, you can start them with the following commands:

Hub

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar hub

Node

(if Hub and Node are running on the same machine)

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar node --detect-drivers

Node

(if Hub and Node are on different machines)

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar node --detect-drivers --publish-events tcp://hub:4442 --subscribe-events tcp://hub:4443

For the demonstration, we are running Hub and Node on the same machine. Once the Hub is started, it opens XPUB and XSUB sockets that bind to tcp://*:4442 and tcp://*.4443 respectively.

Once the node is started using the command mentioned earlier, it connects to the same address (i.e. tcp://*:4442 and tcp://*.4443). The –detect-drivers option automatically detects the Selenium WebDrivers present in the system.

As soon as the instance of Chrome WebDriver is instantiated, it is registered on the Grid

xxxxxxxxxx

DesiredCapabilities caps = new DesiredCapabilities();

caps.setBrowserName("chrome");

/* The execution happens on the Selenium Grid with the address mentioned earlier */

driver = new RemoteWebDriver(new URL(Node), caps);

As seen in the output, a session with Session ID 609efa510550943bdcedc46f4d67d1c6 was created.

xxxxxxxxxx

"sessions": [

         {

           "sessionId": "609efa510550943bdcedc46f4d67d1c6",

           "stereotype": {

             "browserName": "chrome"

           },

           "currentCapabilities": {

             "acceptInsecureCerts": false,

             "browserName": "chrome",

             "browserVersion": "83.0.4103.116",

             "chrome": {

               "chromedriverVersion": "83.0.4103.39 (ccbf011cb2d2b19b506d844400483861342c20cd-refs\u002fbranch-heads\u002f4103@{#416})",

               "userDataDir": "C:\\Users\\Lenovo\\AppData\\Local\\Temp\\scoped_dir9476_1607038509"

             },

             "goog:chromeOptions": {

               "debuggerAddress": "localhost:60557"

             },

             "networkConnectionEnabled": false,

             "pageLoadStrategy": "normal",

             "platformName": "windows",

             "proxy": {

             },

             "setWindowRect": true,

             "strictFileInteractability": false,

             "timeouts": {

               "implicit": 0,

               "pageLoad": 300000,

               "script": 30000

             },

             "unhandledPromptBehavior": "dismiss and notify",

             "webauthn:virtualAuthenticators": true

           }

         }

       ]

‘Fully Distributed’ Selenium Grid

Grid 4 can be started in a fully distributed manner, with each piece running in its own process. I have already covered the architecture of Fully Distributed Selenium Grid in this section of the Selenium 4 tutorial. It will give a general idea about what a fully distributed Grid looks like.

Step 1 – Start the Sessions Map

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar sessions

It maps the Session IDs to the node, the session is currently running on.

Step 2 – Start the Distributor

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar distributor --sessions http://localhost:5556

It listens for Session requests on port 5556.

Step 3 – Start the Router

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar router --sessions http://localhost:5556 --distributor http://localhost:5553

The router gets the Node URI to forward that request to Node. Request for a new session goes to the Distributor. The Router listens for new requests on http://localhost:4444.

Step 4 – Create a Node

Node is the machine where the test will be executed.

xxxxxxxxxx

java -jar selenium-server-4.0.0-alpha-2.jar node --detect-drivers

Once the Node is created, Node details and Node URI are updated in the Sessions Map.

Once the instance of Chrome WebDriver is instantiated, it is registered on the Grid.

xxxxxxxxxx

DesiredCapabilities caps = new DesiredCapabilities();

caps.setBrowserName("chrome");

/* The execution happens on the Selenium Grid with the address mentioned earlier */

driver = new RemoteWebDriver(new URL(Node), caps);

As shown in the screenshot, the session details are also updated in the Sessions Map. The Node URI is 5555.

The output of curl http://localhost:4444/status indicates that a session with Session ID 432562bb885dfc616e260e26dec365e0 was created for Selenium test automation.

xxxxxxxxxx

"sessions": [

         {

           "sessionId": "432562bb885dfc616e260e26dec365e0",

           "stereotype": {

             "browserName": "chrome"

           },

           "currentCapabilities": {

             "acceptInsecureCerts": false,

             "browserName": "chrome",

             "browserVersion": "84.0.4147.89",

             "chrome": {

               "chromedriverVersion": "84.0.4147.30 (48b3e868b4cc0aa7e8149519690b6f6949e110a8-refs\u002fbranch-heads\u002f4147@{#310})",

               "userDataDir": "C:\\Users\\Lenovo\\AppData\\Local\\Temp\\scoped_dir10092_57520474"

             },

             "goog:chromeOptions": {

               "debuggerAddress": "localhost:53155"

             },

             "networkConnectionEnabled": false,

             "pageLoadStrategy": "normal",

             "platformName": "windows",

             "proxy": {

             },

             "setWindowRect": true,

             "strictFileInteractability": false,

             "timeouts": {

               "implicit": 0,

               "pageLoad": 300000,

               "script": 30000

             },

             "unhandledPromptBehavior": "dismiss and notify",

             "webauthn:virtualAuthenticators": true

           }

         }

       ]

Wrapping It Up

I hope this Selenium 4 tutorial was able to bring an entirely fresh perspective to Selenium test automation for distributed testing. Due to the completely new architecture, it supports different modes – Standalone, Hub and Node, and Fully Distributed.

It is more modern than the previous versions of the Grid, as it can be used with modern technologies like Docker and Kubernetes with ease. It is also designed to take the advantage of cloud infrastructures like AWS and Azure. Though Selenium 4 is still in the Alpha phase, there is a lot of excitement amongst the developer and tester community and its features have fared well so far.

Let me know what you think about this tutorial below.

Happy Testing!