Image-Based Test Automation Using Sikuli

Introduction

Sikuli is an open source test automation tool for testing web applications. It is similar to Selenium, but it considers all the web elements of the application as images. This tool is capable of recognizing images and interacting with the elements matching the image GUI.

How to Use Sikuli in an Eclipse-Maven Project

Step 1: Create a Maven project called "SikuliProject".

Step 2: Add the Sikuli jar to the pom.xml file in order to load the Sikuli library to our project.

    <dependency>

        <groupId>com.sikulix</groupId>

        <artifactId>sikulixapi</artifactId>

        <version>1.1.0</version>

    </dependency>

Step 3: Add a Selenium dependency to support loading the browser, opening the application URL, etc.

xxxxxxxxxx

    <dependency>

        <groupId>org.seleniumhq.selenium</groupId>

        <artifactId>selenium-java</artifactId>

        <version>3.141.59</version>

    </dependency>

Step 4: Create a script using two main classes – "Screen" and "Pattern" – and perform testing of web applications.

Screen Class

This is the main class of the Sikuli API. It contains various built-in methods to perform actions on web elements. 

Examples include: click(), doubleclick(), wait(), and type().

Example:

xxxxxxxxxx

Screen screen = new Screen();

screen.click(loginlink);

Below are some commonly used Screen() class methods:

1. click() -> Clicks on an element referred to by the image name.
2. doubleclick() -> Double clicks on an element referred to by the image name.
3. delayClick() -> Clicks an element after the given time in milliseconds.
4. delayType() -> Introduces a delay in milliseconds between each character at next type.
5. type() -> Type the given text as an argument.

Pattern Class

It is used to uniquely identify the web element based on the image stored in a specific location.

Syntax: Pattern p = new Pattern("Location of image saved");

Example:

xxxxxxxxxx

Pattern uname = new Pattern("C:\\Image_screenshots\\userName.png");

Example Demo for Login:

Step 1: Identify the web elements to perform login, capture images, and store them in your hard drive.

Step 2: Images captured for our demo are given below:

• Login link 
• Username textbox
• Password textbox
• Login Button
• Logout Link

Selenium-Sikuli Demo:

xxxxxxxxxx

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.firefox.FirefoxDriver;

import org.sikuli.script.Pattern;

import org.sikuli.script.Screen;

import org.testng.annotations.Test;

public class ImageProcessing {

    @Test

    public void testImage() throws Exception  {

        Screen screen = new Screen();

        Pattern loginlink = new Pattern("C:\\Image_screenshots\\loginLink.png");

        Pattern uname = new Pattern("C:\\Image_screenshots\\userName.png");

        Pattern pwd = new Pattern("C:\\Image_screenshots\\password.png");

        Pattern signin = new Pattern("C:\\Image_screenshots\\loginButton.png");

        Pattern logout = new Pattern("C:\\Image_screenshots\\logoutLink.png");

        System.setProperty("webdriver.gecko.driver","C:\\Selenium\\Drivers\\geckodriver.exe");

        WebDriver driver = new FirefoxDriver();

        driver.manage().window().maximize();

        driver.get("http:\\demowebshop.tricentis.com");

        screen.wait(loginlink,10);

        screen.click(loginlink);

        screen.wait(uname,10);

        screen.type(uname,"remi@abc.com");

        screen.type(pwd,"test@1234");

        screen.click(signin);

        screen.wait(logout,10);

        if (screen.exists("C:\\Image_screenshots\\logoutLink.png")!=null) {

            System.out.println("Element is present");

            screen.click(logout);

        }

    }

}

Conclusion

Sikuli is used widely when Selenium has difficulty recognizing the elements or when there is a need to perform image-based element recognitions. Thanks for reading this article on how to integrate Selenium-Sikuli and perform image-based testing.