Are You Getting What You Need from Your UI Testing Tool?

UI testing has had an upswing in activity in the last few years. New and exciting tools have entered the market, bringing all sorts of innovative approaches to make it simple to do what has traditionally been a pretty complicated process.

Back in the day, we only had big-box solutions like the IBM Rational Suite or Mercury/HP/MicroFocus QTP/UFT. Now we're seeing a lot of people moving away from the "traditional" test automation tools towards open source alternatives, like Selenium, or new innovative AI-powered solutions or frameworks, like mabl, Selenic, or Functionize. This shift is causing lots of hype, but it is also rooted in addressing common complaints and challenges, that you need to make sure to address if you're adopting a new tool.

You may also enjoy:  14 of the Best Automation Testing Tools Available Today 

But of course, it's not as easy as just picking the "best one" and going to town. Whether you've decided to build a framework yourself, to have ultimate control, or you've decided to go into a proprietary framework that offers advanced features like self-healing or parallel execution, there are seven important factors in any solution that you should consider.

1. Is it Selenium?

Sounds like a loaded question, right? It's not. Selenium has quickly become the de-facto standard for organizations that want to automate web UI testing but don't want to be locked into a vendor's platform.

As a community, the Selenium open source project has banded together commercial vendors and open source purists alike to make one of the most powerful frameworks in our industry — and it's being used, a lot.

According to our research, over 64% of users are leveraging Selenium for their UI test automation. Are you?

This is a factor that must be considered. Are you using Selenium? If you aren't, now is a good time to think about the future and how flexible you want to be with your future testing initiatives. Many of the solutions on the market now boast Selenium-like functionality or are based on Selenium, focusing on the value of importing/exporting pure Selenium tests. But they don't operate on pure Selenium, or execute pure Selenium, so you have to work within the constraints of their technology.

2. Does It Use the Page Object Model?

The page object model is a UI testing design paradigm where users can define UI elements in association with the pages on which they are present. Here's a great tutorial about the page object model, if you aren't yet familiar. In order to proactively address issues of maintainability, you want your UI tests to leverage this page object model paradigm as it makes it much easier to maintain your scripts due to the element locations being defined in one place, and then leveraged across your test suite.

The page object methodology is powerful because it helps address issues of maintainability when the application changes. In the page object model, an element on a page is defined in one location, so if you have identified that a specific web element has moved, you don't have to go through the entire script to update each of the individual locators. The page object model allows you to have a single source of truth for an element on a page, to make this all a lot more automated.

3. Does It Recommend Smart Locators?

Most of the maintenance problems that occur when performing UI testing stem from test cases that break due to an application's UI changes. So one of the most important things to do when designing your UI test is to build a method to create reliability and stability for each of your element locators.

Some of the new UI testing tools are taking unique and innovative approaches to this, using various different terms, i.e. "smart locators" or "magic bindings." Regardless, there's lots of potential for you to benefit from all this innovation, in the area of defining multiple locators for a single element or building in the best possible locator-defining strategy.

4. Can the Test Scripts Self-Heal?

Self-healing sounds like a dream state, where our automated tests automatically identify where failures occur and then take it upon themselves to fix the problem. The crazy thing is that this is actually a reality in many of the popular UI testing tools that are out there. By taking advantage of smart locators, these UI testing tools can try out different locators or ways of identifying buttons in the UI automatically at runtime.

Now that my mind has been blown, I see this as a strict necessity in any UI testing solution, because ultimately we're trying to reduce test churn and overall testing cycles. If you've created a thousand UI tests and are running them every single night, you don't want half of them breaking because somebody changed a button from add to "cart" to add to "basket."

5. Can It Integrate Into My Existing Toolchain?

In your software development organization, you might already be writing code in some kind of IDE (i.e. Eclipse, IntelliJ), and you might be building that code with a build system like Jenkins. These will be proven and practiced frameworks that have been established across your organization, so it's clearly beneficial for the UI testing solution you are using to integrate into what you've already built.

Many UI testing tools are their own separate tools, and that's potentially really great because it means that they have focused on building a solution that can solve all of these challenges while having ultimate control of generated test scripts. But vendor-lock (which I'll discuss a little more later) is dangerous, and we've seen organizations moving away from the big-box solutions for this very reason. Because developers and testers are more technically savvy than they've ever been, they want access to their test scripts. So when you're looking at UI testing tools, you can see if your test scripts will be portable, not just with other scripts that have to be integrated into your DevOps pipeline, but able to be truly integrated into your existing framework.

6. Is Support Available?

User experience is especially critical in B2C web applications, where a confusing UI or unreliable customer experience can quickly impact revenue, and organizations understand that it is crucial to ensure that the experience is seamless. With new projects emerging daily and testing requirements being so aggressive, especially with the move to Agile, being able to get yourself out of a hole could mean the difference between releasing on time and missing the window. Your UI testing tool will help you validate that critical customer experience — but if you're trying to do something and you can't figure it out, having a vendor's support team to get you unblocked is a choice that you can make, to have whenever you need it.

There are some open frameworks that can be easily integrated into your existing frameworks, but without great support, what happens when things go wrong just before a critical release, and no one is accountable? Selenium is a great example here — although it is incredibly powerful and has a wonderful community backing it, there's no one to call when things go wrong.

7. Can You Avoid Vendor Lock?

As I said before, 20+ new UI testing tools entered the market in the last few years. What does that mean for the next five?? Everything we're doing right now will no doubt change, especially from a UI testing perspective. Something new and exciting is bound to show up, so it's important to consider now how we facilitate the migration process. A key to this is ensuring that whatever solution you bring in will not lock you into a proprietary framework.

Since Selenium can be easily executed in any pipeline and is implemented as code, it's highly flexible. Some of the new solutions have taken this into consideration, with import/export mechanisms to allow you to switch in and out of their tool, but the import and export needs to validate that it works. It's not as easy as you might think. It seems to me that if a vendor is confident about their capabilities, they should supply this functionality. And when you are looking at UI testing tools, you can ask, "Am I getting locked into their framework?"

What's Next?

If you're looking at UI testing tools and want a handy guide, check out the top 10 new UI testing tools everyone's talking about, and start asking these questions as you consider which tools you should adopt into your software delivery process.