It's not magic: How the CustomWebResolver makes your web application UI testable

With QF-Test, our goal is to make testing web application UIs as easy as testing a native application. This presents some challenges for which we developed a unique solution we call the CustomWebResolver. In this blog post, I will explain why QF-Test is so much better at testing web applications than other tools and what this “CustomWebResolver” has to do with it.

The problem with CSS and XPath selectors

Most browser-based UI testing tools rely on CSS selectors or XPath to select an element to interact with or validate. This approach has several downsides:

• Very verbose: A typical CSS selector looks like this: body > main > div > ul > li:nth-of-type(3) > button.my-action, the same as an XPath would be: .//body/main/div/ul/li[3]/button[@class="my-action"]. Not very readable.
• Not very stable: These selectors rely on the underlying HTML structure of the application to remain the same, otherwise they break.
• Not easy to auto-generate: This becomes important when recording an interaction. Generating a CSS or XPath selector for an HTML element programmatically is easy. Generating a stable, concise and reusable selector, however, must usually be done by hand.
• No Shadow DOM access: It is not possible to use CSS or XPath to select an element encapsulated in a “shadow root”. This is a problem with applications built on the Web Components standard. Instead, JavaScript and a deep understanding of web technologies are required.
• Technology-specific: These selectors are only applicable to HTML, they cannot be used in a cross-platform context.
• No checking of state: With these selectors, you only found the element. Checking if the element has the required state to pass a test needs to be handled separately.

There is a better way: The QF-Test CustomWebResolver

Although you can also work with selectors here, QF-Test offers a better way: A technology we call the CustomWebResolver, or CWR for short, resolves the raw HTML of your application into a semantic tree of well-defined UI components. This offers several advantages:

• No knowledge of HTML required: Test authors are not required to understand the intricacies of the DOM structure of the application to address components.
• Semantic tests: UI components can be addressed based on their semantic function as presented to the user, instead their structural position in the DOM or the HTML code.
• Unified handling of similar components: UI components that have the same semantic function but are implemented differently on the HTML level are still addressed in the same way.
• Checks included: The state of a UI component can automatically be checked according to common criteria like “is it visible”, “what’s the text content?”, “what’s the input value?”, “is it enabled or disabled?”, “is it checked or selected?”, and more.
• Don’t worry about the Shadow DOM: With a CustomWebResolver you never have to think about things like shadow roots – QF-Test just takes care of everything.
• Robust against DOM changes: The CustomWebResolver is much more resilient against changes to the DOM structure. And if changes are necessary, they can be made in a central place instead of everywhere a component is used.
• Cross-platform: Because it is not bound to HTML, the QF-Test component structure can be reused between applications built with Java UI libraries, Native Windows UIs and even iOS and Android apps with little adjustment needed.
• Selection via SmartID: In QF-Test you can select a component via the very compact SmartID syntax like this: #List:&2@#Button:

Okay, but how does this magical CustomWebResolver technology work?

The basis of the CustomWebResolver is the translation of the DOM structure into a tree of components of certain generic classes based on a set of mappings. Let’s untangle that statement.

Generic classes

Any application UI consists of a number of widgets, or components, that serve some purpose. There are usually windows with different panels which contain labels, tables, trees, lists, checkboxes, text inputs, sliders, drop-downs, and so on.

QF-Test comes with a big list of generic classes which can be used to describe almost every component imaginable in an application UI. QF-Test understands their semantics, i.e. how they work, and can automatically offer the right actions and checks per component.

For example, QF-Test knows to offer a “‘checked’ state” check for the generic class CheckBox, but not for e.g. the generic class TextField, and it understands that you can click a CheckBox component to toggle it’s ‘checked’ state. It also understands that a Table or List can contain child elements and gives you easy access to them.

CWR mappings

The job of the CustomWebResolver is to look at the DOM of your application and figure out which HTML elements belong to which generic class. To do this, it uses mappings which are usually defined in a YAML document. The simplest mapping possible looks like this:
 

genericClasses:
- Button: btn

This piece of YAML, when interpreted by the CustomWebResolver, will tell QF-Test to assign every HTML element with the CSS class btn the generic class Button and therefore understand that it’s a button that can be activated as long as it’s not disabled or invisible.

With the CustomWebResolver configured like this, you can now select “OK” buttons using the SmartID #Button:OK, regardless of where this button is or what HTML tag it actually uses, as long as it has the CSS class btn and a label “OK”.

Here is another example of a CustomWebResolver mapping, this time a more complex one:
 

genericClasses:
- ComboBox:
    attribute: role
    attributeValue: combobox
- List:ComboBoxList:
    tag: ul
    parent: ComboBox
- Item:ComboBoxListItem:
    tag: li
    parent: List:ComboBoxList

The mapping above is for a ComboBox, i.e. some kind of drop-down select input. It actually consists of multiple sub-components:

1. The drop-down itself,
2. the list that appears when the drop-down is opened and
3. the individual selectable options.

Thanks to these mappings, the CustomWebResolver can tell QF-Test to handle this kind of drop-down as a single semantic component: QF-Test can automatically select a given option in the list, it can fetch the current selection or check that a given option is the current selection, or it can return the entire list of available options – all through the QF-Test UI without writing any code or messing with the DOM.

The component tree

Finally, once all relevant components are mapped to generic classes, QF-Test can discard the rest of the HTML and you are left with a beautifully simple tree of semantic components, on which you can build your tests.

A side-by-side illustration of how the complexity of the DOM tree of a web application is reduced if all the DIVs and SPANs are replaced with semantic components. The non-optimized tree on the left has way deeper nesting than the optimized, semantic tree.

And how do I configure this CustomWebResolver for my application?

The only thing users of QF-Test have to do is to provide the initial configuration for the CustomWebResolver. We have implemented three ways to make this as easy as possible:

1. For a growing list of web component frameworks, we bundle comprehensive CWR configurations with QF-Test. QF-Test will automatically recognize and translate any components from these frameworks into generic classes, ready for you to use in your tests and checks. We support popular frameworks like Vaadin, Angular Material UI, and Google Web Toolkit. Check out the full list to see if your framework is already supported.
2. If you use a different framework, but your application adheres to the WAI-ARIA standards for web accessibility by marking components with role and aria-* attributes, QF-Test will use these attributes to automatically configure the CustomWebResolver. The better the accessibility of your UI, the better QF-Test will work with it – a double win!
3. Otherwise, you’ll need to write your own CWR mappings. This is not super-difficult, but it requires some knowledge of the HTML structure of your application. Don’t worry, QF-Test does a lot to help you with this. QF-Test comes with a comprehensive user interface which helps you at every step when editing and verifying your configuration. And with the QF-Test UI Inspector, you can see the effect of your CWR configuration in real-time. And if you’re still having trouble, our support team is ready to assist you.

A screenshot of the Edit menu in the QF-Test CustomWebResolver node. It includes all kinds of contextual actions to help you generate a valid CWR configuration in YAML format.

We believe that semantic components and the CustomWebResolver are the best way to build reliable, robust and comprehensive UI tests for web applications. And our customers think so, too.

If you have been using QF-Test for the web already but have not made full use of the power of the CustomWebResolver yet, you should give it a try. And if you are totally new to QF-Test, you can get started with a free trial today!

Read more

• The CustomWebResolver in the QF-Test manual: Improving component recognition with a CustomWebResolver
• Tutorial for configuring your first CustomWebResolver: Example for “CarConfigurator Web” demo
• List of frameworks with built-in CWR: Special support for various web frameworks
• Detailed documentation of everything the CWR can do: The Install CustomWebResolver node