53.1
Proceeding

The MacOS Accessibility Interface is allowing programs to control the GUI elements of native MacOS applications. With QF-Test you can use the framework in script nodes via the Jython module automac (alternatively de.qfs.automac for Groovy, automac for Javascript).

QF-Test provides a package in the standard library with procedures for the most commonly needed interactions with GUI elements for direct and easy use of the API for test development. This chapter describes the standard library package.

You will find the procedures relevant for control of native MacOS elements in the package qfs.automac.

When developing tests for native MacOS applications you generally need to perform the following steps:

53.1.1
Starting the application

The application to be tested may but does not necessarily have to be started via QF-Test. In any case you need to establish a "connection" to the application via the procedure qfs.automac.app.connect. It stores a handle to the accessibility interface of the application in a QF-Test Jython variable. It has the following parameters for searching, respectively starting, the application.

You can specify bundleId to identify the application via the unique bundle id, e.g. com.apple.Calculator. In case the application is already running QF-Test just stores the handle, otherwise it starts the application as well.

You can specify bundleFile to identify the application via the bundle file where the application is stored, e.g. /Applications/Calculator.app. In case the application is already running QF-Test just stores the handle, otherwise it starts the application as well. In that case a QF-Test process will also appear in the list of the QF-Test clients menu »Clients« By stopping that process the application will also be terminated.

You can specify title to identify the application via the window title. The application has to be started beforehand, e.g. via the node Execute shell command.

You can specify processId to identify the application via the process identification number (PID). The application has to be started beforehand, e.g. via the node Execute shell command.

The procedure qfs.automac.helpers.DumpDesktopWindows lists title, PID, bundle id and bundle file of all running applications in the terminal.

53.1.2
Listing the GUI elements of a window

Before you can set up a test you need to get an overview of the GUI elements of the application. You may either use the procedure qfs.automac.helpers.dumpComponents to print the GUI elements to the QF-Test terminal or qfs.automac.helpers.dumpComponentsToFile to write them to a file.

qfs.automac.helpers.dumpComponents prints label, title, role, subrole, type and identifier of the GUI elements, provided the attribute was implemented for the respective GUI element.

All the GUI elements visible on the desktop are organized in a tree structure with the desktop as the root element. The nesting of the components is represented via indentation. The procedure lists the components of the application specified in the procedure call or of the one already connected.

Note The procedure dumpComponents() prints its output to the QF-Test terminal displayed in the botton part of the QF-Test window. The output is not displayed in the terminal or consoles which can be opened separately (client terminal and scripting consoles).

53.1.3
Information on single GUI elements

The procedure qfs.automac.helpers.dumpComponent allows you to print further information for single GUI elements, including a list of the methods available for the element as well as attribute values.

53.1.4
Identifiers for GUI elements

All procedures of the standard library package performing actions on native MacOS applications need to determine the respective GUI element as the first step and then perform the action in a second step. You find the procedures in the package qfs.automac.component. Because all procedures use qfs.automac.component.getControl to identify the GUI element, the parameters of this helper procedure are valid for all the procedures performing an action on a GUI element.

Valid parameters:

If you specify more than one parameter the procedure looks for the GUI element for which all values match.

identifier
Specify identifier for the unique identifier of the GUI element within the window. It has to be set explicitly during application development, which unfortunately does not always happen.
label
The label usually corresponds to the text displayed. It does not have to be unique and you may have to specify other parameters additionally. In case you use the Accessibility Inspector for the analysis of the GUI elements the attribute is called either Label or AXDescription.
title
The title usually corresponds to the text displayed. It does not have to be unique and you may have to specify other parameters additionally.
role
The role is a value from a predefined list of component types, e.g. AXButton.
roleType
roleType specifies the type of the GUI element. In case you use the Accessibility Inspector for the analysis of the GUI elements the attribute is called either Type or AXRoleDescription.
subrole
subrole is an additional specification to role.
index
When there are more than one GUI element matching the given parameters the index specifies the one to pick. index starts at 0.
53.1.5
Actions on GUI elements

You will find procedures in the package qfs.automac.component of the standard library for the most common actions. You are free to enhance the package. We recommend to use a separate test suite for the enhancement and not to change the qfs.qft since we continuously update the standard library and ship a new version with every QF-Test release.

Mouse click
Procedure: qfs.automac.component.click

The procedure waits for the given component and the replays a click event to the GUI element.
Wait for component
Procedure: qfs.automac.component.waitForComponent

The procedure waits for the given component and returns control to the calling node as soon as it finds the component. The given timeout (in milliseconds) is the maximum time to wait. It throws an exception if the component is not found within the given time.
Text input
Procedure: qfs.automac.component.setValue

The procedure waits for the given component and then enters the value. It uses the method setValue() of the Accessibility interface.
Keyboard events
Procedure: qfs.automac.sendKey

The procedure lets you replay any key like a single letter, a digit, a function key, etc, also combined with the modifiers SHIFT, CTRL and ALT. The event is replayed to the component with the focus in application.
Fetch text
Procedure: qfs.automac.component.getValue

The procedure waits for the given component and then fetches the value via the method getValue().
Fetch geometry
Procedure: qfs.automac.component.getGeometry

The procedure fetches and returns the x and y screen coordinates of the upper left corner of the component as well as the width and height. They are returned as comma-separated text.
Check text
Procedure: qfs.automac.component.checkValue

The procedure fetches the text of the GUI elements via the procedure getValue() and compares the value returned with the given text.

Check geometry
Procedure: qfs.automac.component.checkGeometry

The procedure fetches the geometry data via getGeometry() and compares them to the given values.
Image check
Procedure: qfs.automac.component.checkImage

The procedure relies on a file with a reference image. The file needs to have a png format. The procedure determines the screen coordinates of the element via qfs.automac.component.getGeometry. The actual comparison is done via the procedure getPositionOfImage() of the qfs.autoscreen package of the standard library.

Select an item in a menu
Procedure: qfs.automac.menu.selectItem

Especially when single-stepping through the test when debugging it is useful to have a procedure which clicks to a menu and its menu item which can be executed in one step. Thus the application will not loose the focus between steps which might cause the menu to close.