In the following chapter the possibilities of different test automation tools concerning the network integration software Service ON Integrator will be examined.
In the foreground of the tests is the graphical interface of the ServiceOn Integrator (SOI) the portal which is executable as Java WebStart application under Windows as well as under Linux. The biggest part of the existing functionalities that the SOI is offering is available via this interface. The emphasis is on the two GUI automated testing tools QF-Test and Marathon.
The image shows the SOI portal without an open component. The resource tree on the left helps opening the single components (e.g. error handling).
Here the GUI testing tools Marathon and Q-Test will be evaluated whether they are suitable for automating the interaction with the SOI portal and which offers the higher value added.
Since Marathon doesn’t have special functions to start WebStart applications all jar-archives must be saved locally on the test computer, before starting the portal with Marathon. This can happen via e.g. manual starting of the SOP loader (the SOI specific Java WebStart implementation).
All downloaded jar files must be made available in the class path of the Marathon test project what can be automated via a script. A bit more difficult is setting defined Java properties in the jnlp files, but it is possible due to the xml structure of the jnlp files (Java Network Launching Protocol) via a parser.
A further challenge that arose during the software tests was Marathon’s manner of handling component recognition, because the title of the single program windows are created dynamically and it happens sometimes that texts are used multiple times within a window.
The handling of the Marathon GUI testing tool is very developer centered, especially due to the recording completely in Jython. This requires deep knowledge of this scripting language, especially when recordings should be modularized and reused.
Starting the SOI portal can be done via the SOPLoader provided that the jdk used in the SOPLoader’s jdk is instrumented.
Also the component recognition in QF-Test is extremely depending on the language version used, because normally no names for components are assigned in the source code of the project. Due to the possibility of intervening in the component recognition manually, the first tests could achieve first good results. We early found out that the maintenance of component recognition transferred a high development effort to the tests that usually should go into the development of the system.
The following possibility is a great advantage: Accessing components of complex structure elements like e.g. tree structures or tables depending on the situation via an absolute path or the relative position. This enables choosing one element directly as well as checking whether an element with special properties is available without knowing the exact name.
The usability of the QF-Test suite differs radically from other Java GUI test automation tools due to its node structure. This structure gives you an easy overview of recorded or manually configured sequences. Despite of this easy structure sometimes test scenarios need an intervention via scripts. An example therefore is checking of table content that just consist of images. QF-Test offers the option of testing the image of a single row, but this test is then depending on the width of the cell. Listing 12 checks the widths of a cell via the run context rc provided by QF-Test to set it in a simulated “drag and drop sequence” to a defined width.
Additionally many tests just require simple test requests and the comparison with a regular expression or a text section. Here also the Jython script functionality is necessary.
A further interesting option of automation of SOI offers the node for executing a shell command. So also functionalities can be called up from the core system that is not locally available
Both tools follow very different approaches in test administration. Marathon’s user group is consisting of developers and testers with development background due to its source code centering. QF-Test tries to avoid the contact of testers with source code, just when this is necessary. This decoupling should enable that testers with just few programming experience can record and modularize tests. For the few scripting cases it is enough to have some testers with knowledge in Jython.
The issue of component recognition is actually not enough supported by none of those two tools by default. QF-Test delivers the possibility of a fast workaround by a name resolver that offers point by point good results. In the long run you will have to change the application. I think you can accept having the development effort for the goal of testing an application more easily, so to extend the design of the application for a better testability.
To put in a nutshell QF-Test is offering the more promising approach of testing the ServiceOn Integrator.
