QF-Test at Airbus Defence and Space

The continuous deployment chain for the SW product and its variant is illustrated in the figure above. In the build and deployment process, both the SW to be tested and QF-Test are initially packaged and stored in a repository. In addition to QF-Test, the QF-Test package stored in the NEXUS repository consists of two other parts, a product library (Product Library.qft) and a corresponding test suite (Smoke-Test-Suite.qft) for each platform. The product library contains all records to access graphical elements of the Eclipse RCP based client: menus, parts, tables, buttons, sliders etc. The Windows and Components Tree is also stored here, which together with a one-to-one component identifier allows access to the graphical elements of the client. The dynamic flow of the tests varies between the two server solutions (RHEL and Oracle Solaris) and is adapted accordingly in the respective Smoke test suite for each platform. The test suites implemented here are started daily after installation and make use of the basic functions stored in the product library. First, test suites are executed to configure the data transfer between server and client (ports, IP addresses, protocols). Then, the correctness of the implemented interfaces between server and client is verified for each platform by simulating and automatically checking a simple data exchange (images, videos, motion information). The use of Data Driver/Tables facilitates the execution of the same test sequence with different configurations, while keeping the test code maintainabe. The individual procedures are encapsulated in simple try/catch blocks, so unforeseen errors can be easily caught and logged.

Running about 100 tests automatically and in parallel takes about one hour. Running the tests manually used to occupy one person for three hours per configuration. For the standard product and its configuration (two modes), this results in a saving of currently 8 hours per day, time which we happily invest in further automated tests or other activities.

2.3 Conclusion

Through this approach of "write once, test anywhere", we found a big amount of errors that were related to the configuration of the application on the respective operating system. Overall, test automation has a very positive effect on accelerating the development process, since test results are available quickly, and time-consuming manual tests can be eliminated. Furthermore, successful automated test results are considered a handover criterion for the installation and configuration of the SW at the next integration stage.

QF-Test as a tool for GUI test automation is designed for cross-platform use and is very well suited for our technical needs. QF-Test is easy to learn and we were able to integrate it into our operating environment without any problems. Particularly noteworthy is the ability to add the included pixel-based image comparison algorithms for images and videos to the test flow. In addition, we took advantage of QF-Test's exception handling in the flow control of the individual test suites. Finally, the very fast and reliable German support from the manufacturer also helped us out.

2.4 Outlook

For the targeted testing of the product variant already in operational use, further regression tests are planned, which are to check frequently executed parts of the system. This involves the simulation of long-running and asynchronous processes as well as investigations of the handling of large data volumes in order to draw conclusions about the performance (CPU load, memory requirements) of the system. Here, too, the already existing product library can be used, only a new regression test suite has to be created that contains the new use cases. With this planned approach we hope to further increase the quality of the SW and the minimization of the risk that errors are only found at a late stage and are therefore much more expensive to fix.

Author: Thomas Schöning
ISTQB Certified Test Manager
Airbus Defence and Space GmbH
Multi-INT Products & Projects Germany