Situation analysis and challenge

TimeNET is an interactive Software tool that should facilitate the modulation of stochastic Petri nets. In several students’ theses diverse functionalities of this software have been being developed and the already existing ones have been being optimized. The consequence was that TimeNET version 4 had significantly more features than its predecessor TimeNET 4.0. On the other hand this approach had the disadvantage of a huge number of developers working on one project. Every newly developed feature was tested itself, but the testing of the whole TimeNET Software became more difficult after several development cycles.

Conclusion

Comparison of the tools: QF-Test, Ranorex, GUIDancer, Maveryx, Sikuli

The experiences with the test tools have shown that there are differences in their ability for the TimeNet project. Absolutely inappropriate for the use case of “Test automation of the TimeNET-GUI” is the tool Sikuli. That’s why I left it out in the following overview of the different test tools. In the following table are the advantages and disadvantages of the remaining tools. For every criterion every tool reached different number of points.

CriterionQF-TestRanorexGUIDancerMaveryx
Costs---+++++
Supported platform(s)+++-++++++
Testable applications+++++++++-
Specialized in Java+++-++++++
Logfiles and analysis options++++++++++
Automated test execution++++++++
Usability / Training period++-+++

Outlook and conclusion

There are two tools that seem appropriate for the use as GUI test automation tool and that are recommended. This is on the one hand the commercial tool QF-Test, which offers significant reductions for academic projects. On the other hand the tool GUIDancer is an option. Both tools run on Windows, Linux and additionally on macOS and are specialized in Java applications. The decision for one of the tools is primarily depending on the costs. QF-Test is more professional and the extensive support possibilities speak in favor of the use of this tool. But there are costs and we have to think about them for university. If the monetary costs are the decisive argument, you should decide for the cost free GUIDancer.

The author’s recommendation is a further evaluation of both tools: QF-Test and GUIDancer. For this thesis it would have taken too long to evaluate directly on the TimeNET project. This can be the subject of a further thesis. After the decision it is possible to create the first test cases for the TimeNET project. Then a documentation of the creation of test cases would make sense.

The first requirement of this thesis was the creation of a test tool, what wasn’t too useful since there are already a sufficient number of such tools in the market. The result of the evaluation of those is an overview to give an action recommendation for the TimeNET project. Independent from the tool decision the subject test automation should be kept in focus of the project leaders. Since software tests help achieving better software quality and maintain it in the long run.

The whole evaluation report can be found here (German PDF only).

Advanced seminar: Draft of a test automation of TimeNet-GUI - December 2012, Katharina Schröder, Technische Universität Ilmenau, Germany.

(Original German texts and citations are translated into English.)