QF-Test and Java client code coverage analysis

Code coverage analysis can be used in order to analyze to which degree the source code of a program gets executed during test execution. A high percentage of executed code suggests a lower chance of undetected software bugs.

QF-Test alone does not provide a possibility to generate code coverage reports. However it is possible to combine QF-Test with Open Source Tools like JaCoCo. The integration of such tools make it possible to generate such code coverage reports for the teststeps performed by QF-Test.

How to integrate JaCoCo into a QF‑Test test run

In order to integrate JaCoCo into a QF-Test test run, you first need to extract the 'jacocoagent.jar' file from the lib folder of the 'jacoco-$(version).zip' file. This zip file can be downloaded from here.

It is easiest to integrate JaCoCo, if the Java application you are testing is a .jar or a .class file. In this case you simply need to locate the 'Start Java SUT client' node that is starting your Java application. Then, specify

-javaagent:<path to jacocoagent.jar>=destfile=<path+filename to where jacoco should write its statistics>

in the "parameter table" of this node.

In case you have a .exe or a .bat Java application, the above way will not work. In this case two Jython Server Scripts are needed. The first script with the content

jacocojar = "<path to jacocoagent.jar>"
destfile = "<path+filename to where jacoco should write its statistics>"
rc.setProperty("env", "JAVA_TOOL_OPTIONS", "-javaagent:%s=destfile=%s" % (jacocojar, destfile))

rc.setProperty("env", "JAVA_TOOL_OPTIONS", None)

can be inserted after the 'Wait for client to connect' node.

Remarks

• Please note, that JaCoCo is not directly writing its report into the result file. The result file is written, when the application you are testing gets terminated!

• The JaCoCo result file is quite unreadable when opened with most text editors. So in order to analyze the statistics you probably want to use a plugin for your IDE.

• In order to gain control over the application you are testing, QF-Test is injecting some of its own classes into the application. As a result the result file does not only contain code coverage statistics for the classes of your application, but also for some QF-Test classes. Because you probably do not want to test QF-Test but your application, you can simply ignore the statistics created for QF-Test classes.

• Please be sure that the process that is executing your application has the rights to write the JaCoCo result file.

• When specifying a (Windows) path in a Jython/Groovy Script, please replace all "\" characters in the path by "/".