Introduction to QF-Test Plugin Development

Some people may have stumbled over the plugin directory in QF-Test and wondered why it is there and how to use it. In fact, the directory (with its subdirectories qftest and sut) serves multiple purposes:

Upfront, if you place any .jar file containing Java classes in that directory, they will be automatically added to the QF-Test and/or SUT classpath and therefore available for use in server or SUT script nodes, in data driver nodes (e.g. JDBC drivers) etc. If the library is placed in the qftest subdirectory, the contained classes are only available within the main QF-Test process and server scripts, but not within the classpath of the tested application. If you place the library in sut, you can use it within SUT scripts only. A library placed directly in plugin is available in both context.

In addition, it is possible to enhance the QF-Test test execution itself, or to add special functionality. Unforgotten is the holistic plugin, which caused QF-Test to play “characteristic” sounds during test execution so users could listen to their tests running. While this example may be rather playful, we will see how to use this mechanism for more serious tasks.

With QF-Test 8 and the QF-Test Grade plugin 2.1.0 we made the development of QF-Test plugins very comfortable - it’s time to take a deep dive into the topic.

In this example, we will build add a plugin which extends QF-Test with an embedded web server that displays the current execution state. The final result can be found at gitlab.com/qfs/qftest-plugins/webstatus-plugin.

Technically, QF-Test plugins are also .jar files which are placed in the plugin folder. But they contain a specific pointer file in META-INF/services which allows QF-Test to load the plugin class at runtime.

For development, any IDE which supports Gradle builds can be used, for example IntelliJ. To follow along, create a new project, name it webstatus, select Gradle as build system and make sure that at least Java 17 is selected as JDK.

New Project view in IntelliJ

As a first step, we add the QF-Test Gradle plugin to the project by adding it to the plugins section of the build.gradle file:

plugins {  
    id 'java'  
    id 'de.qfs.qftest' version '2.1.0'
}

If we now reload the project (e.g. using the reload button in the “Gradle” IntelliJ panel), Gradle will automatically download the latest QF-Test and add it to the project’s testImplementation dependencies.

IntelliJ Gradle view

You can instruct the plugin to use the QF-Test installation you already have on your system by adding the version = 'local' property in a new qftestsection of the build.gradle file (If you did not install QF-Test at the default location, you can use the versionDir property instead, as described in the documentation of the QF-Test Gradle Plugin):

qftest {  
    version = 'local'  
}

Until now, the instructions have been similar to any project which uses the QF-Test Gradle plugin for test execution. To put the project into “QF-Test plugin development mode”, we need to add another property to the qftest section of build.gradle:

qftest {  
    version = 'local'  
    pluginClass = 'de.qfs.apps.qftest.plugins.WebStatusPlugin'  
}

In our example de.qfs.apps.qftest.plugins.WebStatusPlugin is the fully qualified class name of the plugin class we’re about to develop. Since the QF-Test plugin will only be available in the QF-Test main process, we can tell the Gradle plugin to focus on this development environment:

qftest {  
    version = 'local'  
    pluginClass = 'de.qfs.apps.qftest.plugins.WebStatusPlugin'
    devScopes = ['qftest']  // default: ['qftest','sut']
}

After having setup the build system, we can start developing the plugin itself: Let’s create a new class and let it implement de.qfs.apps.qftest.shared.QFTestPlugin:

Skeleton of the plugin class

Since the Gradle plugin automatically extended the implementationdependencies, the required QF-Test classes are directly available using code completion. As a first step, we simply add a “Hello World” output:

@Overridepublicvoidinit() {  
    System.out.println("Hello plugin world!");  
}

To start up QF-Test including the plugin, we can use the startQFTest task of Gradle - the Gradle Plugin will automatically take care of creating the required services file and make the new class available to QF-Test:

 % ./gradlew startQFTest
Starting a Gradle Daemon

> Task :startQFTest
Starting QF-Test (local)

BUILD SUCCESSFUL in 7s
3 actionable tasks: 3 executed

Our plugin class is compiled, QF-Test starts up - but no output! What went wrong?

By default, the startQFTest task is used to start QF-Test interactively in a “fire-and-forget” manner - no outputs are attached and the Gradle task does not wait until QF-Test is closed. To see the debug output, we must change the fork property of the task, by adding this new definition to the build.gradle:

task startQFTestAndWait(type: StartQFTestTask) {  
    fork = false  
}

Now, we use this new task to start QF-Test, and our output becomes visible:

% ./gradlew startQFTestAndWait

> Task :startQFTestAndWait
Starting QF-Test (local)

Hello plugin world!
<=========----> 75% EXECUTING [17s]
> :startQFTestAndWait

The init() method is called very early during the startup of QF-Test or the SUT, as soon as the class has been loaded. You can distinguish different execution contexts by calling the static methods isRunningInSUT() (i.e. SUT (true) or QF-Test (false)) and isRunningInteractively()(interactive (true) or batch mode (false)) of the class de.qfs.apps.qftest.shared.Util.

If you want to delay execution until QF-Test has loaded or the SUT has been connected, you can listen for the "qftest-loaded" or "sut-loaded"notifications respectively:

import de.qfs.lib.notifications.DefaultNotificationCenter;  
import de.qfs.lib.notifications.Notification;  
  
[...]

@Overridepublicvoidinit() {  
    System.out.println("Plugin init");  
    DefaultNotificationCenter.instance().addObserver(this::qftestLoaded,"qftest-loaded");  
    DefaultNotificationCenter.instance().addObserver(this::sutLoaded,"sut-loaded");  
}  
  
privatevoidqftestLoaded(Notification n) {  
    System.out.println("QF-Test loaded");  
}  
  
privatevoidsutLoaded(Notification n) {  
    System.out.println("SUT loaded");  
}

Notifications are a powerful mechanism in QF-Test to send synchronous messages (= well defined Strings with optional additional data) in a decoupled way. The concept is well known from the Apple Foundation framework.

You can also use notifications in your QF-Test scripts - just type “notifications.” in a script and press CTRL+SPACE to get a documentation about the available UserNotifications and simplified sender and observer methods. We can also use such a UserNotifications with the notification observer technique - e.g. to monitor test execution state in a basic fashion (If compilation fails due to a class file version issue, make sure you’re using at least JDK 17 for your project):

import de.qfs.apps.qftest.extensions.qftest.TestRunEvent;  
import de.qfs.apps.qftest.extensions.qftest.TestSuiteNode;
import de.qfs.apps.qftest.shared.script.modules.UserNotifications;

[...]

publicclassWebStatusPluginimplementsQFTestPlugin {  
  
    publicfinal Queue<TestSuiteNode> nodeStack = new LinkedList<>();  
    publicboolean running = false;  
  
    @Overridepublicvoidinit() {  
        DefaultNotificationCenter.instance().addObserver(this::runStarted,
            UserNotifications.RUN_STARTED);  
        DefaultNotificationCenter.instance().addObserver(this::runStopped, 
            UserNotifications.RUN_STOPPED);  
  
        DefaultNotificationCenter.instance().addObserver(this::nodeEntered, 
            UserNotifications.NODE_ENTERED);  
        DefaultNotificationCenter.instance().addObserver(this::nodeExited, 
            UserNotifications.NODE_EXITED);  
    }  
  
    privatevoidrunStarted(Notification notification) {  
        nodeStack.clear();  
        running = true;  
    }  
  
    privatevoidrunStopped(Notification notification) {  
        running = false;  
    }  
  
    privatevoidnodeEntered(Notification notification) {  
        TestRunEvent event = 
            (TestRunEvent) notification.getUserInfoValue("event");  
        nodeStack.add(event.getNode());  
    }  
  
    privatevoidnodeExited(Notification notification) {  
        nodeStack.poll();  
    }  
  
}

In order to present the current test execution state in the browser, we need an embedded web server delivering this information as a web page. Unfortunately, Java 17 (which is used to execute QF-Test) does not include a simple HTTP server module yet (this has only been introduced with Java 18). But a look in the lib library of QF-Test reveals that the undertow library is bundled, which includes a simple HTTP server class. When we add 'undertow' to the devScopes property, the corresponding library bundled with QF-Test is automatically added to our implementation dependencies:

qftest {  
    version = 'local'  
    pluginClass = 'de.qfs.apps.qftest.plugins.WebStatusPlugin'
    devScopes = ['qftest', 'undertow']
}

With this library, we sketch out a simple WebStatusServer class which returns the current execution state upon request:

package de.qfs.apps.qftest.plugins;  
  
import io.undertow.Undertow;  
import io.undertow.server.HttpServerExchange;    
import io.undertow.util.Headers;
  
publicclassWebStatusServer {  
    publicWebStatusServer(WebStatusPlugin status) {  
        Undertow server = Undertow.builder()  
                .addHttpListener(9000,"localhost")  
                .setHandler(e -> {  
                    if (status.running) {  
                        var node = status.nodeStack.peek();  
                        if (node != null) {  
                            output(e, node.getType() + ": " + node.getName());  
                        } else {  
                            output(e,"Test execution running");  
                        }  
                    } else {  
                        output(e, "Test execution stopped");  
                    }}).build();  
        server.start();
  
        Runtime.getRuntime().addShutdownHook(new Thread(server::stop));

        final Undertow.ListenerInfo listenerInfo = server.getListenerInfo().iterator().next();
        System.out.println("QF-Test status available at " +
                listenerInfo.getProtcol() + ":/" + listenerInfo.getAddress());
    }  
  
    protectedvoidoutput(HttpServerExchange exchange,  
                          String response) {    
        exchange.getResponseHeaders().add(Headers.REFRESH,"1");
        exchange.getResponseSender().send(response);  
        exchange.endExchange();  
    }  
}

We load this web server in the WebStatusPlugin class as soon as QF-Test has loaded:

privatevoidqftestLoaded(Notification notification) {  
    new WebStatusServer(this);  
}

But unfortunately, the start of our plugin now fails:

% ./gradlew startQFTestAndWait

> Task :startQFTestAndWait
Starting QF-Test (local)
1 (...) main de.qfs.lib.notifications.DefaultNotification.getDefaultExceptionHandler().ExceptionHandler.handleException(Throwable,Notification,Observer): (#147) exception: java.lang.NoClassDefFoundError: io/undertow/Undertow (...)

The undertow library appears to be bundled by QF-Test, but is not normally available in the default classpath, so we have to do some reflection calls to add the library to the classpath and instantiate it - luckily the QF-Test de.qfs.lib package has helper methods for these tasks:

import de.qfs.apps.qftest.shared.system.Native;
import de.qfs.lib.util.DynamicClassLoader;  
import de.qfs.lib.util.Reflector;

[...]

privatevoidqftestLoaded(Notification notification) {  
    ClassLoader cl = this.getClass().getClassLoader();  
    try {  
        DynamicClassLoader.loadJarIntoClassLoader(cl, 
                Native.getVersionDir() + "/lib/undertow.jar");
        Class<?> serverClass =  
                cl.loadClass("de.qfs.apps.qftest.plugins.WebStatusServer");  
        Reflector.createInstance(serverClass, WebStatusPlugin.class, this);  
    } catch (Exception ex) {  
        System.err.println("Could not start WebStatus server: ");  
        ex.printStackTrace();  
    }  
}

Now start QF-Test, head over to [http://localhost:9000] and watch our test progress on every automatic reload!

Finally, to ship this great new QF-Test plugin, we build an all-containing .jarfile using ./gradlew jar and copy the resulting file from the build/libsfolder to the plugin/qftest folder of QF-Test, so it is available in all our test runs.

In an upcoming blog post, we’ll see how easy it is to use the QF-Test plugin mechanism to extend the new QF-Test 8 Assertion API. Until then, let this post inspire your creativity!