Extending the QF-Test Assertion API – A practical introduction

Since QF-Test 8 there is a new and comfortable way to check conditions in scripts: The QF-Test Assertion API, which is loosely based on the chai.JS library known from Javascript.

In a Groovy script, for example, we can now write:

def foo = 'bar'  
def beverages = [ tea: [ 'chai', 'matcha', 'oolong' ] ]  
  
expect(foo).to.be.a('String')  
foo.should.be.equal('bar')  
expect(foo).to.have.lengthOf(3)  
expect(beverages).to.have.property('tea').with.lengthOf(3)

The API of the fluid check expressions is well equipped so that most checks can be written very easily. In some special cases, however, you may wish to extend the API, as is possible with chai.JS plugins. For example, it would be nice if you could use the following expression to check whether a given string represents a valid JSON object:

def validJson = '[1,2,3,"s",{}"key":"value"}]'
def invalidJson = '[{]'

validJson.should.be.json()
invalidJson.should.not.be.json()

QF-Test offers an interface for such extensions of the Assertions API as well, which we would like to demonstrate in this blog post.

To enable a dynamic extension of the API, a proxy object is created under the hood for each assertion whenever a test expression is executed. In addition to the standard methods of each assertion, this proxy also supports any methods that have been registered as extensions in the meantime. An interface definition in the form of an interface is first required for these extensions:

package de.qfs.lib.assertions.external;

import de.qfs.lib.assertions.Assertion;
  
publicinterfaceJsonAssertionextendsAssertion {
    Assertion json();
}

Please note that

• the interface is normally placed in the en.qfs.lib.assertions.external package,
• it extends the existing Assertion interface,
• and that each new API method in turn returns the assertion object itself.

This makes it possible to chain the method calls. The code to be executed can now be defined in a separate class that implements the interface and derives it from en.qfs.lib.assertions.AssertionExtension. However, it is easier to do this directly in the interface definition as a defaultimplementation:

default Assertion json() {  
    return self();  
}

The implementation of the method does not yet do very much, it merely returns a reference to its own assertion object, as required. To perform or report the actual check, the inherited method doAssert(condition, message, negatedMessage, expected, actual) must be called:

• condition is a boolean value or a function with a boolean return value that contains or returns the result of the check.
• message is a string that can be used for an error message or for the check performed in QF-Test - the placeholder #{this} is replaced by the checked object, #{exp} by the expected value and #{act} by the actual value (usually identical to the checked object).
• negatedMessage contains the message that is used for checks whose call chain contains a not.
• expected contains the expected value.
• actual contains the actual value.

With the exception of condition, all parameters are optional and can be omitted at the end of the parameter list.

Our assertion method might now look like this:

default Assertion json() {  
    doAssert(this::isObjectJson,  
            "expected #{this} to be a json string",  
            "expected #{this} not to be json string");  
  
    return self();  
}

We have outsourced the actual check to a separate method in which we try to read the input as JSON and return true if successful and false if an error occurs:

defaultbooleanisObjectJson() {  
    try {  
        final Object s = _obj();
        if (s instanceof String) {  
            Json.parse((String) s);  
            returntrue;  
        }  
        returnfalse;  
    } catch (final ParseException ex) {  
        returnfalse;  
    }  
}

Note that the expression _obj() was used to access the object to be checked.

In order to be able to use our new method in check expressions, it must be registered once with the AssertionFactory. This factory is a singleton and provides the instance method registerExtension(extension, overwrite) for this purpose:

• extension is the new assertion interface of the extension. Either a class that implements the interface or an object of such a class can be specified here.
• overwrite determines how to proceed with already registered methods of the same name. If true, the method of the newly registered extension is preferred, with false the previously registered one is.

The easiest way to register an extension is to do so when initializing a QF-Test plugin. A general description of how to build QF-Test plugins can be found in the previous blog article. For the plugin that provides our Assertion API extension, the corresponding plugin class could look like this:

import de.qfs.apps.qftest.shared.QFTestPlugin;  
import de.qfs.lib.assertions.AssertionFactory;  
import de.qfs.lib.assertions.external.JsonAssertion;  
  
publicclassJsonAssertionPluginimplementsQFTestPlugin {  
  
    @Overridepublicvoidinit()  
    {  
        AssertionFactory.instance()
                    .registerExtension(JsonAssertion.class, false);  
    }  
}

If you want to add “properties” to the call chain, please note that the associated methods must each be preceded by a get - the script engines Jython or Groovy then automatically convert the properties in the check expression into the corresponding method call.

As an example, we want to extend our new API to include the check for “primitive” JSON objects:

def primitiveObj = '"String"'
def nonPrimitiveObj = '[1,2,3]'

primitiveObj.should.be.primitive.json()
nonPrimitiveObj.should.not.be.primitive.json()

Then we add the corresponding getPrimitive method to our JsonAssertion interface:

default Assertion getPrimitive() {  
    self().flag("primitive", true);  
    return self();  
}

and add the test method:

defaultbooleanisObjectJson() {    
    finalboolean primitive = AssertionUtil.flagAsBoolean(self(), "primitive");
    
    try {  
        final Object s = _obj();
        if (s instanceof String) {
            final JsonValue value = Json.parse((String) s);  
            
            if (primitive) {  
                finalboolean valueIsObjectOrArray = 
                                  value.isObject() || value.isArray();  
                return ! valueIsObjectOrArray;  
            }  
            
            returntrue;  
        }  
        returnfalse;  
    } catch (final ParseException ex) {  
        returnfalse;  
    }  
}

A typical Assertion API scheme is used here: in the property method, a status value is set in the assertion object with self().flag(name, value), which is then retrieved during evaluation. To achieve this, the self().flag(name) method returns the corresponding status value. To simplify matters, the class en.qfs.lib.assertions.AssertionUtiloffers the static helper methods flagAsBoolean(assertion, name) and flagAsString(assertion, name).

The message for the error case should also be adjusted to reflect the status:

default Assertion json() {  
  
    finalboolean primitive = AssertionUtil.flagAsBoolean(self(), "primitive");  
    final String descriptor = primitive ? "primitive " : "";  
  
    doAssert(this::isObjectJson,  
            "expected #{this} to be a " + descriptor + "json string",  
            "expected #{this} not to be " + descriptor + "json string");  
  
    return self();  
}

Finally, in order to hide the internal details in the stack trace of the AssertionError thrown in the event of an error, the test method can define itself as an entry point (ssfi stands for start stack function indicator)):

default Assertion json() { 
    _ssfi(JsonAssertion.class,"json");
    
    (...)
}

Note: Due to the class loader used, extensions are currently restricted to Server scripts and Groovy SUT scripts.

The code of the entire extension is available on GitLab.