Scripting (Jython, Groovy and JavaScript)
This section explains technical details about the Jython integration in QF-Test and serves as a reference for the whole API exposed by QF-Test for use in Jython, Groovy and JavaScript scripts. For a more gentle introduction including examples please take a look at chapter 11.
Module load-path
The load-path for scripting modules is assembled from various sources in the following order:
- the script directory in the user configuration directory
-
the directory
qftest/qftest-9.0.0/<scriptlanguage>
In addition, during Server script or SUT script node execution, the directory of the containing test suite is prepended to the path.
The directory
qftest/qftest-9.0.0/<scriptlanguage>
contains
internal modules of the specific script language. You should
not modify these files, since they may change in later versions of
QF-Test.
The script directory in the user configuration directory is the place to put your own shared modules. These will be left untouched during an update of QF-Test. You can locate the rescpective script directory via »Help« under "System info" as "dir.<scriptlanguage>".
Modules that are specific to a test suite can also be placed
in the same directory as the test suite. The file extension for all
modules must be .py
.
In Jython you can add additional directories to the load-path by defining the python.path
system property.
The plugin directory
The script languages can also be used to access Java classes and methods beyond the scope of QF-Test by simply importing such classes, e.g.
from java.util import Date from java.text import SimpleDateFormat print SimpleDateFormat("yyyy-MM-dd").format(Date())
The classes available for import are those in the Java class path, i.e. all classes
of the standard Java API and QF-Test's own classes. Note that QF-Test ignores the CLASSPATH
environment variable, but you may define QFTEST_CLASSPATH
with the same value if necessary
(e.g. to start a client application). For the SUT things also depend on the ClassLoader concept in use.
WebStart and Eclipse/RCP in particular make it difficult to import classes directly from
the SUT.
Additionally, there are plugin directories into which you can simply drop a jar file to
make it available to scripts.
QF-Test searches for a directory called plugin
.
You can locate the currently used plugin directory via »Help« under "System info" as "dir.plugin".
The location of the plugin directory can be overridden with the command line argument
-plugindir <directory>
.
Jar files in the main plugin directory are available to both
Server script and SUT script nodes. To make a jar
available solely to Server scripts or solely to
SUT scripts, drop it in the respective subdirectory called
qftest
or sut
instead.
Initialization (Jython)
During QF-Test and SUT startup an embedded Jython interpreter is
created. For QF-Test, the module named qftest
is
imported, for the SUT the module named qfclient
.
Both are based on qfcommon
which contains shared code.
These modules are required to provide the run context interface and
to set up the global namespace.
Next the load-path sys.path
is searched for your
personal initialization files. For QF-Test initialization, the file
called qfserver.py
is loaded, the file called
qfsut.py
is used for the SUT. In both cases
execfile
is used to execute the contents of these files
directly in the global namespace instead of loading them as
modules. This is much more convenient for an initialization file
because everything defined and all modules imported will be directly
available to Server scripts and SUT scripts. Note that
at initialization time no run context is available and no
test suite-specific directory is added to sys.path
.
Namespace environment for script execution (Jython)
The environments in which Server scripts or SUT scripts are executed are defined by the global and local namespaces in effect during execution. Namespaces in Jython are dictionaries which serve as containers for global and local variable bindings.
The global namespace is shared between all scripts run in the same
Jython interpreter. Initially it will contain the classes
TestException
and UserException
, the
module qftest
or qfclient
for QF-Test or
the SUT respectively, and everything defined in or imported by
qfserver.py
or qfsut.py
. When assigning a
value to a variable declared to be global with the
global
statement, that variable is added to the global
namespace and available to scripts run consecutively. Additionally,
QF-Test ensures that all modules imported during script execution are
globally available.
The local namespace is unique for each script and its lifetime is
limited to the script's execution. Upon invocation the local
namespace contains rc
, the interface to QF-Test's
run context, and true
and false
bound to
1
and 0
respectively for better
integration with QF-Test.
Accessing or setting global variables in a different Jython
interpreter is enabled through the methods fromServer
,
fromSUT
, toServer
and toSUT
.
Run context API
The run context object rc
is an interface to
the execution state of the currently running test in QF-Test. Providing
this wrapper instead of directly exposing QF-Test's Java API
leaves us free to change the implementation of QF-Test without
affecting the interface for scripts.
Following is a list of the methods of the run context object
rc
in alphabetical order. The syntax used is a bit of a
mixture of Java and Python. Python doesn't support static
typing, but the parameters are passed on to Java, so they must be
of the correct type to avoid triggering exceptions.
If a parameter is followed by an '=' character and a value, that
value is the default and the parameter is optional.
Note
Please note that the Groovy syntax for keyword parameters is different from Jython and
requires a ':' instead of '='. The tricky bit is that, for example,
rc.logMessage("bla", report=true)
is perfectly legal Groovy code yet
doesn't have the desired effect. The '=' here is an assignment resulting in the value
true, which is simply passed as the second parameter, thus the above is equal to
rc.logMessage("bla", true)
and the true is passed to
dontcompactify
instead of report
. The correct Groovy version
is rc.logMessage("bla", report:true)
.
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The expand
parameter
The methods
getStr
, getObj
, getInt
, getNum
, getBool
,
getPattern
and getJson
support the optional parameter expand
.
This parameter controls, whether to expand the value of the variable recursively, which
means whether to treat substrings of the String value of the variable value which happen to have the QF-Test
variable syntax $(somecharacters) as a variable to be expanded or as simple text.
If the parameter is omitted or null
, the replacement is performed (recursively) if and only if
the variable value is a String. To avoid probems, some strings, e.g. the return value of a Fetch text step,
the client output from the special group ${qftest:client.output.<name>}
or the result of the
standard procedure qfs.utils.readTextFromFile
, are also not automatically expanded, but only
if the expand
parameter is explicitely set to true
.
Note that if you want to
set this parameter, you must use Python keyword syntax to
avoid conflicts e.g. with getStr(String group, String
name)
, i.e. rc.getStr("var", expand=0)
instead of rc.getStr("var", 0)
- otherwise the
property 0
would be taken from the group var
.
- Sample
-
Given the QF-Test variables and values
Variable reference value $(simplevar) foo $(nestedvar) A value: $(simplevar) ${group:var} A value: $(simplevar) Table 50.1: QF-Test variables for the expand
parameter sample belowthe parameter
expand
has the following effect:print rc.getStr("nestedvar", expand=True) # "A value: foo" print rc.getStr("nestedvar", expand=False) # "A value: $(simplevar)" print rc.getStr("group", "var", True) # "A value: foo" print rc.getStr("group", "var", False) # "A value: $(simplevar)"
Example 50.2: Usage of the expand
parameter (Jython script)
The qf
module
In some cases there is no run context available, especially when implementing some of
the extension interfaces described in the following sections. The module qf
enables logging in those cases and also provides some generally useful methods that
can be used without depending on a run context. Following is a list of the methods of
the qf
module in alphabetical order. Unless mentioned otherwise, methods are
available in Groovy and Jython and for both Server script and SUT script
nodes.
Note
Please note that the Groovy syntax for keyword parameters is different from Jython and
requires a ':' instead of '='. The tricky bit is that, for example,
qf.logMessage("bla", report=true)
is perfectly legal Groovy code yet
doesn't have the desired effect. The '=' here is an assignment resulting in the value
true, which is simply passed as the second parameter, thus the above is equal to
qf.logMessage("bla", true)
and the true is passed to
dontcompactify
instead of report
. The correct Groovy version
is qf.logMessage("bla", report:true)
.
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
3.0+50.7
Image API
The Image API provides classes and interfaces to take screenshots, to save or load images or for own image comparisons. The image API is designed so that the different methods in general do not throw any exception. Instead, the different methods are logging warnings.
The ImageWrapper
class
For taking screenshots you can use the Jython class ImageWrapper, located in the module
imagewrapper.py
, which comes with the QF-Test
installation.
Here is a short sample Jython script demonstrating the usage of the Image API:
from imagewrapper import ImageWrapper #create ImageWrapper instance iw = ImageWrapper(rc) #take screenshot of the whole screen currentScreenshot = iw.grabScreenshot() #save screenshot to a file iw.savePng("/tmp/screenshot.png", currentScreenshot)
And the same in Groovy:
import de.qfs.ImageWrapper def iw = new ImageWrapper(rc) def currentScreenshot = iw.grabScreenshot() iw.savePng("/tmp/screenshot.png", currentScreenshot)
Following is a list of the methods of the ImageWrapper
class in alphabetical order. The syntax used is a bit of a
mixture of Java and Python. Python doesn't support static
typing, but the parameters are passed on to Java, so they must be
of the correct type to avoid triggering exceptions.
If a parameter is followed by an '=' character and a value, that
value is the default and the parameter is optional.
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The JSON
module
The JSON module, which is available in all script without dedicated import,
parses a JSON string into a data structure of Maps, Lists and primitive types like Integer,
Double, Boolean and String. Serializing is done via the stringify()
method.
Note: In order to read structured JSON data from a QF-Test variable use the rc.getJson()
call.
|
|
||||||||||||||||||||||||||||||||
Natural Language Assertions
Inspired by Chai.js we have implemented our own assertion API for scripting. It can be used from Groovy, JavaScript and Jython scripts.
Motivation
The idea is to make the checks implemented in the scripts in QF-Test more readable and closer to the human
language. Verifying and validating data when working in the Server or SUT Scripts is usually done via
rc.check()
or the Java keyword assert
. They are fine when working with basic
data types like strings. However, it can become tedious when you have to check complex data types like
structured objects, e.g. created from a JSON string. This is where the QF-Test assertions API makes live a lot easier.
Here are two Groovy script examples where you can see the difference between the natural language
assertions and the traditional rc.check() and assert().
def foo = 'bar'
def beverages = [ tea: [ 'chai', 'matcha', 'oolong' ] ]
expect(foo).to.be.a('String')
foo.should.be.equal('bar')
expect(foo).to.have.lengthOf(3)
expect(beverages).to.have.property('tea').with.lengthOf(3)
def foo = 'bar' def beverages = [ tea: [ 'chai', 'matcha', 'oolong' ] ] rc.check(foo instanceof String,"") rc.checkEqual(foo,'bar',"") rc.checkEqual(foo.length(),3,"") assert(beverages.tea!=null) assert(beverages.tea.size()==3)
check
method and Java assert
API documentation
The QF-Test assertions API has the interfaces Assert
and expect
. In Groovy scripts, also
a direct chaining with should
is available. expect
and assert
support language chains,
the Assert
syntax is more traditional.
The result of an assertion can be either be written to the run log as failed or successful check, or optionally as an exception. Additionally, the result of the last assertion can be retrieved as a boolean value, which can be assigned to a variable. For details please see Result handling.
The API documentation is provided in doc/javadoc/qfaa.zip
. The documentation lists all the
methods available for Assert
. They can also be used with expect
and
should
(in Groovy), where they are part of the language chains. Since the QF-Test assertions API is very
similar to Chai.js, many of the examples on https://www.chaijs.com/api/ will also work with QF-Test. For methods
available in Chai.js but not yet implemented for QF-Test please refer to subsection 50.9.2.3.
When working with Assert
you can use autocompletion and display the documentation
of the available methods
by typing Assert.
and then pressing Ctrl+Space.
For Regular expressions use the module java.util.regex.Pattern
.
Language chains
The biggest advantage comes via the language chains.
They can be used with expect()
and should()
.
The following chainable getters are available:
.to .be .been .is .that .which .and .has .have .with .at .of .same .but .does .still .also
def testObj = [ "name": "test", "sub": [ "name": 'test sub' ], "numbers": [1, 2, 3, 4], "hasNumbers" : true ]; expect(testObj).to.be.an('Object').and.is.ok expect(testObj).to.have.property('sub').that.is.an('Object').and.is.ok expect(testObj.sub).to.have.property('name').that.is.a('String').and.to.equal('test sub') expect(testObj).to.have.property('numbers').that.deep.equals([1, 2, 3, 4]) expect(testObj).to.have.property('hasNumbers', true)
expect
rc.setLocal("jsonData", """ { "Actors": [ { "name": "Tom Cruise", "age": 56, "Born At": "Syracuse, NY", "Birthdate": "July 3, 1962", "photo": "https://jsonformatter.org/img/tom-cruise.jpg", "wife": null, "weight": 67.5, "hasChildren": true, "hasGreyHair": false, "children": [ "Suri", "Isabella Jane", "Connor" ] }, { "name": "Robert Downey Jr.", "age": 53, "Born At": "New York City, NY", "Birthdate": "April 4, 1965", "photo": "https://jsonformatter.org/img/Robert-Downey-Jr.jpg", "wife": "Susan Downey", "weight": 77.1, "hasChildren": true, "hasGreyHair": false, "children": [ "Indio Falconer", "Avri Roel", "Exton Elias" ] } ] }""") def data = rc.getJson("jsonData") data.Actors.should.be.a("ArrayList") expect(data.Actors[0]).to.be.a("LinkedHashMap") Assert.instanceOf(data.Actors[0], "LinkedHashMap", "Bla") data.Actors[0].name.should.be.a("String") data.Actors[0].age.should.be.a("Long") data.Actors[0].weight.should.be.a("Double") data.Actors[0].hasChildren.should.be.a("Boolean") rc.setGlobalJson("gData",data)
should
For the documentation of the chainable getters please refer to: https://www.chaijs.com/api/bdd.
Differences between the QF-Test assertions API and Chai.js
Due to the Java implementation, some syntax in QF-Test differs from Chai.js.
-
As
assert
is a reserved word in Java and Groovy, the QF-TestAssert
is spelled differently, which means the first letter is a capital "A". The same applies to the assertionsTRUE
,FALSE
, andNULL
, which have to be written all-caps. -
All methods with
strict*
prefix use==
for comparison, otherwise Java’s Objectequal()
is used. For a list of allstrict*
methods typeAssert.strict
in the script editor and then press Ctrl+Space. -
Assert.test
replacesassert
:Assert.test('foo' !== 'bar', 'foo is not bar') Assert.test({true}, 'Closures can return true')
Example 50.9: Assert.test(...)
Unavailable assertions
Some of the assertions implemented by Chai.JS can not be directly translated from Javascript to Java, and some assertions are not implemented, yet. Among these are:
Assert
-
isAbove(), isAtLeast(), isBelow(), isAtMost()
-
isNaN(), isNotNan()
-
isUndefined()
-
isFinite()
-
throws(), doesNotThrow()
-
operator()
-
closeTo()
Expect/Should
-
.to.be.above(), .to.be.least(), .to.be.below(), .to.be.most()
-
.to.be.NaN, .not.to.be.NaN
-
.to.be.undefined
-
.to.be.finite
-
.to.throw(), .to.not.throw()
-
.to.be.closeTo()
Result handling
- Result handling with Assert, expect(), should() (System)
- Server (automatically forwarded to SUT) script name: OPT_PLAY_HANDLE_ASSERTION
Possible Values: VAL_PLAY_HANDLE_ASSERTION_AS_CHECK, VAL_PLAY_HANDLE_ASSERTION_WITH_EXCEPTION, VAL_PLAY_HANDLE_ASSERTION_SILENTLY
The option is used to configure return value and logging.
-
“Handle as check” - VAL_PLAY_HANDLE_ASSERTION_AS_CHECK
In case of failure, an error, otherwise a successful check is logged to the run log -
“As Exception” - VAL_PLAY_HANDLE_ASSERTION_WITH_EXCEPTION
An assertion exception is thrown in case of a failure. Can be caught in scripts as Throwable and in Try-Catch nodes asScriptException
-
“As return value” - VAL_PLAY_HANDLE_ASSERTION_SILENTLY
The assertion check will be executed, but no error will be logged or exception will be thrown. Nevertheless, the assertion will be executed and returns a result value (true
orfalse
). When usingexpect
/should
, the result can be accessed with a chained.getResult()
. -
“Handle automatically” - VAL_PLAY_HANDLE_ASSERTION_AUTOMATICALLY (Default value)
Same asVAL_PLAY_HANDLE_ASSERTION_AS_CHECK
, except within a Unit test node, whereVAL_PLAY_HANDLE_ASSERTION_WITH_EXCEPTION
is used to fulfill the JUnit contract.
-
“Handle as check” - VAL_PLAY_HANDLE_ASSERTION_AS_CHECK
Use the option at the beginning of the script:
rc.setOption(Options.OPT_PLAY_HANDLE_ASSERTION, Options.VAL_PLAY_HANDLE_ASSERTION_SILENTLY) def a = 54 def b = 55 def isEqual = Assert.test(a==b, "") if (isEqual) { ... }
If you used fluent assertions, you have to call .getResult()
to query the result:
rc.setOption(Options.OPT_PLAY_HANDLE_ASSERTION, Options.VAL_PLAY_HANDLE_ASSERTION_SILENTLY) a = 54 b = 55 isEqual = expect(a).to.equal(b).getResult() if isEqual.getResult(): ...
Exception handling
All QF-Test Exceptions are
automatically imported inscripts and can be used for
try/except
clauses like
try: com = rc.getComponent("someId") except ComponentNotFoundException: ...
ComponentNotFoundException
in Jython
When working with Groovy you use try/catch
:
try { com = rc.getComponent("someId") } catch (ComponentNotFoundException) { ... }
ComponentNotFoundException
in Groovy
Only the following exceptions should be raised explicitly from
script code (with raise
or throw new
respectively):
-
UserException("Some message here...")
should be used to signal exceptional error conditions. -
BreakException()
orraise BreakException("loopId")
can be used to break out of a Loop or While node, either without parameters to break out of the innermost loop or with the QF-Test loop ID parameter to break out of a specific loop with the respective QF-Test ID. -
ReturnException()
orraise ReturnException("value")
can be used to return - with or without a value - from a Procedure node, similar to executing a Return node. To improve readability, preferably callrc.returnValue(...)
.
Debugging scripts (Jython)
When working with Jython modules you don't have to restart QF-Test
or the SUT after you made changes.
You can simply use reload(<modulename>)
to load the module anew.
Debugging scripts in an embedded Jython interpreter can be tedious. To simplify this task, QF-Test offers an active console window for communicating with each interpreter. For more information please see the last part of section 11.1.
Alternatively, a network connection can be established to talk remotely to the Jython
interpreter - in QF-Test as well as within the SUT - and get an interactive command line.
To enable this feature you must use the command line argument -jythonport <number>
to set
the port number that the Jython interpreter should listen on. For the SUT
-jythonport=<port>
can be defined in the "Extra"
Executable parameters of the Start Java SUT client or
Start SUT client node. You can then connect to the Jython interpreter, for
example with
telnet localhost <port>
Combined with Jython's ability to access the full Java API, this is not only useful for debugging scripts but can also be used to debug the SUT itself.