24.1

Test execution in batch mode

There are a lot of command line arguments when running QF-Test in batch mode; an overview can be found in chapter 43. Here we will present examples showing the most important of them.

The examples are written for the Windows operating system, but you may easily adapt them for the Linux platform. What is different is the path specification and also the syntax for placeholders (subsection 43.2.4): On Linux you can use +X as well as %X. On Windows there's a separate console application qftestc.exe. In contrast to its GUI variant qftest.exe, it waits until the execution of QF-Test has terminated and also displays print output from a 'Server script'. You can use qftestc.exe in place of qftest.exe wherever you'll find it convenient.

24.1.1

Command line usage

Let's start with the most simple QF-Test command to execute a test:

qftest -batch -run c:\mysuites\suiteA.qft

The argument -batch makes QF-Test start without a graphical user interface. The second argument, -run, is the specifier for test execution. Finally, at the end of the command line, you find the test suite to be executed.

Note The argument -run is optional, i. e. the test execution is defined as default for the batch mode.

When running the above command, all top-level 'Test case' and 'Test set' nodes of suiteA.qft will be executed one after another. After the test run you will find a run log file in the current directory; it has the same name as the test suite (except from the extension, which can be .qrl, .qrz or .qzp). The run log file shows the result of the test run.

By specifying -nolog you can suppress the creation of a run log. Probably this only makes sense, if you have extended your test by your own log output (written to a file). Otherwise you'd have to check the result code of QF-Test, whereas 0 means that everything is alright. A positive value in contrast indicates that warnings, errors or exceptions occurred during the test run (see section 43.3). That's why in most situations you'll probably prefer to create a run log and save it at a fixed place in the file system. This can be achieved with the parameter -runlog:

qftest -batch -compact -runlog c:\mylogs\+b c:\mysuites\suiteA.qft

A run log file suiteA.qrz will now be created in the specified directory c:\mylogs. The placeholder +b is responsible for its name being identical with that of the test suite. The additional switch -compact prevents the run log from growing too large: Only the nodes needed for a report and those immediately before an error or an exception are kept in the run log. Especially in case of very long test runs this may help to reduce the amount of required memory. The newer method of using split run logs is even more powerful. For more information about that see section 7.1.

Note Whether the file is indeed created as compressed run log (to be distinguished from the above "compact") with extension .qrz, depends on the system settings. To force the creation of a particular format you can set the file extension explicitly. With -runlog c:\mylogs\+b.qrl, for example, an uncompressed XML file will be produced.

Sometimes you may want to execute not the whole test suite but only parts of it. By using the parameter -test you can run a specific node of the test suite:

qftest -batch -runlog c:\mylogs\+b -test "My test case" c:\mysuites\suiteA.qft

The parameter -test expects the 'QF-Test ID' attribute of the node to follow or the qualified name of a 'Test case' or 'Test set'. If you want to execute several nodes, you can define -test <ID> multiple times. Apart from the node's 'QF-Test ID', -test accepts also the numerical index of a top-level node. For example, -test 0 will run the first child of the 'Test suite' node.

The run log provides a rather technical view of the test run; it is helpful mainly when analyzing errors (cf. section 7.1). The report in contrast contains a summary of the executed test cases and errors (cf. chapter 23) in XML or HTML format. It is created from the run log either in a separate step after running the test or automatically with the test run:

qftest -batch -runlog c:\mylogs\+b -report c:\mylogs\rep_+b_+y+M+d+h+m c:\mysuites\suiteA.qft

In this example the XML and HTML files are saved in a directory which name consists of the test suite and a timestamp like c:\mylogs\rep_suiteA_0806042152. When replacing the argument -report with -report.xml or -report.html respectively, only an XML or HTML report will be created.

Test cases often uses variables to control the execution of the test. For example, you may have defined the variable myvar in the 'Test suite' node of the suite. You can overwrite its default value when running the test suite from the command line:

qftest -batch -variable myvar="Value from command line" -runlog c:\mylogs\+b c:\mysuites\suiteA.qft

If needed, you can specify -variable <name>=<wert> multiple times to set values for different variables.

24.1.2

Windows batch script

Running tests from the command line is fundamental for integrating QF-Test in test management systems (see Interaction with Test Management Tools). Otherwise, living without such a tool, you may find it convenient to embed the command for the test execution into a script. A simple Windows batch script (qfbatch.bat) looks like this:

@echo off
setlocal
if "%1" == "" (
    echo Usage: qfbatch Testsuite
    goto end
) else (
    set suite=%~f1
)
set logdir=c:\mylogs
pushd c:\programs\qftest\qftest-8.0.1\bin

@echo on
.\qftest -batch -compact -runlog %logdir%\+b %suite%
@echo off

if %errorlevel% equ 0 (
  echo Test terminated successfully
  goto end
)
if %errorlevel% equ 1 (
  echo Test terminated with warnings
  goto end
)
if %errorlevel% equ 2 (
  echo Test terminated with errors
  goto end
)
if %errorlevel% equ 3 (
  echo Test terminated with exceptions
  goto end
)
if %errorlevel% leq -1 (
  echo Error %errorlevel%
  goto end
)

:end
popd

Now you can simply run that script with only the file name of the test suite as parameter. Everything else is done automatically: The test suite will be executed, the run log file stored in logdir and finally the script will print out the state of the test run (depending on the QF-Test result code).

3.0+24.1.3

Groovy

Since version QF-Test 3 the language Groovy is part of the release (cf. chapter 11). It is meant mainly for scripting inside QF-Test (Server and SUT scripts), but it can, like Jython, also be used outside of QF-Test. Groovy is probably well suited to create a little test execution management system by yourself. By the way, Groovy simplifies working with Ant, too: Instead of dealing with bulky XML files, which makes it hard to define conditions, you can work with the Groovy AntBuilder. However, that's out of scope here, the following example doesn't rely on Ant but only on the basic Groovy features:

def suite = ''
if (args.size() == 0) {
    println 'Usage: groovy QfExec Testsuite'
    return
}
else {
    suite = args[0]
}

def qftestdir = 'c:\\programs\\qfs\\qftest\\qftest-8.0.1'
def qftest = qftestdir + '\\bin\\qftest.exe'
def command = [qftest,
              "-batch",
              "-compact",
              "-runlog", "c:\\mylogs\\+b",
              suite]
def printStream = { stream ->
    while (true) {
        try {
            stream.eachLine { println it }
        } catch (IOException) {
            break
        }
    }
}

println "Running command: $command"
def proc = command.execute()
new Thread().start() { printStream(proc.in) }
new Thread().start() { printStream(proc.err) }
proc.waitFor()

switch (proc.exitValue()) {
    case '0': println 'Test terminated successfully'; break
    case '1': println 'Test terminated with warnings'; break
    case '2': println 'Test terminated with errors'; break
    case '3': println 'Test terminated with exceptions'; break
    default: println "Error ${proc.exitValue()}"
}

If you have Groovy installed on your computer independently of QF-Test, you can run the example test suite simply via groovy QfExec c:\mysuites\suiteA.qft. Otherwise you can use the Groovy jar file from the QF-Test installation, preferably again with help of a batch script:

@echo off
setlocal
if "%1" == "" (
    echo Usage: qfexec Testsuite
    goto end
)
set qftestdir=c:\programs\qftest\qftest-8.0.1
set scriptfile=QfExec.groovy
java -cp %qftestdir%/lib/groovy-all.jar groovy.ui.GroovyMain %scriptfile% %*
:end

Now execute the test suite with qfexec c:\mysuites\suiteA.qft.

Last update: 9/10/2024 Copyright © 1999-2024 Quality First Software GmbH