QF-Test is primarily a tool for the creation and execution of automated tests. However, it
is rarely possible - or economical - to automate 100% of the required tests for a project.
In most projects some manual tests need to be performed as well. One of the biggest
challenges in testing a project is consolidating the different results and reports of
automated and manual testing to get an overview about the execution status of all tests. To
facilitate reporting the results of manual test execution along with those of automated
testing, QF-Test now offers the capability of tracking manual tests from within itself.
The steps to be performed during a manual test have to be defined in an Excel file which
is read by a test suite called ManualTestRunner.qft. This test suite is
provided along with a sample specification file in the directory
demo/manualtester below the QF-Test installation directory. The test designer
has to specify each step in that Excel file including the expected result. After stepping
through the manual tests QF-Test provides the usual results - a run log, HTML and XML
reports. Additionally, a newly created Excel file with the results of the respective
test run is created. For a detailed description please see section 33.2.
The dialog used for the test execution is called ManualStepDialog and looks
like this:
The title of the dialog shows the name of the test case. The detailed step description and
the expected result are shown in the first two text-boxes. After performing the test the
tester has to specify whether the test succeeded or not. In case the test failed the
tester also has to enter the received result which is intended to show the differences
between the actual and the expected result. This dialog can also be used for your own
purposes, see section 56.1.
Please perform the following steps on your system to launch a manual test from QF-Test.
-
Copy the definition Excel file from
qftest-8.0.1/demo/manualtester/SampleTestDescription.xlsx to your
project location and rename it to a suitable name. We recommend to use the same path on
all test systems. Perhaps you can make use of a shared network drive.
-
Also copy the execution test suite from
qftest-8.0.1/demo/manualtester/ManualTestRunner.qft to your
project location. You may want to rename it as well.
-
Open the Excel file and define the test steps.
-
After saving the changes to the Excel file, open the execution test suite and adapt the
global variable
testFile variable to target your specific Excel file.
-
Turn off the QF-Test debugger. It would only interfere with the steps of the manual tester.
-
Start the test suite via selecting the test suite node and pressing "Start test run".
-
QF-Test will now read the data from the Excel file and open a dialog containing the first
test step.
-
Enter the result of the test step and proceed with executing each test step.
-
At the end of the test execution QF-Test will write a new Excel file containing the test
description and the according results. You can also store the run log of that execution
or create an HTML report.
Please read the comments in the test suite and Excel file carefully, because you can adapt
this concept according to your needs. It is even possible to start only specific tests.
The Excel file has a specific structure which allows you to describe the manual test steps
quite flexibly. The meaning of the columns is explained in the following table:
| | Column | Description |
|---|
| TestCase |
A unique identifier for each test case. If the step belongs to the same test case as the
previous step, just leave this column empty.
| | Type of Test |
Optional definition of the kind or function of the test or step, e.g. a functional
test or a usability test, startup, etc.
| | Comment |
An individual comment for the test case. This comment will be shown in the
run log of QF-Test.
| | Short Description | A short description about the content of the test. | | Step Description | The detailed description of the manual step. | | Expected Result | The description of the expected result of that test step. |
|
| | Table 33.1: Description of the Excel file for the definition of manual tests | |
The Excel file with the results of the manual test execution will contain two additional
columns as follows:
| | Column | Description |
|---|
| Received Result |
The result the tester received during test execution. If a test step fails, the
tester must specify a received result.
| | State |
The state of the test, i.e. PASSED, FAILED, CANCELED or SKIPPED.
|
|
| | Table 33.2: Description of the Excel file with the results of manual tests | |
The ManualTestRunner.qft test suite contains some global variables at
suite-level which provide fine-grained control over test run. These are explained in the
following table. All variables not listed here are used internally by the test suite and
should not be changed.
| | Global Variable | Description |
|---|
| testFile | The path to the test step definition Excel file. | | testSheet | The worksheet of the Excel file containing the test steps. | | resultSheet | The name of the worksheet for the results. | | tests |
A list of tests to be intended to execute. If this variable is empty, all tests will be executed. If
you want to execute only test 5 and 6, you can specify 5,6 or
5-6. It is even possible to specify things like: 1,3-5,7
to execute the tests 1, 3, 4, 5 and 7.
| | defaultState | The default selection of the state. You can set it either to PASSED
or FAILED. All other states will be converted to FAILED. | | testCaseColumn | The heading of the column containing the test case number. | | commentColumn | The heading of the column containing the comment. | | shortDescColumn | The heading of the column containing the short step description. | | stepDescColumn | The heading of the column containing the full step description. | | expResultColumn | The heading of the column containing the expected result. | | recResultColumn | The heading of the column containing the received result. | | stateColumn | The heading of the column containing the state of the test. |
|
| | Table 33.3: Description of the global variables in the ManualTestRunner
test suite | |
An executed test step can be set to one of the following states:
| | Result | Description |
|---|
| PASSED | The test step was successful. | | FAILED | The test step failed. | | CANCELED | The test step was canceled. | | SKIPPED | The test step was skipped. |
|
| | Table 33.4: States of manual test execution | |
