41.2

Test and Sequence nodes

Tests and sequences are the main structural elements of a test suite. 'Test case' nodes represent logical test cases and are implemented as specialized sequences. A sequence is a container that executes its child nodes one by one. It can define variables (see chapter 6) that remain bound during the execution of the children.

Other kinds of sequences differ either in the way their child nodes are executed, or in the restrictions they impose on their child or parent nodes.

41.2.1

'Test case'

A 'Test case' node represents one or more logical test cases. In a sense it is the most important of all QF-Test nodes and everything else only serves to lend structure to 'Test cases' or to implement their logic. Functionally it is a highly specialized 'Sequence' with a number of important extensions and special attributes. A 'Test case' should focus on the actual test it supposed to perform. Setup and cleanup tasks required to ensure that the 'Test case' executes in the required environment and does not interfere with subsequent tests should be implemented in the form of 'Dependencies' as described in section 8.6. Alternatively - or in addition to 'Dependencies' - a 'Test case' can have 'Setup' and 'Cleanup' nodes to be executed before and after the 'Test case'. Because a 'Test case' can be called from a 'Test call' node it is also somewhat similar to a 'Procedure' in that its 'Name' attribute is mandatory and that it has a list of 'Parameter default values' that can be overridden in the calling node. 'Test cases' also play a central role in run logs and test reports. During a test run a 'Test case' node can be executed several times in different contexts and with different parameters. Logically these executions may represent the same or different test cases. By defining a set of 'Characteristic variables' you can specify which variable values are used to differentiate between executions, thus characterizing the run-time environment of the test. The values of these variables at the time of entry to the 'Test case' are stored in the run log. To emphasize the fact that each execution of a 'Test case' node may represent a separate logical test case there is an alternative name attribute called 'Name for run log and report'. Its value may contain references to the 'Characteristic variables' of the test. In the run log or report the test will then be listed with this name, including the expanded run-time variable values. Finally, there are situations in which a test cannot or should not be executed for specific variable settings. If the 'Condition' attribute of the 'Test case' is defined, the 'Test case' will only be executed if that expression expands to a true value. If the 'Test case' is not executed due to the 'Condition' it will be listed as skipped in the report.

Contained in:'Test suite', 'Test set'.

Children: Optional 'Dependency' or 'Dependency reference' as the first element. A 'Setup' may be the next and a 'Cleanup' the last node with an arbitrary number of normal child nodes in between. A 'Test case' that does not contain normal child nodes will be listed as not implemented in the report.

Execution: First the 'Variable definitions' of the 'Test case' are bound on the primary and its 'Parameter default values' on the fallback variable stack (see chapter 6). With these in place the 'Condition' is evaluated and the node will be skipped if a non-empty 'Condition' evaluates to false. Next the dependency of the 'Test case' - possibly inherited from its parent node - is determined and resolved as described in section 8.6. Then the optional 'Setup' is executed once, followed by the normal child nodes and a single execution of the optional 'Cleanup'. An exception raised during the course of the 'Test case' will be caught and passed to the 'Dependency' for handling in a 'Catch' node. Even if the exception is not handled by a 'Dependency' it is not propagated beyond the 'Test case' to prevent aborting the whole test run. The error state is duly noted in run log and report however.

Attributes:

		Figure 41.2:  'Test case' attributes

'Name'

A 'Test case' is identified by its name and the names of its 'Test set' ancestors, so you should assign a name with a meaning that is easy to recognize and remember.

Variable: No

Restrictions: Must not be empty or contain the characters '.' or '#'.

'Name for run log and report'

A separate name to be used for run log and report. This is useful to differentiate between multiple executions with potentially different values for the 'Characteristic variables'.

Variable: Yes

Restrictions: None

'Characteristic variables'

These variables are part of the characteristics of a 'Test set' or 'Test case'. Two executions of a 'Test case' are considered to represent the same logical test case if the run-time values of all 'Characteristic variables' are identical. The run-time values of the 'Characteristic variables' are stored in the run log.

Variable: No

Restrictions: None

'Name for separate run log'

If this attribute is set it marks the node as a breaking point for split run logs and defines the filename for the partial log. When the node finishes, the respective log entry is removed from the main run log and saved as a separate, partial run log. This operation is completely transparent, the main run log retains references to the partial logs and is fully controllable. Please see subsection 7.1.6 for further information about split run logs.

This attribute has no effect if the option Create split run logs is disabled or split run logs are explicitly turned off for batch mode via the -splitlog command line argument.

There is no need to keep the filename unique. If necessary, QF-Test appends a number to the filename to avoid collisions. The filename may contain directories and, similar to specifying the name of a run log in batch mode on the command line, the following placeholders can be used after a '%' or a '+' character:

	Character Replacement % Literal '%' character. + Literal '+' character. i The current runid as specified with -runid <ID>. r The error level of the partial log. w The number of warnings in the partial log. e The number of errors in the partial log. x The number of exceptions in the partial log. t The thread index to which the partial log belongs (for tests run with parallel threads). y The current year (2 digits). Y The current year (4 digits). M The current month (2 digits). d The current day (2 digits). h The current hour (2 digits). m The current minute (2 digits). s The current second (2 digits).
		Table 41.1:  Placeholders for the 'Name for separate run log' attribute

Variable: Yes

Restrictions: None, characters that are illegal for a filename will be replaced with '_'.

'Inherit dependency from parent node'

This option allows inheriting the 'Dependency' from the parent node as a replacement for or in addition to specifying a 'Dependency' for the node itself.

Variable: No

Restrictions: None

'Condition'

If the condition is non-empty it will be evaluated and if the result is false the execution of the current node is aborted. In this case the node will be reported as skipped.

Like the 'Condition' of an 'If' node, the 'Condition' is evaluated by the Jython interpreter, so the same rules apply.

Variable: Yes

Restrictions: Valid syntax

'Script language'

This attribute determines the interpreter in which to run the script, or in other words, the scripting language to use. Possible values are "Jython", "Groovy" and "JavaScript".

Variable: No

Restrictions: None

'Expected to fail if...'

This attribute allows specifying a condition under which the 'Test case' is expected to fail. This is helpful to distinguish new problems from already known ones. Of course the latter should be fixed, but if and when that happens may be outside the tester's sphere of influence.

Most of the time it is sufficient to simply set this attribute to 'true' to mark the 'Test case' as an expected failure. But if the 'Test case' is executed multiple times, e.g. within a 'Data driver' or on multiple systems and fails only in specific cases, the condition should be written so that it evaluates to true for exactly those cases, e.g. ${qftest:windows} for a test that fails on Windows but runs fine on other systems.

If this attribute evaluates to true and the 'Test case' fails with an error, it will be listed separately in the run log, report and on the status line. It still means that there is an error in the application, so the overall percentage of successful tests is not changed.

It is treated as an error if this attribute evaluates to true and the 'Test case' does not fail because this means that either the test is not able to reproduce the problem reliably or that the problem has been fixed and the 'Test case' must be updated accordingly.

Variable: Yes

Restrictions: Valid syntax

'Variable definitions'

These variables are bound on the direct bindings stack (see chapter 6). They remain valid during the execution of the 'Test case's' child nodes and cannot be overridden by a 'Test call' node. See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Parameter default values'

Here you can define default or fallback values for the 'Test case's' parameters (see chapter 6). Defining these values also serves as documentation and is a valuable time-saver when using the dialog to select the 'Test case' for the 'Test name' attribute of a 'Test call'. See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Maximum error level'

When a warning, error or exception occurs during a test run, the state of the corresponding node of the run log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run log node represents the worst of its child nodes' states. Using this attribute, the maximum error state that the run log node for a sequence will propagate, can be limited.

Note This value has no effect on the way exceptions are handled. It only affects the error states of the run log nodes and by that the exit code of QF-Test when run in batch mode (see. section 1.7). It also has no effect on the creation of compact run logs (see command line argument -compact). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute.

Variable: No

Restrictions: None

'Execution timeout'

Time limit for the node's execution in milliseconds. If that limit expires the execution of that node will get interrupted.

Variable: Yes

Restrictions: >= 0

'Border for relative calls'

This flag determines whether relative procedure calls, test calls or dependency references are allowed within that certain node. Relative calls passing that border are not allowed. If that attribute is not specified in the hierarchy, no relative calls are allowed.

Variable: No

Restrictions: None

'QF-Test ID'

When using the command line argument -test <n>|<ID> for execution in batch mode you can specify the 'QF-Test ID' of the node as an alternative to its qualified name.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test suite.

Note For detailed documentation, especially for 'Test set', 'Test case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the button.

You can trigger special behaviors of some nodes using doctags, please see Doctags.

If you enter text in the comment field of a 'Component' node, the node will be considered as 'used' when you want to mark or delete unused components.

Variable: Yes

Restrictions: None

41.2.2

'Test set'

The main purpose of a 'Test set' is to give structure to the 'Test cases' of a test suite. 'Test sets' can be nested. The 'Name' of a 'Test set' is part of the 'Test case's' fully qualified name to which a 'Test call' node refers. 'Test sets' are also callable themselves and thus have a set of 'Parameter default values' that can be overridden in the 'Test call'. One way of structuring 'Test cases' is to collect those with similar setup and cleanup requirements in the same 'Test set' which can provide a 'Dependency' or 'Dependency reference' node to be inherited by the 'Test case' or nested 'Test set' child nodes. If the 'Inherit dependency from parent node' attribute of the 'Test set' is set, the 'Dependency' nodes of the 'Test set's' parent(s) are also inherited. See section 8.6 for information about QF-Test's dependency mechanism. Alternative or additional setup and cleanup is available in the form of 'Setup' and 'Cleanup' nodes which will be executed before and after each of the 'Test set' and 'Test case' child nodes. Another important feature of a 'Test set' is data-driven testing. This can be achieved by adding a 'Data driver' node with one or more 'Data binders' as described in chapter 22. Like a 'Test case', a 'Test set' plays an important role for reports. As it can also be executed several times with different parameter settings, it has a set of 'Characteristic variables' and an alternative 'Name for run log and report' that work just like for a 'Test case'. The same is true for the 'Condition' which can be used to skip an entire 'Test set' depending on current variable values.

Contained in:'Test suite', 'Test set'.

Children: Optional 'Dependency' or 'Dependency reference' as the first child, followed by an optional 'Data driver'. A 'Setup' may come next and a 'Cleanup' at the end with an arbitrary number of 'Test set', 'Test case' and 'Test call' nodes in between.

Execution: First the 'Parameter default values' of the 'Test set' are bound on the fallback variable stack (see chapter 6). With these in place the 'Condition' is evaluated and the node will be skipped if a non-empty 'Condition' evaluates to false. The 'Dependency' or 'Dependency reference' node will not be resolved at this time unless its 'Always execute, even in test suite and test set nodes' attribute is set. If there is a 'Data driver' node, it is executed to create a data driving context, bind one or more 'Data binders' and then repeatedly execute the other child nodes as described in chapter 22. For each loop iteration - or once in case no 'Data driver' is present - the 'Test set' executes each of its 'Test set' or 'Test case' child nodes. If an optional 'Setup' or 'Cleanup' node are present, they are executed before and after each of these child nodes respectively.

Attributes:

		Figure 41.3:  'Test set' attributes

'Name'

The name of a 'Test set' is part of its own identification and of that of the 'Test case' and 'Test set' nodes it contains, so you should assign a name with a meaning that is easy to recognize and remember.

Variable: No

Restrictions: Must not be empty or contain the characters '.' or '#'.

'Name for run log and report'

A separate name to be used for run log and report. This is useful to differentiate between multiple executions with potentially different values for the 'Characteristic variables'.

Variable: Yes

Restrictions: None

'Characteristic variables'

These variables are part of the characteristics of a 'Test set' or 'Test case'. Two executions of a 'Test case' are considered to represent the same logical test case if the run-time values of all 'Characteristic variables' are identical. The run-time values of the 'Characteristic variables' are stored in the run log.

Variable: No

Restrictions: None

'Name for separate run log'

If this attribute is set it marks the node as a breaking point for split run logs and defines the filename for the partial log. When the node finishes, the respective log entry is removed from the main run log and saved as a separate, partial run log. This operation is completely transparent, the main run log retains references to the partial logs and is fully controllable. Please see subsection 7.1.6 for further information about split run logs.

This attribute has no effect if the option Create split run logs is disabled or split run logs are explicitly turned off for batch mode via the -splitlog command line argument.

There is no need to keep the filename unique. If necessary, QF-Test appends a number to the filename to avoid collisions. The filename may contain directories and, similar to specifying the name of a run log in batch mode on the command line, the following placeholders can be used after a '%' or a '+' character:

	Character Replacement % Literal '%' character. + Literal '+' character. i The current runid as specified with -runid <ID>. r The error level of the partial log. w The number of warnings in the partial log. e The number of errors in the partial log. x The number of exceptions in the partial log. t The thread index to which the partial log belongs (for tests run with parallel threads). y The current year (2 digits). Y The current year (4 digits). M The current month (2 digits). d The current day (2 digits). h The current hour (2 digits). m The current minute (2 digits). s The current second (2 digits).
		Table 41.2:  Placeholders for the 'Name for separate run log' attribute

Variable: Yes

Restrictions: None, characters that are illegal for a filename will be replaced with '_'.

'Inherit dependency from parent node'

This option allows inheriting the 'Dependency' from the parent node as a replacement for or in addition to specifying a 'Dependency' for the node itself.

Variable: No

Restrictions: None

'Condition'

If the condition is non-empty it will be evaluated and if the result is false the execution of the current node is aborted. In this case the node will be reported as skipped.

Like the 'Condition' of an 'If' node, the 'Condition' is evaluated by the Jython interpreter, so the same rules apply.

Variable: Yes

Restrictions: Valid syntax

'Script language'

This attribute determines the interpreter in which to run the script, or in other words, the scripting language to use. Possible values are "Jython", "Groovy" and "JavaScript".

Variable: No

Restrictions: None

'Parameter default values'

Here you can define default or fallback values for the 'Test set's' parameters (see chapter 6). Defining these values also serves as documentation and is a valuable time-saver when using the dialog to select the 'Test set' for the 'Test name' attribute of a 'Test call'. See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Maximum error level'

When a warning, error or exception occurs during a test run, the state of the corresponding node of the run log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run log node represents the worst of its child nodes' states. Using this attribute, the maximum error state that the run log node for a sequence will propagate, can be limited.

Note This value has no effect on the way exceptions are handled. It only affects the error states of the run log nodes and by that the exit code of QF-Test when run in batch mode (see. section 1.7). It also has no effect on the creation of compact run logs (see command line argument -compact). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute.

Variable: No

Restrictions: None

'Execution timeout'

Time limit for the node's execution in milliseconds. If that limit expires the execution of that node will get interrupted.

Variable: Yes

Restrictions: >= 0

'Border for relative calls'

This flag determines whether relative procedure calls, test calls or dependency references are allowed within that certain node. Relative calls passing that border are not allowed. If that attribute is not specified in the hierarchy, no relative calls are allowed.

Variable: No

Restrictions: None

'QF-Test ID'

When using the command line argument -test <n>|<ID> for execution in batch mode you can specify the 'QF-Test ID' of the node as an alternative to its qualified name.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test suite.

Note For detailed documentation, especially for 'Test set', 'Test case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the button.

You can trigger special behaviors of some nodes using doctags, please see Doctags.

If you enter text in the comment field of a 'Component' node, the node will be considered as 'used' when you want to mark or delete unused components.

Variable: Yes

Restrictions: None

41.2.3

'Test call'

With this node a call to some other executable test node can be made. Possible targets are 'Test suite', 'Test set' and 'Test case' nodes in the same or a different test suite. Execution will continue in the called node and when finished return to the 'Test call' and thus to its parent node. The name of a 'Test case' or 'Test set' to call is determined by its 'Name' and the 'Names' of its 'Test set' ancestors. These are concatenated with a dot ('.') as separator, starting with the outermost 'Test set' and ending in the 'Test case's' name. Thus to call a 'Test case' named nodeTest in a 'Test set' named Tree that is itself a child of a 'Test set' named Main, the 'Test name' attribute would be set to 'Main.Tree.nodeTest'. A node in a different test suite is addressed by prepending the filename of the test suite followed by a '#' to the 'Test name'. A 'Test suite' node is addressed by a single '.', so calling a whole test suite is done with a 'Test name' attribute of the form 'suitename.qft#.'. It is generally easiest to pick the target node interactively from a dialog by clicking on the button above the 'Test name' attribute. See also section 25.1 for further information about cross-suite calls.

Contained in: All kinds of sequences.

Children: None

Execution: The 'Variable definitions' of the 'Test call' are bound, the target node is determined and execution is passed to it. After the 'Test call' returns, the 'Test call's' variables are unbound again.

Attributes:

		Figure 41.4:  'Test call' Attributes

'Name'

In case of a 'Test case' or 'Test set' it is the full name, created from the 'Name' of its 'Test set' parents and its own 'Name', joined by a dot. A 'Test suite' node is addressed by a single '.' character. The "Select test" button above the attribute brings up a dialog in which you can select the target node interactively. You can also get to this dialog by pressing [Shift-Return] or [Alt-Return] when the focus is in the text field. By selecting the "Copy parameters" checkbox you can adopt the 'Test case's' or 'Test set's' 'Parameter default values' as parameters for the 'Test call' node to save typing.

Variable: Yes

Restrictions: Must not be empty.

'Name for separate run log'

If this attribute is set it marks the node as a breaking point for split run logs and defines the filename for the partial log. When the node finishes, the respective log entry is removed from the main run log and saved as a separate, partial run log. This operation is completely transparent, the main run log retains references to the partial logs and is fully controllable. Please see subsection 7.1.6 for further information about split run logs.

This attribute has no effect if the option Create split run logs is disabled or split run logs are explicitly turned off for batch mode via the -splitlog command line argument.

There is no need to keep the filename unique. If necessary, QF-Test appends a number to the filename to avoid collisions. The filename may contain directories and, similar to specifying the name of a run log in batch mode on the command line, the following placeholders can be used after a '%' or a '+' character:

	Character Replacement % Literal '%' character. + Literal '+' character. i The current runid as specified with -runid <ID>. r The error level of the partial log. w The number of warnings in the partial log. e The number of errors in the partial log. x The number of exceptions in the partial log. t The thread index to which the partial log belongs (for tests run with parallel threads). y The current year (2 digits). Y The current year (4 digits). M The current month (2 digits). d The current day (2 digits). h The current hour (2 digits). m The current minute (2 digits). s The current second (2 digits).
		Table 41.3:  Placeholders for the 'Name for separate run log' attribute

Variable: Yes

Restrictions: None, characters that are illegal for a filename will be replaced with '_'.

'Variable definitions'

This is where you define the parameter values for the target node. These variables are bound on the primary variable stack (see chapter 6) so they override any 'Parameter default values'. See subsection 2.2.5 about how to work with the table.

4.2+ In case you want to re-set the order of the parameters like they are sorted in the called test case or test set, you can select »Re-set parameter order«.

Variable: Variable names no, values yes

Restrictions: None

'Act like a procedure call'

If the 'Test call' node is executed inside a 'Test case', this attribute determines how exceptions are handled in the called node(s). If it is activated a single exception terminates the whole call irrespective of 'Test set' and 'Test case' nesting, just like it would for a regular 'Procedure call' node. If this attribute is unset the special roles of 'Test set' and 'Test case' nodes with their local exception handling is maintained.

Variable: Yes

Restrictions: None

'Execution timeout'

Time limit for the node's execution in milliseconds. If that limit expires the execution of that node will get interrupted.

Variable: Yes

Restrictions: >= 0

'QF-Test ID'

At the moment the 'QF-Test ID' attribute has no meaning for this type of node.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test suite.

Note For detailed documentation, especially for 'Test set', 'Test case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the button.

You can trigger special behaviors of some nodes using doctags, please see Doctags.

If you enter text in the comment field of a 'Component' node, the node will be considered as 'used' when you want to mark or delete unused components.

Variable: Yes

Restrictions: None

41.2.4

'Sequence'

This is the most general kind of sequence. Its children are executed one by one and their number or type is not limited in any way.

Contained in: All kinds of sequences.

Children: Any

Execution: The 'Variable definitions' of the 'Sequence' are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again.

Attributes:

		Figure 41.5:  'Sequence' attributes

'Name'

The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and say something about the function of the sequence.

Variable: No

Restrictions: None

'Variable definitions'

This is where you define the values of the variables that remain bound during the execution of the sequence's child nodes (see chapter 6). See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Maximum error level'

When a warning, error or exception occurs during a test run, the state of the corresponding node of the run log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run log node represents the worst of its child nodes' states. Using this attribute, the maximum error state that the run log node for a sequence will propagate, can be limited.

Note This value has no effect on the way exceptions are handled. It only affects the error states of the run log nodes and by that the exit code of QF-Test when run in batch mode (see. section 1.7). It also has no effect on the creation of compact run logs (see command line argument -compact). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute.

Variable: No

Restrictions: None

'QF-Test ID'

At the moment the 'QF-Test ID' attribute has no meaning for this type of node.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test suite.

Note For detailed documentation, especially for 'Test set', 'Test case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the button.

You can trigger special behaviors of some nodes using doctags, please see Doctags.

If you enter text in the comment field of a 'Component' node, the node will be considered as 'used' when you want to mark or delete unused components.

Variable: Yes

Restrictions: None

41.2.5

'Test step'

A 'Test step' is a special 'Sequence' that serves to divide a 'Test case' into steps that can be documented individually and will show up in the report or testdoc documentation. In contrast to 'Test cases', which should not be nested, 'Test steps' can be nested to any depth.

Contained in: All kinds of sequences.

Children: Any

Execution: The 'Variable definitions' of the 'Test step' are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again.

Attributes:

		Figure 41.6:  'Test step' attributes

'Name'

The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and say something about the function of the sequence.

Variable: No

Restrictions: None

'Name for run log and report'

A separate name to be used for run log and report. This is useful to differentiate between multiple executions of the surrounding 'Test case' with potentially different values for its 'Characteristic variables'.

Variable: Yes

Restrictions: None

'Name for separate run log'

If this attribute is set it marks the node as a breaking point for split run logs and defines the filename for the partial log. When the node finishes, the respective log entry is removed from the main run log and saved as a separate, partial run log. This operation is completely transparent, the main run log retains references to the partial logs and is fully controllable. Please see subsection 7.1.6 for further information about split run logs.

This attribute has no effect if the option Create split run logs is disabled or split run logs are explicitly turned off for batch mode via the -splitlog command line argument.

There is no need to keep the filename unique. If necessary, QF-Test appends a number to the filename to avoid collisions. The filename may contain directories and, similar to specifying the name of a run log in batch mode on the command line, the following placeholders can be used after a '%' or a '+' character:

	Character Replacement % Literal '%' character. + Literal '+' character. i The current runid as specified with -runid <ID>. r The error level of the partial log. w The number of warnings in the partial log. e The number of errors in the partial log. x The number of exceptions in the partial log. t The thread index to which the partial log belongs (for tests run with parallel threads). y The current year (2 digits). Y The current year (4 digits). M The current month (2 digits). d The current day (2 digits). h The current hour (2 digits). m The current minute (2 digits). s The current second (2 digits).
		Table 41.4:  Placeholders for the 'Name for separate run log' attribute

Variable: Yes

Restrictions: None, characters that are illegal for a filename will be replaced with '_'.

'Variable definitions'

This is where you define the values of the variables that remain bound during the execution of the sequence's child nodes (see chapter 6). See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Maximum error level'

When a warning, error or exception occurs during a test run, the state of the corresponding node of the run log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run log node represents the worst of its child nodes' states. Using this attribute, the maximum error state that the run log node for a sequence will propagate, can be limited.

Note This value has no effect on the way exceptions are handled. It only affects the error states of the run log nodes and by that the exit code of QF-Test when run in batch mode (see. section 1.7). It also has no effect on the creation of compact run logs (see command line argument -compact). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute.

Variable: No

Restrictions: None

'Execution timeout'

Time limit for the node's execution in milliseconds. If that limit expires the execution of that node will get interrupted.

Variable: Yes

Restrictions: >= 0

'QF-Test ID'

At the moment the 'QF-Test ID' attribute has no meaning for this type of node.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test suite.

Note For detailed documentation, especially for 'Test set', 'Test case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the button.

You can trigger special behaviors of some nodes using doctags, please see Doctags.

If you enter text in the comment field of a 'Component' node, the node will be considered as 'used' when you want to mark or delete unused components.

Variable: Yes

Restrictions: None

41.2.6

'Sequence with time limit'

This node extends the basic 'Sequence' node with time-constraint checking. Child nodes are executed as usual, but upon completion of the sequence the elapsed time is compared to the time-limit. Exceeding the time limit will result in a warning, error or exception, depending on the value of the attribute 'Error level if time limit exceeded'. Explicit delays like 'Delay before/after' or user interrupts are deducted from the duration before the constraints are checked, unless 'Check realtime' is activated. For report generation, time-constraints are treated like checks. If the 'Comment' attribute starts with an '!' character, the result will be logged in the report. Note The function of this node is to check time constraints in the user-acceptance range, i.e. between a few hundred milliseconds and a few seconds. Real-time constraints of a few milliseconds or less are beyond its limits.

Contained in: All kinds of sequences.

Children: Any

Execution: The 'Variable definitions' of the 'Sequence with time limit' are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again. The elapsed time is compared to the given time limit.

Attributes:

		Figure 41.7:  'Sequence with time limit' attributes

'Name'

The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and say something about the function of the sequence.

Variable: No

Restrictions: None

'Time limit for execution'

The time (in milliseconds) allowed for the execution of the sequence.

Variable: Yes

Restrictions: Must not be negative.

'Check realtime'

Normally explicit delays like 'Delay before/after' or user interrupts are deducted from the duration, before the time constraints are checked. To prevent this deduction and therefore check the real-time, this attribute can be activated.

Variable: No

Restrictions: None

'Error level if time limit exceeded'

This attribute determines what happens in case the time limit is exceeded. If set to "exception", a CheckFailedException will be thrown. Otherwise a message with the respective error-level will be logged in the run log.

Variable: No

Restrictions: None

'Variable definitions'

This is where you define the values of the variables that remain bound during the execution of the sequence's child nodes (see chapter 6). See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Maximum error level'

When a warning, error or exception occurs during a test run, the state of the corresponding node of the run log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run log node represents the worst of its child nodes' states. Using this attribute, the maximum error state that the run log node for a sequence will propagate, can be limited.

Note This value has no effect on the way exceptions are handled. It only affects the error states of the run log nodes and by that the exit code of QF-Test when run in batch mode (see. section 1.7). It also has no effect on the creation of compact run logs (see command line argument -compact). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute.

Variable: No

Restrictions: None

'Execution timeout'

Time limit for the node's execution in milliseconds. If that limit expires the execution of that node will get interrupted.

Variable: Yes

Restrictions: >= 0

'QF-Test ID'

At the moment the 'QF-Test ID' attribute has no meaning for this type of node.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test suite.

Note For detailed documentation, especially for 'Test set', 'Test case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the button.

You can trigger special behaviors of some nodes using doctags, please see Doctags.

If you enter text in the comment field of a 'Component' node, the node will be considered as 'used' when you want to mark or delete unused components.

Variable: Yes

Restrictions: None

41.2.7

'Extras'

This node is a kind of clipboard or playground where the usual restrictions on the parent of a node don't apply. You can add any kind of node here to assemble and try out some test sequences.

Contained in: Root node

Children: Any

Execution: Cannot be executed

Attributes:

		Figure 41.8:  'Extras' attributes

'QF-Test ID'

At the moment the 'QF-Test ID' attribute has no meaning for this type of node.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test suite.

Note For detailed documentation, especially for 'Test set', 'Test case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the button.

You can trigger special behaviors of some nodes using doctags, please see Doctags.

If you enter text in the comment field of a 'Component' node, the node will be considered as 'used' when you want to mark or delete unused components.

Variable: Yes

Restrictions: None

Last update: 9/10/2024 Copyright © 1999-2024 Quality First Software GmbH