18. août 2021
TestRunListeners dans QF-Test
The TestRunListener interface can be used to execute additional actions before or after the execution of each node or in the case of any exception / error.
This actions can (for example) be used for testdocumentation or error analysis.
These actions can for example be used for test documentation or error analysis. In the following some TestRunListeners are introduced (Jython server scripts):
Variable monitoring
The following TestRunListener always logs an error when the value of a certain variable (in the example offset) changes. This is especially helpful when debugging a test suite if it is not clear where exactly the variable is assigned a certain value:
varname = "offset" from de.qfs.apps.qftest.extensions.qftest import AbstractTestRunListener def checkVar(state, var): try: val = rc.lookup(var) if(val != state): state = val rc.logError("Variable '%s': %s" % (var, state)) except: if state != "": rc.logError("Variable '%s' nicht verfügbar" % var) state = "" return state class VarChanger (AbstractTestRunListener): def __init__(self): self.state = None def nodeEntered(self, event): self.state = checkVar(self.state, varname) def nodeExited(self, event): self.state = checkVar(self.state, varname) global varChanger try: rc.removeTestRunListener(varChanger) except: pass varChanger = VarChanger() rc.addTestRunListener(varChanger)
Alternatively, it's easy to adapt this script so that the TestRunListener always logs an error when a certain value is assigned to the variable.
Error sound
It can happen that a user starts several tests on several computers at the same time. The following TestRunListener helps if it can happen that the test execution on a machine - for whatever reason - can come to a standstill; The TestRunListener always plays a tone if the execution of a node takes longer than a certain specified time limit. It is possible to redefine this time limit for a node using a @informUserTimeOut doctag. (Note: The sound output probably only works on Windows)
# A TestRunListener that informs the user when the current testrun has stopped for more then a certain period of time. # some imports we need later from de.qfs.apps.qftest.extensions.qftest import AbstractTestRunListener from java.lang import System, Thread from java.awt import Toolkit import time, re # The default time in ms after which to inform # the user. TIME_IN_MS_AFTER_WHICH_TO_INFORM_THE_USER = 2 * 60 * 1000 # The function that is used in order to inform the user def informUserAction(lastActionTimestamp): """ This function is called whenever the test- runlistener detected the timeout. In oder words, this function should play a sound or do something else in order to inform the user. :lastActionTimestamp: The timestamp of the last action. """ print "=== HEY %s ms passed, since we entered the last node! ===" % (System.currentTimeMillis() - lastActionTimestamp) # output a sound on windows Toolkit.getDefaultToolkit().getDesktopProperty("win.sound.exclamation").run() # the thread that will inform you once the timeout is reached ... class InformingThread(Thread): def __init__(self): self.updateTimeout(TIME_IN_MS_AFTER_WHICH_TO_INFORM_THE_USER) self.stop = False def updateTimeout(self, timeout): self.lastAction = System.currentTimeMillis() self.errorAfterTimestamp = self.lastAction + timeout def run(self): while not self.stop: time.sleep(1) if System.currentTimeMillis() > self.errorAfterTimestamp and not self.stop: try: informUserAction(self.lastAction) except: pass # the testrunlistener that keeps track of when a new node get's enetered ... class InformUserWhenHaveBeenIdleTestRunListener(AbstractTestRunListener): def __init__(self): self.thread = InformingThread() self.thread.start() self.myRegex = re.compile("@informUserTimeOut\\s+(\\d+)", re.DOTALL) def nodeEntered(self, event): timeout = 0 comment = event.getNode().comment if comment != None and comment.find("@informUserTimeOut") != -1: match = self.myRegex.search(comment) if match: timeout = int(match.group(1)) self.thread.updateTimeout(TIME_IN_MS_AFTER_WHICH_TO_INFORM_THE_USER if timeout <= 0 else timeout) def runStopped(self, event): self.thread.stop = True # register the testrun listener global informUserWhenHaveBeenIdleTestRunListener try: informUserWhenHaveBeenIdleTestRunListener.thread.stop = True rc.removeTestRunListener(informUserWhenHaveBeenIdleTestRunListener) except: pass informUserWhenHaveBeenIdleTestRunListener = InformUserWhenHaveBeenIdleTestRunListener() rc.addTestRunListener(informUserWhenHaveBeenIdleTestRunListener)
Counting Check-nodes
The following TestRunListener counts how many check nodes where executed during test execution:
from de.qfs.apps.qftest.extensions.qftest import AbstractTestRunListener class StatisticTestRunner (AbstractTestRunListener): def __init__(self): self.steps, self.checkSteps = 0, 0 def runStopped(self, event): print "steps: %s\ncheckSteps: %s" % (self.steps, self.checkSteps) self.steps, self.checkSteps = 0, 0 # reset counts def nodeEntered(self, event): self.steps += 1 if (event.getNode().getType().startswith("Check")): self.checkSteps += 1 global statisticTestRunner try: rc.removeTestRunListener(statisticTestRunner) except: pass statisticTestRunner = StatisticTestRunner() rc.addTestRunListener(statisticTestRunner)
General remarks
- Similar to SUT Scripts you should avoid calling rc.callProcedure from within a TestRunListener. Although it's often possible to call rc.callProcedure from within a TestRunListener, as this can lead to unexpected problems.
- Instead of derivating from TestRunListener you should always derivate from AbstractTestRunListener.