Developed by Muth Partners Dirk O. Schweier – G. Muth Partners GmbH, Wiesbaden, Germany – November 2017
In projects where test automation and load testing is playing a role, two questions frequently arise.
When load tests are executed with a load test tool, usually the response time is defined by the duration of sending a request and receiving the response. However when the response arrives on the client’s side, additional operations may be needed before this information can be visualized, e.g. with a web browser.
This visualization may be time consuming withinin today’s web applications, because most often HTML-structures are created via scripts on the client dynamically.
In consequence the question when the end user is able to see a reaction in the browser arises. This metric is often called End User Experience and can be measured with this integration.
Usually test automation tools test the functionality of an application in projects that already use test automation. If in addition to already existing automated tests load tests should get executed too, it is certainly advanteagous if these scripts can get reused.
Test automation tools and load test tools have different concepts, so this reuse has certain limitations. An integration between these two tool types can reduce the effort a lot. The test automation tool QF-Test from Quality First Software and the load test tool NeoLoad offer interfaces. On their basis we developed an integration that gives answers to the two above mentioned questions. This integration adds the measurement of End User Experience to existing load test scenarios. The difference between response time and End User Experience shows how much time was spent for content processing and rendering by the browser.
It is possible to migrate already existing testcases developed within QF-Test efficiently and partly automated in NeoLoad. The integration saves time for manual capturing of NeoLoad scripts as the transfered NeoLoad script structure is comparable.
Starting from NeoLoad version 6.1 onwards the possibility of calling up other programs and tests scripts from the advanced actions has been added. These program calls can be integrated into the user path.
For an advanced action “Executable Test Script” the performance tester can deliver with parameters that are transferred into an external software call. The parameters’ order is defined by the parameters’ names.
The advanced actions are provided by the integration of Muth Partners that are matched to those of QF-Test. They support special parameters whose order isn’t important.
Different symbols of the different QF-Test actions help to gasp the user path optically. Via the test execution report the performance tester can access different performance measurements e.g. the duration of the execution. The graph is only uncertain, because the time to access the test execution software is added as well, which is not clear for the evaluation.
The integration makes it possible that both software products test cases can be called up in QF-Test by specialized advanced actions in NeoLoad. The test execution time (e.g. test case nodes or sequences) is delivered from QF-Test to the NeoLoad controller. In the left part of the screenshot on the image above you can see the structure of the test script in QF-Test shown as folders. Below those folders the metrics (e.g. duration or state are visualized).
In the right part of the screenshot there are two graphs. The upper graph shows the duration of the advanced action and the lower of the selected test case node on the left. The graphic shows that the values can vary widely.
The performance tester can change the degree of detail of the values’ recording via the configuration himself.
The advantage of this functionality is that existing scripts from QF-Test can be integrated easily into a load test and the load test with the metric of the End User Experience contains new possibilities of evaluation.
For load tests few substantial business processes are needed that then are replayed in a high number to reach a realistic system load.
When these business processes are already developed for functional tests in QF-Test, it is advantageous to reuse them without great developing effort in NeoLoad. Usually this isn’t that easy since QF-Test tests via the GUI and NeoLoad just sends requests and receives responses. So the scripts differ a lot in their structure. The idea of opening QF-Test respectively often from NeoLoad doesn’t work in many cases since several browser instances on one computer can disturb each other. Via the newly realized integration scripts that are developed, QF-Test test-suites can be imported partly automatically in NeoLoad. Therefore the relevant scripts are executed in QF-Test and recorded by NeoLoad. On the following image on the left handside you can see the script how it is deposited in QF-Test and on the right handside is the corresponding import in NeoLoad.
In the user path it is possible to recover sequences from QF-Test as transactions in which communication with the test system is taking place. Scripts developed in QF-Test can be migrated with this integration without much preparation. When developing in QF-Test there are no precautions needed to ensure for the later import. However, you may have to post-edit the scripts in NeoLoad, because the NeoLoad wizard neither recognizes all dynamic parts nor replaces them all.Additionally you should complete logical actions as branches or loops afterwards in the user path. For later imports or scripts changes in QF-Test you can use the script maintenance functionality of NeoLoad. Here the new recording with the existing user path is created as an updated user path. The updated user path contains logical actions and variables after the import that the performance tester created before.
The advantage of this functionality of integration is the widely automated import from QF-Test to NeoLoad. Furthermore the recognition of sequences and transactions via names and the measurement of End User Experience for a meaningful analysis for load tests.
The integration pre-assumes that QF-Test and the load generator is able to communicate with a QF-Test daemon installed on another computer too. Alternatively the load generator can also communicate with a QF-Test daemon which is installed on another computer.
For test script development with the usage of this integration we recommend a Windows or Linux computer with a minimum of 4GB RAM.
The integration is tested with QF‑Test 4.1.6 and released for this version.
You need at least one valid runtime license of QF‑Test per QF‑Test instance for a load test.
QF-Test can be evaluated with a Trial licensefor four weeks.
The integration is tested with NeoLoad 6.1 and released for this version. For using the Advanced Actions the option “Integrations & Advanced Usage” must be included in the license.
With the Free-License of NeoLoad the integration can be evaluated.
report their
QF-Test experiences
report their
QF-Test experiences
report their
QF-Test experiences
