When will the test tool be amortized?
With test automation, e.g., using tools like QF-Test, test costs can already break even after the third test cycle. The individually reached ROI can vary of course.
In agile software development the ROI is reached faster due to shorter iteration cycles (daily at best) than in classical development and QA approaches.
The initial effort for automating tests is usually greater than doing manual tests.
“So far – and this is just an interim result – we have saved 28,638 hours in our end-to-end test automation in 25 projects with 5 technologies (Web, JavaFX, Electron, Android, SWT) at 10 locations with QF-Test.”
Automated tests and Return on Inverstment (ROI)
Handling the increasing testing effort
Due to the consecutive development cycles more and more functionalities must be tested. The increasing test effort can just be handled with test automation.
In the course of time a great test base is created and for a maximum test coverage further tests are recorded bit by bit when there are new features.
Agile software development has lots of iterations and that’s the reason why test automation is a necessity – in addition to unit tests and single manual tests.
Time savings due to automation in practice: Practical examples of QF-Test users
| Time before automation | Time after automation | Savings | Citation |
|---|---|---|---|
| 8 hours | 1 hour | 87.5% | Thomas Schöning, ISTQB Certified Test Manager, Airbus Defence and Space GmbH, Multi-INT Products & Projects Germany: Running about 100 tests automatically and in parallel takes about one hour. Running the tests manually used to occupy one person for three hours per configuration. For the standard product and its configuration (two modes), this results in a saving of currently 8 hours per day, time which we happily invest in further automated tests or other activities. (see complete case study). |
| 8 weeks | 1 week | 87.5% | Sean Kane, Manager of Test Engineering, Intervoice Inc., Dallas, USA: Now we can regression test the application in 1 week. It used to take 8 weeks. |
| 2 weeks | 5 days | 50% | Gadi Goldbarg, Development Tools, QA Team Leader, Zend, Ramat Gan, Israel: The usage of QF-Test has reduced my test cycle from 2 weeks (plus/minus) to 5 days (since not all is automated yet). |
| 1 day | 3 hours | 62.5% | Denis Gauthier Software Integration, Thales Australia, Melbourne: Regarding the time saving aspect, it usually took me a full day to perform the Non-Regression Tests. It has now been reduced to three hours. (see complete case study). |
| 1 day | less than 1 hour | 87.5% | Phil Cross, Lockheed Martin, Owego New York, USA: Regression testing of our Java application that previously took a full day to execute is now done in less than an hour with QF-Test. |
| 7-8 hours | 1,5 hours | 80% | Heidi Klade, Logistik Pur Software GmbH, Koppl near Salzburg, Austria: Our tests run for about 90 minutes. I suppose that a manual test run would last about 7-8 hours of concentrated work. |
| Reduction of almost 50% | CertiCon, Prague, Czech Republic: In a matter of months we succeeded in reducing the test processing time by almost 50%, adding hundreds of additional test cases. (see complete case study). |
You can find the complete citations here: Advantages for decision-makers
Phases of the test process with high influence on the ROI
The three pink phases of the test process (see image on the left) Test case development, test case execution and maintenance of test cases influence the return on investment (ROI) during the automation of tests the most.
| Phase of the test process | Manual Testing | Automated Testing |
|---|---|---|
| Test planning | Test planning includes the planning of the tests and the provision of the test environment. This is identical for manual and automated testing. | |
| Test case specification | For manual and automated tests you need the analysis and description of the test cases. | |
| Development | In manual testing, you must elaborately create instructions for the testers that consistency can be maintained. | Implementing the test cases with the test tool takes time, money and resources. The initial investment are amortized over the test cycles. |
| Documentation | In manual testing, the test plan correlates with the test instructions. | Automated documentation can be generated quickly from the test cases. |
| Test case managment | With the manual or automated test strategy, documents must be managed in each case. | |
| Test execution | The test execution is the big cost driver here, namely hardware and personnel cause it. Careless errors are also possible. | The test tool is very reliable during test execution. It makes optimal use of the hardware. |
| Management of results | Manually, the results must be entered by hand. | The advantage of automation is the automatic generation of the report. This increases the quality of the report and simplifies and speeds up the report creation process. |
| Maintenance of test cases | In manual testing, one would adjust the test instructions only after fundamental changes. | Test cases in the GUI must of course be adapted, which involves maintenance work. However, the advantage of a test tool is that changes to the graphical user interface only have to be made once. It also enables efficient modularization and reliable recognition of components. |
See for yourself how QF-Test can generate a fast ROI for you!
The supported technologies
-
Any Java application (Java Swing, SWT/Eclipse, JavaFX) cross-platform on Windows, Linux and macOS
-
HTML 5, Web applications cross-browser and headless with CDP connection mode and web driver
-
Native, modern Windows applications (Win32, .Net, Windows Apps, C++)
-
Android and iOS applications with real devices and emulator
-
Testing PDFs with every QF-Test developer license
