In our first article “Automated Testing ROI Part 1: When is Automated Testing More Profitable than Manual One?” (please see the full method of calculation here) we reviewed the advantages and disadvantages of automated testing vs manual one on a general basis regardless where they are done. In Part 2 we would like to take a look if there is any difference, and what it is, when automated testing is done in different countries.
Our customers often wonder if there’s any difference in ROI, its pace and efficiency, depending upon in what country and with what company an order for automated testing is placed. Some are convinced that automated testing ROI in the West, for instance, will vary from that in the East. This question sometimes comes up due to the difference in time and effort by QA engineers.
In order to get this question answered we decided to analyze ROI in different parts of the world: the Eastern Europe and the USA.
Example 1: Basic Test Set Usage
Suppose we have a regular task to work out a set of basic tests which we are going to perform in every new product build. Our goal is a minimal quality assessment and Non-GUI tests running.
Our assumptions for our project are:
- Daily builds and test runs (5 times a week)
- Manual tests take 20 days to design, 8 hours to run
- Only half the manual tests would be run on any given day (4 hours) with the other half run the following day
- Automated tests take 60 days to design and implement, automatically run (zero cost)
- Automation is done with batch scripts and integrated into the build process. Test server requires $1,500 in added hardware, with a useful life of 3 years
- Automated tests need to be maintained every 20 runs, 3 days of work required
- Periods of time (t) selected: 6 months (125 days) and 18 months (375 days)
Example 2: the Full-Scale Set of GUI Tests
Let’s presume we have a goal to quality test the product functionality via GUI.
In this case our assumptions will look like the following:
- A new product with all new tests
- 1 man-year developing manual tests, 3 for automated tests
- 1 person maintenance after 1st year for automated tests
- 3 people full time running manual tests, 1 person for automated
- Fixed costs for automated tests (new hardware, tools licenses) of $15,000 with useful life of 3 years
- Periods of time (t) selected: 12 months (250 days) and 24 months (500 days)
Example 3: We have increased our effort from 3 to 6 man-years.
(The data is the same as in Example 2, with 6 man-years of automated tests instead of 3).
As we see on all three charts above, the ROI in our projects doesn’t vary much from the costs of resources. At the same time the end expenses on automation when placing an order in the Eastern Europe are lower than in the USA and at the same time the ROI is nearly the same.
The bottom line: it’s pretty much the same ball game if you place your order for automated testing in the Eastern Europe or the US – the time for ROI is almost identical but expenses will differ.
We hope our experience will help you understand more on Automated Testing ROI.
