Saturday, July 4, 2009

Manual & Automation

There are two ways of testing – MANUAL & AUTOMATION.

Manual testing is the process of manually testing software for defects. It requires a tester to play the role of an end user, and use most of all features of the application to ensure correct behavior. To ensure completeness of testing, the tester often follows a written test plan that leads them through a set of important test cases.

Test automation is the technique of testing software using software rather than people. A test program is written that exercises the software and identifies its defects. These test programs may be written from scratch, or they may be written utilizing a generic Test automation framework that can be purchased from a third party vendor. Test automation can be used to automate the sometimes menial and time consuming task of following the steps of a use case and reporting the results.

Here are the comparisons between the two ways:

Pros of Automation:
If you have to run a set of tests repeatedly, automation is a huge win for you
It gives you the ability to run automation against code that frequently changes to catch regressions in a timely manner
It gives you the ability to run automation in mainstream scenarios to catch regressions in a timely manner
Aids in testing a large test matrix (different languages on different OS platforms). Automated tests can be run at the same time on different machines, whereas the manual tests would have to be run sequentially.

Cons of Automation:
It costs more to automate. Writing the test cases and writing or configuring the automate framework you’re using costs more initially than running the test manually.
Can’t automate visual references, for example, if you can’t tell the font color via code or the automation tool, it is a manual test.

Pros of Manual:
If the test case only runs twice a coding milestone, it most likely should be a manual test. Less cost than automating it.
It allows the tester to perform more ad-hoc (random testing). In my experiences, more bugs are found via ad-hoc than via automation. And, the more time a tester spends playing with the feature, the greater the odds of finding real user bugs.

Cons of Manual:
Running tests manually can be very time consuming
Each time there is a new build, the tester must rerun all required tests - which after a while would become very mundane and tiresome.

No comments: