Testing accounts for a significant part of any development effort. Estimates put this at between 50% – 80% of producing the first working version of an application. If the lifecycle of an application is considered from inception to retirement, then test and quality assurance costs are an even larger part of the total cost.
Unit testing plays a major role in testing efforts. Studies have proved time and time again that effective unit testing can dramatically reduce the overall cost of identifying and debugging defects as can be seen in the chart below from Capers Jones.

The cost includes the effort taken to prepare the test cases, run the tests, analyse the results and fix any resulting defects. This makes it obvious that the smart way to detect and fix defects is better quality testing at the early stages.

Typical Problems

1. Monotony

There are no two ways about it, unit testing is dull! Unlike fixing defects, which can be like solving a puzzle, testing is an activity which doesn’t have a predictable end. Each test case may or may not reveal a defect. Test runs have to be repeated many times. There is no other activity within the development process which is as repetitive as testing. For this reason it is an activity that no-one enjoys and as a result it is rarely done sufficiently well. Automation of as many of the routine activities as possible will help a long way in reducing the monotony of testing. Defining concrete completeness criteria for testing activity also brings some predictability to testing in that there is now an identifiable goal to be achieved.

2. Poor Documentation of Test Cases

Test cases are as valuable an asset as the application itself. Test cases are modified and re-run many times over the lifetime of an application so documenting the test cases is absolutely essential for effective future use. In practice test cases are rarely documented adequately. When the code gets changed, the test cases may need to be updated and the documentation must be updated. Testing often includes test cases that are made up on the spot during testing. Such test cases are very rarely documented. It is very useful to ensure that test case documentation is updated as part of the testing process. This way documentation continues to be generated while testing is done.

3. Informal Testing Process

While the early part of the development process such as Analysis is well defined and widely practiced, effective testing techniques are not as well known and more rarely implemented. As a result testing continues to be an informal and ill-planned process. Combining Functional (based on specifications) Structured (based on the structure of the code) and Heuristic (based on intuition) testing techniques give much better results than simply using an intuitive approach to testing. Many developers simply run a few intuitive test cases then go into debugging mode. Testing must be mostly systematic and partly intuitive rather than the more usual practice of mostly intuitive and party systematic approach to testing.

4. Poor regression testing

Testing is a very repetitive activity that needs to be repeated whenever a change is made to existing code. For example if Test Case 12 in a large test plan fails, the problem is analysed, fixed and then tested in isolation to validate the fix. The tester then moves on to run Test Case 13. During the maintenance phase it is much better to re-run all the tests successfully completed on the document in question. However for every fix it is much more effective to re-run all the affected test cases (better still all the test cases) for the application in question. Since testing is essentially a manual process, any capability to automate re-running of test cases is invaluable. Automation is the only effective solution to poor regression testing.

Coming Next – Secrets of Effective Testing

In the next article we’ll discuss some of the techniques you can use to minimise the problems above.