Home > UAT > Testing Principles

Testing Principles

"The Art of Software Testing" by Glenford Myers was published in 1979 and covered good practise in software testing. The book was and remains a must read text for anybody doing software testing and was revised edition was published in 2004. Key to both editions is three important principles of software testing:

1. Testing is the process of executing a program with the intent of finding errors.
2. A good test case is one that has a high probability of detecting an as yet undiscovered error.
3. A successful test case is one that detects an as yet undiscovered error.

These sound principles seem counter intuitive to many people when starting testing, especially UAT, and need further explanation.

Testing Is To Find Errors

The 1st principle is "Testing is the process of executing a program with the intent of finding errors". The 1st edition has "finding an error" rather than "finding errors".

The problem people have with this principle is they believe that testing is about showing that a program or system works. They adopt a system of belief in the system rather than being properly sceptical and adversarial. They think that just running a few tests, they believe will work, are sufficient. Whereas they need to run tests designed to make the system fail. Only by doing are they doing useful testing.

It follows from this principle that you do not stop testing just because the tests are failing. This is what you are seeking to and therefore you continue running tests. The only reason to stop is if the system is so badly fouled up that no useful tests can be run but this should be a very rare decision.

Tests Are Designed To Find New Errors

The 2nd principle is "A good test case is one that has a high probability of detecting an as yet undiscovered error". The 1st edition has "finding" rather than "detecting".

The focus of this principle is the design of test cases. The aim should be to focus on trying to find new errors in a system and not on finding the same error again. It has two important aspects:

1. It is a restatement of the Equivalence Partition principle of finding areas of the system which behave the same and testing them once. If they fail it is deemed all that area has failed, if it passes then all have passed.
2. It is also guides the development of test cases in those areas that are most likely to fail. These are:
   • Areas which have already have had faults and
   • Areas which have been extensively modified.

The key is to design the test cases to give the biggest opportunity for them to fail.

Successful Tests Find new Errors

The 3rd principle is "A successful test case is one that detects an as yet undiscovered error". The 1st edition has "uncovers" rather than "detects".

This principle deals with running the test cases and the reaction the tester has to the results. Running a test does not detect an error by itself. All it can do is show is a difference between the Expected Results from the test case and the Actual Results. This difference can then be evaluated to see if an error exists in the system. Therefore when a difference is found the reaction of the tester is joy at having found a potential error. If the expected and actual results are the same, indicating the test has passed, then this is an unsuccessful test case and the tester is unhappy.

The Happy Tester

These principles lead you the concept of happy testers. These are people who are happy when test cases fail in new and unique ways because it will allow them to find an error, and they are unhappy when tests pass because they have found no new errors.

A happy tester strives hard to come up with a list of unique errors with a program or system using the minimal number of tests to do the job.

Other UAT Definitions

• What Is UAT, And Why Do It? - why User Acceptance Testing is important.
• UAT Definition is the formal UAT definition.