7. Item Pass/Fail Criteria
There are four sections for defining the QUALITY of a Test Plan: 2. Introduction, 6. Approach, 7. Item Pass/Fail Criteria, and 9. Test Deliverables.
QUALITY deals with the issues of the quality standards to be applied to the testing plan and not to the software being tested. The plan gives the framework for how the system will be evaluated and under what circumstances it will be released.
The "7. Item Pass/Fail Criteria" section deals with defining when an item has passed or failed. This is not the place to define the detailed pass criteria for each feature, but to describe the process and overall standards for evaluating the test results.
There are various approaches that are used in evaluating test items including the mechanical one of passing or failing a test item depending on whether a predetermined number of incidents of certain types have been found, e.g. "There must be no Type B incidents and no more than 5 type C."
Although a common method it is deeply flawed, especially for UAT. When assessing the system for use in an organisation you are trying to see if it can deliver value to that organisation and not if it works perfectly. The reality is that all computer systems have faults. What you are evaluating is if you can live with those faults.
In this section you describe a process to enable a realistic decision to be made about the test item. In this section of the Test Plan there are four suggested subsections:
- 7.1 Evaluation Team - This sub-section describes the job responsibilities and possibly the names of those on the evaluation team.
- 7.2 Exit Criteria - Defines when to stop testing.
- 7.3 Evaluation Process - Describes the four steps of the evaluation process to be followed.
- 7.4 Requirements Traceability Matrix - Describes the process to record and track incidents back to requirements.
Other Articles on QUALITY Sections
- 2. Introduction gives an overview of the QUALITY process.
- 6. Approach gives an overview of the six steps of the testign process approach.
- 9. Test Deliverables describes what is produced by the testing process.
- Basics of Project Plans gives further information on item 2:"Long or Unrealistic Timescales" and item 4:"Scope Creep" as two of the five key characteristics of a project plan and the importance of balancing them.
- Verification and Validation has further information on item 6:"Poor Testing" by explaining what the terms Verification and Validation mean.