Home > Test Plans > 7.3 Evaluation Process

7.3 Evaluation Process

Section 7, Item Pass/Fail Criteria of a test plan has four sub-sections: 7.1 Evaluation Team, 7.2 Exit Criteria, 7.3 Evaluation Process, and 7.4 Requirements Traceability Matrix.

This sub-section on the 7.3 Evaluation Process describes a four stage process for systematically evaluating the results of the testing in order to make a decision about whether the test item has passed or failed. The stages are:

1. Summarise Testing Results - This deals with taking all Incidents and tracing them back to the requirements they affected.
2. Evaluate Business Scenarios - This is similar but is more organisation oriented as it deals with the effect of Incidents on the business scenarios the organisation will run on this test item. The intention is to evaluate exactly what the effect each open incident will have on the business scenario functionality.
3. Estimate Business Impact - Having identified where the open Incidents will affect each requirements and business scenario the effect on the organisation is then judged. A table is constructed showing the the incident, the effect it will have, the frequency of the impact on the business, and counter measures that could be taken if the item was released with this open incident.
4. Make Acceptance Decision - A decision is then taken as to whether to accept or reject the item. In practise there will be a lot of pressure to accept and item. Therefore it is useful to have a third category of Limited Acceptance, which means that the system is accepted with provisos.

Sample Text for 7.3 Evaluation Process

1. Summarise Testing Results: All incidents, whether open or closed, will be traced back to the requirements on a Requirements Traceability Matrix.
2. Evaluate Business Scenarios: Each open Incident - and closed Incident if agreed it is appropriate - is traced back to the Business Scenarios and an assessment made about the technical impact on them, and whether they can deliver the functionality the organisation needs.
3. Estimate Business Impact: Each open Incident is then checked to see what impact it will have on the organisation. The effect, frequency business impact, and counter measures such as fix schedule and workarounds are analysed and recorded.
4. Make Acceptance Decision: The analyses are then evaluated as to the acceptance decision. This can be one of:
   1. Full Acceptance: The system will be accepted as is. Any outstanding Incidents will be worked around.
   2. Limited System Acceptance: The outstanding Incidents cause too many problems. The system is accepted subject to a timetable of fixes, staff training about workarounds, and similar measures.
   3. System Rejection: This is where the system does not support the Business Scenarios of the organisation.

Other Articles on QUALITY

• 7. Item Pass/Fail Criteria gives an overview of how to evaluate the testing data to make a pass/fail decision.
• Basics of Project Plans gives further information on item 2:"Long or Unrealistic Timescales" and item 4:"Scope Creep" as two of the five key characteristics of a project plan and the importance of balancing them.
• Verification and Validation has further information on item 6:"Poor Testing" by explaining what the terms Verification and Validation mean.