To identify and assess the differences between the expected results
and actual results
To enter change request information into a tracking tool for
assessment, management, and resolution
In the Test Execution activity, the results of test were reviewed to ensure
that the testing executed to completion, and the reported test results were not
influenced by non-target-of-test factors.
In this activity, the test results are analyzed to identify the details regarding
the differences between the expected test results and the actual test results.
Differences indicate potential defects in the target-of-test and should be
entered into a tracking system as change requests, and the appropriate next
corrective actions taken.
Review each reported test failure and identify the necessary information
needed to enter a change request into the appropriate tracking system. The
information entered should be accurate, appropriate, and meet accepted change
request tracking standards or guidelines.
Evaluate Requirements-based Test Coverage
Purpose
To determine if the required (or appropriate) requirements-based test
coverage has been achieved
To evaluate requirements-based test coverage, you need to review the test
results and determine:
The ratio between how many requirement-based tests (test cases) have been
performed in this iteration and a total number of tests for the target for
test.
The ratio of successfully performed test cases.
The objective is to ensure that 100 % of the requirements-based tests
targeted for this iteration have been executed successfully. If this is not
possible or feasible, a different test coverage criteria should be identified,
based upon:
Risk or priority
Acceptable coverage percent
Document the results in a Test Evaluation Report for this iteration.
Evaluate Code-based Test Coverage
Purpose
To determine if the required (or appropriate) code-based test coverage
has been achieved.
To evaluate code-based test coverage, you need to review the test results and
determine:
The ratio between the code that has been executed during test (such as
lines or statements) in this iteration and the total code in the test
target.
The objective is to ensure that 100 % of the code targeted for this iteration
phase has been executed successfully. If this is not possible or feasible, a
different test coverage criteria should be identified, based upon:
Risk or priority
Acceptable coverage percent
Document the results in a Test Evaluation Report for this iteration.
Analyze Defects
Purpose
To evaluate the defects and recommend the appropriate follow-on
activity
To produce objective reports communicating the results of testing
To analyze defects, you need to review and analyze the measures chosen as
part of your defect analysis strategy. The most common defect measures used
include the following different measures (often displayed in the form of a
graph):
Defect Density - the number of defects are shown as a function of
one or two defect attributes (such as status or severity).
Defect Trend - the defect count is shown as a function over time.
Defect Aging - a special defect density report in which the defect
counts are shown as a function of the length of time a defect remained in a
given status (open, new, waiting-for-test, etc.)
Compare the measures from this iteration to the results from the analysis of
previous iterations, to get a view of defect trends.
It is advisable to show the results in diagram form.
Test Completion and Success Criteria
Purpose
To determine if testing has been executed completely and acceptably
To identify the appropriate follow-on test activity
Review the test strategy defined in the test plan. It should present test
criteria in terms of test coverage and, or defect evaluation. Examine the test
results, defects, and the analysis of the defects, and determine if the criteria
have been met.
If the criteria have not been met, there are several alternatives:
Gather additional information:
Produce different reports, such as different defect density reports
Investigate the process to determine if unexpected conditions contributed
to the deviation from the stated test criteria and re-assess the criteria
based upon this new knowledge
Suggest scheduling of additional testing:
Implement new tests to add depth to the test cases
Implement new tests to add greater test coverage
Modify the test criteria:
Review and assess the risk of changing the criteria POST testing
Identify a subset of the software that satisfies the test criteria and
decide whether it can be deployed.
Generate the Test Evaluation Report
Purpose
To produce an objective report communicating the results of testing,
key measures, and test recommendations
The last step in the Evaluate Test activity is to generate the test
evaluation report. This is accomplished by assembling the above
information into a single report and distributing it to the appropriate roles for review.
Roles and Activities >
Tester >
Evaluate Test
