Concepts:  Acceptance Testing

Acceptance testing is the final test action prior to deploying the software. The goal of acceptance testing is to verify that the software is ready and can be used by the end-users to perform those functions and tasks the software was built to do. There are three common strategies for implementing an acceptance test. They are:

• Formal acceptance
• Informal acceptance or alpha test
• Beta test

The strategy you select is often based on the contractual requirements, organizational and corporate standards, and the application domain.

Formal Acceptance Testing

Formal acceptance testing is a highly managed process and is often an extension of the system test. The tests are planned and designed as carefully and in the same detail as system testing. The test cases chosen should be a subset of those performed in system test. It is important not to deviate in any way from the chosen test cases. In many organizations, formal acceptance testing is fully automated.

The activities and artifacts are the same as for system testing. In some organizations, the development organization (or its independent test group), with the representatives of the end-user organization, performs the acceptance test. In other organizations, acceptance testing is completely performed by the end-user organization, or an objective group of people chosen by the end-user organization.

The benefits of this form of testing are:

• The functions and features to be tested are known.
• The details of the tests are known and can be measured.
• The tests can be automated, which permits regression testing.
• The progress of the tests can be measured and monitored.
• The acceptability criteria are known.

The disadvantages include:

• Requires significant resources and planning.
• The tests may be a re-implementation of system tests.
• It may not uncover subjective defects in the software, since you are only looking for defects you expect to find.

Informal Acceptance Testing

In informal acceptance testing, the test procedures for performing the test are not as rigorously defined as for formal acceptance testing. The functions and business tasks to be explored are identified and documented, but there are no particular test cases to follow. The individual tester determines what to do. This approach to acceptance testing is not as controlled as formal testing and is more subjective than the formal one.

Informal acceptance testing is most frequently performed by the end-user organization.

The benefits of this form of testing are:

• The functions and features to be tested are known.
• The progress of the tests can be measured and monitored.
• The acceptability criteria are known.
• You will uncover more subjective defects than with formal acceptance testing.

The disadvantages include:

• Resources, planning, and management resources are required.
• You have no control over which test cases are used.
• End users may conform to the way the system works and not see the defects.
• End users may focus on comparing the new system to a legacy system, rather than looking for defects.
• Resources for acceptance testing are not under the control of the project and may be constricted.

Beta Testing

Beta testing is the least controlled of the three acceptance test strategies. In beta testing, the amount of detail, the data, and approach taken is entirely up to the individual tester. Each tester is responsible for creating their own environment, selecting their data, and determining what functions, features, or tasks to explore. Each tester is responsible for identifying their own criteria for whether to accept the system in its current state or not.

Beta testing is implemented by end users, often with little or no management from the development (or other non end-user) organization. Beta testing is the most subjective of all the acceptance test strategies.

The benefits of this form of testing are:

• Testing is implemented by end users.
• Large volumes of potential test resources.
• Increases customer satisfaction to those who participate.
• You will uncover more subjective defects than with formal or informal acceptance testing.

The disadvantages include:

• Not all functions and / or features may be tested.
• Test progress is difficult to measure.
• End users may conform to the way the system works and not see or report the defects.
• End users may focus on comparing the new system to a legacy system, rather than looking for defects.
• Resources for acceptance testing are not under the control of the project and may be constricted.
• Acceptability criteria are not known.
• You need increased support resources to manage the beta testers.