Alpha, beta and acceptance tests

Alpha, beta and acceptance tests

In the various testing activities that have been described so far, users have played a supporting role for the most part. They have been involved in requirements analysis and reviews, and have played a role in test planning. This is especially true for acceptance test planning if the software is being custom made for an organization. The clients along with test planners design the actual test cases that will be run during acceptance test. Users/clients may also have participated in prototype evaluation, usage profile development, and in the various stages of usability testing. After the software has passed all the system tests and defect repairs have been made, the users take a more active role in the testing process. Developers/testers must keep in mind that the software is being developed to satisfy the users requirements, and no matter how elegant its design it will not be accepted by the users unless it helps them to achieve their goals as specified in the requirements. Alpha, beta, and acceptance tests allow users to evaluate the software in terms of their expectations and goals.

When software is being developed for a specific client, acceptance tests are carried out after system testing. The acceptance tests must be planned carefully with input from the client/users. Acceptance test cases are based on requirements. The user manual is an additional source for test cases. System test cases may be reused. The software must run under real-world conditions on operational hardware and software. The software-under-test should be stressed. For continuous systems the software should be run at least through a 25-hour test cycle. Conditions should be typical for a working day. Typical inputs and illegal inputs should be used and all major functions should be exercised. If the entire suite of tests cannot be run for any reason, then the full set of tests needs to be rerun from the start.

Acceptance tests are a very important milestone for the developers. At this time the clients will determine if the software meets their requirements. Contractual obligations can be satisfied if the client is satisfied with the software. Development organizations will often receive their final payment when acceptance tests have been passed.

Acceptance tests must be rehearsed by the developers/testers. There should be no signs of unprofessional behavior or lack of preparation. Clients do not appreciate surprises. Clients should be received in the development organization as respected guests. They should be provided with documents and other material to help them participate in the acceptance testing process, and to evaluate the results. After acceptance testing the client will point out to the developers which requirement have/have not been satisfied. Some requirements may be deleted, modified, or added due to changing needs. If the client has been involved in prototype evaluations then the changes may be less extensive. If the client is satisfied that the software is usable and reliable, and they give their approval, then the next step is to install the system at the client‘s site. If the client‘s site conditions are different from that of the developers, the developers must set up the system so that it can interface with client software and hardware. Retesting may have to be done to insure that the software works as required in the client‘s environment. This is called installationtest.

If the software has been developed for the mass market (shrinkwrapped software), then testing it for individual clients/users is not practical or even possible in most cases. Very often this type of software undergoes two stages of acceptance test. The first is called alpha test. This test takes place at the developer‘s site. A cross-section of potential users and members of the developer‘s organization are invited to use the software. Developers observe the users and note problems. Beta test sends the software to a cross-section of users who install it and use it under realworld working conditions. The users send records of problems with the software to the development organization where the defects are repaired sometimes in time for the current release.