Test planning

Test planning

A plan can be defined in the following way.

A plan is a document that provides a framework or approach for achieving a set of goals.

In the software domain, plans can be strictly business oriented, for example, long-term plans to support the economic growth of an organization, or they can be more technical in nature, for example, a plan to develop a specific software product. Test plans tend to be more technically oriented. However, a software project plan that may contain a test plan as well will often refer to business goals. In this chapter we focus on planning for execution-based software testing (validation testing).

Test planning is an essential practice for any organization that wishes to develop a test process that is repeatable and manageable. Pursuing the maturity goals embedded in the TMM structure is not a necessary precondition for initiating a test-planning process. However, a test process improvement effort does provide a good framework for adopting this essential practice. Test planning should begin early in the software life cycle, although for many organizations whose test processes are immature this practice is not yet in place. Models such as the V-model, or the Extended/ Modified V-model (Figure 1.5), help to support test planning activities that begin in the requirements phase, and continue on into successive software development phases [2,3].

In order to meet a set of goals, a plan describes what specific tasks must be accomplished, who is responsible for each task, what tools, procedures, and techniques must be used, how much time and effort is needed, and what resources are essential. A plan also contains milestones.

Milestones are tangible events that are expected to occur at a certain time in the project’s lifetime. Managers use them to determine project status.

Tracking the actual occurrence of the milestone events allows a manager to determine if the project is progressing as planned. Finally, a plan should assess the risks involved in carrying out the project. Test plans for software projects are very complex and detailed documents. The planner usually includes the following essential high-level items.

1. Overall test objectives. As testers, why are we testing, what is to be achieved by the tests, and what are the risks associated with testing this product?

2. What to test (scope of the tests). What items, features, procedures, functions, objects, clusters, and subsystems will be tested?

3. Who will test. Who are the personnel responsible for the tests?

4. How to test. What strategies, methods, hardware, software tools, and techniques are going to be applied? What test documents and deliverable should be produced?

5. When to test. What are the schedules for tests? What items need to be available?

6. When to stop testing. It is not economically feasible or practical to plan to test until all defects have been revealed. This is a goal that testers can never be sure they have reached. Because of budgets, scheduling, and customer deadlines, specific conditions must be outlined in the test plan that allow testers/managers to decide when testing is considered to be complete.

Test plans can be organized in several ways depending on organizational policy. There is often a hierarchy of plans that includes several levels of quality assurance and test plans. The complexity of the hierarchy depends on the type, size, risk-proneness, and the mission/safety criticality of software system being developed. All of the quality and testing plans should also be coordinated with the overall software project plan. A sample plan hierarchy is shown in Figure 7.1. At the top of the plan hierarchy there may be a software quality assurance plan. This plan gives an overview of all verification and validation activities for the project, as well as details related to other quality issues such as audits, standards, configuration control, and supplier control.

Below that in the plan hierarchy there may be a master test plan that includes an overall description of all execution-based testing for the software system. A master verification plan for reviews inspections/ walkthroughs would also fit in at this level. The master test plan itself may be a component of the overall project plan or exist as a separate document. Depending on organizational policy, another level of the hierarchy could contain a separate test plan for unit, integration, system, and acceptance tests. In some organizations these are part of the master test plan. The level-based plans give a more detailed view of testing appropriate to that level. The

IEEE Software Engineering Standards Collection has useful descriptions for many of these plans and other test and quality-related documents such as verification and validation plans.

The persons responsible for developing test plans depend on the type of plan under development. Usually staff from one or more groups cooperates in test plan development. For example, the master test plan for execution-based testing may be developed by the project manager, especially if there is no separate testing group. It can also be developed by a tester or software quality assurance manager, but always requires cooperation and input from the project manager. It is essential that development and testing activities be coordinated to allow the project to progress smoothly. The type and organization of the test plan, the test plan hierarchy, and who is responsible for development should be specified in organizational standards or software quality assurance documents.

The remainder of this chapter focuses on the development of a general- purpose execution- based test plan that will be referred to as a ―test plan.‖ The description of the test plan contents is based on a discussion of recommended test plan components appearing in the IEEE Standard for Software Test Documentation: IEEE/ANSI Std 829-1983 . This standard also contains examples of other test-related documents described in this chapter. The reader should note that the IEEE test plan description serves as a guideline to test planners. The actual templates and documents developed by test planners should be tailored to meet organizational needs and conform to organizational goals and policies.