Test plan attachments

Test plan attachments

The previous components of the test plan were principally managerial in nature: tasks, schedules, risks, and so on. A general discussion of technical issues such as test designs and test cases for the items under test appears in Section 5 of the test plan, ‗‗Approach.‘‘ The reader may be puzzled as to where in the test plan are the details needed for organizing and executing the tests.

For example, what are the required inputs, outputs, and procedural steps for each test; where will the tests be stored for each item or feature; will it be tested using a black box, white box, or functional approach? The following components of the test plan contain this detailed information. These documents are generally attached to the test plan.

Test Design Specifications

The IEEE standard for software test documentation describes a test design specification as a test deliverable that specifies the requirements of the test approach . It is used to identity the features covered by this design and associated tests for the features. The test design specification also has links to the associated test cases and test procedures needed to test the features, and also describes in detail pass/fail criteria for the features. The test design specification helps to organize the tests and provides the connection to the actual test inputs/outputs and test steps.

To develop test design specifications many documents such as the requirements, design documents, and user manual are useful. For requirements-based test, developing a requirements traceability matrix is valuable. This helps to insure all requirements are covered by tests, and connects the requirements to the tests. Examples of entries in such a matrix are shown in Table 7.3. Tools called requirements tracers can help to automate traceability tasks . These will be described in Chapter 14. A test design specification should have the following components according to the IEEE standard . They are listed in the order in which the IEEE recommends they appear in the document. The test planner should be sure to list any related documents that may also contain some of this material.

Test Design Specification Identifier

Give each test design specification a unique identifier and a reference to its associated test plan.

Features to Be Tested

Test items, features, and combination of features covered by this test design specification are listed. References to the items in the requirements and/or design document should be included.

Approach Refinements

In the test plan a general description of the approach to be used to test each item was described. In this document the necessary details are added. For example, the specific test techniques to be used to generate test cases are described, and the rational is given for the choices. The test planner also describes how test results will be analyzed. For example, will an automated comparator be used to compare actual and expected results? The relationships among the associated test cases are discussed. This includes any shared constraints and procedural requirements.

Test Case Identification

Each test design specification is associated with a set of test cases and a set of set procedures. The test cases contain input/output information, and the test procedures contain the steps necessary to execute the tests. A test case may be associated with more than one test design specification.

Pass/Fail Criteria

In this section the specific criteria to be used for determining whether the item has passed/failed a test is given.

Test Case Specifications

This series of documents attached to the test plan defines the test cases required to execute the test items named in the associated test design specification. There are several components in this document. IEEE standards require the components to appear in the order shown here, and references should be provided if some of the contents of the test case specification appear in other documents .

Much attention should be placed on developing a quality set of test case specifications. Strategies and techniques, as described in Chapters 4 and 5 of this text, should be applied to accomplish this task. Each test case must be specified correctly so that time is not wasted in analyzing the results of an erroneous test. In addition, the development of test software and test documentation represent a considerable investment of resources for an organization. They should be considered organizational assets and stored in a test repository. Ideally, the test-related deliverables may be recovered from the test repository and reused by different groups for testing and regression testing in subsequent releases of a particular product or for related products. Careful design and referencing to the appropriate test design specification is important to support testing in the current project and for reuse in future projects.

Test Case Specification Identifier

Each test case specification should be assigned a unique identifier.

Test Items

This component names the test items and features to be tested by this test case specification. References to related documents that describe the items and features, and how they are used should be listed: for example the requirements, and design documents, the user manual.

Input Specifications

This component of the test design specification contains the actual inputs needed to execute the test. Inputs may be described as specific values, or as file names, tables, databases, parameters passed by the operating system,and so on. Any special relationships between the inputs should be identified.

Output Specifications

All outputs expected from the test should be identified. If an output is to be a specific value it should be stated. If the output is a specific feature such as a level of performance it also should be stated. The output specifications are necessary to determine whether the item has passed/failed the test.

Special Environmental Needs

Any specific hardware and specific hardware configurations needed to execute this test case should be identified. Special software required to execute the test such as compilers, simulators, and test coverage tools should be described, as well as needed laboratory space and equipment.

Special Procedural Requirements

Describe any special conditions or constraints that apply to the test procedures associated with this test.

Intercase Dependencies

In this section the test planner should describe any relationships between this test case and others, and the nature of the relationship. The test case identifiers of all related tests should be given.

Test Procedure Specifications

A procedure in general is a sequence of steps required to carry out a specific task.

In this attachment to the test plan the planner specifies the steps required to execute a set of test cases. Another way of describing the test procedure specification is that it specifies the steps necessary to analyze a software item in order to evaluate a set of features. The test procedure specification has several subcomponents that the IEEE recommends being included in the order shown below. As noted previously, reference to documents where parts of these components are described must be provided.

Test Procedure Specification Identifier

Each test procedure specification should be assigned a unique identifier.

Purpose

Describe the purpose of this test procedure and reference any test cases it executes.

Specific Requirements

List any special requirements for this procedure, like software, hardware, and special training.

Procedure Steps

Here the actual steps of the procedure are described. Include methods, documents for recording (logging) results, and recording incidents. These will have associations with the test logs and test incident reports that result from a test run. A test incident report is only required when an unexpected output is observed. Steps include

(i) setup: to prepare for execution of the procedure;

(ii) start: to begin execution of the procedure;

(iii)proceed: to continue the execution of the procedure;

(iv)measure: to describe how test measurements related to outputs will be made;

(v)   shut down: to describe actions needed to suspend the test when unexpected events occur;

(vi)           restart: to describe restart points and actions needed to restart the procedure from these points;

(vii)stop: to describe actions needed to bring the procedure to an orderly halt;

(viii)      wrap up: to describe actions necessary to restore the environment;

(ix)           contingencies: plans for handling anomalous events if they occur during execution of this procedure.