Test Case Design Strategies

Test Case Design Strategies

A smart tester who wants to maximize use of time and resources knows that she needs to develop what we will call effective test cases for execution-based testing. By an effective test case we mean one that has a good possibility of revealing a defect (see Principle 2 in Chapter 2). The ability to develop effective test cases is important to an organization evolving toward a higher-quality testing process. It has many positive consequences. For example, if test cases are effective there is (i) a greater probability of detecting defects, (ii) a more efficient use of organizational resources, (iii) a higher probability for test reuse, (iv) closer adherence to testing and project schedules and budgets, and, (v) the possibility for delivery of a higher -quality software product. What are the approaches a tester should use to design effective test cases? To answer the question we must adopt the view that software is an engineered product. Given this view there are two basic strategies that can be used to design test cases. These are called the black box (sometimes called functional or specification) and white box (sometimes called clear or glassbox) test strategies. The approaches are summarized in Figure

Using the black box approach, a tester considers the software-under test to be an opaque box. There is no knowledge of its inner structure (i.e., how it works). The tester only has knowledge of what it does. The size of the software-under-test using this approach can vary from a simple module, member function, or object cluster to a subsystem or a complete Software system. The description of behavior or functionality for the software-under-test may come from a formal specification, an Input/Process/Output Diagram (IPO), or a well-defined set of pre and post conditions. Another source for information is a requirements specification document that usually describes the functionality of the software-under-test and its inputs and expected outputs. The tester provides the specified inputs to the software-under-test, runs the test and then determines if the outputs produced are equivalent to those in the specification. Because the black box approach only considers software behavior and functionality, it is often called functional or specification-based testing. This approach is especially useful for revealing requirements and specification defects.

The white box approach focuses on the inner structure of the software to be tested. To design test cases using this strategy the tester must have knowledge of that structure. The code, or a suitable pseudo codelike representation must be available. The tester selects test cases to exercise specific internal structural elements to determine if they are working properly. For example, test cases are often designed to exercise all statements or true/false branches that occur in a module or member function. Since designing, executing, and analyzing the results of white

box testing is very time consuming, this strategy is usually applied to smaller-sized pieces of software such as a module or member function. The reasons for the size restriction will become more apparent in Chapter 5 where the white box strategy is described in more detail. White box testing methods are especially useful for revealing design and code-based control, logic and sequence defects, initialization defects, and data flow defects.

The smart tester knows that to achieve the goal of providing users with low-defect, high-quality software, both of these strategies should be used to design test cases. Both support the tester with the task of selecting the finite number of test cases that will be applied during test. Neither approach by itself is guaranteed to reveal all defects types we have studied in Chapter 3. The approaches complement each other; each may be useful for revealing certain types of defects. With a suite of test cases designed using both strategies the tester increases the chances of revealing the many different type of defects in the software under test. The tester will also have an effective set of reusable test cases for regression testing (re-test after changes), and for testing new releases of the software.

There is a great deal of material to introduce to the reader relating to both of these strategies. To facilitate the learning process, the material has been partitioned into two chapters. This chapter focuses on black box methods, and Chapter 5 will describe white box methods and how to apply them to design test cases.