Covering Code Logic

Covering Code Logic

Logic-based white box-based test design and use of test data adequacy/ coverage concepts provide two major payoffs for the tester: 

(i) quantitative coverage goals can be proposed, and 

(ii) commercial tool support is readily available to facilitate the tester‘s work .

As described in Section 5.1, testers can use these concepts and tools to decide on the target logic elements (properties or features of the code) and the degree of coverage that makes sense in terms of the type of software, its mission or safety criticalness, and time and resources available. For ex ample, if the tester selects the logic element program statements, this indicates that she will want to design tests that focus on the execution of program statements. If the goal is to satisfy the statement adequacy/ coverage criterion, then the tester should develop a set of test cases so that when the module is executed, all (100%) of the statements in the module are executed at least once. In terms of a flow graph model of the code, satisfying this criterion requires that all the nodes in the graph are exercised at least once by the test cases. For the code in Figure 5.2 and its corresponding flow graph in Figure 5.3 a tester would have to develop test cases that exercise nodes 1-8 in the flow graph. If the tests achieve this goal, the test data would satisfy the statement adequacy criterion. In addition to statements, the other logic structures are also associated with corresponding adequacy/coverage criteria. For example, to achieve complete (100%) decision (branch) coverage test cases must be designed

/* pos_sum nds the sum of all positive numbers (greater than zero) stored in an integer

array a. Input parameters are num_of_entries, an integer, and a, an array of integers with num_of_entries elements. The output parameter is the integer sume */

1. pos_sum(a, num_of_entries, sum)

2. sum 0

3. inti 1

4. while (i < num_of_entries)

5. if a[i] > 0

6.   sum sum a[i]

endif

7. i = i + 1

end while

8. end pos_sum

FIG. 5.2 Code sample with branch and loop.

so that each decision element in the code (if-then, case, loop) executes with all possible outcomes at least once. In terms of the control flow model, this requires that all the edges in the corresponding flow graph must be exercised at least once. Complete decision coverage is considered to be a stronger coverage goal than statement coverage since its satisfaction results in satisfying statement coverage as well (covering all the edges in a flow graph will ensure coverage of the nodes). In fact, the statement coverage goal is so weak that it is not considered to be very useful for revealing defects. For example, if the defect is a missing statement it may remain undetected by tests satisfying complete statement coverage. The reader should be aware that in spite of the weakness, even this minimal coverage goal is not required in many test plans. Decision (branch) coverage for the code example in Figure 5.2, requires test cases to be developed for the two decision statements, that is, the four true/false edges in the control flow graph of Figure 5.3. Input values must ensure execution the true/false possibilities for the decisions in line 4 (while loop) and line 5 (if statement). Note that the if statement has a full else component, that is, there is no else part. However, we include a test that covers both the true and false conditions for the statement.

A possible test case that satisfies 100% decision coverage is shown in Table 5.1. The reader should note that the test satisfies both the branch adequacy criterion and the statement adequacy criterion, since all the statements 1-8 would be executed by this test case. Also note that for this code example, as well as any other code component, there may be several sets of test cases that could satisfy a selected criterion. This code example represents a special case in that it was feasible to achieve both branch and statement coverage with one test case. Since one of the inputs, , is an array, it was possible to assign both positive and negative values to the elements of , thus allowing coverage of both the true/false branches of the if statement. Since more than one iteration of the while loop was also possible, both the true and false branches of this loop could also be covered by one test case. Finally, note that the code in the example does not contain any checks on the validity of the input parameters. For simplicity it is assumed that the calling module does the checking.

In Figure 5.2 we have simple predicates or conditions in the branch and loop instructions. However, decision statements may contain multiple conditions, for example, the statement

If (x MIN and y MAX and (not INT Z))

has three conditions in its predicate: (i) x MIN, (ii) y MAX, and (iii) not INT Z. Decision coverage only requires that we exercise at least once all the possible outcomes for the branch or loop predicates as a whole, not for each individual condition contained in a compound predicate. There are other coverage criteria requiring at least one execution of the all possible conditions and combinations of decisions/conditions. The names of the criteria reflect the extent of condition/decision coverage. For example, condition coverage requires that the tester insure that each individual condition in a compound predicate takes on all possible values at least once during execution of the test cases. More stringent coverage criteria also require exercising all possible combinations of decisions and conditions in the code. All of the coverage criterion described so far can be arranged in a hierarchy of strengths from weakest to strongest as follows: statement, decision, decision/condition. The implication for this approach to test design is that the stronger the criterion, the more defects will be revealed by the tests. Below is a simple example showing the test cases for a decision statement with a compound predicate.

if(age <65 and married true) do X

do Y .........

else

do Z

The criteria described above do not require the coverage of all the possible combinations of conditions. This is represented by yet another criterion called multiple condition coverage where all possible combinations of condition outcomes in each decision must occur at least once when the test cases are executed. That means the tester needs to satisfy the following combinations for the example decision statement:

Condition 1         Condition 2

True                     True

True                     False

False                    True

False                    False

In most cases the stronger the coverage criterion, the larger the number of test cases that must be developed to insure complete coverage. For code with multiple decisions and conditions the complexity of test case design increases with the strength of the coverage criterion. The tester must decide, based on the type of code, reliability requirements, and resources available which criterion to select, since the stronger the criterion selected the more resources are usually required to satisfy it.