Designing integration tests

Designing integration tests

Integration tests for procedural software can be designed using a black or white box approach. Both are recommended. Some unit tests can be reused. Since many errors occur at module interfaces, test designers need to focus on exercising all input/output parameter pairs, and all calling relationships. The tester needs to insure the parameters are of the correct type and in the correct order. The author has had the personal experience of spending many hours trying to locate a fault that was due to an incorrect ordering of parameters in the calling routine. The tester must also insure that once the parameters are passed to a routine they are used correctly. For example, in Figure 6.9, Procedure_b is being integrated with Procedure_a. Procedure_a calls Procedure_b with two input parameters in3, in4. Procedure_b uses those parameters and then returns a value for the output parameter out1. Terms such as lhs and rhs could be any variable or expression. The reader should interpret the use of the variables in the broadest sense. The parameters could be involved in a number of def and/or use data flow patterns. The actual usage patterns of the parameters must be checked at integration time. Data flow-based (def-use paths) and control flow (branch coverage) test data generation methods are useful here to insure that the

input parameters, in3, in4, are used properly in Procedure_b. Again data flow methods (def-use pairs) could also be used to check that the proper sequence of data flow operations is being carried out to generate the correct value for out1 that flows back to Procedure_a.

Black box tests are useful in this example for checking the behavior of the pair of procedures. For this example test input values for the input parameters in1 and in2 should be provided, and the outcome in out2 should be examined for correctness. For conventional systems, input/output parameters and calling relationships will appear in a structure chart built during detailed design. Testers must insure that test cases are designed so that all modules in the structure chart are called at least once, and all called modules are called by every caller. The reader can visualize these as coverage criteria for integration test. Coverage requirements for the internal logic of each of the integrated units should be achieved during unit tests. Some black box tests used for module integration may be reusable from unit testing. However, when units are integrated and subsystems are to be tested as a whole, new tests will have to be designed to cover their functionality and adherence to performance and other requirements (see example above).

Sources for development of black box or functional tests at the integration level are the requirements documents and the user manual. Testers need to work with requirements analysts to insure that the requirements are testable, accurate, and complete. Black box tests should be developed to insure proper functionally and ability to handle subsystem stress. For example, in a transaction-based subsystem the testers want to determine the limits in number of transactions that can be handled. The tester also wants to observe subsystem actions when excessive amounts of transactions are generated. Performance issues such as the time requirements for a transaction should also be subjected to test. These will be repeated when the software is assembled as a whole and is undergoing system test.

Integration testing of clusters of classes also involves building test harnesses which in this case are special classes of objects built especially for testing. Whereas in class testing we evaluated intraclass method interactions, at the cluster level we test interclass method interaction as well. We want to insure that messages are being passed properly to interfacing objects, object state transitions are correct when specific events occur, and that the clusters are performing their required functions. Unlike procedural-oriented systems, integration for object-oriented systems usually does not occur one unit at a time. A group of cooperating classes is selected for test as a cluster. In their object-oriented testing framework the method is the entity selected for unit test. The methods and the classes they belong to are connected into clusters of classes that are represented by a directed graph that has two special types of entities.These are method-message paths, and atomic systems functions that represent input port events. A method-message path is described as a sequence of method executions linked by messages. An atomic system function is an input port event (start event) followed by a set of method messages paths and terminated by an output port event (system response).Murphy et al. define clusters as classes that are closely coupled and work together to provide a unified behavior [5]. Some examples of clusters are groups of classes that produce a report, or monitor and control a device.Scenarios of operation from the design document associated with a cluster are used to develop test cases. Murphy and his co-authors have developed a tool that can be used for class and cluster testing.