VALIDATION AND TESTING :
systems need testing to ensure that they work as they are intended. But bugs
can be subtle, particularly in embedded systems, where specialized hardware and
real-time responsiveness make programming more challenging. Fortunately, there
are many available techniques for software testing that can help us generate a
comprehensive set of tests to ensure that our system works properly.
question we must ask ourselves is how much testing is enough. Clearly, we
cannot test the program for every possible combination of inputs. Because we
cannot implement an infinite number of tests, we naturally ask ourselves what a
reasonable standard of thoroughness is. One of the major contributions of
software testing is to provide us with standards of thoroughness that make
sense. Following these standards does not guarantee that we will find all bugs.
major types of testing strategies:
generate tests without looking at the internal structure of the program.
known as white-box) methods generate tests based on the program
control/data flow graph extracted from a program’s source code is an important
tool in developing clear-box tests for the program. To adequately test the
program, we must exercise both its control and data operations.
to execute and evaluate these tests, we must be able to control variables in
the program and observe the results of computations, much as in manufacturing
testing. In general, we may need to modify the program to make it more
testable. By adding new inputs and outputs, we can usually substantially reduce
the effort required to find and execute the test.
what we are testing, we must accomplish the following three things in a test:
Provide the program with inputs that exercise the
test we are interested in.
Execute the program to perform the test.
Examine the outputs to determine whether the test
tests are generated without knowledge of the code being tested. When used
alone, black-box tests have a low probability of finding all the bugs in a
program. But when used in conjunction with clear-box tests they help provide a
well-rounded test set, since black-box tests are likely to uncover errors that
are unlikely to be found by tests extracted from the code structure.
tests can really work. For instance, when asked to test an instrument whose
front panel was run by a microcontroller, one acquaintance of the author used
his hand to depress all the buttons simultaneously. The front panel immediately
locked up. This situation could occur in practice if the instrument were placed
face-down on a table, but discovery of this bug would be very unlikely via clear-box
important technique is to take tests directly from the specification for the
code under design. The specification should state which outputs are expected
for certain inputs. Tests should be created that provide specified outputs and
evaluate whether the results also satisfy the inputs.
test every possible input combination, but some rules of thumb help us select
reasonable sets of inputs. When an input can range across a set of values, it
is a very good idea to test at the ends of the range. For example, if an input
must be between 1 and 10, 0, 1, 10, and 11 are all important values to test. We
should be sure to consider tests both within and outside the range, such as,
testing values within the range and outside the range. We may want to consider
tests well outside the valid range as well as boundary-condition tests.
tests form one category of black-box test. Random values are generated with a
distribution. The expected values are computed independently of the
system, and then the test inputs are applied. A large number of tests must be
applied for the results to be statistically significant, but the tests are easy
to generate. Another scenario is to test certain types of data values. For
example, integer valued inputs can be generated at interesting values such as
0, 1, and values near the maximum end of the data range. Illegal values can be
tested as well.
tests form an extremely important category of tests. When tests are created
during earlier stages in the system design or for previous versions
of the system, those tests should be saved to apply to the later versions of
the system. Clearly, unless the system specification changed, the new system
should be able to pass old tests. In some cases old bugs can creep back into
systems, such as when an old version of a software module is inadvertently
installed. In other cases regression tests simply exercise the code in
different ways than would be done for the current version of the code and
therefore possibly exercise different bugs.