Testing
Testing is a process activity
that homes in on product quality: making the product failure free or failure
tolerant. Each software problem (especially when it relates to security) has
the potential not only for making software fail but also for adversely
affecting a business or a life. Thomas Young, head of NASA's investigation of
the Mars lander failure, noted that "One of the things we kept in mind
during the course of our review is that in the conduct of space missions, you
get only one strike, not three. Even if thousands of functions are carried out
flawlessly, just one mistake can be catastrophic to a mission" [NAS00]. This same sentiment is true for
security: The failure of one control exposes a vulnerability that is not
ameliorated by any number of functioning controls. Testers improve software
quality by finding as many faults as possible and by writing up their findings
carefully so that developers can locate the causes and repair the problems if
possible.
Do not ignore a point from
Thompson [THO03]: Security testing is
hard. Side effects, dependencies, unpredictable users, and flawed implementation
bases (languages, compilers, infrastructure) all contribute to this difficulty.
But the essential complication with security testing is that we cannot look at
just the one behavior the program gets right; we also have to look for the
hundreds of ways the program might go wrong.
Testing usually involves
several stages. First, each program component is tested on its own, isolated
from the other components in the system. Such testing, known as module testing,
component testing, or unit testing, verifies that the component functions
properly with the types of input expected from a study of the component's
design. Unit testing is done in a
controlled environment whenever possible so that the test team can feed a
predetermined set of data to the component being tested and observe what output
actions and data are produced. In addition, the test team checks the internal
data structures, logic, and boundary conditions for the input and output data.
When collections of
components have been subjected to unit testing, the next step is ensuring that
the interfaces among the components are defined and handled properly. Indeed,
interface mismatch can be a significant security vulnerability. Integration testing is the process of
verifying that the system components work together as described in the system
and program design specifications.
Once we are sure that
information is passed among components in accordance with the design, we test
the system to ensure that it has the desired functionality. A function test
evaluates the system to determine whether the functions described by the
requirements specification are actually performed by the integrated system. The
result is a functioning system.
The function test compares the system being
built with the functions described in the developers' requirements
specification. Then, a performance test compares
the system with the remainder of these software and hardware requirements. It
is during the function and performance
tests that security requirements are examined, and the testers confirm that the
system is as secure as it is required to be.
When the performance test is
complete, developers are certain that the system functions according to their
understanding of the system description. The next step is conferring with the
customer to make certain that the system works according to customer
expectations. Developers join the customer to perform an acceptance test, in
which the system is checked against the customer's requirements description.
Upon completion of acceptance testing, the accepted system is installed in the
environment in which it will be used. A final installation test is run to make
sure that the system still functions as it should. However, security
requirements often state that a system should not do something. As Sidebar 3-7 demonstrates, it is difficult to
demonstrate absence rather than presence.
The objective of unit and
integration testing is to ensure that the code implemented the design properly;
that is, that the programmers have written code to do what the designers
intended. System testing has a very different objective: to ensure that the
system does what the customer wants it to do. Regression testing, an aspect of
system testing, is particularly important for security purposes. After a change
is made to enhance the system or fix a problem, regression testing ensures that all remaining functions are still
working and that performance has not been degraded by the change.
Each of the types of tests listed here can be
performed from two perspectives: black box and clear box (sometimes called
white box). Black-box testing treats
a system or its components as black boxes; testers cannot "see
inside" the system, so they apply particular inputs and verify that they get the expected output. Clear-box testing allows visibility. Here, testers can examine the
design and code directly, generating test cases based on the code's actual
construction. Thus, clear-box testing knows that component X uses CASE
statements and can look for instances in which the input causes control to drop
through to an unexpected line. Black-box testing must rely more on the required
inputs and outputs because the actual code is not available for scrutiny.
Sidebar
3-7: Absence vs. Presence
Pfleeger [PFL97] points out that
security requirements resemble those for any other computing task, with one
seemingly insignificant difference. Whereas most requirements say "the
system will do this," security requirements add the phrase "and
nothing more." As we pointed out in Chapter 1, security awareness
calls for more than a little caution when a creative developer takes liberties
with the system's specification. Ordinarily, we do not worry if a programmer or
designer adds a little something extra. For instance, if the requirement calls
for generating a file list on a disk, the "something more" might be
sorting the list in alphabetical order or displaying the date it was created.
But we would never expect someone to meet the requirement by displaying the
list and then erasing all the files on the disk!
If we could determine easily whether an
addition was harmful, we could just disallow harmful additions. But
unfortunately we cannot. For security reasons, we must state explicitly the
phrase "and nothing more" and leave room for negotiation in the
requirements definition on any proposed extensions.
It is natural for programmers to want to
exercise their creativity in extending and expanding the requirements. But
apparently benign choices, such as storing a value in a global variable or
writing to a temporary file, can have serious security implications. And
sometimes the best design approach for security is counterintuitive. For
example, one cryptosystem attack depends on measuring the time to perform an
encryption. That is, an efficient implementation can undermine the system's
security. The solution, oddly enough, is to artificially pad the encryption
process with unnecessary computation so that short computations complete as
slowly as long ones.
In another instance, an enthusiastic
programmer added parity checking to a cryptographic procedure. Because the keys
were generated randomly, the result was that 255 of the 256 encryptions failed
the parity check, leading to the substitution of a fixed keyso that 255 of
every 256 encryptions were being performed under the same key!
No
technology can automatically distinguish between malicious and benign code. For
this reason, we have to rely on a combination of approaches, including
human-intensive ones, to help us detect when we are going beyond the scope of
the requirements and threatening the system's security.
The mix of techniques
appropriate for testing a given system depends on the system's size,
application domain, amount of risk, and many
other factors. But understanding the effectiveness of each technique
helps us know what is right for each particular system. For example, Olsen [OLS93] describes the development at Contel IPC
of a system containing 184,000 lines of code. He tracked faults discovered
during various activities, and found differences:
17.3 percent of the faults
were found during inspections of the system design
19.1 percent during component
design inspection
15.1 percent during code
inspection
29.4 percent during
integration testing
16.6 percent during system
and regression testing
Only 0.1 percent of the
faults were revealed after the system was placed in the field. Thus, Olsen's
work shows the importance of using different techniques to uncover different
kinds of faults during development; it is not enough to rely on a single method
for catching all problems.
Who does the testing? From a
security standpoint, independent testing
is highly desirable; it may prevent a developer from attempting to hide
something in a routine or keep a subsystem from controlling the tests that will
be applied to it. Thus, independent testing increases the likelihood that a
test will expose the effect of a hidden feature.
One type of testing is unique
to computer security: penetration
testing. In this form of testing, testers specifically try to make software
fail. That is, instead of testing to see that software does do what it is
expected to (as is the goal in the other types of testing we just listed), the
testers try to see if the software does what it is not supposed to do, which is
to fail or, more specifically, fail to enforce security. Because penetration
testing usually applies to full systems, not individual applications, we study
penetration testing in Chapter 5.
Related Topics
Privacy Policy, Terms and Conditions, DMCA Policy and Compliant
Copyright © 2018-2024 BrainKart.com; All Rights Reserved. Developed by Therithal info, Chennai.