Design and I/O System in Five Easy Pieces

Design and I/O System in Five Easy Pieces

The art of I/O system design is to find a design that meets goals for cost, dependability, and variety of devices while avoiding bottlenecks to I/O performance. Avoiding bottlenecks means that components must be balanced between main memory and the I/O device, because performance and hence effective cost/performance can only be as good as the weakest link in the I/O chain. Finally, storage must be dependable, adding new constraints on proposed designs.

In designing an I/O system, analyze performance, cost, capacity, and availability using varying I/O connection schemes and different numbers of I/O devices of each type. Here is one series of steps to follow in designing an I/O system. The answers for each step may be dictated by market requirements or simply by cost, performance, and availability goals.

1.     List the different types of I/O devices to be connected to the machine, or list the standard buses that the machine will support.

2.     List the physical requirements for each I/O device. Requirements include size, power, connectors, bus slots, expansion cabinets, and so on.

3.     List the cost of each I/O device, including the portion of cost of any controller needed for this device.

4.     List the reliability of each I/O device.

5.     Record the CPU resource demands of each I/O device.

This list should include

Ø Clock cycles for instructions used to initiate an I/O, to support operation of an I/O device (such as handling interrupts), and complete I/O

Ø CPU clock stalls due to waiting for I/O to finish using the memory, bus, or cache

Ø CPU clock cycles to recover from an I/O activity, such as a cache flush

List the memory and I/O bus resource demands of each I/O device. Even when the CPU is not using memory, the bandwidth of main memory and the I/O bus is limited.

The final step is assessing the performance and availability of the different ways to organize these I/O devices. Performance can only be properly evaluated with simulation, though it may be estimated using queuing theory. Reliability can be calculated assuming I/O devices fail independently and are that MTTFs are exponentially distributed. Availability can be computed from reliability by estimating MTTF for the devices, taking into account the time from failure to repair.

Cost/performance goals affect the selection of the I/O scheme and physical design. Performance can be measured either as megabytes per second or I/Os per second, depending on the needs of the application. For high performance, the only limits should be speed of I/O devices, number of I/O devices, and speed of memory and CPU. For low cost, the only expenses should be those for the I/O devices themselves and for cabling to the CPU. Cost/performance design, of course, tries for the best of both worlds. Availability goals depend in part on the cost of unavailability to an organization.

To make these ideas clearer, the next dozen pages go through five examples. Each looks at constructing a disk array with about 2 terabytes of capacity for user data with two sizes of disks. To offer a gentle introduction to I/O design and evaluation, the examples evolve in realism.

To try to avoid getting lost in the details, let’s start with an overview of the five examples:

Ø Naive cost-performance design and evaluation: The first example calculates cost-performance of an I/O system for the two types of disks. It ignores dependability concerns, and makes the simplifying assumption of allowing 100% utilization of I/O resources. This example is also the longest.

Ø Availability of the first example: The second example calculates the poor availability of this naive I/O design.

Ø Response times of the first example: The third example uses queuing theory to calculate the impact on response time of trying to use 100% of an I/O resource.

Ø More realistic cost-performance design and evaluation: Since the third example shows the folly of 100% utilization, the fourth example changes the design to obey common rules of thumb on utilization of I/O resources. It then evaluates cost-performance.

Ø More realistic design for availability and its evaluation: Since the second example shows the poor availability when dependability is ignored, this final example uses a RAID 5 design. It then calculates availability and performance.