Why Parallel Computing?

Chapter 1

Why Parallel Computing?

From 1986 to 2002 the performance of microprocessors increased, on average, 50% per year. This unprecedented increase meant that users and software developers could often simply wait for the next generation of microprocessors in order to obtain increased performance from an application program. Since 2002, however, single-processor performance improvement has slowed to about 20% per year. This difference is dramatic: at 50% per year, performance will increase by almost a factor of 60 in 10 years, while at 20%, it will only increase by about a factor of 6.

Furthermore, this difference in performance increase has been associated with a dramatic change in processor design. By 2005, most of the major manufacturers of microprocessors had decided that the road to rapidly increasing performance lay in the direction of parallelism. Rather than trying to continue to develop ever-faster monolithic processors, manufacturers started putting multiple complete processors on a single integrated circuit.

This change has a very important consequence for software developers: simply adding more processors will not magically improve the performance of the vast majority of serial programs, that is, programs that were written to run on a single processor. Such programs are unaware of the existence of multiple processors, and the performance of such a program on a system with multiple processors will be effectively the same as its performance on a single processor of the multiprocessor system.

All of this raises a number of questions:

       Why do we care? Aren’t single processor systems fast enough? After all, 20% per year is still a pretty significant performance improvement.

       Why can’t microprocessor manufacturers continue to develop much faster sin-gle processor systems? Why build parallel systems? Why build systems with multiple processors?

       Why can’t we write programs that will automatically convert serial programs into parallel programs, that is, programs that take advantage of the presence of multiple processors?

Let’s take a brief look at each of these questions. Keep in mind, though, that some of the answers aren’t carved in stone. For example, 20% per year may be more than adequate for many applications.