Characteristics of Multiprocessor Systems
processors with multiple cores are now prevalent, it is also becoming more
common to encounter systems with multiple processors. As soon as there are
multiple processors in a system, accessing memory becomes more complex. Not
only can data be held in memory, but it can also be held in the caches of one
of the other processors. For code to execute correctly, there should be only a
single up-to-date version of each item of data; this feature is called cache coherence.
common approach to providing cache coherence is called snooping. Each proces-sor broadcasts the address that it wants to
either read or write. The other processors watch for these broadcasts. When
they see that the address of data they hold can take one of two actions, they
can return the data if the other processor wants to read the data and they have
the most recent copy. If the other processor wants to store a new value for the
data, they can invalidate their copy.
this is not the only issue that appears when dealing with multiple proces-sors.
Other concerns are memory layout and latency.
system with two processors. The system could be configured with all the memory
attached to one processor or the memory evenly shared between the two
processors. Figure 1.18 shows these two alternatives.
between processor and memory will increase the latency of any memory access by
that processor. So if only one processor has memory attached, then that
proces-sor will see low memory latency, and the other processor will see higher
memory latency. In the situation where both processors have memory attached,
then they will have both local memory that is low cost to access and remote
memory that is higher cost to access.
systems where memory is attached to multiple processors, there are two options
for reducing the performance impact. One approach is to interleave memory,
often at a cache line boundary, so that for most applications, half the memory
accesses will see the short memory access, and half will see the long memory
access; so, on average, applica-tions will record memory latency that is the
average of the two extremes. This approach typifies what is known as a uniform memory architecture (UMA), where
all the processors see the same memory latency.
approach is to accept that different regions of memory will have different
access costs for the processors in a system and then to make the operating
system aware of this hardware characteristic. With operating system support,
this can lead to applica-tions usually seeing the lower memory cost of
accessing local memory. A system with this architecture is often referred to as
having cache coherent nonuniform memory
operating system to manage ccNUMA memory characteristics effectively, it has to
do a number of things. First, it needs to be aware of the locality structure of
the system so that for each processor it is able to allocate memory with low
access latencies. The second challenge is that once a process has been run on a
particular processor, the operating system needs to keep scheduling that
process to that processor. If the operating system fails to achieve this second
requirement, then all the locally allocated memory will become remote memory
when the process gets migrated.
an application running on the first processor of a two-processor system. The
operating system may have allocated memory to be local to this first processor.
Figure 1.19 shows this configuration of an application running on a processor
and using local memory. The shading in this figure illustrates the application
running on processor 1 and accessing memory directly attached to that
processor. Hence, the process sees local memory latency for all memory
application will get good performance because the data that it frequently
accesses will be held in the memory with the lowest access latency. However, if
that application then gets migrated to the second processor, the application
will be accessing data that is remotely held and will see a corresponding drop
in performance. Figure 1.20 shows an application using remote memory. The
shading in the figure shows the application run-ning on processor 2 but
accessing data held on memory that is attached to processor 1. Hence, all
memory accesses for the application will fetch remote data; the fetches of data
will take longer to complete, and the application will run more slowly.