Scaling to Multiple Consumers or Producers

Scaling to Multiple Consumers or Producers

The code in Listing 8.21 works for a single producer and consumer without synchronization because one thread is responsible for adding items to the circular buffer and one thread is responsible for removing items. If the code were to be scaled to multiple con-sumers or producers, this would no longer be true, and in general, the code would require some kind of locking.

There are some cases where the use of synchronization could be avoided. If the code scales to either multiple consumers or multiple producers, but not both, then the number of circular buffers could also be scaled. This would maintain the one-to-one relationship between producers and circular buffers and between circular buffers and consumers. If there were multiple producers and multiple consumers, then synchronization could be avoided if each of these were paired. So, one producer can feed only a single consumer, and that consumer can take work from only a single producer. Finally, if there was a sin-gle circular buffer for every producer-consumer pair, then synchronization could again be avoided.

All of the previous scenarios reflect different trade-offs, memory footprints, and run-time behaviors. Assuming that the amount of work does indeed need to scale, it is most likely that some kind of “many producer to many consumer” mapping will be necessary. It would be possible to code an NxM system of queues that would provide a circular buffer for every producer-consumer pair, but this approach would likely be less efficient than using some amount of either atomic operations or mutex locks.