An Introduction to Parallel Programming

Chapter 1 : Why Parallel Computing

Why Parallel Computing?
Why We Need Ever-Increasing Performance
Why We�re Building Parallel Systems
Why we Need to Write Parallel Programs
How Do We Write Parallel Programs?
Concurrent, Parallel, Distributed

Chapter 2 : Parallel Hardware and Parallel Software

Parallel Hardware and Parallel Software
Some Background: von Neumann architecture, Processes, multitasking, and threads
Modifications to the Von Neumann Model
Parallel Hardware
Parallel Software
Input and Output
Performance of Parallel Programming
Parallel Program Design with example
Writing and Running Parallel Programs
Assumptions - Parallel Programming

Chapter 3 : Distributed Memory Programming with MPI

Distributed-Memory Programming with MPI
The Trapezoidal Rule in MPI
Dealing with I/O
Collective Communication
MPI Derived Datatypes
Performance Evaluation of MPI Programs
A Parallel Sorting Algorithm

Chapter 4 : Shared Memory Programming with Pthreads

Shared-Memory Programming with Pthreads
Processes, Threads, and Pthreads
Pthreads - Hello, World Program
Matrix-Vector Multiplication
Critical Sections
Busy-Waiting
Mutexes
Producer-Consumer Synchronization and Semaphores
Barriers and Condition Variables
Read-Write Locks
Caches, Cache Coherence, and False Sharing
Thread-Safety
Shared-Memory Programming with OpenMP
The Trapezoidal Rule
Scope of Variables
The Reduction Clause
The parallel For Directive
More About Loops in Openmp: Sorting
Scheduling Loops
Producers and Consumers
Caches, Cache Coherence, and False Sharing
Thread-Safety
Parallel Program Development
Two n-Body Solvers
Parallelizing the basic solver using OpenMP
Parallelizing the reduced solver using OpenMP
Evaluating the OpenMP codes
Parallelizing the solvers using pthreads
Parallelizing the basic solver using MPI
Parallelizing the reduced solver using MPI
Performance of the MPI solvers
Tree Search
Recursive depth-first search
Nonrecursive depth-first search
Data structures for the serial implementations
Performance of the serial implementations
Parallelizing tree search
A static parallelization of tree search using pthreads
A dynamic parallelization of tree search using pthreads
Evaluating the Pthreads tree-search programs
Parallelizing the tree-search programs using OpenMP
Performance of the OpenMP implementations
Implementation of tree search using MPI and static partitioning
Implementation of tree search using MPI and dynamic partitioning
Which API?

Important questions and answers, Question Paper download, Online Study Material, Lecturing Notes, Assignment, Reference, Wiki