Parallel and Distributed Learning
(CSC-418)
Lecture 7: OpenMP
Saima Sultana
Senior Lecturer
Department of Computer Science
College of Computer Science and Information
Systems
Saima.sultana@iobm.edu.pk
LECTURE
• Open MP
OPENMP
• OpenMP is an acronym for Open Multi-Processing.
• OpenMP is a directive-based Application
Programming Interface (API) for developing parallel
programs on shared memory architectures.
• OpenMP is a library for parallel programming on
shared-memory processors (SMP).
• It supports C, C++, and Fortran, and works through
high-level constructs to simplify parallel code.
• The program structure has sequential sections and
parallel sections managed by a master thread and
slave threads.
PARALLEL PROGRAMMING
WITH OPENMP
• OpenMP is an implementation of multithreading, a method of
parallelizing whereby a primary thread (a series of
instructions executed consecutively) forks a specified
number of sub-threads and the system divides a task among
them.
• A special #pragma omp parallel directive is used to define
parallel sections.
• Threads are spawned when a parallel section is reached, and
each thread executes independently.
• Threads are identified by IDs, with the master thread being
ID 0.
COMPILING OPENMP CODE
• OpenMP is compiled using the -fopenmp
flag.Compilers like gcc and g++ support
OpenMP, while the native macOS compiler
clang.
PRIVATE AND SHARED
VARIABLES
• Variables in a parallel section can be private
(unique to each thread) or shared (accessible
by all threads).
• Private variables are not initialized or
maintained outside the parallel region.
• Shared variables can lead to race conditions if
not handled carefully.
SYNCHRONIZATION IN OPENMP
• OpenMP provides several synchronization
mechanisms:
• critical: Code is executed by only one
thread at a time.
• atomic: Ensures atomic memory updates
for performance benefits.
• barrier: Forces threads to wait until all
threads have reached a specific point.
• nowait: Allows threads to proceed without
waiting for others.
PARALLELIZING LOOPS
• OpenMP can parallelize loops with the #pragma omp for
directive.
• The loop iterations are distributed across threads, which
can be statically or dynamically scheduled.
• Loop scheduling types:
• static: Iterations are equally divided among threads.
• dynamic: Iterations are assigned to threads as they finish their
tasks.
• guided: Iterations decrease in size exponentially for load balancing.
ADVANCED DIRECTIVES
• There are additional advanced directives like
sections for splitting tasks, ordered loops, and
critical or atomic sections for shared memory
updates.
PERFORMANCE
CONSIDERATION
• Using too many synchronization mechanisms
(like critical and atomic) can serialize code and
degrade performance.
• OpenMP works best when loops are easily
parallelizable, with no data dependencies
between iterations.
