FACULTY OF ENGINEERING SCIENCES AND TECHNOLOGY
CSC 431 – Parallel and Distributed Computing
Spring 2024
Instructor Dr. Lubna Aziz
Designation Assistant Professor
Department Computer Science
Office Location Faculty Cubicle # 2, 1st Floor
Email Lubna.aziz@iqra.edu.pk
Office Hours Friday 1115 to 1300 (for students’ consultation and tutoring)
Batch/Intake Fall 2019
Prerequisite(s) Data Communication and Computer Networks, Object oriented Programming
Credit Hours 3+0
Course Objective(s)
This Parallel and Distributed Computing course aims to equip students with a solid theoretical understanding of parallel and
distributed computing principles. This includes exploring various architectures, programming paradigms, algorithms, and
their analysis, along with addressing key challenges like concurrency control and performance optimization.
Course Contents
Asynchronous/synchronous computation/communication, concurrency control, fault tolerance, GPU architecture and
programming, heterogeneity, interconnection topologies, load balancing, memory consistency model, memory hierarchies,
Message passing interface (MPI), MIMD/SIMD, multithreaded programming, parallel algorithms & architectures, parallel
I/O, performance analysis and tuning, power, programming models (data parallel, task parallel, process-centric,
shared/distributed memory), scalability and performance studies, scheduling, storage systems, synchronization, and tools
(Cuda, Swift, Globus, Condor, Amazon AWS, OpenStack, Cilk, gdb, threads, MPICH, OpenMP, Hadoop, FUSE).
Sustainable Development Goals (SDGs)
A course in parallel and distributed computing equips students with foundational knowledge applicable to achieving multiple
Sustainable Development Goals.
SDG 7: Affordable and Clean Energy: The course covers techniques for optimizing programs for energy efficiency, which
directly contributes to this goal.
SDG 9: Industry, Innovation and Infrastructure: Understanding how to design scalable distributed systems prepares
students to develop efficient and sustainable infrastructure solutions.
SDG 12: Responsible Consumption and Production: The course emphasizes techniques that minimize resource waste in
FACULTY OF ENGINEERING SCIENCES AND TECHNOLOGY
distributed systems, aligning with responsible production practices.
Course Textbook / Reference Books and Supplementary Reading Material
S No Book Title Author(s)
1. Distributed and Parallel Computing Hesham ElRewini, Manning Greenwich
2. Introduction to Parallel Computing” W.P. Peterson, W.P. Peterson, P.Abenz , Oxford University
Press
3. Parallel Scientific Computing in C++ and MPI G.E. Karniadakis, Cambridge University Press
Grading Breakup and Policy
Assignments/ Projects/CEP 15%
Quizzes 15%
Mid Semester Examination 30%
End Semester Examination 40%
Course Learning Outcomes (CLOs)
CLO 1 Explain the suitability of parallel and distributed computing for specific problems.
CLO 2 Apply programming models like Open MP, MPI and CUDA-C/Python to develop portable parallel
programs.
CLO 3 Analyze parallel program efficiency using speedup and scalability metrics.
CLO/PLO Mapping and Assessment
EE-111 Related PLOs Level of Teaching CLO Attainment
CLOs Learning Methods checked in
CLO1 GA-1: Academic Education Instructions Quiz-1, Assignment
C2 (Presentation and Class
PLO-1: Engineering Knowledge activity, discussion), Exam
(understand)
(Mid & Final)
CLO2 GA-2: Knowledge for Solving C3 (Apply) Instructions Quiz-2, Assignment 2,
Computing Problems Exam (Mid & Final)
FACULTY OF ENGINEERING SCIENCES AND TECHNOLOGY
PLO-1: Engineering Knowledge
CLO3 GA-3: Problem Analysis Instructions Quiz 3, Mid, Exam
C4 (Analyze)
PLO-2: Problem Analysis
Weekly Course Plan
Week Topic Covered Remarks
Introduction to Parallel and Distributed Computing
History of computers, operating systems and sequential algorithms.
Review: Types of processors (RISC & CISC)
Flynn’s taxonomy
1 SISD, SIMD (vector and array processors), MIMD (shared and CLO1-PLO1
distributed memory), GPU
Design Goal of Distributed System, and Application (high
performance computing, information integration system and
pervasive system)
Concurrent systems Reading Assignment
Multitasking Systems, system API for multiprogramming
2
Multithreading Systems, system APIs for thread management
Multiprocessing Systems, shared memory architecture
Concurrency Control
Quiz
3 Mutual exclusion and synchronization
CLO1-PLO1
System APIs for concurrency control
Data Distribution Techniques
4 Inter process communications using PIPS/FIF/Shared Memory CLO2-PLO1
Network Sockets
Parallel Random-Access Machines
Overview of Parallel Random Access Machine, PRAM Models,
5 CLO2-PLO1
EREW PRAM Algorithms
Analysis of ERCW-PRAM, Algorithms
Parallel Random-Access Machines CLO2-PLO1
6 Analysis of CRCW-PRAM Algorithms
Analysis of CREW-PRAM Algorithms
FACULTY OF ENGINEERING SCIENCES AND TECHNOLOGY
7 Communication in Distribution system Quiz
Distributed Computing
Cluster Computing, GRID Computing, Overview of Available
8 tools,
Overview of Message Passing Systems MPI/MPICH &
Applications
Mid Term Examination Week
Message Passing Interface (MPI)
9 Six Basic APIs to implement distributed memory applications
APIs for group management and communications
Assignment
10 Reading Assignment / class activity
Message Passing Interface (MPI)
APIs for data distribution and collections
11 Quiz
Collective operations
Converting PRAM models into MPI Programs
General-Purpose Graphics Processor Architectures
12 GPU Architecture
Algorithmic design for GPU Programming
General-Purpose Graphics Processor Architectures
13
GPU Programming using CUDA-C/ CUDA-Python
Election Algorithm Assignment
14 Bully and Ring algorithm.
CLO 2
Gossip-based coordination: aggregation
Presentation, Assessment, and Class participation
15
Final Term Examination Week
