CSEP524: Parallel Computation

Basics

Software

UW CSE Fedora 17 Home Virtual Machine image:

This is the officially supported operating system for assignments in this course. Though you're welcome to work using your native operating system, the burden of installing software, making sure it works correctly, and that your solutions work for the VM will be on you.

Pthreads: (included with the above)

OpenMP: (included with the above)

Chapel: chapel-fedora17-1.6.1.1.tar.gz

(This is a pre-release of the Chapel 1.6.1 sources, pre-compiled for the Fedora 17 VM; unpack and follow the directions in README, mentally substituting "1.6.1.1" for "1.6.1" where necessary and skipping step 2 if you're using the VM)

MPI: Setup MPI
This describes how to install MPI and how to run it locally and on our course VM cluster.

Widgets

• Class Discussion Board

• Dropbox

• Gradebook

• Anonymous Feedback Form

• Course Speed (red = too slow, green = too fast)

  Lectures	Readings	Assignments

Lectures

• Lecture #1 (Motivation, Definitions, Course Overview, Metrics, Embarassing Parallelism, starting/stopping Pthreads)

• Lecture #2 (Course Details, Introduction to Chapel)
  • Chapel Background
  • Chapel Basics (serial Chapel)
  • Chapel Compiler Schematic and directory overview
  • Chapel Task Parallelism

• Lecture #3 (Assignment #1 debrief, False Sharing, Computational Intensity, Load Balance, Block and Cyclic evaluation and alternatives, Multidimensional Distributions, Measuring Load Imbalance, RRWW bugs, Pthreads and Chapel Synchronization Primitives)

• Lecture #4 (Pthreads vs. Chapel comparison; Unbalanced Block Distributions; Deadlock, Livelock, and dealing with them; Memory Consistency Models; Data Races; Fences; Reductions) [audio (CSE NetID required)]
  • Lecture #4 Housekeeping

• Lecture #5 (Search and Eurekas; Task vs. Data Parallelism; Mapping Tasks to Threads; Reductions on Arrays; Scans; Barriers; Data Parallelism in Chapel; OpenMP; Atomic/Lock-Free Programming Concepts) [audio (CSE NetID required)]
  • Alex Duran's OpenMP Slides [abridged version from lecture | full version]

• Lecture #6 (Shared Memory wrap-up; ccNUMA architectures; distributed memory; networks; Flynn's taxonomy; SPMD; MPI) [audio (CSE NetID required)]
  • Rajeev Thakur's MPI Slides

• Lecture #7 (MPI wrap-up, Multigrid Method, Abstract Machine Models, Intro to PGAS, Chapel multi-locale computations and distributions, Smith-Waterman)
  • Chapel Locales
  • Chapel Domain Maps

• Lecture #8 (PGAS: UPC, CAF, Chapel; single-sided communication and active messages; Smith-Waterman algorithm)
  • Competition => Collaboration (a runtime-centric view of parallel computation), Simon Kahan (guest lecture)

• Lecture #9 (Amdahl's Law; Processor Technology Trends; Hierarchical Locales; Parallel Programming Model Evaluations; All-to-all communications; approaches to FFTs; STM; Wrap-up; Bonus slides: Finite Element Method; Partitioning Irregular workloads/Graph Partitioning; HPF, ZPL, and their impact on Chapel's design; User-definable Chapel features)
  • Fast Multipole Method: program flow and data structures