CSEP524: Parallel Computation

Course Description

With the advent of multicore processors and accelerators, parallel programming has moved out of supercomputer centers and onto the desktop. Though parallel programming was once required only by scientists and programmers at the high-end, we are now firmly in an era where every performance-minded programmer has the potential to benefit from parallel computing. Issues of locality and scalability, once a concern only on large-scale systems, are now issues of mainstream concern as our processor architectures increasingly resemble supercomputers-on-a-chip.

By way of introduction, I've spent the past two decades working in parallel computing, primarily within the 'supercomputing' or high-performance computing (HPC) community -- first as a graduate student at UW and currently as an R&D engineer at Cray Inc. here in Seattle.

In this course, I hope to pass along some of the knowledge that I've picked up over the years, focusing primarily on topics related to parallel programming, which is my main interest: parallel languages and models, algorithms, performance and scalability, hazards, and so forth.

As the first paragraph above argues, regardless of whether you're an engineer working on large-scale numerical simulations at Boeing, a desktop software engineer, or somewhere in-between (e.g., cloud or datacenter computing), due to architectural trends, there are lessons to be learned from the high-end that will be increasingly applicable to your life as an engineer today and going forward.

I plan to teach this course through a combination of principles and practice. The principles will be exercised through thought and paper & pencil exercises. The practice will be done through programming assignments using programming models such as OpenMP, MPI, UPC, CUDA/OpenCL/OpenACC, and Chapel (an emerging parallel language that I work on at Cray). We'll be using Lin and Snyder's Principles of Parallel Programming as our primary text, though I we'll also be using other supplementary papers and materials as we go.

I've included a list of likely topics we'll cover just below. I'm also very open to covering other topics of interest by request.