19
Moving Data In Parallel Computation
Two
views of data motion in parallel computation
–It
should be transparent -- shared memory
•Data
movement is complex … simplify by eliminating it
•Analogous mechanisms (VM, paging, caching) have proved their
worth and show that amortizing costs works
–It is the programmer’s responsibility to move data to wherever it is
needed -- message passing
•Data movement is complex ... rely on programmer to do it well
•Message
passing is universal -- it works on any machine while shared memory needs special
hardware
Many furious battles have taken place
over this issue … at the
moment message passing is the state-of-the-art