PL/B: Programming for locality and large scale parallelism. George Alm á si Luiz A. DeRose José E. Moreira David A. Padua. Overview. Concepts Examples Thoughts about implementation Conclusions. A programming system for distributed-memory machines: Focus: numerical computing . - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Cannon’s Algorithm written down in PL/Bfunction [c] = cannon(a,b) % a, b are assumed to be distributed on an n*n grid.
% create an n*n distributed hta for matrix c. c{1:n,1:n} = zeros(p,p); % communication
% “parallelogram shift” rows of a, columns of b for i=2:n a{i:n,:} = cshift(a(i:n,:},dim=2,shift=1); % communication b{:,i:n} = cshift(b{:,i:n},dim=1,shift=1); % communication end
% main loop: parallel multiplications, column shift a, row shift b for k=1:n c{:,:} = c{:,:}+a{:,:}*b{:,:}; % computation a{:,:} = cshift(a{:,:},dim=2, shift=1); % communication b{:,:} = cshift(b{:,:},dim=1,shift=1); % communication endend
A “Distributed Array Virtual Machine” implemented on backed nodes Multiple types of memory (local, shared, co-arrays etc) Similar to UPC, OpenMP runtimes DAVM instruction set (bytecode?)
A MATLAB™ based frontend The MATLAB interpreter runs the show HTA code can be compiled into AVM code and distributed to backend A MATLAB “toolbox” contains the new data types Possible changes to MATLAB syntax: as few as we can get away
Q: is PL/B a toy language? A: it is as expressive as SPMD
Subsumes a large part of MPI:a{1} = b{2} is a message sent from rank 2 to 1.x = sum(a{:}) MPI_Reducex{:} = a MPI_Bcast
Many important algorithms can be formulated “better”
Q: Is PL/B still Matlab™ or a new beast?
PL/B defines a new data type and operators
MATLAB is a polymorphic language:New data type is compatible (drop-in replacement) with existing data typesNew data types bring new functionalityThink “toolbox” – Matlab users are familiar with the concept
Porting code to PL/B: Changes are going to be fairly localizedThe code will keep working during transition
Q: Debugging and profiling PL/B A: Debugging PL/B should not be
different from debugging a regular MATLAB program.
Q: Performance? A: PL/B has a better chance of
scaling than a regular MPI program Most communication primitives are high-level and are going to be optimized. Writing low-level communication code in PL/B is possible, but not a natural thing to do Implementation easy for most primitives (MPI)