CS 584 Lecture 14 n Assignment –Program due now –Check the web »Create program model & do calculations »Additional calculation problems n Paper presentations.

CS 584 Lecture 14CS 584 Lecture 14

AssignmentAssignment– Program due nowProgram due now– Check the webCheck the web

» Create program model & do calculationsCreate program model & do calculations

» Additional calculation problemsAdditional calculation problems

Paper presentations on Friday by:Paper presentations on Friday by:– Matt Probst Matt Probst – Glenn JuddGlenn Judd

High Level Parallel High Level Parallel ProgrammingProgramming

Message Passing is considered to be low-Message Passing is considered to be low-level parallel programming.level parallel programming.

Why not have a high level parallel Why not have a high level parallel programming language?programming language?– Easier?Easier?– More efficient?More efficient?– Same performance?Same performance?

High Level Parallel High Level Parallel ProgrammingProgramming

Many high level languages introducedMany high level languages introduced– SR (University of Arizona)SR (University of Arizona)– HPFHPF– C*C*– othersothers

A high level language compiles to message A high level language compiles to message passing code which is then compiled to passing code which is then compiled to machine code to be executed.machine code to be executed.

Data Parallel ProgrammingData Parallel Programming

Parallel & scalar dataParallel & scalar data Based on operations being performed on Based on operations being performed on

each data element of a parallel variable.each data element of a parallel variable. A statement executed on parallel data is A statement executed on parallel data is

performed on each element individuallyperformed on each element individually Functions are provided to include the Functions are provided to include the

element index in the operationelement index in the operation

C*C*

Data parallel languageData parallel language Based on standard C with extensionsBased on standard C with extensions Produce code for both SIMD and MIMDProduce code for both SIMD and MIMD Accept any standard C program and Accept any standard C program and

compile it correctly.compile it correctly.

C*C*

Presents a Presents a global viewglobal view abstraction of the abstraction of the parallel machine.parallel machine.

Data can be Data can be parallelparallel or or scalarscalar– scalar is defaultscalar is default

Parallel data is acted upon by new or Parallel data is acted upon by new or overloaded operators and statementsoverloaded operators and statements

Parallel variables are seen and operated on as Parallel variables are seen and operated on as monolithic vectors or arrays.monolithic vectors or arrays.

C* Additions to CC* Additions to C

Reserved wordsReserved words– bool, dimof, everywhere, overload, pcoord, bool, dimof, everywhere, overload, pcoord,

shape, where, with, and othersshape, where, with, and others New operatorsNew operators

– <?, <?=, >?, >?=, %%<?, <?=, >?, >?=, %%– [] index operator is used as a unary prefix[] index operator is used as a unary prefix

Overloaded OperatorsOverloaded Operators– reductions, etc.reductions, etc.

Parallel VariablesParallel Variables

A A shapeshape is an indication of an arrangement is an indication of an arrangement of a parallel variableof a parallel variable

Parallel variables are declared in 2 StepsParallel variables are declared in 2 Steps– Declare the shapeDeclare the shape– Declare a variable that is based on the shapeDeclare a variable that is based on the shape

The compiler automatically distributes the The compiler automatically distributes the variable across the architecture.variable across the architecture.

Declaring Parallel VariablesDeclaring Parallel Variables

Shape DeclarationShape Declaration

– Gives the compiler clues on partitioningGives the compiler clues on partitioning– Uses left indexing followed by shape nameUses left indexing followed by shape name– shape [10]Sb, [50][30]Scshape [10]Sb, [50][30]Sc

Variable declarationVariable declaration– Declare type and shape of each variableDeclare type and shape of each variable– int ai1:Sb, ai2:Scint ai1:Sb, ai2:Sc

Parallel VariablesParallel Variables

Shapes can be used for all variable types Shapes can be used for all variable types including structures and unions.including structures and unions.

Shapes can be dynamically created just the Shapes can be dynamically created just the same as arrays are created.same as arrays are created.– Single dimension onlySingle dimension only– Fake multiple dimensionsFake multiple dimensions

Using Parallel VariablesUsing Parallel Variables

Addressed just like arrays only backwardsAddressed just like arrays only backwards– index firstindex first– [5][6]ai2 = 23;[5][6]ai2 = 23;– for (I = 0; I < 10; I++)for (I = 0; I < 10; I++)

[I]c = [I]a + [I]b;[I]c = [I]a + [I]b;

Parallel Variable OperationsParallel Variable Operations

Parallel to scalar reductionParallel to scalar reduction– scalar op parallel_variablescalar op parallel_variable– x += ai1;x += ai1;– OperatorsOperators

Result is combined with scalar value!!Result is combined with scalar value!!

+= sum &= bitwise AND <?= min-= neg. sum ^= bitwise XOR >?= max*= product |= bitwise OR/= 1/product

ContextualizationContextualization

Sets up a boolean mask to prevent an Sets up a boolean mask to prevent an operation on certain elements of a shapeoperation on certain elements of a shape

Where statementWhere statement– where (where (where-expressionwhere-expression) )

then-bodythen-body– where (where (where-expressionwhere-expression) )

then-bodythen-body else else else-bodyelse-body

ContextualizationContextualization

The The where-expressionwhere-expression must be based on the must be based on the shape that will be operated on.shape that will be operated on.

ExampleExample– where (b >= 3)where (b >= 3)

c = b + ac = b + a Assignment only occurs in the Assignment only occurs in the activeactive

positions.positions.

The with statementThe with statement

The with statement is used for selecting the The with statement is used for selecting the current shape to operate on.current shape to operate on.

The with-statement must be used to select a The with-statement must be used to select a current shape before any parallel code may current shape before any parallel code may be executedbe executed

with (with (shape-expressionshape-expression))shape-bodyshape-body

pcoordpcoord

Works on the current shapeWorks on the current shape Returns the element numberReturns the element number You supply which dimension you are You supply which dimension you are

interested in.interested in. int:current pcoord(int dim)int:current pcoord(int dim)

– dim == 0 returns the rowdim == 0 returns the row– dim == 1 returns the columndim == 1 returns the column

#include <stdio.h>#define SIZE 1000

shape [SIZE]span;

main(){double sum;double width;

with(span) {double x:span;x = (pcoord (0) + 0.5) * 1.0/SIZE;sum = (+= (4.0/(1 + x*x)));

}sum *= 1.0/SIZE;printf("Estimation is %lf\n", sum);

}

LindaLinda

Consists of several operations that work on Consists of several operations that work on a global data space (tuple space)a global data space (tuple space)

The operations have been added to several The operations have been added to several languages.languages.

MIMD programming modelMIMD programming model– Interaction is through tuple spaceInteraction is through tuple space

TuplesTuples

A A tupletuple is an object consisting of: is an object consisting of:– a keya key– zero or more argumentszero or more arguments

» Example ("jim", 88, 1.5)Example ("jim", 88, 1.5)

The key is used for matchingThe key is used for matching

Tuple SpaceTuple Space

Global data spaceGlobal data space Collection of tuplesCollection of tuples Tuples may be:Tuples may be:

– inserted inserted (out)(out)– read read (rd and rdp)(rd and rdp)– deleteddeleted (in and inp)(in and inp)– evaluatedevaluated (eval)(eval)

» forks a new worker forks a new worker

Tuple SpaceTuple Space

Tuple SpaceTuple Space

Updating a tuple Updating a tuple – delete - modify - insertdelete - modify - insert

Duplicate key entries is allowedDuplicate key entries is allowed– Non-determinismNon-determinism

inp and rdp guarantee to locate a matching inp and rdp guarantee to locate a matching tuple iff a matching tuple must have been tuple iff a matching tuple must have been added and could not have been removed added and could not have been removed before the requestbefore the request

Example ProgramsExample Programs

Database searchDatabase search Master-WorkerMaster-Worker Divide and Conquer Divide and Conquer

procedure manager count = 0 until EOF do

read datum from fileOUT("datum", datum)count++

enddo best = 0.0 for j = 1 to count

IN("score", value)if (value > best)

best = value endfor for j = 1 to numworkers

OUT("datum", "stop") endforend

procedure worker IN("datum", datum) until datum == "stop"

value = compare(datum, target)OUT("score", value)IN("datum", datum)

enddoend

Tuple SpaceTuple Space

Perfect candidate for a database.Perfect candidate for a database. Simplifies parallel programming?Simplifies parallel programming? Performance?Performance?

– Consider the implementation of the tuple space.Consider the implementation of the tuple space.

Tuple Space ImplementationTuple Space Implementation

CentralCentral

What advantages/disadvantages does this What advantages/disadvantages does this implementation present?implementation present?

Tuple Space ImplementationTuple Space Implementation

DistributedDistributed

What advantages/disadvantages does this What advantages/disadvantages does this implementation present?implementation present?