A Multiple Associative Model to Support Branches in Data Parallel Applications Wittaya Chantamas and Johnnie W. Baker Department of Computer Science Kent State University, Kent, OHIO 44242 USA Telephone: (330) 672-9055 Fax: (330) 672-7824 [email protected] and [email protected]
A Multiple Associative Model to Support Branches in Data Parallel Applications Wittaya Chantamas and Johnnie W. Baker Department of Computer Science Kent.
Dec 22, 2015
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
A Multiple Associative Model to Support Branches in Data Parallel Applications
Wittaya Chantamas and Johnnie W. BakerDepartment of Computer ScienceKent State University, Kent, OHIO 44242 USATelephone: (330) 672-9055Fax: (330) [email protected] and [email protected]
MASC - Spring 2007
Outline
SIMD and Branches Single Instruction Multiple Data (SIMD) A data parallel program contains branches
MASC Computational Model Multiple Associative Computing (MASC) Model MASC model with manager-worker paradigm The power of MASC model
With variations of MASC With other models
ASC language compiler support for the MASC model MASC Algorithm
Shapes algorithm Modified ASC Quick Hull algorithm for the MASC model with
manager-worker paradigm
MASC - Spring 2007
SIMD and Branches
Most SIMD computers allow masking of PEs while determining whether or not that PE should participate in the operation in a parallel IF-THEN-ELSE statement
The THEN and ELSE parts have to be executed sequentially
MASC - Spring 2007
SIMD and Branches
A traditional SIMD computer IF ( a parallel condition)
THEN Statement Block AELSE Statement Block B
Suppose we have 14 PEs
MASC - Spring 2007
SIMD and Branches
A traditional SIMD computer IF ( a parallel condition)
THEN Statement Block AELSE Statement Block B
Suppose we have 14 PEs
MASC - Spring 2007
SIMD and Branches
A traditional SIMD computer IF ( a parallel condition)
THEN Statement Block AELSE Statement Block B
Suppose we have 14 PEs
MASC - Spring 2007
SIMD and Branches
A traditional SIMD computer IF ( a parallel condition)
THEN Statement Block AELSE Statement Block B
Suppose we have 14 PEs
MASC - Spring 2007
SIMD and Branches
Question: How can we improve the execution of the branches if we can have more than one instruction stream?
One probable answer: Execute each part of the branches simultaneously by using each of the instruction streams
MASC - Spring 2007
SIMD and Branches
The MASC computational model IF ( a parallel condition)
THEN Statement Block AELSE Statement Block B
Suppose we have 14 PEs
MASC - Spring 2007
SIMD and Branches
The MASC computational model IF ( a parallel condition)
THEN Statement Block AELSE Statement Block B
Suppose we have 14 PEs
MASC - Spring 2007
Outline
SIMD and Branches Single Instruction Multiple Data (SIMD) A data parallel program contains branches
MASC Computational Model Multiple Associative Computing (MASC) Model MASC model with manager-worker paradigm The power of MASC model
With variations of MASC With other models
ASC language compiler support for the MASC model MASC Algorithm
Shapes algorithm Modified ASC Quick Hull algorithm for the MASC model with
manager-worker paradigm
MASC - Spring 2007
MASC Computational Model
An extension of the associative computing model or ASC ASC model was created to capture Aspro and Staran ‘s style
of programming The associative properties
Broadcast data in constant time Constant time global reduction of
Boolean values using AND/OR Integer values using MAX/MIN
Constant time data search Provides content addressable data Eliminates need for sorting and indexing.
Pick one responder in constant time Supported in hardware with the broadcast and reduction network
MASC - Spring 2007
MASC Computational Model
The basic components of the model A set of instruction
streams An array of cells (PE
+ Memory) IS-to-Cell broadcast
and reduction networks (one for each IS, preferable)
An IS-to-IS Network A simple cell
network
Inst
ruct
ion
Str
eam
Net
wor
k
Bro
adca
st a
nd R
educ
tion
Net
wor
ks
Cel
l Net
wor
k
ManagerInstruction
Stream
WorkerInstruction
Stream
WorkerInstruction
Stream
WorkerInstruction
Stream
Cells
MemoryProcessingElement
ProcessingElement
ProcessingElement
ProcessingElement
ProcessingElement
ProcessingElement
Memory
Memory
Memory
Memory
Memory
MASC - Spring 2007
MASC Using Manager-Worker Paradigm
A variation of the MASC model Two types of ISs
A manager-IS (ID 0 ) Managing the work pool of tasks Coordinating and assigning subtasks (using a FORK
operation) Combining finished tasks (using a JOIN operation) from
worker-ISs
Identical worker-ISs (ID 1 to ID m) Executing tasks in an associative (e.g., data parallel,
SIMD) fashion using the PEs currently assigned to it
MASC - Spring 2007
MASC Using Manager-Worker Paradigm
The IS networkAn IS-to-IS broadcast/reduction
networkThe manager-IS can use the network
to perform a Min/MAX or logical reduction on ISs in constant time (pick one idle worker-IS)
The cell network is optional
MASC - Spring 2007
MASC Using Manager-Worker Paradigm
A Cell is a simple ALU + its local memory IS-Selector Register (an (lg m)-bit register if there are m ISs)
Holding the instruction stream ID, to which that PE is currently listening
The register can be set or reset by the instruction stream that the PE is listening to or by the data tested in that PE
Task-History Stack (use the memory in each cell) Holding the task ID The default content is empty The top of the stack always shows
the current task the PE is currently executing, the task that has just been finished, or a new task that has not yet assigned to a worker-IS
At any point in time, each PE listens to exactly one IS
MASC - Spring 2007
MASC Using Manager-Worker Paradigm
A task is broken down into subtasks No interaction between subtasks during their executions
A FORK operation Generates one or more subtasks from a branch by partitioning
PEs into group based on the parallel condition New task id will be push into the Task-History Stack Those subtasks will be assigned to worker-ISs to be executed
concurrently by setting the IS-Selector register of PEs in the corresponding group to the ID of the worker-IS
A JOIN operation Recombines subtasks into the original parent task (i.e., the one
existing prior to the fork) after they have been successfully executed by popping top of the Task-History Stack
MASC - Spring 2007
MASC Using Manager-Worker Paradigm
A work pool (WP-Q)Containing tasks ready to be executed
Work Pool
Worker-ISWorker-IS
MASC - Spring 2007
Fork Operation
Wittaya Chantamas, 08/24/2004
input: task 2, ( task 5 if TRUE, task 6 if FALSE)
1
3
2
4
6
5
Fork1
Fork2
Fork3
Join2
Join3
Join1
TRUE FALSE
MASC - Spring 2007
Join Operation
Wittaya Chantamas, 08/24/2004
input: ( task 5, task 6)
1
3
2
4
6
5
Fork1
Fork2
Fork3
Join2
Join3
Join1
TRUE FALSE
MASC - Spring 2007
The Power of MASC Model
Among the variations of the MASC model, the original MASC model with a simple cell network (1-d, 2-d, or hypercube) has the same power as A MASC model without any cell network
1-d cell network can be simulated in the MASC without any cell network in O(1) with a polynomial blow-up in size (PEs and ISs)
A proof of the 2-d and hypercube network case is similar to the case of 1-d cell network
A MASC model with manager-worker paradigm (We believed! Need further proof.)
MASC - Spring 2007
The Power of MASC Model
Comparing to other models, the MASC model has the same power as
Basic and Segmenting Reconfigurable Multiple Bus Machine (RMBM)
CRCW-PRAM A restriction version of RM A Mesh with Multiple Broadcasting (MMB)
is less powerful than Fused and Extended RMBM Reconfigurable Mesh (RM) Linear Mesh (LM)
MASC - Spring 2007
ASC Language Compiler Support for the MASC Model
The MASC model needs a multiple IS support from the ASC An extension of the ASC language compiler for the MASC
model A MASC directive
Concurrent data parallel executions of different paths in a branch can be achieved by using the directive/* .masc fork */
A user has a tight control Not all different paths in branches will be executed concurrently Only those in branches with directives will
Considered as a comment by the ASC compiler (will show in .lst file, not show in .iob file)
No need for a new ASC compiler in order to run an ASC program in MASC system
Need another extension if wanted to add a parallel case statement support
MASC - Spring 2007
A parallel IF-THEN-ELSE statement in the ASC language
IF condition expression
THEN statement block A
ELSE statement block B
ENDIF;
Tests thecondition
expression
Executesthe THEN
part
Executesthe ELSE
part
The instruction streamexecutes the statement block Ausing cells that satisfy the test.
The instruction streamexecutes the statement block Busing cells that do not satisfythe test.
The instruction stream teststhe condition expression.
MASC - Spring 2007
main testint parallel b[$], c[$], d[$];logical parallel BCD[$];associate b[$], c[$], d[$] with BCD[$];
read b[$] c[$] d[$] in BCD[$];b[$] = c[$] + 2;c[$] = d[$] - 3;
/* will be no fork here */if (b[$] .lt. c[$]) then
b[$] = c[$];d[$] = 4;
else c[$] = b[$];
b[$] = d[$];endif;c[$] = d[$];d[$] = c[$];
end;
M100 0000
W110 0000
M111 0000
M1000000
W1100000
a structure a structure codecode
.MI_BEGIN W1100000.MI_BEGIN W1100000beg_of_stmt 1c00 6 0 beg_of_stmt 1c00 6 0 beg_read 5a00 SYSOT beg_read 5a00 SYSOT BCD B,C,D, BCD B,C,D, …… beg_of_stmt 1c00 20 0 beg_of_stmt 1c00 20 0 mvpa_ 4812 C Dmvpa_ 4812 C D.MI_END W1100000.MI_END W1100000
M1110000
MASC - Spring 2007
A parallel IF-THEN-ELSE statement in the ASC language
/* .MASC fork */
IF condition expression
THEN statement block A
ELSE statement block B
ENDIF;
Tests thecondition
expression
Executesthe THEN
part
Executesthe ELSE
part
A workerinstruction streamexecutes thestatement blockA using cells thatsatisfy the test.
Another workerinstruction streamexecutes thestatement block Busing cells that donot satisfy the test.
The manager instructionstream tests the conditionexpression.
Does aFork
Does a Join
The manager instructionstream assigns two tasks totwo idle worker instructionstreams.
The manager instruction streamcombines the tasks backtogether.
MASC - Spring 2007
main testint parallel b[$], c[$], d[$];logical parallel BCD[$];associate b[$], c[$], d[$] with BCD[$];
read b[$] c[$] d[$] in BCD[$];b[$] = c[$] + 2;c[$] = d[$] - 3;
/*.MASC FORK */if (b[$] .lt. c[$]) then
b[$] = c[$];d[$] = 4;
else c[$] = b[$];
b[$] = d[$];endif;c[$] = d[$];d[$] = c[$];
end;
M100 0000
W110 0000
M111 0000
W111 1000
W111 2000
W111 X100
M111 X110
M1000000
W1100000
W1111000
M1110000
W111X100
M111X110
a structure a structure codecode
.MI_BEGIN W1112000beg_of_stmt 1c00 16 0 beg_of_stmt 1c00 16 0 mvpa_ 4812 B C mvpa_ 4812 B C beg_of_stmt 1c00 17 0 beg_of_stmt 1c00 17 0 mvpa_ 4812 D Bmvpa_ 4812 D B.MI_END W1112000
W1112000
MASC - Spring 2007
Outline
SIMD and Branches Single Instruction Multiple Data (SIMD) A data parallel program contains branches
MASC Computational Model Multiple Associative Computing (MASC) Model MASC model with manager-worker paradigm The power of MASC model
With variations of MASC With other models
ASC language compiler support for the MASC model MASC Algorithm
Shapes algorithm Modified ASC Quick Hull algorithm for the MASC model with
manager-worker paradigm
MASC - Spring 2007
Shape Problem
The testing problem To compute area of basic shapes in a database Can use the MASC model to solve this problem Each type of shapes required different equation to
compute the area Areas of each shape types can be compute
simultaneously by partitioning PEs in to groups (triangle, rectangle, or circle) and using one IS to compute the areas for each group
MASC - Spring 2007
MASC Quick Hull Algorithm
The convex hull problemThe convex hull of a set of points S is the
smallest convex set containing S. In particular, each point of set S is either on the boundary of or in the interior of the convex hull
Modified ASC Quick Hull algorithm for the MASC model with a limited number of ISs and using manager-worker paradigm with work pool
MASC - Spring 2007
MASC Quick Hull Algorithm
Algorithm MASC Quick Hull (for the upper hull)Input: A set of points S given as (x,y) coordinates, each
PE holds one point in SOutput: vertices of the upper convex hull The manager assigns the initialization task (i.e.,
task 0) to a worker IS to find two extreme points, X-min point (w) and X-max point (e) Two points (w and e) in the convex hull are
identified The manager creates task we and places it in the
work pool. The PEs associated with this task are the ones whose point lies above segment we
MASC - Spring 2007
MASC Quick Hull Algorithm
The manager assigns each task pq in the work pool to a worker IS to find another point in the convex hull using the PEs assigned to this task. Another point (r) in the convex hull is identified
The manager places task pr and task rq in the work pool. The PEs associated with each task are the ones whose point lies above corresponding line segment
The manager continues to execute 2 steps above until there are no active tasks and no tasks remain in the work pool, and then terminates the algorithm
MASC - Spring 2007
MASC Quick Hull Algorithm
F F
J
J
J
JT 0 T we
T pr
T rq
F
M-IS: Fork Task 0
W-IS: Execute Task 0
M-IS: Join Task 0
M-IS: Fork Task WE
W-IS: Execute Task WE
M-IS: Join Task WE
M-IS: Fork Task PR and Task RQ
W-IS: Execute Task RQ
W-IS: Execute Task PR
M-IS: Join Task PR
M-IS: Join Task RQ
MASC - Spring 2007
MASC Quick Hull Algorithm
Timing n is the number of points in S and m is the number
of instruction streams Still O(n) in the worst case If we assume that on the average O(lg n) is the
number of convex hull points, the average case running time is O((lg lg n)(lg n)/m)
Producing a constant speedup of approximately m over the 1-IS version of the same algorithm for the average case
MASC - Spring 2007
Conclusion
Traditional SIMD executes each part of branches of a data parallel program sequentially
MASC can execute most or all parts of the branches simultaneously if there are enough instruction streams
The original MASC model with a simple cell network is as powerful as a model without any cell network or with manager/worker paradigm
The MASC model is as powerful as many computational models such as PRAM and some versions of RMBM
An extension of the ASC compiler is required to take the benefit of having multiple ISs
Some problems can take the advantage of having more than one instruction stream. Some do not.
Related Documents
