(C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2.

(C) P. H. Welch, 2003 1

Paradigmsof

Parallelism

Chapter 2

(C) P. H. Welch, 2003 2

•Geometric (90%)•Algorithmic (70%)•Farming (99%)•???

+ mixtures of the above

Levels of efficiencythat should be obtained

Paradigms of Parallelism

(C) P. H. Welch, 2003 3

Fox’s Wall

specification

•A large pile of bricks•One bricklayer – OK!•Three bricklayers???

need organizing

(C) P. H. Welch, 2003 4

Geometric Distribution

• Task partitioned geometrically amongst workers

• For the main, each worker works away independently

• But needs to interact with neighbouring workers when working on boundaries of allotted task

(C) P. H. Welch, 2003 5

Geometric Distribution

Green can work fastest(no half bricks to cut)

Red works as fast as it can

Blue is an apprentice(and slow)

(C) P. H. Welch, 2003 6

Geometric Distribution

Green can work fastest(but is now held up!)

Red works as fast as it can

Blue is an apprentice(and slow)

(C) P. H. Welch, 2003 7

Algorithmic (Pipeline)Distribution

• Functional distribution of tasks to workers

• Need to interact with neighbouring workers continually (to input data and pass on results)

• Like geometric distribution but work area is all boundary

(C) P. H. Welch, 2003 8

Pipeline Distribution

• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the

“pipe”

(C) P. H. Welch, 2003 9

Pipeline Distribution

• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the

“pipe”

(C) P. H. Welch, 2003 10

Pipeline Distribution

• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the

“pipe”

(C) P. H. Welch, 2003 11

Pipeline Distribution

• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the

“pipe”

(C) P. H. Welch, 2003 12

Pipeline Distribution

• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the

“pipe”

(C) P. H. Welch, 2003 13

Pipeline Distribution

• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the

“pipe”

(C) P. H. Welch, 2003 14

Farming Distribution

• Each worker gets work from a single source (“farmer”)

• Each worker sends completed work to a single “harvester”

• Each worker’s work is not dependent on any particular other worker

• Each worker’s rate of work may be different

(C) P. H. Welch, 2003 15

Farming Distribution

• All workers always busy

• Work at your own pace

• Your work must not be (tightly) dependent on other work

(C) P. H. Welch, 2003 16

• All workers always busy

• Work at your own pace

• Your work must not be (tightly) dependent on other work

Farming Distribution

(C) P. H. Welch, 2003 17

• All workers always busy

• Work at your own pace

• Your work must not be (tightly) dependent on other work

Farming Distribution

(C) P. H. Welch, 2003 18

Take n particles with different masses and initial positions and velocities in 3-space.

Assume some inter-particle forces (e.g., gravity).

Display their movements.

n-Body Problem

(C) P. H. Welch, 2003 19

C0

C2

C1

Ci

Ci+1

Ci+2

Cn-1

C3

(positionvelocity)

Each cell knows all constants (e.g. themasses of all particles).Cell Ci knows the current position andvelocity of particle i(its variables).

Geometric Distribution

(C) P. H. Welch, 2003 20

C0

C2

C1

Ci

Ci+1

Ci+2

Cn-1

C3

(positionvelocity)

Each cell knows all constants (e.g. themasses of all particles).Cell Ci knows the current position andvelocity of particle i(its variables).

graphics

Geometric Distribution

(C) P. H. Welch, 2003 21

pump

<unsorted items>end.marker

<unsorted items>end.marker

<sorted items>end.marker

<sorted items>end.marker

Sort Pump

(C) P. H. Welch, 2003 22

cell

<items>end.marker<items>

end.marker

<items> flow through the cell one at a time. The cell hangs on to the largest item is sees – passing smaller ones out. When the end.marker arrives, thecell finally outputs what it was holding (followed by the end.marker).

Sort Cell

(C) P. H. Welch, 2003 23

• Rate of input of unsorted items.

• Cell cycle time.

• Inter-cell transfer rate.

(n-1)This starts out with up to n items. With (n-1) separate (silicon)cells, its rate of output of sorted items is:-

minimum

This performance is independent of n.

Pipeline Distribution

(C) P. H. Welch, 2003 24

(0,2)

(-2,0) (2,0)

(0,-2)

c

Z0 = cZi+1 = Zi + c

Given c, find first |M such that|ZM| > 2.

Give up is M = 511.

Mandelbrot

(C) P. H. Welch, 2003 25

Colour the point c according to the value of M such that:-

M

0 511

some colour spectrumwhite black

Mandelbrot

(C) P. H. Welch, 2003 26

work packets

results

farmer

worker

worker

harvester

worker

Standard Process Farming

(C) P. H. Welch, 2003 27

Work Packet

(x, y) a point in the complex plane

a horizontal resolutionn number of points to colour

Results

(x,y) (x + n.a,y)

Computer the Mandelbrot number Mfor each of these points.

Mandelbrot

(C) P. H. Welch, 2003 28

in.work in.result

buffer

prompter

out.work

work

catch

mux

out.result

worker

Farm Worker Harness

* omitted from the last worker!* omitted from the last worker!

**

* omitted from the first worker!* omitted from the first worker!

**