Multiprocess Synchronization Algorithms (20225241) Lecturer: Danny Hendler The Mutual Exclusion problem.

Multiprocess Synchronization

Algorithms (20225241)

Lecturer: Danny Hendler

The Mutual Exclusion problem

2

The mutual exclusion problem(Dijkstra, 1965)

We need to devise a protocolthat guarantees mutuallyexclusive access by processesto a shared resource (such asa file, printer, etc.)

3

The problem model (reads/writes)

• Shared-memory multiprocessor: multiple processes

• Processes can apply Atomic reads and writes to shared registers

• Completely asynchronous

4

Mutex: formal definition

loop foreverRemainder codeEntry codeCritical section (CS)Exit code

end loop

Remainder code

Entry code

CS

Exit code

5

Mutex Requirements

• Mutual exclusion: No two processes are at their CS at the same time.

• Deadlock-freedom: If a process is trying to enter its critical section, then some process eventually enters its critical section.

• Starvation-freedom (optional): If a process is trying to enter its critical section, then this process must eventually enter its critical section.

6

An additional assumption

Processes do not stop while performing the entry, CS, or exit

code.

7

Incorrect algorithm 1.

Program for process 0

1. await turn=02. CS of process 03. turn:=1


1. await turn=12. CS of process 13. turn:=0

initially: turn=0

Does algorithm1 satisfy mutex?

Does it satisfy deadlock-freedom?

Yes

No

8


Program for both processes

1. await lock=02. lock:=13. CS4. lock:=0

initially: lock=0



No

Yes

9


initially: flag[0]=false, flag[1]=false




1. flag[0]:=true2. await flag[1]=false3. CS of process 04. flag[0]:=false


1. flag[1]:=true2. await flag[0]=false3. CS of process 14. flag[1]:=false

No

Yes

10

Peterson’s 2-process algorithm(Peterson, 1981)

initially: b[0]=false, b[1]=false, turn=0 or 1


1. b[0]:=true2. turn:=03. await (b[1]=false or

turn=1)4. CS5. b[0]:=false


1. b[1]:=true2. turn:=13. await (b[0]=false or

turn=0)4. CS5. b[1]:=false

11

Schematic for Peterson’s 2-process algorithm

Indicate participationb[i]:=true

Barrierturn:=i

Is there contention?b[1-i]=true?

yes

no

Critical Section

Exit codeb[i]:=false

First to cross barrier?turn=1-i?

no, maybe

yes

Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

12

Let’s prove that Peterson’s 2-process algorithm satisfies both mutual-exclusion and

starvation-freedom.

13

Kessel’s single-writer algorithm

(Kessels, 1982)

initially: b[0]=false, b[1]=false, turn[0], turn[1]=0 or 1


1. b[0]:=true2. local[0]:=turn[1]3. turn[0]:=local[0]4. Await (b[1]=false or

local[0]<>turn[1]5. CS6. b[0]:=false


1. b[1]:=true2. local[1]:=1-turn[0]3. turn[1]:=local[1]4. Await (b[0]=false or

local[1]=turn[0]5. CS6. b[1]:=false

A single-writer register is a register that can be written by a single process only.


14

Mutual exclusion for n processes:

Tournament trees0

0 1

0 1 2 3

0 1 2 3 4 5 6 7

Level 0

Level 1

Level 2

Processes

A tree-node is identified by: [level, node#]Synchronization Algorithms and Concurrent Programming

Gadi Taubenfeld © 2006

15

Tournament tree based on Peterson’s 2-process alg.

Program for process i1. node:=i2. For level = o to log n-1 do3. id:=node mod 24. node:= node/2 5. b[level,2node+id]:=true6. turn[level,node]:=id7. await (b[level,2node+1-id]=false or

turn[level,node]=1-id)8. od 9. CS10. for level=log n –1 downto 0 do11. node:= i/2level 12. b[level,node]:=false13. od

VariablesPer node: b[level, 2node], b[level, 2node+1], turn[level,node]Per process (local): level, node, id.

16

The tournament tree using Peterson’s 2-process algorithm satisfies both mutual-exclusion

and starvation-freedom.

17

Contention-free step complexity

The worst-case number of steps for a process to enter the CS when it runs by itself.

What’s the contention-free step complexity of Peterson’s tournament

tree? log n

Can we do better?

18

Lamport’s fast mutual exclusion algorithmVariables

Fast-lock, slow-lock initially 0want[i] initially false

Program for process i1. want[i]:=true2. fast-lock:=i3. if slow-lock<>0 then4. want[i]:=false5. await slow-lock:=06. goto 17. slow-lock:=i8. if fast-lock <> i then9. want[i]:=false10. for j:=1 to n do await want[j] = false od11. if slow-lock <> i then12. await slow-lock = 013. goto 114. CS15. slow-lock:=0• want[i]:=false

19

Schematic for Lamport’s fast mutual exclusion

Indicate contentionwant[i]:=true, fast-lock:=i

Is there contention?slow-lock< > 0?

yes

Barrierslow-lock:=i

CS

Wait until CS is releasedwant[I]:=false, await slow-

lock:=0

Is there contention?fast-lock < > i?

no

no

EXIT

yes Wait until no other process can cross the Barrier

Not last to cross Barrier?slow-lock < > i?

yesWait until CS is released

no


20

Lamport’s fast mutual exclusion algorithm satisfies both mutual-exclusion and

deadlock-freedom.

21

First in First Out (FIFO)

entry code

exit code

criticalsection

remainder• Mutual Exclusion

• Deadlock-freedom

• Starvation-freedom

doorway

waiting

• FIFO: if process p is waiting and process q has not yet started the doorway, then q will not enter the CS before p.


23

time

Lamport’s Bakery Algorithm

0 0 0 0 0 0

doorway

1 2 3 4 5 n

CS

exit

1

1

2 2

2 2

1

1

0

2

2

0

3

3

2

2

0

4

4waiting

en

try

remainder


24

Implementation 1code of process i , i {1 ,..., n}

number[i] := 1 + max {number[j] | (1 j n)}for j := 1 to n (<> i) { await (number[j] = 0) (number[j] > number[i])}critical sectionnumber[i] := 0

1 2 3 4 n

number integer0 0 0 0 0 0

Answer: No, it can deadlock!

Does this implementation work?


25

time

Implementation 1: deadlock

0 0 0 0 0 0

doorway

1 2 3 4 5 n

CS

exit

1

1

2 2

2 2

1

1

0

waiting

en

try

remainder

deadlock


26

number[i] := 1 + max {number[j] | (1 j n)}for j := 1 to n (<> i) { await (number[j] = 0) (number[j],j) number[i],i)

// lexicographical order

}critical sectionnumber[i] := 0

1 2 3 4 n


Answer: It does not satisfy mutual exclusion!

Implementation 2code of process i , i {1 ,..., n}

Does this implementation work?


27

time

Implementation 2: no mutual exclusion

0 0 0 0 0

doorway

1 2 3 4 5 n

CS

exit

0

1

0 0

2 2

1

1

0

2 2

waiting

en

try

remainder

1 2 2

0


28

The Bakery Algorithmcode of process i , i {1 ,..., n}

1: choosing[i] := true2: number[i] := 1 + max {number[j] | (1 j n)}3: choosing[i] := false4: for j := 1 to n do5: await choosing[j] = false 6: await (number[j] = 0) (number[j],j) (number[i],i)7: od8: critical section9: number[i] := 0

1 2 3 4 n

choosing bitsfalse


false false false false false

Doorway

Waiting Bakery

29

Computing the maximumcode of process i , i {0 ,..., n-1}

local1 := 0for local2 := 1 to n { local3 := number[local2] if local1 < local3 then {local1 := local3}}number[i] := 1+local1

0123

n-1

numberchoosingfalsefalsefalsefalsefalse

false

00000

0

The correctness of the Bakery algorithm depends on an implicit assumption on the implementation of computing the maximum (statement 2). Below we give a correct implementation. For each process, three additional local registers are used. They are named local1, local2, local3 and their initial values are immaterial.


30

Question: Computing the maximum

code of process i , i {0 ,..., n-1}

local1 := ifor local2 := 1 to n { if number[local1] < number[local2] then {local1 := local2}}number[i] := 1+ number[local1]

0123

n-1

numberchoosingfalsefalsefalsefalsefalse

false

00000

0

Is the following implementation also correct? That is, does the Bakery algorithm solves the mutual exclusion problem when the following implementation is used? Justify your answer.For each process, two additional local registers are used. They are named local1, local2, and their initial values are immaterial.


31

time

0 0 0 0

doorway

1 2 3 4 5 n

CS

exit

1 1

1 1

1

1

0

1

waiting

en

try

remainder 00

?

1

1

?local1 2

1


The 2nd maximum alg. doesn’t work

32

Properties of the Bakery algorithm

• Satisfies Mutual exclusion and first-come-first-served.

• The size of number[i] is unbounded.– In practice this is not a problem, 16 bits

registers will give us ticket numbers which can grow up to 2^16, a number that in practice will never be reached.

• There is no need to assume that operations on the same memory location occur in some definite order; it works correctly even when it is allowed for reads which are concurrent with writes to return an arbitrary value.

The Black-White Bakery AlgorithmThe Black-White Bakery AlgorithmBounding the space of the Bakery

Algorithm

Bakery (FIFO, unbounded)

The Black-White Bakery Algorithm

FIFOBounded space+ one bit


34

time

The Black-White Bakery AlgorithmThe Black-White Bakery Algorithm

0 0 0 0 0

doorway

1 2 3 4 5 n

CS

exit

0

1

0 0

2 2

1

1

0

2

2

0

1

2

2

0

2waiting

en

try

remainder

1 20 201 2

1

1

00

color bit


35

The Black-White Bakery Algorithm

1 2 3 4 n

choosing

Data StructuresData Structures

mycolor

number

color bit

bits

bits

{0,1,...,n}

{black,white}


36

The Black-White Bakery Algorithmcode of process i , i {1 ,..., n}

choosing[i] := truemycolor[i] := colornumber[i] := 1 + max{number[j] | (1 j n) (mycolor[j] = mycolor[i])}choosing[i] := falsefor j := 0 to n do await choosing[j] = false if mycolor[j] = mycolor[i] then await (number[j] = 0) (number[j],j) (number[i],i) (mycolor[j] mycolor[i]) else await (number[j] = 0) (mycolor[i] color) (mycolor[j] = mycolor[i]) fi odcritical sectionif mycolor[i] = black then color := white else color := black finumber[i] := 0


37

A space lower bound fordeadlock-free mutex

How many registers must an n-process deadlock-free mutual exclusion algorithm use if it can only usesingle-writer registers?

We now prove that the same result holds for multi-reader-multi-writer registers, regardless of their

size.

38

Some definitions required for the proof

• Configuration

• A quiescent configuration

• Indistinguishable configurations

• A P-quiescent configuration

• A covered register

• An execution

39

Example of indistinguishability

Execution x is indistinuishable from execution y to process p

execution x• p reads 5 from

r1• q writes 6 to r1• p writes 7 to r1• q writes 8 to r1• p reads 8 from

r1

execution y• p reads 5 from

r1• p writes 7 to r1• q writes 6 to r1• q reads 6 from

r1• q writes 8 to r1• p reads 8 from

r18r1 8r1

• q write 6 to r1

6

The values of the shared registers must also be the same


40

Illustration for Lemma 1

C

pi-quiescent,W covered by P

Quiescent

D

C ~ Dpi

(by pi)

C1pi in CS

pi in CS

(by pi) D1

(By P)Q

Quiescent

(By pj)

R

Pj in CS

(By P) Q1

Q ~ Q1

pj

(By pj)

Z

Both pi, pj in CS!

Based on the proof in “Distributed Computing”, by Hagit Attiya & Jennifer Welch

41

Illustration for the simple part of Lemma 2

C1

{pk,…,pn-1}-quiescent p0…pk-1 cover W

pk runs until it covers x

' (pk only)

D1Quiescent

{pk+1,…,pn-1}-quiescent W U {x} covered

C'2

D‘1 ~ D1

{p0…pk-1}

(by p0… pk-1) C2

{pk,…,pn-1}-quiescentp0…pk-1 cover W


D'1

p0… pk write to W and exit

x is coveredP-{pk} in remainder

42

Illustration for the general part of Lemma 2


D0 D'i'i

quiescent

D1

1

quiescent

C11

{pk,…,pn-1}-quiescent p0…pk-1 cover W1

C22

{pk,…,pn-1}-quiescent p0…pk-1 cover W2

Ci

2… i

{pk,…,pn-1}-quiescent p0…pk-1 cover Wi

{pk+1,…,pn-1}-quiescent W U {x} covered

C’j

i+1i+1… j

D‘i ~ Di

{pk+1,…,pn-1}-i

Di

quiescent

Cj

i+1i+1… j

{pk+1,…,pn-1}-quiescent p0…pk-1 cover Wi

43

A matching upper bound:the one-bit algorithminitially: b[i]:=false

Program for process i• repeat• b[i]:=true; j:=1• while (b[i] = true) and (j < i) do• if (b[j]=true then • b[i]:=false• await b[j]=false• j:=j+1• until b[i]=true• for (j:=i+1 to n) do• await b[j]=false • Critical Section• b[i]=false

44

Read-Modify-Write (RMW) operations

Read-modify-write (w, f)do atomically prev:=w w:=f(prev) return prev

Fetch-and-add(w, Δ)do atomically prev:=w w:= prev+Δ return prev

Test-and-set(w)do atomically prev:=w w:=1 return prev

45

Mutual exclusion using test-and-set

Program for process I

• await test&set(v) = 0• Critical Section• v:=0

initially: v:=0

Mutual exclusion?Yes

Deadlock-freedom?

NoStarvation-freedom?

Yes

46

Mutual exclusion using general RMW

Program for process I

• position:=RMW(v, <v.first, v.last+1> )

• repeat• queue:=v• until queue.first = position.last• Critical Section• RMW(v, <v.first+1, v.last> )

initially: v:=<0,0>

How many bits does this algorithm require?

Unbounded number, but can be improved to 2 log2 n

47

Lower bound on the number

of bits required for mutual exclusion

Multiprocess Synchronization Algorithms (20225241) Lecturer: Danny Hendler The Mutual Exclusion problem.

Documents