Top Banner
Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
198

Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Dec 16, 2015

Download

Documents

Linda Poole
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Concurrent Queues and Stacks

Companion slides forThe Art of Multiprocessor

Programmingby Maurice Herlihy & Nir Shavit

Page 2: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

2

The Five-Fold Path

• Coarse-grained locking• Fine-grained locking• Optimistic synchronization• Lazy synchronization• Lock-free synchronization

Page 3: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

3

Another Fundamental Problem

• We told you about – Sets implemented using linked lists

• Next: queues• Next: stacks

Page 4: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

4

Queues & Stacks

• Both: pool of items• Queue

– enq() & deq()– First-in-first-out (FIFO) order

• Stack– push() & pop()– Last-in-first-out (LIFO) order

Page 5: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

5

Bounded vs Unbounded

• Bounded– Fixed capacity– Good when resources an issue

• Unbounded– Holds any number of objects

Page 6: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

6

Blocking vs Non-Blocking

• Problem cases:– Removing from empty pool– Adding to full (bounded) pool

• Blocking– Caller waits until state changes

• Non-Blocking– Method throws exception

Page 7: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

7

This Lecture

• Bounded, Blocking, Lock-based Queue

• Unbounded, Non-Blocking, Lock-free Queue

• Monitors• ABA problem• Unbounded Non-Blocking Lock-free

Stack• Elimination-Backoff Stack

Page 8: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

8

Queue: Concurrency

enq(x) y=deq()

enq() and deq() work at

different ends of the object

tail head

Page 9: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

9

Concurrency

enq(x)

Challenge: what if the queue is empty or full?

y=deq()ta

ilhead

Page 10: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

10

Bounded Queue

Sentinel

head

tail

Page 11: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

11

Bounded Queue

head

tail

First actual item

Page 12: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

12

Bounded Queue

head

tail

Lock out other deq() calls

deqLock

Page 13: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

13

Bounded Queue

head

tail

Lock out other enq() calls

deqLock

enqLock

Page 14: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

14

Not Done Yet

head

tail

deqLock

enqLock

Need to tell whether queue is

full or empty

Page 15: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

15

Not Done Yet

head

tail

deqLock

enqLock

Permission to enqueue 8 items

permits

8

Page 16: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

16

Not Done Yet

head

tail

deqLock

enqLock

Incremented by deq()Decremented by enq()

permits

8

Page 17: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

17

Enqueuer

head

tail

deqLock

enqLock

permits

8

Lock enqLock

Page 18: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

18

Enqueuer

head

tail

deqLock

enqLock

permits

8

Read permits

OK

Page 19: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

19

Enqueuer

head

tail

deqLock

enqLock

permits

8

No need to lock

tail

Page 20: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

20

Enqueuer

head

tail

deqLock

enqLock

permits

8

Enqueue Node

Page 21: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

21

Enqueuer

head

tail

deqLock

enqLock

permits

87

getAndDecrement()

Page 22: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

22

Enqueuer

head

tail

deqLock

enqLock

permits

8 Release lock7

Page 23: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

23

Enqueuer

head

tail

deqLock

enqLock

permits

7

If queue was empty, notify/signal waiting

dequeuers

Page 24: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

24

Unsuccesful Enqueuer

head

tail

deqLock

enqLock

permits

0

Uh-oh

Read permits

Page 25: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

25

Dequeuer

head

tail

deqLock

enqLock

permits

8

Lock deqLock

Page 26: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

26

Dequeuer

head

tail

deqLock

enqLock

permits

7

Read sentinel’s next field

OK

Page 27: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

27

Dequeuer

head

tail

deqLock

enqLock

permits

7

Read value

Page 28: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

28

Dequeuer

head

tail

deqLock

enqLock

permits

7

Make first Node new sentinel

Page 29: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

29

Dequeuer

head

tail

deqLock

enqLock

permits

8

Increment permits

Page 30: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

30

Dequeuer

head

tail

deqLock

enqLock

permits

7Release deqLock

Page 31: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

31

Unsuccesful Dequeuer

head

tail

deqLock

enqLock

permits

8

Read sentinel’s next field

uh-oh

Page 32: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

32

Bounded Queue

public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}

Page 33: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

33

Bounded Queue

public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}

Enq & deq locks

Page 34: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

34

Digression: Monitor Locks

• Java Synchronized objects and Java ReentrantLocks are monitors

• Allow blocking on a condition rather than spinning

• Threads: – acquire and release lock– wait on a condition

Page 35: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

35

public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}

The Java Lock Interface

Acquire lock

Page 36: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

36

public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}

The Java Lock Interface

Release lock

Page 37: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

37

public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}

The Java Lock Interface

Try for lock, but not too hard

Page 38: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

38

public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}

The Java Lock Interface

Create condition to wait on

Page 39: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

39

The Java Lock Interface

public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}

Guess what this method does?

Page 40: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

40

Lock Conditions

public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }

Page 41: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

41

public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }

Lock Conditions

Release lock and wait on condition

Page 42: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

42

public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }

Lock Conditions

Wake up one waiting thread

Page 43: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

43

public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }

Lock Conditions

Wake up all waiting threads

Page 44: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

44

Await

• Releases lock associated with q• Sleeps (gives up processor)• Awakens (resumes running)• Reacquires lock & returns

q.await()

Page 45: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

45

Signal

• Awakens one waiting thread– Which will reacquire lock

q.signal();

Page 46: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

46

Signal All

• Awakens all waiting threads– Which will each reacquire lock

q.signalAll();

Page 47: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

47

A Monitor Lock

Cri

tical S

ecti

on

waiting roomlock()

unLock()

Page 48: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

48

Unsuccessful Deq

Cri

tical S

ecti

on

waiting roomlock(

)

await()

deq()

Oh no, empty!

Page 49: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

49

Another One

Cri

tical S

ecti

on

waiting roomlock(

)

await()

deq()

Oh no, empty!

Page 50: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

50

Enqueuer to the Rescue

Cri

tical S

ecti

on

waiting roomlock()

signalAll()

enq( )

unLock()

Yawn!Yawn!

Page 51: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

51

Yawn!

Monitor Signalling

Cri

tical S

ecti

on

waiting room

Yawn!

Awakened thread might still lose lock to outside contender…

Page 52: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

52

Dequeurs Signalled

Cri

tical S

ecti

on

waiting room

Found it

Yawn!

Page 53: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

53

Yawn!

Dequeurs Signalled

Cri

tical S

ecti

on

waiting room

Still empty!

Page 54: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

54

Dollar Short + Day Late

Cri

tical S

ecti

on

waiting room

Page 55: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007 55

Lost Wake-Up

Cri

tical S

ecti

on

waiting roomlock()

signal ()enq( )

unLock()

Yawn!

Page 56: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

56

Lost Wake-Up

Cri

tical S

ecti

on

waiting roomlock()

enq( )

unLock()

Yawn!

Page 57: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

57

Lost Wake-Up

Cri

tical S

ecti

on

waiting room

Yawn!

Page 58: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

58

Lost Wake-Up

Cri

tical S

ecti

on

waiting room

Found it

Page 59: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

59

What’s Wrong Here?

Cri

tical S

ecti

on

waiting room

zzzz….!

Page 60: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Solution to Lost Wakeup

• Always use signalAll and notifyAll • Not signal and notify

Art of Multiprocessor Programming© Herlihy-Shavit

2007

60

Page 61: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

61

public class Queue<T> {

int head = 0, tail = 0; T[QSIZE] items;

public synchronized T deq() { while (tail – head == 0) this.wait(); T result = items[head % QSIZE]; head++; this.notifyAll(); return result; } …}}

Java Synchronized Methods

Page 62: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

62

public class Queue<T> {

int head = 0, tail = 0; T[QSIZE] items;

public synchronized T deq() { while (tail – head == 0) this.wait(); T result = items[head % QSIZE]; head++; this.notifyAll(); return result; } …}}

Java Synchronized Methods

Each object has an implicit lock with an

implicit condition

Page 63: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

63

public class Queue<T> {

int head = 0, tail = 0; T[QSIZE] items;

public synchronized T deq() { while (tail – head == 0) this.wait(); T result = items[head % QSIZE]; head++; this.notifyAll(); return result; } …}}

Java Synchronized Methods

Lock on entry, unlock on

return

Page 64: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

64

public class Queue<T> {

int head = 0, tail = 0; T[QSIZE] items;

public synchronized T deq() { while (tail – head == 0) this.wait(); T result = items[head % QSIZE]; head++; this.notifyAll(); return result; } …}}

Java Synchronized Methods

Wait on implicit condition

Page 65: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

65

public class Queue<T> {

int head = 0, tail = 0; T[QSIZE] items;

public synchronized T deq() { while (tail – head == 0) this.wait(); T result = items[head % QSIZE]; head++; this.notifyAll(); return result; } …}}

Java Synchronized Methods

Signal all threads waiting on condition

Page 66: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

66

(Pop!) The Bounded Queue

public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}

Page 67: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

67

Bounded Queue Fields

public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}

Enq & deq locks

Page 68: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

68

Bounded Queue Fields

public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}

Enq lock’s associated condition

Page 69: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

69

Bounded Queue Fields

public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}

Num permits: 0 to capacity

Page 70: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

70

Bounded Queue Fields

public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}

Head and Tail

Page 71: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

71

Enq Method Part Onepublic void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}

Page 72: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

72

public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}

Enq Method Part One

Lock and unlock enq

lock

Page 73: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

73

public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}

Enq Method Part One

If queue is full, patiently await further

instructions …

Page 74: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

74

public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}

Be Afraid

How do we know the permits field won’t

change?

Page 75: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

75

public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}

Enq Method Part One

Add new node

Page 76: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

76

public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}

Enq Method Part One

If queue was empty, wake frustrated dequeuers

Page 77: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

77

Enq Method Part Deux

public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }

Page 78: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

78

Enq Method Part Deux

public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }Are there dequeuers to be signaled?

Page 79: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

79

public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }

Enq Method Part Deux

Lock and unlock deq

lock

Page 80: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

80

public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }

Enq Method Part Deux

Signal dequeuers that queue no longer

empty

Page 81: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

81

The Enq() & Deq() Methods

• Share no locks– That’s good

• But do share an atomic counter– Accessed on every method call– That’s not so good

• Can we alleviate this bottleneck?

Page 82: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

82

Split the Counter

• The enq() method– Decrements only– Cares only if value is zero

• The deq() method– Increments only– Cares only if value is capacity

Page 83: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

83

Split Counter

• Enqueuer decrements enqSidePermits• Dequeuer increments deqSidePermits• When enqueuer runs out

– Locks deqLock– Transfers permits

• Intermittent synchronization– Not with each method call– Need both locks! (careful …)

Page 84: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

84

A Lock-Free Queue

Sentinel

head

tail

Page 85: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

85

Compare and Set

CAS

Page 86: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

86

Enqueue

head

tail

Enq( )

Page 87: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

87

Enqueue

head

tail

Page 88: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

88

Logical Enqueue

head

tail

CAS

Page 89: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

89

Physical Enqueue

head

tail

Enqueue Node

CAS

Page 90: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

90

Enqueue

• These two steps are not atomic• The tail field refers to either

– Actual last Node (good)– Penultimate Node (not so good)

• Be prepared!

Page 91: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

200791

Enqueue

• What do you do if you find– A trailing tail?

• Stop and help fix it– If tail node has non-null next field– CAS the queue’s tail field to tail.next

• As in the universal construction

Page 92: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

92

When CASs Fail

• During logical enqueue– Abandon hope, restart– Still lock-free (why?)

• During physical enqueue– Ignore it (why?)

Page 93: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

93

Dequeuer

head

tail

Read value

Page 94: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

94

Dequeuer

head

tail

Make first Node new sentinel

CAS

Page 95: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

95

Memory Reuse?

• What do we do with nodes after we dequeue them?

• Java: let garbage collector deal?• Suppose there is no GC, or we

prefer not to use it?

Page 96: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

96

Dequeuer

head

tail

CAS

Can recycle

Page 97: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

97

Simple Solution

• Each thread has a free list of unused queue nodes

• Allocate node: pop from list• Free node: push onto list• Deal with underflow somehow …

Page 98: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

98

Why Recycling is Hard

Free pool

head tail

Want to redirect

tailfrom

grey to red

zzz…

Page 99: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

99

Both Nodes Reclaimed

Free pool

zzz

head tail

Page 100: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

100

One Node Recycled

Free pool

Yawn!

head tail

Page 101: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

101

Why Recycling is Hard

Free pool

CAShead tail

OK, here I go!

Page 102: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

102

Final State

Free pool

zOMG what went wrong?

head tail

Bad news

Page 103: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

103

The Dreaded ABA Problem

Head pointer has value AThread reads value A

head tail

Page 104: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

104

Dreaded ABA continued

zzz Head pointer has value BNode A freed

head tail

Page 105: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

105

Dreaded ABA continued

Yawn! Head pointer has value A againNode A recycled & reinitialized

head tail

Page 106: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

106

Dreaded ABA continued

CAS succeeds because pointer matcheseven though pointer’s meaning has changed

CAShead tail

Page 107: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

107

The Dreaded ABA Problem

• Is a result of CAS() semantics– I blame Sun, Intel, AMD, …

• Not with Load-Locked/Store-Conditional– Good for IBM?

Page 108: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

108

Dreaded ABA – A Solution

• Tag each pointer with a counter• Unique over lifetime of node• Pointer size vs word size issues• Overflow?

– Don’t worry be happy?– Bounded tags?

• AtomicStampedReference class

Page 109: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

109

Atomic Stamped Reference

• AtomicStampedReference class– Java.util.concurrent.atomic package

address S

Stamp

Reference

Can get reference and stamp atomically

Page 110: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

110

Concurrent Stack

• Methods– push(x) – pop()

• Last-in, First-out (LIFO) order• Lock-Free!

Page 111: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

111

Empty Stack

Top

Page 112: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

112

Push

Top

Page 113: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

113

Push

TopCAS

Page 114: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

114

Push

Top

Page 115: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

115

Push

Top

Page 116: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

116

Push

Top

Page 117: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

117

Push

TopCAS

Page 118: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

118

Push

Top

Page 119: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

119

Pop

Top

Page 120: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

120

Pop

TopCAS

Page 121: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

121

Pop

TopCAS

mine!

Page 122: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

122

Pop

TopCAS

Page 123: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

123

Pop

Top

Page 124: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

124

public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff(); }}

Lock-free Stack

Page 125: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

125

public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public Boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) }public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff()}}

Lock-free Stack

tryPush attempts to push a node

Page 126: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

126

public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff()}}

Lock-free Stack

Read top value

Page 127: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

127

public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff()}}

Lock-free Stack

current top will be new node’s successor

Page 128: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

128

public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff()}}

Lock-free Stack

Try to swing top, return success or failure

Page 129: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

129

public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff()}}

Lock-free Stack

Push calls tryPush

Page 130: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

130

public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff()}}

Lock-free Stack

Create new node

Page 131: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

131

public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff()}}

Lock-free Stack

If tryPush() fails,back off before retrying

Makes scheduling benevolent so Method is effectively wait-free

Page 132: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

132

Lock-free Stack

• Good– No locking

• Bad– Without GC, fear ABA– Without backoff, huge contention at

top – In any case, no parallelism

Page 133: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

133

Big Question

• Are stacks inherently sequential?• Reasons why

– Every pop() call fights for top item

• Reasons why not– Stay tuned …

Page 134: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

134

Elimination-Backoff Stack

• How to– “turn contention into parallelism”

• Replace familiar– exponential backoff

• With alternative– elimination-backoff

Page 135: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

135

Observation

Push( )

Pop()

linearizable stack

After an equal number of pushes and pops, stack stays the same

Yes!

Page 136: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

136

Idea: Elimination Array

Push( )

Pop()

stack

Pick at random

Pick at random

Elimination Array

Page 137: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

137

Push Collides With Pop

Push( )

Pop()

stack

continue

continue

No need to access stack

Yes!

Page 138: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

138

No Collision

Push( )

Pop()

stack

If no collision, access stack

If pushes collide or pops collide access stack

Page 139: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

139

Elimination-Backoff Stack

• Lock-free stack + elimination array• Access Lock-free stack,

– If uncontended, apply operation – if contended, back off to elimination

array and attempt elimination

Page 140: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

140

Elimination-Backoff Stack

Push( )

Pop()TopCAS

If CAS fails, back off

Page 141: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

141

Dynamic Range and Delay

Push( )

Pick random range and max time to wait for collision based on level ofcontention encountered

Page 142: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

142

Linearizability

• Un-eliminated calls– linearized as before

• Eliminated calls:– linearize pop() immediately after

matching push()

• Combination is a linearizable stack

Page 143: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

143

Un-Eliminated Linearizability

push(v1)

timetime

linearizable

push(v1)

pop(v1)pop(v1)

Page 144: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

144

Eliminated Linearizability

pop(v2)push(v1)

push(v2)

timetime

push(v2)

linearizable

pop(v2)push(v1)

pop(v1)pop(v1)

Collision Point

Red calls are eliminated

Page 145: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

145

Backoff Has Dual Effect

• Elimination introduces parallelism• Backoff onto array cuts contention

on lock-free stack• Elimination in array cuts down

total number of threads ever accessing lock-free stack

Page 146: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

146

public class EliminationArray { private static final int duration = ...; private static final int timeUnit = ...; Exchanger<T>[] exchanger; public EliminationArray(int capacity) { exchanger = new Exchanger[capacity]; for (int i = 0; i < capacity; i++) exchanger[i] = new Exchanger<T>(); … } …}

Elimination Array

Page 147: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

147

public class EliminationArray { private static final int duration = ...; private static final int timeUnit = ...; Exchanger<T>[] exchanger; public EliminationArray(int capacity) { exchanger = new Exchanger[capacity]; for (int i = 0; i < capacity; i++) exchanger[i] = new Exchanger<T>(); … } …}

Elimination Array

An array of exchangers

Page 148: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

148

Lock-free Exchanger

EMPTY

Page 149: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

149

EMPTY

Lock-free Exchanger

CAS

Page 150: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

150

WAITING

Lock-free Exchanger

Page 151: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

151

Lock-free ExchangerIn search of partner …

WAITING

Page 152: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

152

WAITING

Lock-free Exchanger

Slot

Still waiting …

Try to exchange

item and set state to BUSY

CAS

Page 153: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

153

BUSY

Lock-free Exchanger

Slot

Partner showed up, take item and reset to

EMPTY

item stamp/state

Page 154: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

154

EMPTYBUSY

Lock-free Exchanger

Slot

item stamp/state

Partner showed up, take item and reset to

EMPTY

Page 155: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

155

public class Exchanger<T> { AtomicStampedReference<T> slot = new AtomicStampedReference<T>(null, 0);

A Lock-Free Exchanger

Page 156: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

156

public class Exchanger<T> { AtomicStampedReference<T> slot = new AtomicStampedReference<T>(null, 0);

A Lock-Free Exchanger

Atomically modifiable reference + time stamp

Page 157: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

157

Atomic Stamped Reference

• AtomicStampedReference class– Java.util.concurrent.atomic package

• In C or C++:

address S

Stamp

Reference

Page 158: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

158

Extracting Reference & Stamp

Public T get(int[] stampHolder);

Page 159: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

159

Extracting Reference & Stamp

Public T get(int[] stampHolder);

Returns reference to object of type T

Returns stamp at array index

0!

Page 160: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

160

Exchanger Status

enum Status {EMPTY, WAITING, BUSY};

Page 161: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

161

Exchanger Status

enum Status {EMPTY, WAITING, BUSY};

Nothing yet

Page 162: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

162

Exchange Status

enum Status {EMPTY, WAITING, BUSY};

Nothing yet

One thread is waiting for

rendez-vous

Page 163: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

163

Exchange Status

enum Status {EMPTY, WAITING, BUSY};

Nothing yet

One thread is waiting for

rendez-vous Other threads busy with rendez-

vous

Page 164: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

164

public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } }

The Exchange

Page 165: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

165

public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } }

The Exchange

Item & timeout

Page 166: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

166

public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } }

The Exchange

Array to hold timestamp

Page 167: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

167

public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging } }}

The Exchange

Loop until timeout

Page 168: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

168

public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging } }}

The Exchange

Get other’s item and timestamp

Page 169: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

169

public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } }}

The Exchange

Exchanger slot has three states

Page 170: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

170

case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; }} break;

Exchanger State EMPTY

Page 171: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

171

case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; }} break;

Exchanger State EMPTY

Slot is free, try to insert myItem and change state to WAITING

Page 172: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

172

case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; }} break;

Exchanger State EMPTY

Loop while still time left to attempt exchange

Page 173: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

173

case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; }} break;

Exchanger State EMPTY

Get item and stamp in slot and check if state changed to BUSY

Page 174: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

174

case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; }} break;

Exchanger State EMPTY

If successful reset slot state to EMPTY

Page 175: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

175

case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; }} break;

Exchanger State EMPTY

and return item found in slot

Page 176: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

176

case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; }} break;

Exchanger State EMPTY

Otherwise we ran out of time, try to reset state to EMPTY, if successful time out

Page 177: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

177

case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; }} break;

Exchanger State EMPTY

If reset failed, someone showed up after all, take that item

Page 178: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

178

case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; }} break;

Exchanger State EMPTY

Set slot to EMPTY with new timestamp and return item found

Page 179: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

179

case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; }} break;

Exchanger State EMPTY

If initial CAS failed then someone else changed state from EMPTY to WAITING so retry from start

Page 180: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

180

case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break;case BUSY: // others in middle of exchanging break;default: // impossible break;}

States WAITING and BUSY

Page 181: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

181

case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break;case BUSY: // others in middle of exchanging break;default: // impossible break;}

States WAITING and BUSY

Someone is waiting for an exchange, so try to CAS in my item in & change state to BUSY

Page 182: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

182

case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break;case BUSY: // others in middle of exchanging break;default: // impossible break;}

States WAITING and BUSY

If successful, return that item. Otherwise another thread got it.

Page 183: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

183

case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break;case BUSY: // others in middle of exchanging break;default: // impossible break;}

States WAITING and BUSY

Other threads using slot, so start over.

Page 184: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

184

The Exchanger Slot

• Exchanger is lock-free• Because the only way an exchange

can fail is if others repeatedly succeeded or no-one showed up

• The slot we need does not require symmetric exchange

Page 185: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

185

public class EliminationArray {…public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }}

Elimination Array

Page 186: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

186

public class EliminationArray {…public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }}

Elimination Array

Visit elimination array with value and range

Page 187: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

187

public class EliminationArray {…public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }}

Elimination Array

Pick a random array entry

Page 188: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

188

public class EliminationArray {…public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }}

Elimination Array

Exchange value or time out

Page 189: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

189

public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; }}

Elimination Stack Push

Page 190: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

190

public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; }}

Elimination Stack Push

First try to push

Page 191: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

191

public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; }}

Elimination Stack Push

If failed back-off to try to eliminate

Page 192: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

192

public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; }}

Elimination Stack Push

Value being pushed and range to try

Page 193: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

193

public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; }}

Elimination Stack Push

Only a pop inserts null value so elimination was successful

Page 194: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

194

public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; }}

Elimination Stack Push

Else retry push on lock-free stack

Page 195: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

195

public T pop() { ... while (true) { if (tryPop()) { return returnNode.value; } else try { T otherValue = eliminationArray.visit(null,policy.Range; if otherValue != null) { return otherValue; } }}}

Elimination Stack Pop

Page 196: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

196

public T pop() { ... while (true) { if (tryPop()) { return returnNode.value; } else try { T otherValue = eliminationArray.visit(null,policy.Range; if ( otherValue != null) { return otherValue; } }}}

Elimination Stack Pop

Like push, non-null value pushed by other thread, so elimination

succeeds

Page 197: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

197

Summary• We saw both lock-based and lock-

free implementations of queues and stacks

• Not every data structure that looks sequential is sequential.– Linearizable stack not inherently

sequential • ABA is a real problem, fear it.

Page 198: Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Art of Multiprocessor Programming© Herlihy-Shavit

2007

198

         This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.

• You are free:– to Share — to copy, distribute and transmit the work – to Remix — to adapt the work

• Under the following conditions:– Attribution. You must attribute the work to “The Art of

Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work).

– Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license.

• For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to– http://creativecommons.org/licenses/by-sa/3.0/.

• Any of the above conditions can be waived if you get permission from the copyright holder.

• Nothing in this license impairs or restricts the author's moral rights.