Art of Multiprocessor Programming
1
Concurrent Queues
Companion slides forThe Art of Multiprocessor Programming
by Maurice Herlihy & Nir Shavit
Modified by Rajeev Alur For CIS 640 University of Pennsylvania
Art of Multiprocessor Programming
2Art of Multiprocessor Programming
2
Filterclass Filter implements Lock { int[] level; // level[i] for thread i int[] victim; // victim[L] for level L
public Filter(int n) { level = new int[n]; victim = new int[n]; for (int i = 1; i < n; i++) { level[i] = 0; }}…
}
level
victim
n-1
n-1
0
1
0 0 0 0 0 04
2
2
Thread 2 at level 4
0
4
Art of Multiprocessor Programming
3Art of Multiprocessor Programming
3
Filterclass Filter implements Lock { …
public void lock(){ for (int L = 1; L < n; L++) { level[i] = L; victim[L] = i;
while ((k != i level[k] >= L) && victim[L] == i ); }} public void unlock() { level[i] = 0; }}
Art of Multiprocessor Programming
44
Queues & Stacks
• pool of items
Art of Multiprocessor Programming
55
Queues
deq()/ enq( )
Total orderFirst in
First out
Art of Multiprocessor Programming
66
Stacls
pop()/
push( )
Total orderLast in
First out
Art of Multiprocessor Programming
77
Bounded
• Fixed capacity• Good when resources an issue
Art of Multiprocessor Programming
88
Unbounded
• Unlimited capacity• Often more convenient
…
Art of Multiprocessor Programming
9
Blockingzzz
…Block on attempt to remove from empty
stack or queue
Art of Multiprocessor Programming
10
Blockingzzz
…Block on attempt to add to full bounded stack or
queue
Art of Multiprocessor Programming
11
Non-BlockingOuch
! Throw exception on attempt to remove
from empty stack or queue
Art of Multiprocessor Programming
1212
Queue: Concurrency
enq(x) y=deq()
enq() and deq() work at
different ends of the object
tail head
Art of Multiprocessor Programming
1313
Concurrency
enq(x)
Challenge: what if the queue is empty or full?
y=deq()ta
ilhead
Art of Multiprocessor Programming
1414
Bounded Queue
Sentinel
head
tail
Art of Multiprocessor Programming
1515
Bounded Queue
head
tail
First actual item
Art of Multiprocessor Programming
1616
Bounded Queue
head
tail
Lock out other deq() calls
deqLock
Art of Multiprocessor Programming
1717
Bounded Queue
head
tail
Lock out other enq() calls
deqLock
enqLock
Art of Multiprocessor Programming
18Art of Multiprocessor Programming
18
Not Done Yet
head
tail
deqLock
enqLock
Need to tell whether queue is
full or empty
Art of Multiprocessor Programming
1919
Not Done Yet
head
tail
deqLock
enqLock
Permission to enqueue 8 items
permits
8
Art of Multiprocessor Programming
2020
Not Done Yet
head
tail
deqLock
enqLock
Incremented by deq()Decremented by enq()
permits
8
Art of Multiprocessor Programming
2121
Enqueuer
head
tail
deqLock
enqLock
permits
8
Lock enqLock
Art of Multiprocessor Programming
2222
Enqueuer
head
tail
deqLock
enqLock
permits
8
Read permits
OK
Art of Multiprocessor Programming
2323
Enqueuer
head
tail
deqLock
enqLock
permits
8
No need to lock tail
Art of Multiprocessor Programming
2424
Enqueuer
head
tail
deqLock
enqLock
permits
8
Enqueue Node
Art of Multiprocessor Programming
2525
Enqueuer
head
tail
deqLock
enqLock
permits
87
getAndDecrement()
Art of Multiprocessor Programming
2626
Enqueuer
head
tail
deqLock
enqLock
permits
8 Release lock7
Art of Multiprocessor Programming
2727
Enqueuer
head
tail
deqLock
enqLock
permits
7
If queue was empty, notify waiting
dequeuers
Art of Multiprocessor Programming
2828
Unsuccesful Enqueuer
head
tail
deqLock
enqLock
permits
0
Uh-oh
Read permits
Art of Multiprocessor Programming
2929
Dequeuer
head
tail
deqLock
enqLock
permits
8
Lock deqLock
Art of Multiprocessor Programming
3030
Dequeuer
head
tail
deqLock
enqLock
permits
7
Read sentinel’s next field
OK
Art of Multiprocessor Programming
3131
Dequeuer
head
tail
deqLock
enqLock
permits
7
Read value
Art of Multiprocessor Programming
3232
Dequeuer
head
tail
deqLock
enqLock
permits
7
Make first Node new sentinel
Art of Multiprocessor Programming
3333
Dequeuer
head
tail
deqLock
enqLock
permits
8
Increment permits
Art of Multiprocessor Programming
3434
Dequeuer
head
tail
deqLock
enqLock
permits
7Release deqLock
Art of Multiprocessor Programming
3535
Unsuccesful Dequeuer
head
tail
deqLock
enqLock
permits
8
Read sentinel’s next field
uh-oh
Art of Multiprocessor Programming
3636
Bounded Queue
public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}
Art of Multiprocessor Programming
3737
Bounded Queue
public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}
Enq & deq locks
Art of Multiprocessor Programming
3838
Digression: Monitor Locks
• Java Synchronized objects and Java ReentrantLocks are monitors
• Allow blocking on a condition rather than spinning
• Threads: – acquire and release lock– wait on a condition
Art of Multiprocessor Programming
3939
public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}
The Java Lock Interface
Acquire lock
Art of Multiprocessor Programming
4040
public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}
The Java Lock Interface
Release lock
Art of Multiprocessor Programming
4141
public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}
The Java Lock Interface
Try for lock, but not too hard
Art of Multiprocessor Programming
4242
public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}
The Java Lock Interface
Create condition to wait on
Art of Multiprocessor Programming
4343
The Java Lock Interface
public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}
Guess what this method does?
Art of Multiprocessor Programming
4444
Lock Conditions
public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }
Art of Multiprocessor Programming
4545
public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }
Lock Conditions
Release lock and wait on condition
Art of Multiprocessor Programming
4646
public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }
Lock Conditions
Wake up one waiting thread
Art of Multiprocessor Programming
4747
public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }
Lock Conditions
Wake up all waiting threads
Art of Multiprocessor Programming
4848
Await
• Releases lock associated with q• Sleeps (gives up processor)• Awakens (resumes running)• Reacquires lock & returns
q.await()
Art of Multiprocessor Programming
4949
Signal
• Awakens one waiting thread– Which will reacquire lock
q.signal();
Art of Multiprocessor Programming
5050
Signal All
• Awakens all waiting threads– Which will each reacquire lock
q.signalAll();
Art of Multiprocessor Programming
5151
A Monitor Lock
Cri
tical S
ecti
on
waiting roomLock(
)
unLock()
Art of Multiprocessor Programming
5252
Unsuccessful Deq
Cri
tical S
ecti
on
waiting roomLock
()
await()
Deq()
Oh no, Empty!
Art of Multiprocessor Programming
5353
Another One
Cri
tical S
ecti
on
waiting roomLock
()
await()
Deq()
Oh no, Empty!
Art of Multiprocessor Programming
5454
Enqueur to the Rescue
Cri
tical S
ecti
on
waiting roomLock(
)
signalAll()
Enq( )
unLock()
Yawn!Yawn!
Art of Multiprocessor Programming
5555
Yawn!
Monitor Signalling
Cri
tical S
ecti
on
waiting room
Yawn!
Awakend thread might still lose lock to outside contender…
Art of Multiprocessor Programming
5656
Dequeurs Signalled
Cri
tical S
ecti
on
waiting room
Found it
Yawn!
Art of Multiprocessor Programming
5757
The Bounded Queue
public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}
Art of Multiprocessor Programming
5858
Bounded Queue Fields
public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}
Enq & deq locks
Art of Multiprocessor Programming
5959
Bounded Queue Fields
public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}
Enq lock’s associated condition
Art of Multiprocessor Programming
6060
Bounded Queue Fields
public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}
Num permits: 0 to capacity
Art of Multiprocessor Programming
6161
Bounded Queue Fields
public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}
Head and Tail
Art of Multiprocessor Programming
6262
Enq Method Part Onepublic void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}
Art of Multiprocessor Programming
6363
public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}
Enq Method Part One
Lock and unlock enq
lock
Art of Multiprocessor Programming
6464
public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}
Enq Method Part One
If queue is full, patiently await further
instructions …
Art of Multiprocessor Programming
6565
public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}
Be Afraid
How do we know the permits field won’t
change?
Art of Multiprocessor Programming
6666
public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}
Enq Method Part One
Add new node
Art of Multiprocessor Programming
6767
public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}
Enq Method Part One
If queue was empty, wake frustrated dequeuers
Art of Multiprocessor Programming
6868
Enq Method Part Deux
public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }
Art of Multiprocessor Programming
6969
Enq Method Part Deux
public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }Are there dequeuers to be signaled?
Art of Multiprocessor Programming
7070
public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }
Enq Method Part Deux
Lock and unlock deq
lock
Art of Multiprocessor Programming
7171
public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }
Enq Method Part Deux
Signal dequeuers that queue no longer
empty
Art of Multiprocessor Programming
7272
The Enq() & Deq() Methods
• Share no locks– That’s good
• But do share an atomic counter– Accessed on every method call– That’s not so good
• Can we alleviate this bottleneck?– By splitting the counter
Art of Multiprocessor Programming
7373
A Lock-Free Queue (Michael&Scott)
Sentinel
head
tail
Art of Multiprocessor Programming
7474
Compare and Set
CAS
Art of Multiprocessor Programming
7575
Enqueue
head
tail
Enq( )
Art of Multiprocessor Programming
7676
Enqueue
head
tail
Art of Multiprocessor Programming
7777
Logical Enqueue
head
tail
CAS
Art of Multiprocessor Programming
7878
Physical Enqueue
head
tail
Enqueue Node
CAS
Art of Multiprocessor Programming
7979
Enqueue
• These two steps are not atomic• The tail field refers to either
– Actual last Node (good)– Penultimate Node (not so good)
• Be prepared!
Art of Multiprocessor Programming
8080
Enqueue
• What do you do if you find– A trailing tail?
• Stop and help fix it– If tail node has non-null next field– CAS the queue’s tail field to tail.next
Art of Multiprocessor Programming
8181
When CASs Fail
• During logical enqueue– Abandon hope, restart– Still lock-free (why?)
• During physical enqueue– Ignore it (why?)
Art of Multiprocessor Programming
8282
Dequeuer
head
tail
Read value
Art of Multiprocessor Programming
8383
Dequeuer
head
tail
Make first Node new sentinel
CAS
Art of Multiprocessor Programming
8484
Enq Method public void enq(T x) { Node node = new Node(x); while (true) { Node last = tail.get(); Node next = last.next.get(); if (last == tail.get()) { if (next == null) { if (last.next.compareAndSet(next,node)) { tail.compareAndSet(last, node); return; } } else { tail.compareAndSet(last, next); } } }
Art of Multiprocessor Programming
8585
Memory Reuse?
• What do we do with nodes after we dequeue them?
• Java: let garbage collector deal?• Suppose there is no GC, or we
prefer not to use it?
Art of Multiprocessor Programming
8686
Dequeuer
head
tail
CAS
Can recycle
Art of Multiprocessor Programming
8787
Simple Solution
• Each thread has a free list of unused queue nodes
• Allocate node: pop from list• Free node: push onto list• Deal with underflow somehow …
Art of Multiprocessor Programming
8888
Why Recycling is Hard
Free pool
head tail
Want to redirect
tailfrom
grey to red
zzz…
Art of Multiprocessor Programming
8989
Both Nodes Reclaimed
Free pool
zzz
head tail
Art of Multiprocessor Programming
9090
One Node Recycled
Free pool
Yawn!
head tail
Art of Multiprocessor Programming
9191
Why Recycling is Hard
Free pool
CAShead tail
OK, here I go!
Art of Multiprocessor Programming
9292
Epic FAIL
Free pool
zOMG what went wrong?
head tail
Bad news
Art of Multiprocessor Programming
9393
The Dreaded ABA Problem
Head pointer has value AThread reads value A
head tail
Art of Multiprocessor Programming
9494
Dreaded ABA continued
zzz Head pointer has value BNode A freed
head tail
Art of Multiprocessor Programming
9595
Dreaded ABA continued
Yawn! Head pointer has value A againNode A recycled & reinitialized
head tail
Art of Multiprocessor Programming
9696
Dreaded ABA continued
CAS succeeds because pointer matcheseven though pointer’s meaning has changed
CAShead tail
Art of Multiprocessor Programming
9797
The Dreaded ABA Problem
• Is a result of CAS() semantics– blame Sun, Intel, AMD, …
• Not with Load-Locked/Store-Conditional– Good for IBM?
Art of Multiprocessor Programming
9898
Dreaded ABA – A Solution
• Tag each pointer with a counter• Unique over lifetime of node• Pointer size vs word size issues• Overflow?
– Don’t worry be happy?– Bounded tags?
• AtomicStampedReference class
Art of Multiprocessor Programming
9999
Atomic Stamped Reference
• AtomicStampedReference class– Java.util.concurrent.atomic package
address S
Stamp
Reference
Can get reference and stamp atomically