שירן חליבה Concurrent Queues. Outline: Some definitions 3 queue implementations : A Bounded Partial Queue An Unbounded Total Queue An Unbounded Lock-Free.

שירן חליבה

Concurrent Queues

Outline:Some definitions

3 queue implementations :

A Bounded Partial Queue

An Unbounded Total Queue

An Unbounded Lock-Free Queue

Introduction and some definitions :

Pools show up in many places in concurrent systems. For example, in many applications, one or more producer threads produce items to be consumed by one or more consumer threads.

To allow consumers to keep up, we can place a buffer between the producers and the consumers.

Often, pools act as producer–consumer buffers.A pool allows the same item to appear more

than once.

Introduction and some definitions cont .

A queue is a special kind of pool with FIFO fairness.

It provides an enq(x) method that puts item x at one end of the queue, called the tail, and a deq() method that removes and returns the item at the other end of the queue, called the head.

Bounded vs. Unbounded

A pool can be bounded or unbounded.Bounded

Fixed capacityGood when resources an issue

UnboundedHolds any number of objects

Blocking vs. Non-BlockingProblem cases:

Removing from empty poolAdding to full (bounded) pool

BlockingCaller waits until state changes

Non-BlockingMethod throws exception

Total vs. Partial Pool methods may be total or partial.A method is total if calls do not wait for

certain conditions to become true. For example, a get() call that tries to remove an item from an empty pool immediately returns a failure code or throws an exception. A total interface makes sense when the producer (or consumer) thread has something better to do than wait for the method call to take effect.

Total vs. Partial A method is partial if calls may wait for

conditions to hold. For example, a partial get() call that tries to remove an item from an empty pool blocks until an item is available to return. A partial interface makes sense when the producer (or consumer) has nothing better to do than to wait for the pool to become nonempty (or non full).

Queue: Concurrency

enq(x) y=deq)(

enq)( and deq)( work at

different ends of the object

tail head

Concurrency

enq(x)

Challenge: what if the queue is

empty or full?

y=deq)(ta

ilhead

A Bounded Partial Queue

head

tail

deqLock

enqLock

Permission to enqueue 8 items

permits

8

Lock out other enq() calls

Lock out other deq() calls

First actual itemSentinel

Enqueuer

head

tail

deqLock

enqLock

permits

8

Lock enqLockRead permits

OK

No need to lock tail?

Enqueuer

head

tail

deqLock

enqLock

permits

8

Enqueue Node

7

getAndDecrement)()Why atomic?(

Enqueuer

head

tail

deqLock

enqLock

permits

8 Release lock7

If queue was empty, notify waiting

dequeuers

Unsuccesful Enqueuer

head

tail

deqLock

enqLock

permits

0Uh-oh

Read permits

Dequeuer

head

tail

deqLock

enqLock

permits

7

Lock deqLockRead sentinel’s

next field

OK

Dequeuer

head

tail

deqLock

enqLock

permits

7

Read value

Dequeuer

head

tail

deqLock

enqLock

permits

7

Make first Node new sentinel

Dequeuer

head

tail

deqLock

enqLock

permits

7Release deqLock

Dequeuer

head

tail

deqLock

enqLock

permits

8

Increment permits(no need

lock?)Answer: we had to hold the lock while enqueuing to prevent lots of enqueuers from proceeding without noticing that the capacity had been exceeded. Dequeuers will notice the queue is empty when they observe that the sentinel’s next field is null

Unsuccesful Dequeuer

head

tail

deqLock

enqLock

permits

8

Read sentinel’s next field

uh-oh

Bounded Queue

public class BoundedQueue<T{ > ReentrantLock enqLock, deqLock;

Condition notEmptyCondition, notFullCondition; AtomicInteger permits;

Node head ; Node tail ;

int capacity; enqLock = new ReentrantLock;)(

notFullCondition = enqLock.newCondition;)( deqLock = new ReentrantLock;)(

notEmptyCondition = deqLock.newCondition;)(}

The ReentrantLock is a monitor (The mechanism that Java uses to

support synchronization ). Allows blocking on a condition rather than spinning.

How do we use it?

(*More on monitors: http://www.artima.com/insidejvm/ed2/threadsync

h.html)

Lock Conditions

public interface Condition{ void await;)(

boolean await(long time, TimeUnit unit);…

void signal ;)( void signalAll;)(

}

Await

Releases lock associated with qSleeps (gives up processor)Awakens (resumes running)Reacquires lock & returns

q.await()

Signal

Awakens one waiting threadWhich will reacquire lock

q.signal();

A Monitor Lock

Cri

tical S

ecti

on

waiting room

Lock()

unLock()

Unsuccessful Deq

Cri

tical S

ecti

on

waiting room

Lock)(

await)(

Deq)(

Oh no, Empty!

Another One

Cri

tical S

ecti

on

waiting room

Lock)(

await)(

Deq)(

Oh no, Empty!

Enqueur to the Rescue

Cri

tical S

ecti

on

waiting room

Lock()

signalAll()Enq) (

unLock()

Yawn!Yawn!

Yawn!

Monitor Signalling

Cri

tical S

ecti

on

waiting room

Yawn!

Awakend thread might still lose lock to

outside contender …

Dequeurs Signalled

Cri

tical S

ecti

on

waiting room

Found it

Yawn!

Yawn!

Dequeurs Signalled

Cri

tical S

ecti

on

waiting room

Still empty!

Dollar Short + Day Late

Cri

tical S

ecti

on

waiting room

Why not signal)(?

Lost Wake-Up

Cri

tical S

ecti

on

waiting room

Lock()

signal() Enq) (

unLock()

Yawn!

Lost Wake-Up

Cri

tical S

ecti

on

waiting room

Lock()

Enq) (

unLock()

Yawn!

Lost Wake-Up

Cri

tical S

ecti

on

waiting room

Yawn!

Lost Wake-Up

Cri

tical S

ecti

on

waiting room

Found it

What’s Wrong Here?

Cri

tical S

ecti

on

waiting room

zzzz!.…

Enq Methodpublic void enq(T x){

boolean mustWakeDequeuers = false ; enqLock.lock;)(

try { while (permits.get() == 0)

notFullCondition.await ;)( Node e = new Node(x);

tail.next = e; tail = e;

if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true;

} finally{ enqLock.unlock;)(

} … }

Cont…

public void enq(T x){ …

if (mustWakeDequeuers){ deqLock.lock;)(

try{ notEmptyCondition.signalAll;)(

} finally{ deqLock.unlock;)(

} } }

The Enq() & Deq() MethodsShare no locks

That’s goodBut do share an atomic counter

Accessed on every method callThat’s not so good

Can we alleviate this bottleneck?

What is the problem?

Split the CounterThe enq)( method

Decrements onlyCares only if value is zero

The deq)( methodIncrements onlyCares only if value is capacity

Split CounterEnqueuer decrements enqSidePermitsDequeuer increments deqSidePermitsWhen enqueuer runs out

Locks deqLockTransfers permits(dequeuer doesn't need permits- check

head.next)Intermittent(תקופתי) synchronization

Not with each method callNeed both locks! (careful …)

An Unbounded Total Queue

Queue can hold an unbounded number of items.

The enq() method always enqueues its item.

The deq() throws EmptyException if there is no item to dequeue.

No deadlock- each method acquires only one lock.

Both the enq() and deq() methods are total as they do not wait for the queue to become empty or full.

An Unbounded Total Queue

A Lock-Free Queue

Sentinelhead

tail

•Extension of the unbounded total queue•Quicker threads help the slower threads•Each node’s next field is an: AtomicReference<Node>•The queue itself consists of two AtomicReference<Node> fields: head and tail

Compare and Set

CAS

LockFreeQueue<T> class

Enqueue

head

tail

Enq) (

Enqueue

head

tail

Logical Enqueue

head

tail

CAS

Physical Enqueue

head

tail

Enqueue Node

CAS

EnqueueThese two steps are not atomicThe tail field refers to either

Actual last Node (good)Penultimate* Node (not so good)

Be prepared!

(*Penultimate :next to the last)

EnqueueWhat do you do if you find

A trailing tail?Stop and fix it

If tail node has non-null next fieldCAS the queue’s tail field to tail.next

When CASs FailDuring logical enqueue

Abandon hope, restartDuring physical enqueue

Ignore it (why?)

LockFreeQueue<T> class

Enq()Creates a new node with the new value to be enqueuedreads tail, and finds the node that appears to be lastchecks whether that node has a successorIf not - appends the new node by calling compareAndSet()If the compareAndSet() succeeds, the thread uses a

second compareAndSet() to advance tail to the new nodesecond compareAndSet() call fails, the thread can still

return successfullyIf the tail node has a successor , then the method tries to

“help”other threads by advancing tail to refer directly to the successor before trying again to insert its own node.

Dequeuer

head

tail

Read value

Dequeuer

head

tail

Make first Node new sentinel

CAS

What is the problem here?

LockFreeQueue<T> class

Deq()If the queue is nonempty(the next field of the

head node is not null), the dequeuer calls compareAndSet() to change head from the sentinel node to its successor

before advancing head one must make sure that tail is not left referring to the sentinel node which is about to be removed from the queue

test: if head equals tail and the (sentinel) node they refer to has a non-null next field, then the tail is deemed to be lagging behind.

deq() then attempts to help make tail consistent by swinging it to the sentinel node’s successor , and only then updates head to remove the sentinel

SummaryA thread fails to enqueue or dequeue a

node only if another thread’s method call succeeds in changing the reference, so some method call always completes.

As it turns out, being lock-free substantially enhances the performance of queue implementations, and the lock-free algorithms tend to outperform the most efficient blocking ones.