Linked Lists: Locking, Lock- Free, and Beyond … Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Modified by.
Post on 05-Jan-2016
221 Views
Preview:
Transcript
Linked Lists: Locking, Lock-Free, and Beyond …
Companion slides forThe Art of Multiprocessor Programming
by Maurice Herlihy & Nir Shavit
Modified by Rajeev Alur for CIS640 University of Pennsylvania
Art of Multiprocessor Programming 2
Today: Concurrent Objects
• Adding threads should not lower throughput– Contention effects– Mostly fixed by Queue locks
• Should increase throughput– Not possible if inherently sequential– Surprising things are parallelizable
Art of Multiprocessor Programming 3
Coarse-Grained Synchronization
• Each method locks the object– Avoid contention using queue locks
Art of Multiprocessor Programming 4
Coarse-Grained Synchronization
• Each method locks the object– Avoid contention using queue locks – Easy to reason about
• In simple cases
Art of Multiprocessor Programming 5
Coarse-Grained Synchronization
• Each method locks the object– Avoid contention using queue locks – Easy to reason about
• In simple cases
• So, are we done?
Art of Multiprocessor Programming 6
Coarse-Grained Synchronization
• Sequential bottleneck– Threads “stand in line”
Art of Multiprocessor Programming 7
Coarse-Grained Synchronization
• Sequential bottleneck– Threads “stand in line”
• Adding more threads– Does not improve throughput– Struggle to keep it from getting worse
Art of Multiprocessor Programming 8
Coarse-Grained Synchronization
• Sequential bottleneck– Threads “stand in line”
• Adding more threads– Does not improve throughput– Struggle to keep it from getting worse
• So why even use a multiprocessor?– Well, some apps inherently parallel …
Art of Multiprocessor Programming 9
This Lecture
• Introduce four “patterns”– Bag of tricks …– Methods that work more than once …
Art of Multiprocessor Programming 10
This Lecture
• Introduce four “patterns”– Bag of tricks …– Methods that work more than once …
• For highly-concurrent objects– Concurrent access– More threads, more throughput
Art of Multiprocessor Programming 11
First:Fine-Grained Synchronization• Instead of using a single lock …• Split object into
– Independently-synchronized components
• Methods conflict when they access– The same component …– At the same time
Art of Multiprocessor Programming 12
Second:Optimistic Synchronization
• Search without locking …• If you find it, lock and check …
– OK: we are done– Oops: start over
• Evaluation– Usually cheaper than locking, but– Mistakes are expensive
Art of Multiprocessor Programming 13
Third:Lazy Synchronization
• Postpone hard work• Removing components is tricky
– Logical removal• Mark component to be deleted
– Physical removal• Do what needs to be done
Art of Multiprocessor Programming 14
Fourth:Lock-Free Synchronization
• Don’t use locks at all– Use compareAndSet() & relatives …
• Advantages– No Scheduler Assumptions/Support
• Disadvantages– Complex– Sometimes high overhead
Art of Multiprocessor Programming 15
Linked List
• Illustrate these patterns …• Using a list-based Set
– Common application– Building block for other apps
Art of Multiprocessor Programming 16
Set Interface
• Unordered collection of items• No duplicates• Methods
– add(x) put x in set– remove(x) take x out of set– contains(x) tests if x in set
Art of Multiprocessor Programming 17
List-Based Sets
public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(T x);}
Art of Multiprocessor Programming 18
List-Based Sets
public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(T x);}
Add item to set
Art of Multiprocessor Programming 19
List-Based Sets
public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(Tt x);}
Remove item from set
Art of Multiprocessor Programming 20
List-Based Sets
public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(T x);}
Is item in set?
Art of Multiprocessor Programming 21
List Node
public class Node { public T item; public int key; public Node next;}
Art of Multiprocessor Programming 22
List Node
public class Node { public T item; public int key; public Node next;}
item of interest
Art of Multiprocessor Programming 23
List Node
public class Node { public T item; public int key; public Node next;}
Usually hash code
Art of Multiprocessor Programming 24
List Node
public class Node { public T item; public int key; public Node next;}
Reference to next node
Art of Multiprocessor Programming 25
The List-Based Set
a b c
Sorted with Sentinel nodes(min & max possible keys)
-∞
+∞
Art of Multiprocessor Programming 26
Invariants
• Sentinel nodes– tail reachable from head
• Sorted• No duplicates
Art of Multiprocessor Programming 27
Sequential List Based Set
a c d
a b c
Add()
Remove()
Art of Multiprocessor Programming 28
Sequential List Based Set
a c d
b
a b c
Add()
Remove()
Art of Multiprocessor Programming 29
Coarse Grained Locking
a b d
Art of Multiprocessor Programming 30
Coarse Grained Locking
a b d
c
Art of Multiprocessor Programming 31
honk!
Coarse Grained Locking
a b d
c
Simple but hotspot + bottleneck
honk!
Art of Multiprocessor Programming 32
Coarse-Grained Locking
• Easy, same as synchronized methods
– “One lock to rule them all …”
Art of Multiprocessor Programming 33
Coarse-Grained Locking
• Easy, same as synchronized methods
– “One lock to rule them all …”
• Simple, clearly correct– Deserves respect!
• Works poorly with contention– Queue locks help– But bottleneck still an issue
Art of Multiprocessor Programming 34
Fine-grained Locking
• Requires careful thought
• Split object into pieces– Each piece has own lock– Methods that work on disjoint pieces
need not exclude each other
Art of Multiprocessor Programming 35
Hand-over-Hand locking
a b c
Art of Multiprocessor Programming 36
Hand-over-Hand locking
a b c
Art of Multiprocessor Programming 37
Hand-over-Hand locking
a b c
Art of Multiprocessor Programming 38
Hand-over-Hand locking
a b c
Art of Multiprocessor Programming 39
Hand-over-Hand locking
a b c
Art of Multiprocessor Programming 40
Removing a Node
a b c d
remove(b)
Art of Multiprocessor Programming 41
Removing a Node
a b c d
remove(b)
Art of Multiprocessor Programming 42
Removing a Node
a b c d
remove(b)
Art of Multiprocessor Programming 43
Removing a Node
a b c d
remove(b)
Art of Multiprocessor Programming 44
Removing a Node
a b c d
remove(b)
Art of Multiprocessor Programming 45
Removing a Node
a c d
remove(b)
Why hold 2 locks?
Art of Multiprocessor Programming 46
Concurrent Removes
a b c d
remove(c)remove(
b)
Art of Multiprocessor Programming 47
Concurrent Removes
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 48
Concurrent Removes
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 49
Concurrent Removes
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 50
Concurrent Removes
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 51
Concurrent Removes
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 52
Concurrent Removes
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 53
Concurrent Removes
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 54
Concurrent Removes
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 55
Concurrent Removes
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 56
Uh, Oh
a c d
remove(b)
remove(c)
Art of Multiprocessor Programming 57
Uh, Oh
a c d
Bad news, c not removed
remove(b)
remove(c)
Art of Multiprocessor Programming 58
Problem
• To delete node c– Swing node b’s next field to d
• Problem is,– Someone deleting b concurrently could direct a pointer
to c
ba c
ba c
Art of Multiprocessor Programming 59
Insight
• If a node is locked– No one can delete node’s successor
• If a thread locks– Node to be deleted– And its predecessor– Then it works
Art of Multiprocessor Programming 60
Hand-Over-Hand Again
a b c d
remove(b)
Art of Multiprocessor Programming 61
Hand-Over-Hand Again
a b c d
remove(b)
Art of Multiprocessor Programming 62
Hand-Over-Hand Again
a b c d
remove(b)
Art of Multiprocessor Programming 63
Hand-Over-Hand Again
a b c d
remove(b)
Found it!
Art of Multiprocessor Programming 64
Hand-Over-Hand Again
a b c d
remove(b)
Found it!
Art of Multiprocessor Programming 65
Hand-Over-Hand Again
a c d
remove(b)
Art of Multiprocessor Programming 66
Removing a Node
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 67
Removing a Node
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 68
Removing a Node
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 69
Removing a Node
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 70
Removing a Node
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 71
Removing a Node
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 72
Removing a Node
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 73
Removing a Node
a b c d
remove(b)
remove(c)
Art of Multiprocessor Programming 74
Removing a Node
a b c d
Must acquire Lock of
b
remove(c)
Art of Multiprocessor Programming 75
Removing a Node
a b c d
Cannot acquire lock of b
remove(c)
Art of Multiprocessor Programming 76
Removing a Node
a b c d
Wait!
remove(c)
Art of Multiprocessor Programming 77
Removing a Node
a b d
Proceed to
remove(b)
Art of Multiprocessor Programming 78
Removing a Node
a b d
remove(b)
Art of Multiprocessor Programming 79
Removing a Node
a b d
remove(b)
Art of Multiprocessor Programming 80
Removing a Node
a d
remove(b)
Art of Multiprocessor Programming 81
Removing a Node
a d
Art of Multiprocessor Programming 82
Remove method
public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }}
Art of Multiprocessor Programming 83
Remove method
public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }}
Key used to order node
Art of Multiprocessor Programming 84
Remove method
public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { currNode.unlock(); predNode.unlock(); }}
Predecessor and current nodes
Art of Multiprocessor Programming 85
Remove method
public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }}
Make sure locks
released
Art of Multiprocessor Programming 86
Remove method
public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }}
Everything else
Art of Multiprocessor Programming 87
Remove method
try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); …} finally { … }
Art of Multiprocessor Programming 88
Remove method
try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); …} finally { … }
lock pred == head
Art of Multiprocessor Programming 89
Remove method
try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); …} finally { … }
Lock current
Art of Multiprocessor Programming 90
Remove method
try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); …} finally { … }
Traversing list
Art of Multiprocessor Programming 91
Remove: searching
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
Art of Multiprocessor Programming 92
Remove: searching
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
Search key range
Art of Multiprocessor Programming 93
Remove: searching
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
At start of each loop: curr and pred
locked
Art of Multiprocessor Programming 94
Remove: searching
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;If item found, remove
node
Art of Multiprocessor Programming 95
Remove: searching
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;If node found, remove
it
Art of Multiprocessor Programming 96
Remove: searching
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
Unlock predecessor
Art of Multiprocessor Programming 97
Remove: searching
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
Only one node locked!
Art of Multiprocessor Programming 98
Remove: searching
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
demote current
Art of Multiprocessor Programming 99
Remove: searching
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false;
Find and lock new current
Art of Multiprocessor Programming 100
Remove: searching
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false;
Lock invariant restored
Art of Multiprocessor Programming 101
Remove: searching
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
Otherwise, not present
Art of Multiprocessor Programming 103
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
Why remove() is linearizable
•pred reachable from head•curr is pred.next•So curr.item is in the set
Art of Multiprocessor Programming 104
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
Why remove() is linearizable
Linearization point ifitem is present
Art of Multiprocessor Programming 105
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
Why remove() is linearizable
Node locked, so no other thread can remove it ….
Art of Multiprocessor Programming 106
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
Why remove() is linearizable
Item not present
Art of Multiprocessor Programming 107
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
Why remove() is linearizable
•pred reachable from head•curr is pred.next•pred.key < key •key < curr.key
Art of Multiprocessor Programming 108
while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;
Why remove() is linearizable
Linearization point
Art of Multiprocessor Programming 109
Adding Nodes
• To add node e– Must lock predecessor– Must lock successor
• Neither can be deleted– (Is successor lock actually required?)
Art of Multiprocessor Programming 110
Drawbacks
• Better than coarse-grained lock– Threads can traverse in parallel
• Still not ideal– Long chain of acquire/release– Inefficient
Art of Multiprocessor Programming 111
Optimistic Synchronization
• Find nodes without locking• Lock nodes• Check that everything is OK
Art of Multiprocessor Programming 112
Optimistic: Traverse without Locking
b d ea
add(c)
Aha!
Art of Multiprocessor Programming 113
Optimistic: Lock and Load
b d ea
add(c)
Art of Multiprocessor Programming 114
Optimistic: Lock and Load
b d ea
add(c)
c
Art of Multiprocessor Programming 115
What could go wrong?
b d ea
add(c)
Aha!
Art of Multiprocessor Programming 116
What could go wrong?
b d ea
add(c)
Art of Multiprocessor Programming 117
What could go wrong?
b d ea
remove(b)
Art of Multiprocessor Programming 118
What could go wrong?
b d ea
remove(b)
Art of Multiprocessor Programming 119
What could go wrong?
b d ea
add(c)
Art of Multiprocessor Programming 120
What could go wrong?
b d ea
add(c)
c
Art of Multiprocessor Programming 121
What could go wrong?
d ea
add(c) Uh-oh
Art of Multiprocessor Programming 122
Validate – Part 1
b d ea
add(c)
Yes, b still
reachable from head
Art of Multiprocessor Programming 123
What Else Could Go Wrong?
b d ea
add(c)
Aha!
Art of Multiprocessor Programming 124
What Else Coould Go Wrong?
b d ea
add(c)
add(b’)
Art of Multiprocessor Programming 125
What Else Coould Go Wrong?
b d ea
add(c)
add(b’)b’
Art of Multiprocessor Programming 126
What Else Could Go Wrong?
b d ea
add(c)
b’
Art of Multiprocessor Programming 127
What Else Could Go Wrong?
b d ea
add(c)
c
Art of Multiprocessor Programming 128
Validate Part 2(while holding locks)
b d ea
add(c)
Yes, b still
points to d
Art of Multiprocessor Programming 129
Optimistic: Linearization Point
b d ea
add(c)
c
Art of Multiprocessor Programming 130
Correctness
• If– Nodes b and c both locked– Node b still accessible– Node c still successor to b
• Then– Neither will be deleted– OK to delete and return true
Art of Multiprocessor Programming 131
Unsuccessful Remove
a b d e
remove(c)
Aha!
Art of Multiprocessor Programming 132
Validate (1)
a b d e
Yes, b still reachable from head
remove(c)
Art of Multiprocessor Programming 133
Validate (2)
a b d e
remove(c)
Yes, b still points to
d
Art of Multiprocessor Programming 134
OK Computer
a b d e
remove(c)
return false
Art of Multiprocessor Programming 135
Correctness
• If– Nodes b and d both locked– Node b still accessible– Node d still successor to b
• Then– Neither will be deleted– No thread can add c after b– OK to return false
Art of Multiprocessor Programming 136
Validationprivate boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false;}
Art of Multiprocessor Programming 137
private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false;}
Validation
Predecessor & current
nodes
Art of Multiprocessor Programming 138
private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false;}
Validation
Begin at the beginning
Art of Multiprocessor Programming 139
private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false;}
Validation
Search range of keys
Art of Multiprocessor Programming 140
private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false;}
Validation
Predecessor reachable
Art of Multiprocessor Programming 141
private boolean validate(Node pred, Node curry) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false;}
Validation
Is current node next?
Art of Multiprocessor Programming 142
private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false;}
Validation
Otherwise move on
Art of Multiprocessor Programming 143
private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false;}
ValidationPredecessor not
reachable
Art of Multiprocessor Programming 144
Remove: searchingpublic boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } …
Art of Multiprocessor Programming 145
public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } …
Remove: searching
Search key
Art of Multiprocessor Programming 146
public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } …
Remove: searching
Retry on synchronization conflict
Art of Multiprocessor Programming 147
public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } …
Remove: searching
Examine predecessor and current nodes
Art of Multiprocessor Programming 148
public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } …
Remove: searching
Search by key
Art of Multiprocessor Programming 149
public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } …
Remove: searching
Stop if we find item
Art of Multiprocessor Programming 150
public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } …
Remove: searching
Move along
Art of Multiprocessor Programming 151
On Exit from Loop
• If item is present– curr holds item– pred just before curr
• If item is absent– curr has first higher key– pred just before curr
• Assuming no synchronization problems
Art of Multiprocessor Programming 152
Remove Methodtry { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally {
pred.unlock();curr.unlock();
}}}
Art of Multiprocessor Programming 153
try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally {
pred.unlock();curr.unlock();
}}}
Remove Method
Always unlock
Art of Multiprocessor Programming 154
try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally {
pred.unlock();curr.unlock();
}}}
Remove Method
Lock both nodes
Art of Multiprocessor Programming 155
try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally {
pred.unlock();curr.unlock();
}}}
Remove Method
Check for synchronization
conflicts
Art of Multiprocessor Programming 156
try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally {
pred.unlock();curr.unlock();
}}}
Remove Method
target found, remove node
Art of Multiprocessor Programming 157
try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally {
pred.unlock();curr.unlock();
}}}
Remove Method
target not found
Art of Multiprocessor Programming 158
Optimistic List
• Limited hot-spots– Targets of add(), remove(), contains()– No contention on traversals
• Moreover– Traversals are wait-free– Food for thought …
Art of Multiprocessor Programming 159
So Far, So Good
• Much less lock acquisition/release– Performance– Concurrency
• Problems– Need to traverse list twice– contains() method acquires locks
Art of Multiprocessor Programming 160
Evaluation
• Optimistic is effective if– cost of scanning twice without locks
is less than– cost of scanning once with locks
• Drawback– contains() acquires locks– 90% of calls in many apps
Art of Multiprocessor Programming 161
Lazy List
• Like optimistic, except– Scan once– contains(x) never locks …
• Key insight– Removing nodes causes trouble– Do it “lazily”
Art of Multiprocessor Programming 162
Lazy List
• remove()– Scans list (as before)– Locks predecessor & current (as before)
• Logical delete– Marks current node as removed (new!)
• Physical delete– Redirects predecessor’s next (as before)
Art of Multiprocessor Programming 163
Lazy Removal
aa b c d
c
Art of Multiprocessor Programming 164
Lazy Removal
aa b d
Present in list
c
Art of Multiprocessor Programming 165
Lazy Removal
aa b d
Logically deleted
Art of Multiprocessor Programming 166
Lazy Removal
aa b c d
Physically deleted
Art of Multiprocessor Programming 167
Lazy Removal
aa b d
Physically deleted
Art of Multiprocessor Programming 168
Lazy List
• All Methods– Scan through locked and marked
nodes– Removing a node doesn’t slow down
other method calls …
• Must still lock pred and curr nodes.
Art of Multiprocessor Programming 169
Validation
• No need to rescan list!• Check that pred is not marked• Check that curr is not marked• Check that pred points to curr
Art of Multiprocessor Programming 170
Business as Usual
a b c
Art of Multiprocessor Programming 171
Business as Usual
a b c
Art of Multiprocessor Programming 172
Business as Usual
a b c
Art of Multiprocessor Programming 173
Business as Usual
a b c
remove(b)
Art of Multiprocessor Programming 174
Business as Usual
a b c
a not marked
Art of Multiprocessor Programming 175
Business as Usual
a b c
a still points to b
Art of Multiprocessor Programming 176
Business as Usual
a b c
Logical
delete
Art of Multiprocessor Programming 177
Business as Usual
a b c
physical
delete
Art of Multiprocessor Programming 178
Business as Usual
a b c
Art of Multiprocessor Programming 179
Validationprivate boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); }
Art of Multiprocessor Programming 180
private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); }
List Validate Method
Predecessor not Logically removed
Art of Multiprocessor Programming 181
private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); }
List Validate Method
Current not Logically removed
Art of Multiprocessor Programming 182
private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); }
List Validate Method
Predecessor stillPoints to current
Art of Multiprocessor Programming 183
Removetry { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally {
pred.unlock();curr.unlock();
}}}
Art of Multiprocessor Programming 184
Removetry { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally {
pred.unlock();curr.unlock();
}}}
Validate as before
Art of Multiprocessor Programming 185
Removetry { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally {
pred.unlock();curr.unlock();
}}}
Key found
Art of Multiprocessor Programming 186
Removetry { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally {
pred.unlock();curr.unlock();
}}}
Logical remove
Art of Multiprocessor Programming 187
Removetry { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally {
pred.unlock();curr.unlock();
}}}
physical remove
Art of Multiprocessor Programming 188
Containspublic boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked;}
Art of Multiprocessor Programming 189
Containspublic boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked;}
Start at the head
Art of Multiprocessor Programming 190
Containspublic boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked;}
Search key range
Art of Multiprocessor Programming 191
Containspublic boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked;}
Traverse without locking(nodes may have been
removed)
Art of Multiprocessor Programming 192
Containspublic boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked;}
Present and undeleted?
Art of Multiprocessor Programming 193
Summary: Wait-free Contains
a 0 0 0a b c 0e1d
Use Mark bit + list ordering 1. Not marked in the set2. Marked or missing not in the set
Art of Multiprocessor Programming 194
Lazy List
a 0 0 0a b c 0e1d
Lazy add() and remove() + Wait-free contains()
Art of Multiprocessor Programming 195
Evaluation
• Good:– contains() doesn’t lock– In fact, its wait-free! – Good because typically high % contains()– Uncontended calls don’t re-traverse
• Bad– Contended add() and remove() calls do
re-traverse– Traffic jam if one thread delays
Art of Multiprocessor Programming 196
Traffic Jam
• Any concurrent data structure based on mutual exclusion has a weakness
• If one thread– Enters critical section– And “eats the big muffin”
• Cache miss, page fault, descheduled …
– Everyone else using that lock is stuck!– Need to trust the scheduler….
Art of Multiprocessor Programming 197
Reminder: Lock-Free Data Structures
• No matter what …– Guarantees minimal progress in any
execution– i.e. Some thread will always complete
a method call– Even if others halt at malicious times– Implies that implementation can’t use
locks
Art of Multiprocessor Programming 198
Lock-free Lists
• Next logical step– Wait-free contains()– lock-free add() and remove()
• Use only compareAndSet()– What could go wrong?
Art of Multiprocessor Programming 199
Remove Using CAS
a 0 0 0a b c 0e1c
Logical Removal =Set Mark Bit
PhysicalRemovalCAS pointer
Use CAS to verify pointer is correct
Not enough!
Art of Multiprocessor Programming 200
Problem…
a 0 0 0a b c 0e1c
Logical Removal =Set Mark Bit
PhysicalRemovalCAS
0dProblem: d not added to list…Must Prevent manipulation of removed node’s pointer
Node added BeforePhysical Removal CAS
Art of Multiprocessor Programming 201
The Solution: Combine Bit and Pointer
a 0 0 0a b c 0e1c
Logical Removal =Set Mark Bit
PhysicalRemovalCAS
0d
Mark-Bit and Pointerare CASed together(AtomicMarkableReference)
Fail CAS: Node not added after logical Removal
Art of Multiprocessor Programming 202
Solution
• Use AtomicMarkableReference• Atomically
– Swing reference and– Update flag
• Remove in two steps– Set mark bit in next field– Redirect predecessor’s pointer
Art of Multiprocessor Programming 203
Marking a Node
• AtomicMarkableReference class– Java.util.concurrent.atomic package
address F
mark bit
Reference
Art of Multiprocessor Programming 204
Extracting Reference & Mark
Public Object get(boolean[] marked);
Art of Multiprocessor Programming 205
Extracting Reference & Mark
Public Object get(boolean[] marked);
Returns reference
Returns mark at array index
0!
Art of Multiprocessor Programming 206
Extracting Mark Only
public boolean isMarked();
Value of mark
Art of Multiprocessor Programming 207
Changing State
Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark);
Art of Multiprocessor Programming 208
Changing State
Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark);
If this is the current reference
…
And this is the current mark …
Art of Multiprocessor Programming 209
Changing State
Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark);
…then change to this new reference …
… and this new mark
Art of Multiprocessor Programming 210
Changing State
public boolean attemptMark( Object expectedRef, boolean updateMark);
Art of Multiprocessor Programming 211
Changing State
public boolean attemptMark( Object expectedRef, boolean updateMark);
If this is the current reference …
Art of Multiprocessor Programming 212
Changing State
public boolean attemptMark( Object expectedRef, boolean updateMark);
.. then change to this new mark.
Art of Multiprocessor Programming 213
Removing a Node
a b c d
remove c
CAS
Art of Multiprocessor Programming 214
Removing a Node
a b d
remove b
remove c
cCASCAS
failed
Art of Multiprocessor Programming 215
Removing a Node
a b d
remove b
remove c
c
Art of Multiprocessor Programming 216
Removing a Node
a d
remove b
remove c
Art of Multiprocessor Programming 217
Traversing the List
• Q: what do you do when you find a “logically” deleted node in your path?
• A: finish the job.– CAS the predecessor’s next field– Proceed (repeat as needed)
Art of Multiprocessor Programming 218
Lock-Free Traversal(only Add and Remove)
a b c dCAS
Uh-oh
pred currpred curr
Art of Multiprocessor Programming 219
The Window Class
class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; }}
Art of Multiprocessor Programming 220
The Window Class
class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; }}
A container for pred and current
values
Art of Multiprocessor Programming 221
Using the Find Method
Window window = find(head, key); Node pred = window.pred; curr = window.curr;
Art of Multiprocessor Programming 222
Using the Find Method
Window window = find(head, key); Node pred = window.pred; curr = window.curr;
Find returns window
Art of Multiprocessor Programming 223
Using the Find Method
Window window = find(head, key); Node pred = window.pred; curr = window.curr;
Extract pred and curr
Art of Multiprocessor Programming© Herlihy-Shavit 2007
224
The Find Method
Window window = find(item);
At some instant,
pred curr succ
itemor …
Art of Multiprocessor Programming© Herlihy-Shavit 2007
225
The Find Method
Window window = find(item);
At some instant,
predcurr= null
succ
item not in list
Art of Multiprocessor Programming 226
Removepublic boolean remove(T item) {Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true;}}}
Art of Multiprocessor Programming 227
Removepublic boolean remove(T item) {Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true;}}}
Keep trying
Art of Multiprocessor Programming 228
Removepublic boolean remove(T item) {Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true;}}} Find neighbors
Art of Multiprocessor Programming 229
Removepublic boolean remove(T item) {Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true;}}} She’s not there …
Art of Multiprocessor Programming 230
Removepublic boolean remove(T item) {Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true;}}}
Try to mark node as deleted
Art of Multiprocessor Programming 231
Removepublic boolean remove(T item) {Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true;}}}
If it doesn’t work, just retry, if it does, job
essentially done
Art of Multiprocessor Programming 232
Removepublic boolean remove(T item) {Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true;}}}
Try to advance reference(if we don’t succeed, someone else did or will).
a
Art of Multiprocessor Programming 233
Addpublic boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;}}}}
Art of Multiprocessor Programming 234
Addpublic boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;}}}} Item already there.
Art of Multiprocessor Programming 235
Addpublic boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;}}}}
create new node
Art of Multiprocessor Programming 236
Addpublic boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;}}}}
Install new node, else retry loop
Art of Multiprocessor Programming 237
Wait-free Contains
public boolean contains(Tt item) { boolean marked; int key = item.hashCode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) }
Art of Multiprocessor Programming 238
Wait-free Contains
public boolean contains(T item) { boolean marked; int key = item.hashCode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) }
Only diff is that we get and check
marked
Art of Multiprocessor Programming 239
Lock-free Findpublic Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; }}}
Art of Multiprocessor Programming 240
Lock-free Findpublic Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; }}}
If list changes
while traversed, start overLock-Free
because we start over
only if someone
else makes progress
Art of Multiprocessor Programming 241
public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; }}}
Lock-free Find
Start looking from head
Art of Multiprocessor Programming 242
public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; }}}
Lock-free Find
Move down the list
Art of Multiprocessor Programming 243
public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; }}}
Lock-free Find
Get ref to successor and current deleted bit
Art of Multiprocessor Programming 244
public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; }}}
Lock-free Find
Try to remove deleted nodes in path…code details
soon
Art of Multiprocessor Programming 245
public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; }}}
Lock-free Find
If curr key that is greater or equal, return pred and
curr
Art of Multiprocessor Programming 246
public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; }}}
Lock-free Find
Otherwise advance window and loop again
Art of Multiprocessor Programming 247
Lock-free Find
retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); }…
Art of Multiprocessor Programming 248
Lock-free Find
retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); }…
Try to snip out node
Art of Multiprocessor Programming 249
Lock-free Find
retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); }…
if predecessor’s next field changed must
retry whole traversal
Art of Multiprocessor Programming 250
Lock-free Find
retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); }…
Otherwise move on to check if next node
deleted
Art of Multiprocessor Programming 251
Performance
On 16 node shared memory machineBenchmark throughput of Java List-based Setalgs. Vary % of Contains() method Calls.
Art of Multiprocessor Programming 252
High Contains Ratio
Lock-free Lazy list
Course GrainedFine Lock-coupling
Art of Multiprocessor Programming 253
Low Contains Ratio
Lock-free
Lazy list
Course GrainedFine Lock-coupling
Art of Multiprocessor Programming 254
As Contains Ratio Increases
Lock-free
Lazy list
Course GrainedFine Lock-coupling
% Contains()
Art of Multiprocessor Programming 255
Summary
• Coarse-grained locking• Fine-grained locking• Optimistic synchronization• Lock-free synchronization
top related