Top Banner
Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories
30

Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

Jan 17, 2016

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

Skiplist-based Concurrent Priority Queues

Itay LotanStanford University

Nir ShavitSun Microsystems Laboratories

Page 2: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

2

Priority Queues For Large-Scale MultiProcessor Machines

• Priority Queue – A data structure that allows n asynchronous processes to insert an item, or delete the item with highest priority

• Large Scale Machine – Hundreds of Processors– Usually different architecture than small scale

machines

Page 3: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

3

Fixed Set Priority Queue

• A priority queue with a fixed and predetermined set of priorities

• Used by operating systems to distribute CPU time.

• There is a scalable solution - Funnel Tree Funnel Tree [Shavit, Zemach][Shavit, Zemach]

Page 4: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

4

General Priority Queue

• Supports an unlimited range of priorities

• Between any two priorities may be another

• Useful for – Heuristic searches– Graph searches

• Best solution – HeapHeap algorithm of Hunt et Hunt et alal

• Works better than binary search treesbinary search trees

Page 5: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

5

Heaps [Hunt et alHunt et al]

Page 6: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

6

Scalability Problem of Heaps

Page 7: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

7

New Approach - SkipQueues

• Based on Skip ListSkip List of PughPugh

• All locking is distributed and local

• Balancing of structure is probabilistic

• Deletions evenly distributed, minimal contention

• No need to pre-allocate memory

• All operations in expected logarithmic time

Page 8: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

8

Coming up Next …

1. The inside story on concurrent Skip ListsSkip Lists [PughPugh]

2. How to make a SkipQueueSkipQueue

3. Experimental Results

Page 9: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

9

Skip List Structure [Pugh]

Page 10: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

10

Inserting an Element

• InsertInsert (key_t k):– Randomly choose a level l – Find at each level of the list up to l the largest

key smaller than k– For each level from 1 to l

• Acquire a lock on the item found before

• Insert the new key after it

• Release lock

Page 11: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

11

Skip List - InsertionNew Item

Page 12: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

12

Deleting an Element

• DeleteDelete (key_t k)– Find k at all levels at which it appears.– For each level from l to 1

• Acquire locks on k and the item before it.

• Remove k

• Release locks

Page 13: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

13

Skip List - Deletion

Page 14: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

14

SkipQueues

• Problem: How to allow Delete_MinDelete_Min operations without creating a bottle neck around the first element?

Page 15: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

15

Our Solution

• Observation - The lowest level of the Skip Skip ListList contains all the elements in ascending order

• Processes can advance down this list and “logically” delete the first available key

• Each process can then delete the key that it previously deleted “logically”

Page 16: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

16

Delete_Min Operation

• Traverse the bottom level of the Skip ListSkip List structure

• For each traversed Item– Try to SWAP its Deleted flag

– If it was not already deleted, return the key of the item

– Else, go on to next item

• If at the end of the list, return EMPTY• No locking at all

Page 17: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

17

From Skip List to SkipQueue

Page 18: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

18

Performance Benchmarks

• We compared the performance of 3 structures:

1. The HeapHeap structure of Hunt et alHunt et al

2.2. FunnelListFunnelList – a simple linked list, access to which is governed by a combining-funnelcombining-funnel [Shavit, ZemachShavit, Zemach] structure

3.3. SkipQueueSkipQueue structure as described before

Page 19: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

19

Benchmark Methodology

• We used the ProteusProteus multiprocessor simulator by Brewer et alBrewer et al

• We simulated a 256 processors machine similar to the MIT Alewife

• Processors alternate between performing small amount of work and accessing the queue.

Page 20: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

20

Benchmark Methodology Cont’d

• Processors randomly choose whether to insert or delete

• Priorities of inserted items are chosen uniformly at random

• We measured the average latency of InsertInsert and Delete_MinDelete_Min operations

Page 21: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

21

Small Structure - Deletions

Page 22: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

22

Small Structure - Insertions

Page 23: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

23

Large Structure - Deletions

Page 24: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

24

Large Structure - Insertions

Page 25: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

25

Large Structure Benchmark With 70% Deletions - Deletions

Page 26: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

26

Large Structure Benchmark With 70% Deletions - Insertions

Page 27: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

27

Conclusions

• SkipQueuesSkipQueues scale significantly better than HeapsHeaps

• SkipQueuesSkipQueues are highly distributed – no hot spots or bottle necks

• Deletes are 3 times faster and Inserts are 10 times faster when concurrency reaches 256 processors

• Future Directions– Implementation without locks– Experimenting with other data structures

Page 28: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

28

Delete_Min Cont’d

• In order to assure a stronger ordering property we added the following:– Each item is time–stamped when its insertion is

completed– Each Delete_MinDelete_Min operation notes the time at

which it begins– The Delete_MinDelete_Min operation will only try to

“logically” delete items that were inserted before it started.

Page 29: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

29

Serialization Order

• Each successful Delete_MinDelete_Min is ordered at the time its “logical” deletion was completed

• Each unsuccessful Delete_MinDelete_Min that returned EMPTYEMPTY is ordered at the time of its return instruction

• Each uncompleted Delete_MinDelete_Min is ordered at the time of its invocation

Page 30: Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories.

30

SkipQueue Specification

• For every Delete_MinDelete_Min operation– Let I be the set of keys inserted by Insert

operations serialized before it– Let D be the set of keys removed by

Delete_MinDelete_Min operations serialized before it

• The returned value is the smallest in the set I – D, or EMPTYEMPTY if I – D is empty.