1 ΔΟΜΕΣ ΔΕΔΟΜΕΝΩΝ & ΑΡΧΕΙΩΝ Priority Queues (ουρές προτεραιότητας) HEAPS (σωροί) – binary heaps HeapSort (ταξινόμηση σωρού)
Jan 19, 2016
1
ΔΟΜΕΣ ΔΕΔΟΜΕΝΩΝ & ΑΡΧΕΙΩΝ
Priority Queues (ουρές
προτεραιότητας)
HEAPS (σωροί) – binary heaps
HeapSort (ταξινόμηση σωρού)
2
Priority Queues
• A priority queue is an ADT where: – Each element has an associated priority– Efficient extraction of the highest-priority
element is supported.
• Typical operations of a priority queue are:– Enqueue (insert an element)– Dequeue (remove the highest-priority element)– Max (return the highest-priority element)– Meld (join two priority queues into one)
• Implementation : Binary Heap
3
Priority Queues: Applications
• Scheduling Examples: – Printer scheduling.
• Jobs sent to a printer by faculty have a higher priority than jobs sent by students
– Process scheduling.• Several processes require CPU time. Priority may
be based on various factors:– Shortest jobs have higher priority– Jobs with a shorter remaining time have higher priority– Jobs that have waited the longest have higher priority– etc.
4
Priority Queues: Applications
• Discrete-event simulations– Simulations of systems modeled as collections of
entities that interact with one another through events that occur at discrete times.• The priority assigned to an event is its start
time.– Examples:
• Truck fleet model – Consider the event "Truck leaves city A" with
priority 10:00am. Its execution (dequeue) triggers events "Truck arrives at depot" with priority 6:00pm, and "Truck has unloaded" with priority 7:00pm, which are then enqueued, based on their priorities.
• Circuit validation
5
Example
• How many aircraft are there in the air at peak traffic?
• How would you represent all the aircraft?– Arrays– Linked Lists– Trees
6
Characteristics of Flights
• Created and removed dynamically– Need to add and remove aircraft in realtime
• Different aircraft have different priorities– Different weights associated with
different aircraft– Length of time aircraft has been flying– Emergencies– Special aircraft (military/dignitaries)
7
Array Based Representation
• Removing Aircraft
•Removing takes 1 step, but copying takes
n
•Sorting Aircraft
– Insertion Sort -> n2 -> 25 * 106 operations
– Merge Sort -> n log n -> 20 * 103
operations
Heaps and priority queues
A heap is a data structure used to implement an efficient priority queue. The idea is to make it efficient to extract the element with the highest priority the next item in the queue to be processed.We could use a sorted linked list, with O(1) operations to remove the highest priority node and O(N) to insert a node. Using a tree structure will involve both operations being O(log2N) which is faster.
9
Tree Based Representation
10
Perfect Binary Tree
• For binary tree with height h– All nodes at levels h–1 or less have 2 children
(full)
h = 2 h = 4h = 3
h = 1
11
Complete Binary Trees
• For binary tree with height h – All nodes at levels h–2 or less have 2 children
(full)– All leaves on level h are as far left as possible
h = 2
h = 3
h = 1
12
Complete Binary Trees
h = 4
13
Complete Trees
• Nodes can be numbered in level-by-level order:
1
2 3
4 5 6 7
8 9 10 11 12
The left child of the i-th node is the node 2*i and the right child is the node 2*i + 1
The parent of the i-th node is the node i / 2
14
Complete Trees (cont’d)
• It is convenient to store a complete binary tree in an array (or an ArrayList) in the order of nodes
Argentina
Brazil Chile
Egypt Dominica Greece Italy
Haiti France
1
2 3
4 5 6 7
8 9
15
Heaps
• A complete binary tree is a full binary tree that has all leaves at the same depth
• Is the tree below complete?
• Heaps are nearly complete binary trees
16
Heaps
A heap is defined only on “nearly complete binary tress. What is a “nearly complete binary tree”?
A complete binary tree is one where all the internal nodes have exactly 2 children, and all the leaves are on the same level. An internal node comprises all the nodes except the leaf nodes.
Leaf nodes are nodes without children.
Leaf node
Interior node
Edge
17
Heaps
Height of a node: The number of edges starting at that node, and going down to the furthest leaf.
Height of the heap: The maximum number of edges from the root to a leaf.
Root
Height of blue node = 1
Height of root = 3
18
Heaps
Nearly complete – if not complete, then the only unfilled level is filled in from left to right.
23
4 56 7
8 9 10 11 12 13
1
23
4 56 7
8 9 10 11 12
1
Parent(i)
return i/2Left(i)
return 2i
Parent(i)
return 2i + 1
19
Heaps
• Two key properties– Complete binary tree– Value at node
• Smaller than or equal to values in subtrees (min-heaps)
• Example heap– X Y– X Z
Y
X
Z
20
Heap & Non-heap Examples
Heaps Non-heaps
6
2
22
8 45 25
6
2
22
8 45 25
8
6 45
5
6 22
25
5
5 45
5
21
Heap Properties
• Key operations– Insert ( X )– getSmallest ( )
• Key applications– Heapsort– Priority queue
22
Heap Operations – Insert( X )
• Algorithm1. Add X to end of tree2. While (X < parent)
Swap X with parent // X bubbles up tree
• Complexity– # of swaps proportional to height of tree– O( log(n) )
23
Heap Insert Example
• Insert ( 20 )
10
30 25
37
10
30 25
37 20
10
20 25
37 30
1) Insert to end of tree
2) Compare to parent, swap if parent key larger
3) Insert complete
24
Heap Insert Example
• Insert ( 8 )
10
30 25
37
10
30 25
37 8
10
8 25
37 30
8
10 25
37 30
1) Insert to end of tree
2) Compare to parent, swap if parent key larger
3) Insert complete
25
Heap Operation – getSmallest()
• Algorithm1. Get (remove) smallest node at root2. Replace root with X at end of tree3. While ( X > child )
Swap X with smallest child // X drops down tree
4. Return smallest node
• Complexity– # swaps proportional to height of tree– O( log(n) )
26
Heap GetSmallest Example
• getSmallest ()
8
10 25
37
30
10 25
37
10
30 25
3730
1) Replace root with end of tree
2) Compare node to children, if larger swap
with smallest child
3) Repeat swap if needed
27
Heap GetSmallest Example
• getSmallest ()
8
10 25
30
37
10 25
30
10
37 25
3037
1) Replace root with end of tree
2) Compare node to children, if larger swap
with smallest child
3) Repeat swap if needed
10
30 25
37
28
Binary Heaps
• Binary heap = – a binary tree that is – complete
• every level except possibly the bottom one is completely filled and the leaves in the bottom level are as far left as possible
– satisfies the heap property:• the key stored in every node is greater than
or equal to the keys stored in its children– this is called a max-heap. If the key at each
node is smaller than or equal to the keys of its children, then we have a min-heap.
29
Heaps
• The heap property of a tree is a condition that must be true for the tree to be considered a heap.
• Min-heap property: for min-heaps, requiresA[parent(i)] A[i]So, the root of any subtree holds the least value in that subtree.
• Max-heap property: for max-heaps, requiresA[parent(i)] A[i]The root of any subtree holds the greatest value in the subtree.
30
Heaps - example
• Looks like a max heap, but max-heap property is violated at indices 11 and 12. Why? because those nodes’ parents have larger smaller values.
2
83
84
85
76
87
1
8
69
710
511
812
9
1
9
31
Heaps
• The algorithm for removeMin uses the reheap-down procedure
• The algorithm for add uses the reheap-up procedure
Remove the root and place the last leaf at the root. Starting at the root, swap the node with its smaller child, as many times as needed to repair the heap.
Add a leaf. Starting at the last leaf, swap the node with its parent as many times as needed to repair the heap.
• Both adding and removing an item takes time proportional to the height of the heap (e.g., 20 for 1,000,000 nodes)
32
The removeMin Algorithm
France
Brazil Chile
Egypt Dominica
Greece Italy
Haiti
1
2 3
4 5 6 7
8
Brazil Chile
Egypt
Dominica Greece Italy
Haiti France
1
2 3
4 5 6 7
8 9
Argentina
Brazil
France Chile
Egypt Dominica Greece Italy
Haiti
1
2 3
4 5 6 7
8
Brazil
Dominica Chile
Egypt France Greece Italy
Haiti
1
2 3
4 5 6 7
8
Step 1: the root is
removed
Step 2: the last leaf replaces the root
Step 3: “reheap down”: the new root value moves down, swapping places with the smaller
child
33
The add Algorithm
Step 1: the new value is added as the last leaf
Step 2: “reheap up”: the new value moves up, swapping places with its parent
Brazil
Dominica Chile
Egypt France Greece Italy
Haiti
1
2 3
4 5 6 7
8 China
9
Brazil
Dominica Chile
China France Greece Italy
Haiti
1
2 3
4 5 6 7
8 Egypt
9
Brazil
China Chile
Dominica France Greece Italy
Haiti
1
2 3
4 5 6 7
8 Egypt
9
34
Binary Tree -> Array representation
• Use Breadth-First-Search• – The root node is A( 1 )• – Node i is A( i )
35
HeapsAssume we are given a tree represented by matrix A, and such that i length[A], and the trees rooted at Left(i) and Right(i) are heaps. Then, to create a heap rooted at A[i], use procedure Max-Heapify(A,i):
Max-Heapify(A,i)1. l Left(i) ) * l=left* 2. r Right(i)3. if l heap-size[A] and A[l] > A[i]4. then largest l5. else largest i6. if r heap-size[A] and A[r] > A[largest]7. then largest r8. if largest i9. then swap( A[i], A[largest])10. Max-Heapify(A, largest)
36
Heaps
• So, what must we do to build a heap? We call Max-Heapify(A,i) for every i starting at last node and going to the root.
• Why top-down?• Because Max-Heapify() moves the larger node
upward into the root. If we start at the top and go to larger node indices, the highest a node can move is to the root of that subtree. But what if there is a violation between the subtree’s root and its parent?
• So, we must start from the bottom and go towards the root to fix the problems caused by moving larger nodes into the root of subtrees.
37
Max-Heapify(A,1)
20 9
19 18 6 7
17 16 17 16 1
10
10 9
19 18 6 7
17 16 17 16 1
20
10 9
19 18 6 7
17 16 17 16 1
20
19 9
10 18 6 7
17 16 17 16 1
20
19 9
10 18 6 7
17 16 17 16 1
20
19 9
17 18 6 7
10 16 17 16 1
20
38
Heaps• In practice, heaps are usually implemented
as arrays:16
14 10
8 7 9 3
2 4 1
16 14 10 8 7 9 3 2 4 1A = =
39
Heaps
• To represent a complete binary tree as an array: – The root node is A[1]– Node i is A[i]– The parent of node i is A[i/2] (note: integer
divide)– The left child of node i is A[2i]– The right child of node i is A[2i + 1] 16
14 10
8 7 9 3
2 4 1
16 14 10 8 7 9 3 2 4 1A = =
40
Referencing Heap Elements
• So…
Parent(i) { return i/2; }
Left(i) { return 2*i; }
right(i) { return 2*i + 1; }
41
The Heap Property
• Heaps also satisfy the heap property:A[Parent( i )] A[ i ] for all nodes i > 1– In other words, the value of a node is at most the
value of its parent– Where is the largest element in a heap stored?
42
Heap Operations: Heapify()
• Heapify(): maintain the heap property
– Given: a node i in the heap with children l and r
– Given: two subtrees rooted at l and r, assumed to
be heaps
– Problem: The subtree rooted at i may violate the
heap property
– Action: let the value of the parent node “float
down” so subtree at i satisfies the heap property
43
Heapify() Example
16
4 10
14 7 9 3
2 8 1
16 4 10 14 7 9 3 2 8 1A =
44
Heapify() Example
16
4 10
14 7 9 3
2 8 1
16 10 14 7 9 3 2 8 1A = 4
45
Heapify() Example
16
4 10
14 7 9 3
2 8 1
16 10 7 9 3 2 8 1A = 4 14
46
Heapify() Example
16
14 10
4 7 9 3
2 8 1
16 14 10 4 7 9 3 2 8 1A =
47
Heapify() Example
16
14 10
4 7 9 3
2 8 1
16 14 10 7 9 3 2 8 1A = 4
48
Heapify() Example
16
14 10
4 7 9 3
2 8 1
16 14 10 7 9 3 2 1A = 4 8
49
Heapify() Example
16
14 10
8 7 9 3
2 4 1
16 14 10 8 7 9 3 2 4 1A =
50
Heapify() Example
16
14 10
8 7 9 3
2 4 1
16 14 10 8 7 9 3 2 1A = 4
51
Heapify() Example
16
14 10
8 7 9 3
2 4 1
16 14 10 8 7 9 3 2 4 1A =
52
HeapSort
53
Heap Application – Heapsort
• Use heaps to sort values– Heap keeps track of smallest element in heap
• Algorithm1. Create heap2. Insert values in heap3. Remove values from heap (in ascending order)
• Complexity– O( nlog(n))
54
Heapsort Example
• Input– 11, 5, 13, 6, 1
• View heap during insert, removal– As tree– As array
55
Heapsort – Insert Values
56
Heapsort – Remove Values
57
Heapsort – Insert in to Array 1
• Input– 11, 5, 13, 6, 1
Index = 0 1 2 3 4
Insert 11 11
58
Heapsort – Insert in to Array 2
• Input– 11, 5, 13, 6, 1
Index = 0 1 2 3 4
Insert 5 11 5
Swap 5 11
59
Heapsort – Insert in to Array 3
• Input– 11, 5, 13, 6, 1
Index = 0 1 2 3 4
Insert 13 5 11 13
60
Heapsort – Insert in to Array 4
• Input– 11, 5, 13, 6, 1
Index = 0 1 2 3 4
Insert 6 5 11 13 6
Swap 5 6 13 11
…
61
Heapsort – Remove from Array 1
• Input– 11, 5, 13, 6, 1
Index = 0 1 2 3 4
Remove root 1 5 13 11 6
Replace 6 5 13 11
Swap w/ child 5 6 13 11
62
Heapsort – Remove from Array 2
• Input– 11, 5, 13, 6, 1
Index = 0 1 2 3 4
Remove root 5 6 13 11
Replace 11 6 13
Swap w/ child 6 11 13
63
Example of Heapsort(A)
Illustrate the operation of Heapsort(A) on the array A = <27, 17, 3, 16, 13, 10, 1, 5, 7, 12, 4, 8, 9, 0>
Note: Since there a 14 nodes, and log 8 = 3 < log 14 < log
16 = 4, there are 4 levels (0,1,2 and 3) in the tree
17 3
16 13 10 1
5 7 12 4 8 9
27
0
64
17 3
16 13 10 1
5 7 12 4 8 9
27
0
Heapsort(A)
1. Build-Max-Heap(A)
2. for i length[A] downto 2
3. exchange A[1] A[i]
4. heap-size[A] heap-size[A]-1
5. Max-Heapify(A,1)
Build-Max-Heap(A)
1. heap-size[A] length[A]
2. for i length[A]/2 downto 1
3. do Max-Heapify(A,i)
17 10
16 13 3 1
5 7 12 4 8 9
27
0
17 10
16 13 9 1
5 7 12 4 8 3
27
0
Build-Max-Heap(A) is done:
A is a max-heap.
Now onto lines 2-5 of Heapsort
65
17 10
16 13 9 1
5 7 12 4 8 3
27
0
Heapsort(A)
1. Build-Max-Heap(A)
2. for i length[A] downto 2
3. exchange A[1] A[i]
4. heap-size[A] heap-size[A]-1
5. Max-Heapify(A,1)
17 10
16 13 9 1
5 7 12 4 8 3
0
27
exchange A[1] A[i]
0 10
16 13 9 1
5 7 12 4 8 3
17
27
Max-Heapify(A,1)
16 10
0 13 9 1
5 7 12 4 8 3
17
27
16 10
7 13 9 1
5 0 12 4 8 3
17
27
66
16 10
7 13 9 1
5 0 12 4 8 17
3
27
exchange A[1] A[i] Max-Heapify(A,1)
16 10
7 13 9 1
5 0 12 4 8 17
3
27
3 10
7 13 9 1
5 0 12 4 8 17
16
27
13 10
7 3 9 1
5 0 12 4 8 17
16
27
13 10
7 12 9 1
5 0 3 4 8 17
16
27
67
exchange A[1] A[i] Max-Heapify(A,1)
13 10
7 12 9 1
5 0 3 4 16 17
8
27
13 10
7 12 9 1
5 0 3 4 16 17
8
27
8 10
7 12 9 1
5 0 3 4 16 17
13
27
12 10
7 8 9 1
5 0 3 4 16 17
13
27
12 10
7 4 9 1
5 0 3 8 16 17
13
27
etc...
Notice last 3 elements in array are the largest in A, and they are also sorted in increasing order.
68
Example of Heap-Extract-Max(A)
17 10
16 13 9 1
5 7 12 4 8 3
27
0Heap-Extract-Max(A)
1. if heap-size[A] < 1
2. then error “heap underflow”
3. max A[1]
4. A[1] A[heap-size[A] ]
5. heap-size[A] heap-size[A] - 1
6. Max-Heapify(A,1)
7. return max
17 10
16 13 9 1
5 7 12 4 8 3
0
0
0 10
16 13 9 1
5 7 12 4 8 3
17
0
16 10
0 13 9 1
5 7 12 4 8 3
17
0
16 10
7 13 9 1
5 0 12 4 8 3
17
0
Example Heap Sort
Let us look at this example: we must convert the unordered array with n = 10 elements into a max-heap
None of the leaf nodes need tobe percolated down, and the firstnon-leaf node is in position n/2
Thus we start with position 10/2 = 5
Example Heap Sort
We compare 3 with its child and swap them
Example Heap Sort
We compare 17 with its two children and swap it with the maximum child (70)
Example Heap Sort
We compare 28 with its two children, 63 and 34, and swap it with the largest child
Example Heap Sort
We compare 52 with its children, swap it with the largest– Recursing, no further swaps are needed
Example Heap Sort
Finally, we swap the root with its largest child, and recurse, swapping 46 again with 81, and then again with 70
Heap Sort Example
We have now converted the unsorted array
into a max-heap:
Heap Sort Example
Suppose we pop the maximum element of this heap
This leaves a gap at the back of the array:
Heap Sort Example
This is the last entry in the array, so why not fill it with the largest element?
Repeat this process: pop the maximum element, and then insert it at the end of the array:
Heap Sort Example
Repeat this process– Pop and append 70
– Pop and append 63
Heap Sort Example
We have the 4 largest elements in order– Pop and append 52
– Pop and append 46
Heap Sort Example
Continuing...– Pop and append 34
– Pop and append 28
Finally, we can pop 17, insert it into the 2nd location, and the resulting array is sorted
Heap Sort Example
82
Heapsort
• For a binary heap, building the heap takes O(N) time.
• In a binary heap, deleteMin gives the smallest element. – Running time of O(logN).
• Basic strategy for heap sort given N elements– Build a binary heap from N elements - O(N)– Perform N deleteMin operations O(N*logN)
• Put the result of deleteMins in a new array. – The new array is sorted.
83
Heapsort
• The problem is that: – This strategy uses a new array – This means waste of space for large N.
• A way for using only one array. – When minimum item is deleted, put it to the end of the
array. – After N deleteMins:
• The original array will have the elements in sorted order from max to min.
– If you want items in increasing order (from min to max), then use a Max-Heap.
84
Example
• Assume the input that is to be sorted: – 31, 41, 59, 26, 53, 58, 97– N = 7.
• We need first to build a binary heap (a Max-Heap)– A max-Heap is heap that has the largest item in
root.
85
Example97
53
26 41 58 31
5997 53 59 26 41 58 31
10 2 3 4 5 6 7 8 9 10
59
53
26 41 31
5859 53 58 26 41 31 97
10 2 3 4 5 6 7 8 9 10
97
deleteMax
Max-Heap after BuildHeap()
Heap after first DeleteMin()
86
Example
• After 5 more deleteMins, the array will look like the following.
26 31 41 53 58 59 97
10 2 3 4 5 6 7 8 9 10
This is a sorted array
87
template <class Comparable> void heapsort( vector<Comparable> & a ) {/* 1*/ for( int i = a.size( ) / 2; i >= 0; i-- ) /* buildHeap *//* 2*/ percDown( a, i, a.size( ) );
/* 3*/ for( int j = a.size( ) - 1; j > 0; j-- ) {/* 4*/ swap( a[ 0 ], a[ j ] ); /* deleteMax *//* 5*/ percDown( a, 0, j ); } }
j is running from N -1 to 1
i is running from N/2 to 0
Swap first and last elements
88
template <class Comparable> void percDown( vector<Comparable> & a, int i, int n ) { int child; Comparable tmp;
/* 1*/ for( tmp = a[ i ]; leftChild( i ) < n; i = child ) {/* 2*/ child = leftChild( i );/* 3*/ if( child != n - 1 && a[ child ] < a[ child + 1 ] )/* 4*/ child++;/* 5*/ if( tmp < a[ child ] )/* 6*/ a[ i ] = a[ child ]; else/* 7*/ break; }/* 8*/ a[ i ] = tmp; }
Logical size of heapPoint from where to start
percolate down
89
97 53 59 26 41 58 31
10 2 3 4 5 6 7 8 9
a.size = 7
swap a[0] with a[6]
31 53 59 26 41 58 97
10 2 3 4 5 6 7 8 9
a.size = 7
percolatedown(a, 0, 6)
j=6
heapsort()
90
31 53 59 26 41 58 97
10 2 3 4 5 6 7 8 9
i=0
59 53 26 41 58 97
10 2 3 4 5 6 7 8 9
tmp = 31, n=6
59 53 26 41 58 97
10 2 3 4 5 6 7 8 9
i=2
59 53 58 26 41 31 97
10 2 3 4 5 6 7 8 9
tmp = 31, n=6
percDown()
Go outof for loop!
91
59 53 58 26 41 31 97
10 2 3 4 5 6 7 8 9
j=5
31 53 58 26 41 59 97
10 2 3 4 5 6 7 8 9
j=5
Swap(a[0], a[5])
percolateDown(a, 0, 5)
heapsort()
92
31 53 58 26 41 59 97
10 2 3 4 5 6 7 8 9
i=0
58 53 31 26 41 59 97
10 2 3 4 5 6 7 8 9
percDown()
Applications of Priority Queues
• The Selection Problem: find k-th largest number in a list of N elements
• Straightforward solution: not efficient, O(N2)– Algorithm A: sort N numbers in decreasing order,
then return the kth element – cost O(N2)– Algorithm B: read and sort k numbers in decreasing
order, then read the rest elements one after another and compare to the kth element, if larger then replace it the correct place of the remaining k-1 elements. Finally return the kth element – cost O(kN), because k*k (read first k and sort) + k*(N-k) (the remaining N-k is larger and larger). Here k could be N/2, hence total O(N2) in worst case.
93
• Solution with priority queue:– Algorithm A: (assume find kth smallest) apply
buildHeap algorithm to the array (O(N)), then perform k deleteMin operations (k.O(log N)). The last element extracted from the heap is the answer. Total cost: O(N+k log N). Here k could be N/2, hence total O(NlogN) in worst case.
– Algorithm B: the first k elements are placed into the heap in total time O(k) with a call to buildHeap. The time to process each remaining element is O(1), to test if the element goes into S, plus O(log k), to delete Sk and insert the element if this is necessary. Thus, the total time is O(k+(n-k)log k) = O(N log k). It is also O(N log N) in worst case
94
Source code 1
Source code 2
Source code 3
Source code 4
Source code 5
Source code 6
Source code 7