CS 253: Algorithms

CS 253: Algorithms

Chapter 6

HeapsortAppendix B.5

Credit: Dr. George Bebis

2

Special Types of Trees

Def: Full binary tree a binary tree in which each node is either a leaf or has degree exactly 2.

Def: Complete binary treea binary tree in which all leaves are on the same level and all internal nodes have degree 2.

Full binary tree

214 8

1

167

4

3

9 1012

Complete binary tree

2

1

16

4

3

9 10

Definitions Height of a node = the number of edges on the

longest simple path from the node down to a leaf

Level of a node = the length of a path from the root to the node

Height of tree = height of root node

214 8

116

4

39 10

Height of root = 3

Height of (2)= 1 Level of (10)= 2

Useful Properties

214 8

116

4

39 10

Height of root = 3

Height of (2)= 1 Level of (10)= 2

• There are at most 2l nodes at level (or depth) l of a binary tree

• A binary tree of height h has at most (2h+1 – 1) nodes

• **here A binary tree with n nodes has height at least lgn. (see Ex 6.1-1&2)

1212

122 11

0

h

hh

i

in

5

The Heap Data StructureDef: A heap is a nearly complete binary tree

with the following two properties:◦ Structural property: all levels are full, except

possibly the last one, which is filled from left to right◦ Order (heap) property: for any node x,

Parent(x) ≥ x

Heap

57

8

42

From the heap property, it follows that:

The root is the maximum element of the heap!

Array Representation of Heaps

A heap can be stored as an array A.

◦ Root of tree is A[1]◦ Left child of A[i] = A[2i]◦ Right child of A[i] = A[2i

+ 1]◦ Parent of A[i] = A[ i/2 ]◦ Heapsize[A] ≤ length[A]

The elements in the subarray A[(n/2+1) .. n] are leaves

7

Heap Types

Max-heaps (largest element at root), have the max-heap property: ◦ for all nodes i, excluding the root: A[PARENT(i)] ≥ A[i]

Min-heaps (smallest element at root), have the min-heap property:◦ for all nodes i, excluding the root: A[PARENT(i)] ≤ A[i]

8

Operations on HeapsMaintain/Restore the max-heap property

◦ MAX-HEAPIFY

Create a max-heap from an unordered array◦ BUILD-MAX-HEAP

Sort an array in place◦ HEAPSORT

Priority queues

9

Maintaining the Heap PropertyMAX-HEAPIFY

Suppose a node i is smaller than a child and Left and Right subtrees of i are max-

heaps

How do you fix it?

To eliminate the violation: Exchange node i with the larger child Move down the tree Continue until node is not smaller than

children

◦ MAX-HEAPIFY

ExampleMAX-HEAPIFY(A, 2, 10)

A[2] violates the heap property

A[2] A[4]

A[4] violates the heap property

A[4] A[9]

Heap property restored

Maintaining the Heap Property

Assumptions:◦Left and Right

subtrees of i are max-heaps

◦A[i] may be smaller than its children

Alg: MAX-HEAPIFY(A, i, n)1. l ← LEFT(i)2. r ← RIGHT(i)3. if l ≤ n and A[l] > A[i]4. then largest ←l5. else largest ←i6. if r ≤ n and A[r] > A[largest]7. then largest ←r8. if largest i9. then exchange A[i] ↔

A[largest]10. MAX-HEAPIFY(A,

largest, n)

MAX-HEAPIFY Running Time It traces a path from the root to a leaf (longest path

length h)

At each level, it makes two comparisons

Total number of comparisons = 2h

Running Time of MAX-HEAPIFY = O(2h) = O(lgn)

13

Building a Heap

Alg: BUILD-MAX-HEAP(A)1. n = length[A]2. for i ← n/2 downto 13. do MAX-HEAPIFY(A, i, n)

Convert an array A[1 … n] into a max-heap (n = length[A])The elements in the subarray A[(n/2+1) .. n] are leavesApply MAX-HEAPIFY on elements between 1 and n/2 (bottom up)

214 8

116

7

4

39 10

1

2 3

4 5 6 7

8 9 10

4 1 3 2 16 9 10 14 8 7A:

Example: A4 1 3 2 16 9 10 14 8 7

214 8

116

7

4

39 10

1

2 3

4 5 6 7

8 9 10

142 8

116

7

4

109 3

1

2 3

4 5 6 7

8 9 10

214 8

116

7

4

39 10

1

2 3

4 5 6 7

8 9 10 142 8

116

7

4

39 10

1

2 3

4 5 6 7

8 9 10

142 8

167

1

4

109 3

1

2 3

4 5 6 7

8 9 10 82 4

147

1

16

109 3

1

2 3

4 5 6 7

8 9 10

i = 5 i = 4 i = 3

i = 2 i = 1

15

Running Time of BUILD MAX HEAP

Running time: O(nlgn)This is not an asymptotically tight upper bound! Why?

Alg: BUILD-MAX-HEAP(A)

1. n = length[A]

2. for i ← n/2 downto 1

3. do MAX-HEAPIFY(A, i, n)

O(lgn) O(n)

Running Time of BUILD MAX HEAP MAX-HEAPIFY takes O(h) the cost of MAX-HEAPIFY on a

node i proportional to the height of the node i in the tree

Height Levelh0 = 3 (lgn)

h1 = 2

h2 = 1

h3 = 0

i = 0

i = 1

i = 2

i = 3 (lgn)

No. of nodes20

21

22

23

hi = h – i height of the heap rooted at level ini = 2i number of nodes at level i

)(2)(00

nOihhnnTh

i

ii

h

ii

see the next slide

i

h

iihnnT

0

)( Cost of HEAPIFY at level i (# of nodes at that level)

ihh

i

i 0

2 Replace the values of ni and hi computed before

hh

iih

ih 220

Multiply by 2h both at the nominator and denominator

and write 2i as (1/2-i)

h

kk

h k0 2

2 Change variables: k = h - i

0 2k

k

kn The sum above is smaller than the sum of all elements to and h = lgn

)(nO Running time of BUILD-MAX-HEAP

T(n) = O(n)

Running Time of BUILD MAX HEAP

2)2/11(

2/1

n x=1/2 in the summation

20 )1( x

xkxk

k

18

Goal:◦ Sort an array using heap representations

Idea:◦ Build a max-heap from the array◦ Swap the root (the maximum element) with the

last element in the array◦ “Discard” this last node by decreasing the heap

size◦ Call MAX-HEAPIFY on the new root◦ Repeat this process until only one node remains

Heapsort

19

Example: A=[7, 4, 3, 1, 2]

MAX-HEAPIFY(A, 1, 4) MAX-HEAPIFY(A, 1, 3) MAX-HEAPIFY(A, 1, 2)

MAX-HEAPIFY(A, 1, 1)

20

Alg: HEAPSORT(A)

1. BUILD-MAX-HEAP(A)2. for i ← length[A] downto 23. do exchange A[1] ↔ A[i]4. MAX-HEAPIFY(A, 1, i - 1)

Running time: O(nlgn) Can be shown to be Θ(nlgn)

O(n)

O(lgn)

n-1 times

Priority QueuesEach element is associated with a value (priority)The key with the highest (lowest) priority is extracted first

Support the following operations:◦ INSERT(S, x) : inserts element x into set S◦ EXTRACT-MAX(S) : removes and returns element of S with largest

key◦ MAXIMUM(S) : returns element of S with largest key◦ INCREASE-KEY(S, x, k) : increases value of element x’s key to k

(Assume k ≥ x’s current key value)

14 5

22

HEAP-MAXIMUMGoal:

◦ Return the largest element of the heap

Alg: HEAP-MAXIMUM(A)1. return A[1]

Running time: O(1)

Heap A:

Heap-Maximum(A) returns 7

23

HEAP-EXTRACT-MAXGoal:

◦ Extract the largest element of the heap (i.e., return the max value and also remove that element from the heap

Idea: ◦ Exchange the root element with the last◦ Decrease the size of the heap by 1 element◦ Call MAX-HEAPIFY on the new root, on a heap of

size n-1Heap A: Root is the largest element

Example: HEAP-EXTRACT-MAX

82 4

147

1

16

109 3

max = 168

2 4

147

1

109 3

Heap size decreased with 1

42 1

87

14

109 3

Call MAX-HEAPIFY(A, 1, n-1)

25

HEAP-EXTRACT-MAX

Alg: HEAP-EXTRACT-MAX(A, n)1. if n < 1

2. then error “heap underflow”3. max ← A[1]4. A[1] ← A[n]5. MAX-HEAPIFY(A, 1, n-1) % remakes heap6. return max

Running time: O(lgn)

HEAP-INCREASE-KEYGoal:

◦ Increases the key of an element i in the heap

Idea:◦ Increment the key of A[i] to its new value◦ If the max-heap property does not hold

anymore: traverse a path toward the root to find the proper place for the newly increased key

82 4

147

1

16

109 3i

Key [i] ← 15

Example: HEAP-INCREASE-KEY

142 8

157

1

16

109 3

i

82 4

147

1

16

109 3i

Key [i ] ← 15

82 15

147

1

16

109 3i

152 8

147

1

16

109 3

i

Alg: HEAP-INCREASE-KEY(A, i, key)

1. if key < A[i]2. then error “new key is smaller than current

key”3. A[i] ← key4. while i > 1 and A[PARENT(i)] < A[i]5. do exchange A[i] ↔ A[PARENT(i)]6. i ← PARENT(i)

Running time: O(lgn)

82 4

147

1

16

109 3i

Key [i] ← 15

29

-

MAX-HEAP-INSERTGoal:

Inserts a new element into a max-heap

Idea: Expand the max-heap with a

new element whose key is -

Calls HEAP-INCREASE-KEY to set the key of the new node to its correct value and maintain the max-heap property

82 4

147

1

16

109 3

158

2 4

147

1

16

109 3

30

Example: MAX-HEAP-INSERT

-8

2 4

147

1

16

109 3

Insert value 15:Start by inserting -

158

2 4

147

1

16

109 3

Increase the key to 15Call HEAP-INCREASE-KEY on A[11] = 15

78

2 4

1415

1

16

109 3

The restored heap containingthe newly added element

78

2 4

1514

1

16

109 3

Alg: MAX-HEAP-INSERT(A, key, n)

1. heap-size[A] ← n + 12. A[n + 1] ← - 3. HEAP-INCREASE-KEY(A, n + 1, key)

Running time: O(lgn)

-8

2 4

147

1

16

109 3

32

SummaryWe can perform the following operations on

heaps:

◦ MAX-HEAPIFY O(lgn)◦ BUILD-MAX-HEAP O(n)◦ HEAP-SORT O(nlgn)◦ MAX-HEAP-INSERT O(lgn)◦ HEAP-EXTRACT-MAX O(lgn)◦ HEAP-INCREASE-KEY O(lgn)◦ HEAP-MAXIMUM O(1)

Average:

O(lgn)

33

Priority Queue Using Linked List

Average: O(n)

Increase key: O(n)

Extract max key: O(1)

12 4

Problemsa. What is the maximum number of

nodes in a max heap of height h?

b.What is the maximum number of leaves?

c. What is the maximum number of internal nodes?

d.Assuming the data in a max-heap are distinct, what are the possible locations of the second-largest element?

35

Demonstrate, step by step, the operation of Build-Heap on the array A = [5, 3, 17, 10, 84, 19, 6, 22, 9]

Let A be a heap of size n. Give the most efficient algorithm for the following tasks:

(a) Find the sum of all elements(b) Find the sum of the largest lgn elements

Problems