UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2001

UMass Lowell Computer Science 91.503

Analysis of Algorithms Prof. Karen Daniels

Fall, 2001

UMass Lowell Computer Science 91.503

Analysis of Algorithms Prof. Karen Daniels

Fall, 2001

Lecture 11Lecture 11Tuesday, 12/4/01Tuesday, 12/4/01

Advanced Data StructuresAdvanced Data StructuresChapters 20-21Chapters 20-21

Relevant Sections of ChaptersRelevant Sections of Chapters

Ch20 Binomial HeapsYou’re responsible for material in this chapter that we discuss in lecture. (Note that this includes all sections.)

Ch21 Fibonacci HeapsYou’re responsible for material in this chapter that we discuss in lecture. (Note that this includes all sections.)

NoteNote that Chapter 22 has been removed. that Chapter 22 has been removed.

Ch7 HeapSort

OverviewOverview

Chapter 7: Heap ReviewChapter 7: Heap Review

Chapter 20: Binomial HeapsChapter 20: Binomial Heaps

Chapter 21: Fibonacci HeapsChapter 21: Fibonacci Heaps

Chapter 7Chapter 7

Heap ReviewHeap Review

Review of Heap BasicsReview of Heap Basics

Structure:Structure: Nearly complete binary treeNearly complete binary tree Convenient array Convenient array

representationrepresentation HEAP Property: HEAP Property: (for MAX HEAP)(for MAX HEAP)

Parent’s label not less than Parent’s label not less than that of each child that of each child

1616

1414 1010

88 77 99 33

22 44 11

1616 1414 1010 88 77 99 33 22 44 11

1 2 3 4 5 6 7 8 9 101 2 3 4 5 6 7 8 9 10

Operations on a HeapOperations on a Heap

HEAPIFY:HEAPIFY: for a given node that is the root of a subtree, if both subtrees of for a given node that is the root of a subtree, if both subtrees of

that node are already HEAPs, HEAPIFY enforces the HEAP that node are already HEAPs, HEAPIFY enforces the HEAP PROPERTY via “downward swaps” so that the node together PROPERTY via “downward swaps” so that the node together with its subtrees form a HEAPwith its subtrees form a HEAP

BUILD-HEAP:BUILD-HEAP: builds a HEAP from scratch using HEAPIFYbuilds a HEAP from scratch using HEAPIFY

HEAPSORT: HEAPSORT: sorts an array by first using BUILD-HEAP sorts an array by first using BUILD-HEAP

then repeatedly swapping out root and calling then repeatedly swapping out root and calling HEAPIFYHEAPIFY

assuming array representationassuming array representation

Operations on a HeapOperations on a Heap

HEAPIFY:HEAPIFY:

BUILD-HEAP:BUILD-HEAP:

HEAPSORT: HEAPSORT:

assuming array representationassuming array representation

Asymptotic worst-case Asymptotic worst-case running time is in O(lg n).running time is in O(lg n).

Asymptotic worst-Asymptotic worst-case running time is case running time is in O(n lg n). in O(n lg n). However this is a However this is a loose bound! loose bound! Time is also in O(n).Time is also in O(n).

Asymptotic worst-case Asymptotic worst-case running time is in O(n lg n). running time is in O(n lg n).

For a node at height h, time is in O(h).For a node at height h, time is in O(h).

O(n)O(n)++

O(lgn)O(lgn)nO(lgn)nO(lgn)

T(n) = T(2n/3) + T(n) = T(2n/3) + (1) (1) is in O(is in O(lgnlgn) using ) using Master TheoremMaster Theorem

)(

2

lg

01

hOnn

hh

n

hh

hnO

lg

0 2

)()2(20

nOnOh

nOh

h

Operations on a HeapOperations on a Heap

PRIORITY QUEUE SUPPORT:PRIORITY QUEUE SUPPORT: HEAP-INSERTHEAP-INSERT

adds new leaf to the tree and then “swaps up” to restore adds new leaf to the tree and then “swaps up” to restore HEAP PROPERTYHEAP PROPERTY

HEAP- MAXIMUMHEAP- MAXIMUM HEAP PROPERTY guarantees that maximum is at the HEAP PROPERTY guarantees that maximum is at the

root of a MAX HEAProot of a MAX HEAP HEAP- EXTRACT-MAXHEAP- EXTRACT-MAX

removes the maximum value from the root by swapping removes the maximum value from the root by swapping it outit out

restores HEAP PROPERTY using HEAPIFYrestores HEAP PROPERTY using HEAPIFY

assuming array representationassuming array representation

ApplicationsApplications: Job Scheduling, Event Scheduling: Job Scheduling, Event Scheduling

Operations on a HeapOperations on a Heap

PRIORITY QUEUE SUPPORT:PRIORITY QUEUE SUPPORT: HEAP-INSERTHEAP-INSERT

HEAP- MAXIMUMHEAP- MAXIMUM

HEAP- EXTRACT-MAXHEAP- EXTRACT-MAX

assuming array representationassuming array representation

Asymptotic worst-case Asymptotic worst-case running time is in O(lg n).running time is in O(lg n).

For a node at height h, time is in O(h).For a node at height h, time is in O(h).

Asymptotic worst-case Asymptotic worst-case running time is in O(1).running time is in O(1).

Asymptotic worst-case Asymptotic worst-case running time is in O(lg n).running time is in O(lg n).

For a node at height h, time is in O(h).For a node at height h, time is in O(h).O(1)O(1)

++

O(lgn)O(lgn)

Building a Heap usingHEAPIFY vs. HEAP-INSERTBuilding a Heap usingHEAPIFY vs. HEAP-INSERT

HEAPIFYHEAPIFY swaps swaps downdown compares parent with compares parent with both both

childrenchildren before each swap before each swap

1616

1414 1010

88 77 99 33

22 44 11

Asymptotic worst-case running time of BUILD-HEAP using HEAPIFY is in Asymptotic worst-case running time of BUILD-HEAP using HEAPIFY is in O(n). However, using HEAP-INSERT the time would only be in O(n lg n). O(n). However, using HEAP-INSERT the time would only be in O(n lg n).

HEAP-INSERTHEAP-INSERT swaps swaps upup compares parent with compares parent with one one

childchild before each swap before each swap

)(

2

lg

01

hOnn

hh

n

hh

hnO

lg

0 2

)()2(20

nOnOh

nOh

h

number number of levelsof levels

number of number of nodes in nodes in this levelthis level

maximum maximum number of number of swaps = swaps = length of length of path from path from this level this level down to down to leafleaf

)(lg

2

lg

01

hnnn

hh

number number of levelsof levels

number of nodes number of nodes in this levelin this level

maximum maximum number of number of swaps = swaps = length of length of path from path from this level this level up to rootup to root

hn

nnOn

hh

n

hh

lg

01

lg

0 22

1)lg(

O(n) as in O(n) as in HEAPIFYHEAPIFY

22

1

2

1

2

1

0

lg

0

lg

0

h

hn

h

hn

hh

)lg( nnO

Chapter 20Chapter 20

Binomial HeapsBinomial Heaps

Mergeable Heap OperationsMergeable Heap Operations

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

If UNION not needed, binary heap (Ch7) sufficesIf UNION not needed, binary heap (Ch7) suffices

Mergable Heaps supporting fast UNIONMergable Heaps supporting fast UNION

Mergeable Heap OperationsMergeable Heap Operations

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Mergable Heaps supporting fast UNIONMergable Heaps supporting fast UNION

(inefficient SEARCH)(inefficient SEARCH)

search root list use min pointer

UNION add to root list; update min pointer

orphan, reverse, UNION

orphan, consolidate = degree roots

orphan, consolidate = degree roots

swap up if heap property violation

swap with root; swap down if heap property violation

swap up if heap property violation

swap up if heap property violation

DECREASE-KEY, EXTRACT-MIN

DECREASE-KEY, EXTRACT-MIN

add to root list; update min pointercascading cut if heap property violation

Binomial Tree DefinitionBinomial Tree Definition

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Binomial Heap is a collection of Binomial Heap is a collection of Binomial TreesBinomial Trees

Binomial Tree BBinomial Tree Bkk is an ordered tree defined recursively is an ordered tree defined recursively(children are ordered)(children are ordered)

Binomial Tree PropertiesBinomial Tree Properties

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Binomial Tree PropertiesBinomial Tree Properties

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Binomial Tree PropertiesBinomial Tree Properties

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Proof: (continued)Proof: (continued)

Binomial Heap DefinitionBinomial Heap Definition

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

root degrees increase along root listroot degrees increase along root list

Binomial Heap Operations:MAKE-HEAP, MINIMUMBinomial Heap Operations:MAKE-HEAP, MINIMUM

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

MAKE-HEAP worst-case MAKE-HEAP worst-case running time is in running time is in (1)(1)

HEAP-MINIMUM worst-case HEAP-MINIMUM worst-case running time is in running time is in (lgn)(lgn)

( ( assume no keys with valueassume no keys with value ))Minimum key must be in a root node due Minimum key must be in a root node due to heap-ordering.to heap-ordering.

There are at most There are at most roots to check.roots to check. 1lg n

search root list

Binomial Heap Operations:UNIONBinomial Heap Operations:UNION

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Links 2 binomial heaps Links 2 binomial heaps whose roots have same whose roots have same degreedegree

Merge root lists into single linked list sorted by nondecreasing degree

Link roots of equal degree until at most one root remains of each degree

Pointers into root list:Pointers into root list:

HEAP-UNION worst-case running HEAP-UNION worst-case running time is in time is in (lgn)(lgn)

orphan, consolidate = degree roots

Binomial Heap Operations:UNION (continued)

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

3 roots of same degree3 roots of same degree

Binomial Heap Operations:UNION (continued)

Binomial Heap Operations:UNION (continued)

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Binomial Heap Operations:UNION (continued)

Binomial Heap Operations:UNION (continued)

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Binomial Heap Operations:INSERTBinomial Heap Operations:INSERT

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

HEAP-INSERT worst-case running time is in HEAP-INSERT worst-case running time is in (lgn)(lgn)

UNION

Binomial Heap Operations:EXTRACT-MINBinomial Heap Operations:EXTRACT-MIN

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

HEAP-EXTRACT-MIN worst-case HEAP-EXTRACT-MIN worst-case running time is in running time is in (lgn)(lgn)

orphan, reverse, UNION

Binomial Heap Operations:DECREASE-KEYBinomial Heap Operations:DECREASE-KEY

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

HEAP-DECREASE-KEY worst-case HEAP-DECREASE-KEY worst-case running time is in running time is in (lgn)(lgn)

If violate heap property, swap up.If violate heap property, swap up.

No change in structure.No change in structure.

swap up if heap property violation

Binomial Heap Operations: DELETEBinomial Heap Operations: DELETE

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

HEAP-DELETE worst-case running time is in HEAP-DELETE worst-case running time is in (lgn)(lgn)

DECREASE-KEY, EXTRACT-MIN

Chapter 21Chapter 21

Fibonacci HeapsFibonacci Heaps

Mergeable Heap OperationsMergeable Heap Operations

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Mergable Heaps supporting fast UNIONMergable Heaps supporting fast UNION

(inefficient SEARCH)(inefficient SEARCH)

search root list

UNION

orphan, reverse, UNIONorphan, consolidate = degree roots

swap up if heap property violation

swap with root; swap down if heap property violation

swap up if heap property violation

swap up if heap property violation

DECREASE-KEY, EXTRACT-MIN

use min pointer

add to root list; update min pointer

orphan, consolidate = degree roots

DECREASE-KEY, EXTRACT-MIN

add to root list; update min pointercascading cut if heap property violation

O(D(n))O(D(n))

O(D(n))O(D(n))

Fibonacci Heap BasicsFibonacci Heap Basics

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

For asymptotically fast MST, For asymptotically fast MST, shortest pathsshortest paths

Collection of treesCollection of trees

Relaxed structureRelaxed structure

Lazy: delay work Lazy: delay work

Amortized (potential) costAmortized (potential) cost

Circular linked listsCircular linked lists

Minimum rootMinimum root

t(H) = #trees in root listt(H) = #trees in root list

m(H) = #marked nodesm(H) = #marked nodes

Heap Potential Function

Potential Method (review)Potential Method (review)

Potential MethodPotential Method amortized cost can differ across operations amortized cost can differ across operations (as in accounting method)(as in accounting method) overcharge some operations early in sequence overcharge some operations early in sequence (as in accounting method)(as in accounting method) store overcharge as “potential energy” of data structure as a wholestore overcharge as “potential energy” of data structure as a whole

(unlike accounting method)(unlike accounting method)

Let cLet cii be actual cost of be actual cost of iith operationth operation Let DLet Dii be data structure after applying be data structure after applying iith operationth operation Let Let (D(Di i ) be potential associated with D) be potential associated with Dii Amortized cost of Amortized cost of iith operation:th operation: Total amortized cost of n operations:Total amortized cost of n operations:

Must have:Must have: to “pay in advance” to “pay in advance”

)()(ˆ 1 iiii DDcc

)()())()((ˆ 01

111

DDcDDcc n

n

iiii

n

ii

n

ii

)()( 0DDn

terms terms telescopetelescope

Unordered Binomial Tree PropertiesUnordered Binomial Tree Properties

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

DIFFERENCEDIFFERENCE

DECREASE-KEY DECREASE-KEY Fibonacci Heap Fibonacci Heap operation may violate operation may violate Unordered Binomial Unordered Binomial Tree properties.Tree properties.

Fibonacci Heap Operations:INSERTFibonacci Heap Operations:INSERT

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Increase in potential = Increase in potential =

Amortized cost is in O(1)Amortized cost is in O(1)Actual cost is in O(1)Actual cost is in O(1)

add to root list; update min pointer

Heap Potential Functiont (H) = #trees in root listt (H) = #trees in root listm(H) = #marked nodesm(H) = #marked nodes

Fibonacci Heap Operations:UNIONFibonacci Heap Operations:UNION

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.add to root list; update min pointer

Heap Potential Function

t (H) = #trees in root listt (H) = #trees in root listm(H) = #marked nodesm(H) = #marked nodes

Fibonacci Heap Operations:EXTRACT-MINFibonacci Heap Operations:EXTRACT-MIN

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

disassembledisassemble

Consolidate wherever Consolidate wherever possiblepossible: :

Link roots of = degree Link roots of = degree until at most one root until at most one root remains of each degree remains of each degree

Process one tree at a Process one tree at a time, starting with new time, starting with new “min”.“min”.

Next, update “min”Next, update “min”

First, disassemble old First, disassemble old min tree.min tree.

orphan, consolidate = degree roots

Fibonacci Heap Operations:EXTRACT-MIN ConsolidationFibonacci Heap Operations:EXTRACT-MIN Consolidation

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Degree 0 mergeDegree 0 merge: :

Combine 7 with 23 Combine 7 with 23

Degree 1 mergeDegree 1 merge: :

(keep going) Combine (keep going) Combine 7/23 with 17/30 7/23 with 17/30

Degree 2 mergeDegree 2 merge: :

(keep going) Combine (keep going) Combine 7… with 24…7… with 24…

Fibonacci Heap Operations:EXTRACT-MIN ConsolidationFibonacci Heap Operations:EXTRACT-MIN Consolidation

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Fibonacci Heap Operations:EXTRACT-MIN PseudocodeFibonacci Heap Operations:EXTRACT-MIN Pseudocode

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Fibonacci Heap Operations:EXTRACT-MIN PseudocodeFibonacci Heap Operations:EXTRACT-MIN Pseudocode

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

may do may do multiple multiple degree degree mergesmerges

Fibonacci Heap Operations:EXTRACT-MIN AnalysisFibonacci Heap Operations:EXTRACT-MIN Analysis

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.Amortized cost is in O(lgn)Amortized cost is in O(lgn)

D(n) = upper bound on maximum degree of any node in an n-node D(n) = upper bound on maximum degree of any node in an n-node Fibonacci heap (shown in Section 21.3 to be in O(lgn))Fibonacci heap (shown in Section 21.3 to be in O(lgn))

O(D(n))O(D(n))

Heap Potential Functiont (H) = #trees in root listt (H) = #trees in root listm(H) = #marked nodesm(H) = #marked nodes

Actual WorkActual Work::>= O(D(n)) since at most D(n) children of minimum node>= O(D(n)) since at most D(n) children of minimum nodeWhen CONSOLIDATE is called, size of root list <= D(n) + t(H) - 1When CONSOLIDATE is called, size of root list <= D(n) + t(H) - 1Work in CONSOLIDATE’s for loop in O(D(n) + t(H)) due to tree linking in each iterationWork in CONSOLIDATE’s for loop in O(D(n) + t(H)) due to tree linking in each iterationTotal Actual Cost is in O(D(n) + t(H)) Total Actual Cost is in O(D(n) + t(H))

Fibonacci Heap Operations: DECREASE-KEY PseudoCodeFibonacci Heap Operations: DECREASE-KEY PseudoCode

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.cascading cut if heap property violation

Fibonacci Heap Operations: DECREASE-KEY AnalysisFibonacci Heap Operations: DECREASE-KEY Analysis

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Heap Potential Function

t (H) = #trees in root listt (H) = #trees in root listm(H) = #marked nodesm(H) = #marked nodes

As soon as 2nd child of x is lost, cut x from parent, making it a new root.As soon as 2nd child of x is lost, cut x from parent, making it a new root.

Actual WorkActual Work::Dominated by cost of CASCADING-CUT Dominated by cost of CASCADING-CUT Assume CASCADING-CUT called recursively c timesAssume CASCADING-CUT called recursively c timesTotal Actual Cost is in O(c) Total Actual Cost is in O(c)

Amortized cost is in Amortized cost is in (1)(1)

Change in potential is at most::Change in potential is at most::t(H) to start with, c-1 added from cuts, t(H) to start with, c-1 added from cuts, + tree rooted at x+ tree rooted at x

c-1 unmarked by c-1 unmarked by cascading cuts cascading cuts + at most 1 from last call + at most 1 from last call to CASCADING-CUTto CASCADING-CUT

Fibonacci Heap Operations: DELETEFibonacci Heap Operations: DELETE

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Amortized cost is in Amortized cost is in (lgn)(lgn)

DECREASE-KEY, EXTRACT-MIN

O(D(n))O(D(n))

Bounding the Maximum DegreeBounding the Maximum Degree

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

EXTRACT-MIN (& DELETE) O(lgn) bounds require D(n) in O(lgn)

Show:Show: Cutting node when it loses 2nd childCutting node when it loses 2nd child nnD log)(

2/)51(

Link only occurs in Link only occurs in EXTRACT-MIN, EXTRACT-MIN, DELETEDELETE

Bounding the Maximum DegreeBounding the Maximum Degree

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Bounding the Maximum DegreeBounding the Maximum Degree

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

kkF 2

Exercise 2.2-8Exercise 2.2-8

Bounding the Maximum DegreeBounding the Maximum Degree

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Thus, EXTRACT-MIN (& DELETE) have O(lgn) time bounds.