7/30/2019 Algorithm Ch6 Heapsort
1/41
AlgorithmsChapter 6 Heapsort
Assistant Professor: ChingChi Lin
Department of Computer Science and Engineering
National Taiwan Ocean University
7/30/2019 Algorithm Ch6 Heapsort
2/41
Outline` Heaps
` Maintainingtheheapproperty
` Buildingaheap
` Theheapsort algorithm
` Priorityqueues
2
7/30/2019 Algorithm Ch6 Heapsort
3/41
The purpose of this chapter` Inthischapter,weintroducetheheapsort algorithm.
` withworstcase runningtimeO(nlgn)
`
an
in
place
sorting
algorithm:
only
a
constant
number
of
array
elementsarestoredoutsidetheinputarrayatanytime.
` thus,requireatmostO(1)additionalmemory
` Wealso
introduce
the heap data
structure.
` anusefuldatastructureforheapsort
` makesanefficientpriorityqueue
3
7/30/2019 Algorithm Ch6 Heapsort
4/41
Heaps` The(Binary) heap datastructureisanarray objectthatcanbe
viewedasanearlycompletebinarytree.
` Abinary
tree
with
n nodes
and
depth
kis
complete iff its
nodes
correspondtothenodesnumberedfrom1ton inthefullbinary
treeofdepthk.
16
14 10
8 7 9 3
2 4 1
1
2 3
4 5 6 7
8 9 10
1423978101416
8 9 107654321
4
7/30/2019 Algorithm Ch6 Heapsort
5/41
Binary tree representations
5
1
2
4
8 9
5
10 11
3
6
12 13
7
14 15
1
2
4
8 9
5
3
6 7
10
Afullbinarytree ofheight3. Acompletebinarytreewith10nodes
andheight
3.
7/30/2019 Algorithm Ch6 Heapsort
6/41
Attributes of a Heap` AnarrayAthatpresentsaheapwithtwoattributes:
` length[A]: thenumberofelementsinthearray.
` heapsize[A]: the
number
of
elements
in
the
heap
stored
with
arrayA.
` length[A] heapsize[A]
length[A]=heapsize[A]=10
16
14 10
8 7 9 3
2 4 1
1
2 3
4 5 6 7
8 9 10
1423978101416
8 9 107654321
6
7/30/2019 Algorithm Ch6 Heapsort
7/41
Basic procedures1/2` Ifacompletebinarytreewithn nodesisrepresented
sequentially,thenforanynodewithindexi,1 i n,wehave
`
A[1]
is
the
root of
the
tree` theparentPARENT(i)isat i/2 ifi 1
` theleftchildLEFT(i)isat2i
` therightchildRIGHT(i)isat2i+1
7
16
14 10
8 7 9 3
2 4 1
1
2 3
4 5 6 7
8 9 10
1423978101416
8 9 107654321
7/30/2019 Algorithm Ch6 Heapsort
8/41
Basic procedures2/2` TheLEFT procedurecancompute2iinoneinstructionbysimply
shiftingthebinaryrepresentationofileftonebitposition.
` Similarly,the
RIGHT procedure
can
quickly
compute
2i+1
by
shiftingthebinaryrepresentationofileftonebitpositionand
addingina1astheloworderbit.
` ThePARENT procedure
can
compute
i/2 by
shifting
iright
one
bitposition.
8
7/30/2019 Algorithm Ch6 Heapsort
9/41
Heap properties` Therearetwokindofbinaryheaps:maxheapsandminheaps.
` Inamaxheap,themaxheapproperty isthatforeverynodei
other
than
the
root,
` thelargestelementinamaxheapisstoredattheroot
` thesubtree rootedatanodecontainsvaluesnolargerthanthat
contained
at
the
node
itself` Inaminheap,theminheapproperty isthatforeverynodei
otherthantheroot,
` thesmallest
element
in
amin
heap
is
at
the
root
` thesubtree rootedatanodecontainsvaluesnosmallerthanthat
containedatthenodeitself
9
A[PARENT(i)] A[i] .
A[PARENT(i)] A[i] .
7/30/2019 Algorithm Ch6 Heapsort
10/41
Max and min heaps
10
14
12 7
10 8 6
9
6 3
5
30
25
2
7 4
10 8 6
10
20 83
50
11
21
MaxHeaps
MinHeaps
7/30/2019 Algorithm Ch6 Heapsort
11/41
The height of a heap` Theheight ofanodeinaheapisthenumberofedgesonthe
longestsimpledownwardpathfromthenodetoaleaf,and
theheightoftheheaptobetheheightoftheroot,thatis
(lgn).
` Forexample:
` theheightofnode2is2
` theheight
of
the
heap
is
3
11
16
14 10
8 7 9 3
2 4 1
1
2 3
4 5 6 7
8 9 10
7/30/2019 Algorithm Ch6 Heapsort
12/41
The remainder of this chapter` Weshallpresentssomebasicproceduresintheremainder
ofthischapter.
`
The
MAX
HEAPIFY procedure,
which
runs
in
O(lgn)
time,
is
the
keytomaintainingthemaxheapproperty.
` TheBUILDMAXHEAP procedure,whichrunsinO(n)time,
producesamaxheapfromanunorderedinputarray.
` TheHEAPSORT procedure,
which
runs
in
O(n lgn)
time,
sorts
an
arrayinplace.
` TheMAXHEAPINSERT,HEAPEXTRACTMAX,HEAPINCREASEKEY,
andHEAPMAXIMUM procedures,whichruninO(lgn)time,
allowthe
heap
data
structure
to
be
used
as
apriority
queue.
12
7/30/2019 Algorithm Ch6 Heapsort
13/41
Outline` Heaps
` Maintainingtheheapproperty
` Buildingaheap
` Theheapsort algorithm
` Priorityqueues
13
7/30/2019 Algorithm Ch6 Heapsort
14/41
The MAXHEAPIFY procedure1/2` MAXHEAPIFY isanimportantsubroutineformanipulatingmax
heaps.
`
Input:
an
array
A
and
an
index
i` Output:thesubtree rootedatindexibecomesamaxheap
` Assume:thebinarytreesrootedatLEFT(i) andRIGHT(i) are
maxheaps,butA[i]maybesmallerthanitschildren
` Method:let
the
value
at
A[i]
float
down in
the
max
heap
14
16
4 10
14 7 9 3
2 8 1
1
2 3
4 5 6 7
8 9 10
16
14 10
8 7 9 3
2 4 1
1
2 3
4 5 6 7
8 9 10
MAXHEAPIFYi
7/30/2019 Algorithm Ch6 Heapsort
15/41
The MAXHEAPIFY procedure2/2MAXHEAPIFY(A,i)
1. l LEFT(i)
2. r RIGHT(i)
3. ifl heapsize[A]andA[l]>A[i]
4. then largest l
5. else largest i
6. ifr heapsize[A]anda[r]>A[largest]
7. then largest r
8. iflargest i
9. then exchangeA[i] A[largest]
10. MAX
HEAPIFY (A,
largest)
15
7/30/2019 Algorithm Ch6 Heapsort
16/41
An example of MAXHEAPIFY procedure
16
16
4 10
14 7 9 3
2 8 1
1
2 3
4 5 6 7
8 9 10
16
14 10
4 7 9 3
2 8 1
1
2 3
4 5 6 7
8 9 10
16
14 10
8 7 9 3
2 4 1
1
2 3
4 5 6 7
8 9 10
i
i
i
7/30/2019 Algorithm Ch6 Heapsort
17/41
The time complexity` Ittakes(1)timetofixuptherelationshipsamongthe
elementsA[i],A[LEFT(i)],andA[RIGHT(i)].
`
Also,
we
need
to
run
MAX
HEAPIFY on
a
subtree rooted
at
one
ofthechildrenofnodei.
` Thechildrenssubtrees eachhavesizeatmost2n/3
` worstcaseoccurswhenthelastrowofthetreeisexactlyhalffull
` TherunningtimeofMAXHEAPIFY is
` solveit
by
case
2of
the
master
theorem
` Alternatively,wecancharacterizetherunningtimeofMAX
HEAPIFY onanodeofheighthasO(h).
17
T(n)=T(2n/3)+(1)= O(lg n)
7/30/2019 Algorithm Ch6 Heapsort
18/41
Outline` Heaps
` Maintainingtheheapproperty
` Buildingaheap
` Theheapsort algorithm
` Priorityqueues
18
7/30/2019 Algorithm Ch6 Heapsort
19/41
Building a Heap
19
` WecanusetheMAXHEAPIFY proceduretoconvertanarray
A=[1..n]intoamaxheapinabottomupmanner.
`
The
elements
in
the
subarray A[(n/2+1)n ]
are
all
leaves of
thetree,andsoeachisa1elementheap.
` TheprocedureBUILDMAXHEAP goesthroughtheremaining
nodesofthetreeandrunsMAXHEAPIFY oneachone.
BUILDMAXHEAP(A)
1. heapsize[A] length[A]
2. for i length[A]/2 downto 1
3. do MAXHEAPIFY(A,i)
7/30/2019 Algorithm Ch6 Heapsort
20/41
An example
20
A 7814109162314
8 9 107654321
4
1 3
2 16 9 10
14 8 7
1
2 3
4 5 6
8 9 10
i7
7/30/2019 Algorithm Ch6 Heapsort
21/41
4
1 3
2 16 9 10
14 8 7
[1]
[2] [3]
[5][4]
[7][6]
[8] [9] [10]
4
1 3
2 16 9 10
14 8 7
[1]
[2] [3]
[5][4]
[7][6]
[8] [9] [10]
4
1 3
14 16 9 10
2 8 7
[1]
[2] [3]
[5][4]
[7][6]
[8] [9] [10]
4
1 10
14 16 9 3
2 8 7
[1]
[2] [3]
[5]
[7][6]
[8] [9] [10]
4
16 10
14 1 9 3
2 8 7
[1]
[2] [3]
[5]
[7][6]
[8] [9] [10]
4
16 10
14 7 9 3
2 8 1
[1]
[2] [3]
[5]
[7][6]
[8] [9] [10]
[4] [4][4]
MAXHEAPIFY(A,5) MAXHEAPIFY(A,4) MAXHEAPIFY(A,3)
MAXHEAPIFY(A,1) MAXHEAPIFY(A,2)
7/30/2019 Algorithm Ch6 Heapsort
22/41
16
4 10
14 7 9 3
2 8 1
[1]
[2] [3]
[5]
[7][6]
[8] [9] [10]
[4]
16
14 10
4 7 9 3
2 8 1
[1]
[2] [3]
[5]
[7][6]
[8] [9] [10]
[4]
16
14 10
8 7 9 3
2 4 1
[1]
[2] [3]
[5]
[7][6]
[8] [9] [10]
[4]
maxheap
7/30/2019 Algorithm Ch6 Heapsort
23/41
Correctness1/2
23
` ToshowwhyBUILDMAXHEAP workcorrectly,weusethe
followingloopinvariant:
` Atthestartofeachiterationoftheforloopoflines23,each
nodei+1,i+2,,n istherootofamaxheap.
` Weneedtoshowthat
` thisinvariant
is
true
prior
to
the
first
loop
iteration
` eachiterationoftheloopmaintainstheinvariant
` theinvariantprovidesausefulpropertytoshowcorrectness
whentheloopterminates.
BUILDMAXHEAP(A)
1. heapsize[A] length[A]
2. for i length[A]/2 downto 1
3. do MAXHEAPIFY(A,i)
7/30/2019 Algorithm Ch6 Heapsort
24/41
Correctness2/2
24
` Initialization: Priortothefirstiterationoftheloop,i=n/2.n/2+1,n isaleafandisthustherootofa
trivialmaxheap.
` Maintenance:Bythe loopinvariant,thechildrenofnodeiarebothrootsofmaxheaps.Thisispreciselythe
conditionrequiredforthecallMAXHEAPIFY(A,i)
tomake
node
ia
max
heap
root.
Moreover,
the
MAXHEAPIFY callpreservesthepropertythat
nodesi+1,i+2,...,nareallrootsofmaxheaps.
` Termination: Attermination,i=0.Bytheloopinvariant,eachnode1,2,,n istherootofamaxheap.
Inparticular,node1is.
7/30/2019 Algorithm Ch6 Heapsort
25/41
Time complexity1/2` Analysis1:
` EachcalltoMAXHEAPIFY costsO(lgn),andthereareO(n)such
calls.
` Thus,therunningtimeisO(n lgn).Thisupperbound,through
correct,isnotasymptoticallytight.
` Analysis2:
` Forannelementheap,heightislgn andatmostn /2h+1
nodesofanyheighth.
` ThetimerequiredbyMAXHEAPIFY whencalledonanodeof
heighth is
O(h).
` Thetotalcostis
.2
)(2
lg
0
lg
01
=
==
+
n
h
h
n
h
h
hnOhO
n
25
7/30/2019 Algorithm Ch6 Heapsort
26/41
Time complexity2/2` Thelastsummationyields
` Thus,therunningtimeofBUILDMAXHEAP canbeboundedas
` Wecanbuildamaxheapfromanunorderedarrayinlinear
time.
2)2/11(
2/1
2 20
=
=
=hh
h
)(2)(2 0
lg
01 nO
h
nOhO
n
h
h
n
h
h =
=
==+
26
7/30/2019 Algorithm Ch6 Heapsort
27/41
Outline` Heaps
` Maintainingtheheapproperty
` Buildingaheap
` Theheapsort algorithm
` Priorityqueues
27
7/30/2019 Algorithm Ch6 Heapsort
28/41
The heapsort algorithm` Sincethemaximumelementofthearrayisstoredattheroot,
A[1]wecanexchange itwithA[n].
`
If
we
now
discard A[n],
we
observe
that
A[1...(n
1)]
can
easily
bemadeintoamaxheap.
` ThechildrenoftherootA[1]remainmaxheaps,butthenew
rootA[1]elementmayviolatethemaxheapproperty,sowe
needto
readjust the
max
heap.
That
is
to
call
MAX
HEAPIFY(A,
1).
28
HEAPSORT(A)
1. BUILDMAXHEAP(A)
2. for i length[A] downto 2
3. do exchangeA[1] A[i]
4. heapsize[A] heapsize[A]1
5. MAXHEAPIFY(A,1)
7/30/2019 Algorithm Ch6 Heapsort
29/41
9
An example
29
16
14 10
8 7 9 3
2 4 1
1
2 3
4 5 6 7
8 10i
A 1614109874321
8 9 107654321
7/30/2019 Algorithm Ch6 Heapsort
30/41
1
14 10
8 7 9 3
2 4 16
[1]
[2] [3]
[5]
[7][6]
[8] [9] [10]
[4]
Initialheap
16
14 10
8 7 9 3
2 4 1
[1]
[2] [3]
[5]
[7][6]
[8] [9] [10]
[4]
Exchange
Heapsize=10
Sorted=[16]
1
14 10
8 7 9 3
2 4 16
[1]
[2] [3]
[5]
[7][6]
[9] [10]
Discard
Heapsize=9
Sorted=[16]
[8]
14
8 10
4 7 9 3
2 1 16
[1]
[2] [3]
[5]
[7][6]
[9] [10][8]
ReadjustHeapsize=9
Sorted=[16]
1
8 10
4 7 9 3
2 14 16
[1]
[2] [3]
[5]
[7][6]
[9] [10][8]
ExchangeHeapsize=9
Sorted=[14,16]
1
8 10
4 7 9 3
2 14 16
[1]
[2] [3]
[5]
[7][6]
[9] [10][8]
DiscardHeapsize=8
Sorted=[14,16]
[4]
[4][4][4]
7/30/2019 Algorithm Ch6 Heapsort
31/41
[8]
Heapsize=7
Sorted=[10,14,16]
8
7 3
4 2 1 9
10 14 16
[1]
[2] [3]
[5]
[7][6]
[9] [10]
Heapsize=6
Sorted=[9,10,14,16]
[8]
Heapsize=5
Sorted=[8,9,10,14,16]
Heapsize=4
Sorted=[7,8,9,10,14,16]
Readjust
Heapsize=8
Sorted=[14,16]
10
8 9
4 7 1 3
2 14 16
[1]
[2] [3]
[5]
[7][6]
[9] [10][8]
9
8 3
4 7 1 2
10 14 16
[1]
[2] [3]
[5]
[7][6]
[9] [10]
[4][4] [4]
7
4 3
1 2 8 9
10 14 16
[1]
[2] [3]
[5]
[7][6]
[9] [10][8]
[4]
4
2 3
1 7 8 9
10 14 16
[1]
[2] [3]
[5]
[7][6]
[9] [10][8]
[4]
3
2 1
4 7 8 9
10 14 16
[1]
[2] [3]
[5]
[7][6]
[9] [10][8]
[4]
Heapsize=3
Sorted=[4,7,8,9,10,14,16]
7/30/2019 Algorithm Ch6 Heapsort
32/41
Time complexity
32
` TheHEAPSORT proceduretakesO(n lgn) time
` thecalltoBUILDMAXHEAP takesO(n)time
` eachofthen1callstoMAXHEAPIFY takesO(lgn)time
7/30/2019 Algorithm Ch6 Heapsort
33/41
Outline` Heaps
` Maintainingtheheapproperty
` Buildingaheap
` Theheapsort algorithm
` Priorityqueues
33
7/30/2019 Algorithm Ch6 Heapsort
34/41
Heap implementation of priority queues
34
` Heapsefficientlyimplementpriorityqueues.
` Therearetwokindsofpriorityqueues:maxpriorityqueues
andminpriorityqueues.
` Wewillfocushereonhowtoimplementmaxpriorityqueues,
whichareinturnbasedonmaxheaps.
` Apriorityqueue isadatastructureformaintainingasetS of
elements,each
with
an
associated
value
called
akey.
7/30/2019 Algorithm Ch6 Heapsort
35/41
Priority queues
35
` Amaxpriorityqueue supportsthefollowingoperations.
` INSERT(S,x): insertstheelementxintothesetS.
` MAXIMUM(S): returnsthe
element
of
S with
the
largest
key.
` EXTRACTMAX(S): removesandreturnstheelementofS with
thelargestkey.
` INCREASEKEY(S,
x,
k):
increases
value
of
element
xs key
to
the
newvaluek.Assumek xs currentkeyvalue.
7/30/2019 Algorithm Ch6 Heapsort
36/41
Finding the maximum element
` MAXIMUM(S): returnstheelementofS withthelargestkey.
` Gettingthemaximumelementiseasy:itstheroot.
` TherunningtimeofHEAPMAXIMUM is(1).
HEAP
MAXIMUM(A)1. returnA[1]
36
7/30/2019 Algorithm Ch6 Heapsort
37/41
Extracting max element
` EXTRACTMAX(S):removesandreturnstheelementofS with
thelargestkey.
` Analysis: constanttime
assignments
+time
for
MAX
HEAPIFY.
` TherunningtimeofHEAPEXTRACTMAX isO(lgn).
HEAPEXTRACTMAX(A)
1. ifheapsize[A]
7/30/2019 Algorithm Ch6 Heapsort
38/41
Increasing key value
` INCREASEKEY(S,x,k):increasesvalueofelementxs keytok.
Assumek xs currentkeyvalue.
` Analysis: thepath
traced
from
the
node
updated
to
the
root
haslengthO(lgn).
` TherunningtimeisO(lgn).
HEAP
INCREASE
KEY (A,
i,
key)1. ifkey1and
A[PARENT(i)]