t13Mergesort

8/8/2019 t13Mergesort

1/36

Dale Roberts

MergesortMergesort

Dale Roberts, Lecturer

Computer Science, IUPUI

E-mail: [email protected]

Department ofComputer and Information Science,School of Science, IUPUI


2/36


3/36


4/36

Dale Roberts

ImpactofBetterAlgorithmsImpactofBetterAlgorithms

Example1: NExample1: N--bodybody--simulation.simulation.Simulate gravitational interactions among N bodies.Simulate gravitational interactions among N bodies.

physicists want N = # atoms in universephysicists want N = # atoms in universe

Brute force method: NBrute force method: N22 steps.steps.

Appel (1981).Appel (1981). N log N steps, enables new research.N log N steps, enables new research.

Example2: Discrete FourierTransform(DFT).Example2: Discrete FourierTransform(DFT).

Breaks down waveforms (sound) into periodic components.Breaks down waveforms (sound) into periodic components.

foundation of signal processingfoundation of signal processing

CD players, JPEG, analyzing astronomical data, etc.CD players, JPEG, analyzing astronomical data, etc.

Grade school method: NGrade school method: N22 steps.steps.

RungeRunge--Knig (1924), CooleyKnig (1924), Cooley--Tukey (1965).Tukey (1965).FFT algorithm: N log N steps, enables new technology.FFT algorithm: N log N steps, enables new technology.


5/36

Dale Roberts

MergesortMergesort

Mergesort (divideMergesort (divide--andand--conquer)conquer)

Divide array into two halves.Divide array into two halves.

A L G O R I T H M S

divideA L G O R I T H M S


6/36

Dale Roberts

MergesortMergesort



Recursively sort each half.Recursively sort each half.

sort

A L G O R I T H M S


A G L O R H I M S T


7/36

Dale Roberts

MergesortMergesort



Recursively sort each half.Recursively sort each half.

Merge two halves to make sorted whole.Merge two halves to make sorted whole.

merge

sort

A L G O R I T H M S


A G L O R H I M S T

A G H I L M O R S T


8/36

Dale Roberts

auxiliary array

smallest smallest

A G L O R H I M S T

MergingMerging

Merge.Merge.

Keep track of smallest element in each sorted half.Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.Insert smallest of two elements into auxiliary array.

Repeat until done.Repeat until done.

A


9/36

Dale Roberts

auxiliary array

smallest smallest

A G L O R H I M S T

A

MergingMerging

Merge.Merge.




G


10/36

Dale Roberts

auxiliary array

smallest smallest

A G L O R H I M S T

A G

MergingMerging

Merge.Merge.




H


11/36

Dale Roberts

auxiliary array

smallest smallest

A G L O R H I M S T

A G H

MergingMerging

Merge.Merge.




I


12/36

Dale Roberts

auxiliary array

smallest smallest

A G L O R H I M S T

A G H I

MergingMerging

Merge.Merge.




L


13/36

Dale Roberts

auxiliary array

smallest smallest

A G L O R H I M S T

A G H I L

MergingMerging

Merge.Merge.




M


14/36

Dale Roberts

auxiliary array

smallest smallest

A G L O R H I M S T

A G H I L M

MergingMerging

Merge.Merge.




O


15/36


16/36

Dale Roberts

auxiliary array

first half

exhaustedsmallest

A G L O R H I M S T

A G H I L M O R

MergingMerging

Merge.Merge.




S


17/36

Dale Roberts

auxiliary array

first half

exhaustedsmallest

A G L O R H I M S T

A G H I L M O R S

MergingMerging

Merge.Merge.




T


18/36

Dale Roberts

auxiliary array

first half

exhausted

second half

exhausted

A G L O R H I M S T

A G H I L M O R S T

MergingMerging

Merge.Merge.





19/36

Dale Roberts

ImplementingMergesortImplementingMergesort

Item aux[MAXN];

void mergesort(Item a[], int left, int right) {

int mid = (right + left) / 2;if (right


20/36


21/36

Dale Roberts

ImplementingMergesortImplementingMergesort

void merge(Item a[], int left, int mid, int right) {

int i, j, k;

for (i = mid+1; i > left; i--)

aux[i-1] = a[i-1];

for (j = mid; j < right; j++)aux[right+mid-j] = a[j+1];

for (k = left; k


22/36

Dale Roberts

Mergesort DemoMergesort Demo

MergesortMergesortTheauxilliary array usedinthemergingTheauxilliary array usedinthemerging

operationisshowntotherightofthearray a[],goingoperationisshowntotherightofthearray a[],goingfrom(N+1,1)to(2N,2N).from(N+1,1)to(2N,2N).

ThedemoisadynamicrepresentationofthealgorithmThedemoisadynamicrepresentationofthealgorithm

inaction,sortinganarray acontainingapermutationofinaction,sortinganarray acontainingapermutationof

theintegers1through N. Foreach i,thearray elementtheintegers1through N. Foreach i,thearray elementa[i]isdepictedasablackdotplottedatposition(i,a[i]).a[i]isdepictedasablackdotplottedatposition(i,a[i]).

Thus,theendresultofeach sortisadiagonalofblackThus,theendresultofeach sortisadiagonalofblack

dotsgoingfrom(1,1)atthebottomleftto(N, N)atthedotsgoingfrom(1,1)atthebottomleftto(N, N)atthe

topright.Each timeanelementismoved,agreendotistopright.Each timeanelementismoved,agreendotis

leftatitsoldposition.Thusthemovingblackdotsgivealeftatitsoldposition.Thusthemovingblackdotsgiveadynamicrepresentationoftheprogressofthesortanddynamicrepresentationoftheprogressofthesortand

thegreendotsgivea history ofthedatathegreendotsgivea history ofthedata--movementcost.movementcost.


23/36

Dale Roberts

ComputationalComplexityComputationalComplexity

Frameworktostudy efficiency ofalgorithms.Frameworktostudy efficiency ofalgorithms. Example =Example =

sorting.sorting.

MACHINE MODEL = count fundamental operations.MACHINE MODEL = count fundamental operations.

count number of comparisonscount number of comparisons

UPPER BOUND = algorithm to solve the problem (worstUPPER BOUND = algorithm to solve the problem (worst--case).case).

N logN log22 N from mergesortN from mergesort

LOWER BOUND = proof that no algorithm can do better.LOWER BOUND = proof that no algorithm can do better.

N logN log22 NN -- N logN log22 ee

OPTIMAL ALGORITHM: lower bound ~ upper bound.OPTIMAL ALGORITHM: lower bound ~ upper bound.

mergesortmergesort


24/36

Dale Roberts

DecisionTreeDecisionTree

print

a1,a2,a3

a1


25/36

Dale Roberts

Comparison BasedSorting LowerBoundComparison BasedSorting LowerBound

Theorem.Theorem. Any comparisonbasedsortingalgorithmmustAny comparisonbasedsortingalgorithmmust

useuse;;((N logN log22N)comparisons.N)comparisons.

Proof.Proof. Worstcasedictatedby tree height h.Worstcasedictatedby tree height h.N! different orderings.N! different orderings.

One (or more) leaves corresponding to each ordering.One (or more) leaves corresponding to each ordering.

Binary tree with N! leaves must have heightBinary tree with N! leaves must have height

Foodforthought.Foodforthought. Whatifwedon'tusecomparisons?Whatifwedon'tusecomparisons?Stay tuned for radix sort.Stay tuned for radix sort.

eNNN

eN

Nh

N

22

2

2

loglog

)/(log

)!(log

!

u

u

Stirling'sformula


26/36

Dale Roberts

MergesortAnalysisMergesortAnalysis

How longdoesmergesorttake?How longdoesmergesorttake?

Bottleneck = merging (and copying).Bottleneck = merging (and copying).

merging two files of size N/2 requires N comparisonsmerging two files of size N/2 requires N comparisons

T(N) = comparisons to mergesort N elements.T(N) = comparisons to mergesort N elements.

to make analysis cleaner, assume N is a power of 2to make analysis cleaner, assume N is a power of 2

Claim.Claim. T(N) = N logT(N) = N log22 N.N.Note: same number of comparisons for ANY file.Note: same number of comparisons for ANY file.

even already sortedeven already sorted

We'll prove several different ways to illustrate standard techniques.We'll prove several different ways to illustrate standard techniques.

]

!

! otherwise)2/(2

1if0

)T(

merginghalvesbothsorting

NNT

N

N


27/36

Dale Roberts

ProfilingMergesortEmpiricallyProfilingMergesortEmpirically

void merge(Item a[], int left, int mid, int right){

int i, j, k;

for (i = mid+1;i > left;i--)

aux[i-1] = a[i-1];

for (j = mid;j < right;j++)

aux[right+mid-j] = a[j+1];

for (k = left;k


28/36

Dale Roberts

SortingAnalysisSummarySortingAnalysisSummary

Runningtimeestimates:Runningtimeestimates:

Home pc executes 10Home pc executes 1088 comparisons/second.comparisons/second.

Supercomputer executes 10Supercomputer executes 101212 comparisons/second.comparisons/second.

computer

home

super

thousand

instant

instant

million

2.8 hours

1second

billion

317 years

1.6 weeks

InsertionSort(N2)

thousand

instant

instant

million

1sec

instant

billion

18min

instant

Mergesort(N log N)

thousand

instantinstant

million

0.3secinstant

billion

6mininstant

Quicksort(N log N)


29/36

Dale Roberts

AcknowledgementsAcknowledgements

Sorting methods are discussed in our Sedgewick text. Slides and demos are

from our texts website at princeton.edu.Special thanks to Kevin Wayne in helping to prepare this material.


30/36

Dale Roberts

ExtraSlidesExtraSlides


31/36

Dale Roberts

Proofby PictureofRecursionTreeProofby PictureofRecursionTree

T(N)

T(N/2)T(N/2)

T(N/4)T(N/4)T(N/4) T(N/4)

T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2)

N

T(N / 2k)

2(N/2)

4(N/4)

2k(N /

2k)

N/2(2)

...

...

log2N

N log2N

]

!

! otherwise)2/(2

1if0

)T(merging

halvesbothsorting

NNT

N

N


32/36

Dale Roberts

Proofby TelescopingProofby Telescoping

Claim.Claim. T(N) = N logT(N) = N log22 N (when N isapowerof2).N (when N isapowerof2).

Proof.Proof. ForN > 1:ForN > 1:

]

!

! otherwise)2/(2

1if0

)T(

merginghalvesbothsorting

NNT

N

N

N

NN

NNT

N

NT

N

NT

N

NT

N

NT

N

2

log

log

11/

)/(

114/

)4/(

12/

)2/(

1)2/(2)(

2

!

!

!

!

!

.

.


33/36

Dale Roberts

Mathematical InductionMathematical Induction

Mathematicalinduction.Mathematicalinduction.

Powerful and general proof technique in discrete mathematics.Powerful and general proof technique in discrete mathematics.To prove a theorem true for all integers kTo prove a theorem true for all integers k uu 0:0:

Base case: prove it to be true for N = 0.Base case: prove it to be true for N = 0.

Induction hypothesis: assuming it is true for arbitrary NInduction hypothesis: assuming it is true for arbitrary N

Induction step: show it is true for N + 1Induction step: show it is true for N + 1

Claim:Claim: 0+1+2+3+...+ N = N(N+1) / 2 forall N0+1+2+3+...+ N = N(N+1) / 2 forall N uu 00..

Proof:Proof: (by mathematicalinduction)(by mathematicalinduction)Base case (N = 0).Base case (N = 0).

0 = 0(0+1) / 2.0 = 0(0+1) / 2.

Induction hypothesis: assume 0 + 1 + 2 + . . . + N = N(N+1) / 2Induction hypothesis: assume 0 + 1 + 2 + . . . + N = N(N+1) / 2Induction step: 0 + 1 + . . . + N + N + 1 = (0 + 1 + . . . + N) + N+1Induction step: 0 + 1 + . . . + N + N + 1 = (0 + 1 + . . . + N) + N+1

= N (N+1) /2 + N+1= N (N+1) /2 + N+1= (N+2)(N+1) / 2= (N+2)(N+1) / 2


34/36

Dale Roberts

Proofby InductionProofby Induction

Claim.Claim. T(N) = N logT(N) = N log22 N (when N isapowerof2).N (when N isapowerof2).

Proof.Proof. (by inductionon N)(by inductionon N)

Base case: N = 1.Base case: N = 1.Inductive hypothesis: T(N) = N logInductive hypothesis: T(N) = N log22 N.N.

Goal: show that T(2N) = 2N logGoal: show that T(2N) = 2N log22 (2N).(2N).

)2(l2

21)2(l2

2l2

2)(2)2(

2

2

2

NN

NNN

NNN

NNTNT

!

!

!

!

]

!

! o is)2/(2

1i0

)T(

m gi galv soso i g

NNT

N

N


35/36

Dale Roberts


WhatifN isnotapowerof2?WhatifN isnotapowerof2?

T(N) satisfies following recurrence.T(N) satisfies following recurrence.

Claim.Claim. T(N)T(N) ee NN loglog22NN..Proof.Proof. Seesupplementalslides.Seesupplementalslides.

- ]

!

! ot is2/2/

1if0

)T(

m gi galfig tsolvalfl ftsolv

NNTNT

N

N


36/36

Dale Roberts


Claim.Claim. T(N)T(N) ee NN loglog22NN..

Proof.Proof. (by inductionon N)(by inductionon N)

Base case: N = 1.Base case: N = 1.

Define nDefine n11 == --N / 2N / 2 ,, nn22 == N / 2N / 2..

Induction step: assume true for 1, 2, . . . , NInduction step: assume true for 1, 2, . . . , N 1.1.

NN

NNN

NnN

Nnnnn

Nnnnn

NnTnTNT

2

2

22

222221

222121

21

log

)1log(

log

loglog

loglog

)()()(

!

e

!

e

e

e

1loglog

2/2

2/

222

log

2

2

e

e

!

Nn

Nn

N

- ]

!

! ot is2/2/

1if0

)T(

m gi galfig tsolvalfl ftsolv

NNTNT

N

N

t13Mergesort

Documents