Chapter 2 Analysis Framework 2.1 – 2.2. Homework Remember questions due Wed. Remember questions due Wed. Read sections 2.1 and 2.2 Read sections 2.1 and.

Chapter 2Chapter 2

Analysis Framework 2.1 – 2.2Analysis Framework 2.1 – 2.2

HomeworkHomework

Remember questions due Wed.Remember questions due Wed. Read sections 2.1 and 2.2Read sections 2.1 and 2.2 pages 41-50 and 52-59pages 41-50 and 52-59

Agenda: Analysis of Agenda: Analysis of AlgorithmsAlgorithms BlackboardBlackboard Issues:Issues:

– CorrectnessCorrectness– Time efficiencyTime efficiency– Space efficiencySpace efficiency– OptimalityOptimality

Approaches: Approaches: – Theoretical analysisTheoretical analysis– Empirical analysisEmpirical analysis

Time EfficiencyTime Efficiency

Time efficiency is analyzed by determining the Time efficiency is analyzed by determining the number of repetitions number of repetitions

of the of the basic operationbasic operation

as a function of as a function of input sizeinput size

Theoretical analysis of Theoretical analysis of time efficiencytime efficiency

Basic operation:Basic operation: the operation that contributes the operation that contributes most towards the running time of the most towards the running time of the algorithm.algorithm.

TT((nn) ) ≈≈ ccopopCC((nn))running time execution timefor basic operation

Number of times basic operation is

executed

input size

Input size and basic Input size and basic operation examplesoperation examples

ProblemProblem Input size measureInput size measure Basic operationBasic operation

Search for key in Search for key in list of list of nn items items

Number of items in Number of items in list list nn

Key comparisonKey comparison

Multiply two Multiply two matrices of floating matrices of floating point numberspoint numbers

Dimensions of Dimensions of matricesmatrices

Floating point Floating point multiplicationmultiplication

Compute Compute aann nnFloating point Floating point multiplicationmultiplication

Graph problemGraph problem #vertices and/or #vertices and/or edgesedges

Visiting a vertex or Visiting a vertex or traversing an edgetraversing an edge

Empirical analysis of Empirical analysis of time efficiencytime efficiency Select a specific (typical) sample of Select a specific (typical) sample of

inputsinputs Use physical unit of time (e.g., Use physical unit of time (e.g.,

milliseconds) milliseconds)

OROR Count actual number of basic Count actual number of basic

operations operations Analyze the empirical dataAnalyze the empirical data

Best-case, average-Best-case, average-case, worst-casecase, worst-case

For some algorithms efficiency depends on type of input:For some algorithms efficiency depends on type of input: Worst case:Worst case: W(W(nn) – maximum over inputs of size ) – maximum over inputs of size nn Best case:Best case: B(B(nn) – minimum over inputs of size ) – minimum over inputs of size nn

Average case:Average case: A(A(nn) – “average” over inputs of size ) – “average” over inputs of size nn– Number of times the basic operation will be executed on typical Number of times the basic operation will be executed on typical

inputinput– NOT the average of worst and best caseNOT the average of worst and best case– Expected number of basic operations repetitions considered as a Expected number of basic operations repetitions considered as a

random variable under some assumption about the probability random variable under some assumption about the probability distribution of all possible inputs of size distribution of all possible inputs of size nn

Example: Sequential Example: Sequential searchsearch Problem:Problem: Given a list of Given a list of nn elements and a elements and a

search key search key KK, find an element equal to , find an element equal to KK, if , if any.any.

Algorithm:Algorithm: Scan the list and compare its Scan the list and compare its successive elements with successive elements with KK until either a until either a matching element is found (matching element is found (successful searchsuccessful search) ) of the list is exhausted (of the list is exhausted (unsuccessful searchunsuccessful search))

Worst caseWorst case Best caseBest case Average caseAverage case

Types of formulas for Types of formulas for basic operation countbasic operation count

Exact formulaExact formula

e.g., C(e.g., C(nn) = ) = nn((nn-1)/2-1)/2

Formula indicating order of growth with Formula indicating order of growth with specific multiplicative constantspecific multiplicative constant

e.g., C(e.g., C(nn) ) ≈≈ 0.5 0.5 nn22

Formula indicating order of growth with Formula indicating order of growth with unknown multiplicative constantunknown multiplicative constant

e.g., C(e.g., C(nn) ) ≈≈ cncn22

Order of growth Order of growth

Most important: Order of growth within a Most important: Order of growth within a constant multiple as constant multiple as nn→∞→∞

Example:Example:– How much faster will algorithm run on How much faster will algorithm run on

computer that is twice as fast?computer that is twice as fast?– How much longer does it take to solve problem How much longer does it take to solve problem

of double input size?of double input size? See table 2.1 See table 2.1

Table 2.1Table 2.1

Day 4: AgendaDay 4: Agenda

O, O, ΘΘ, , ΩΩ LimitsLimits DefinitionsDefinitions ExamplesExamples Code Code O(?) O(?)

HomeworkHomework

Due on Monday 11:59PMDue on Monday 11:59PM Electronic submission see Electronic submission see

website.website. Try to log into BlackboardTry to log into Blackboard Finish reading 2.1 and 2.2Finish reading 2.1 and 2.2 page 60-61, questions 2, 3, 4, 5, page 60-61, questions 2, 3, 4, 5,

& 9& 9

Asymptotic growth rateAsymptotic growth rate

A way of comparing functions that ignoresA way of comparing functions that ignoresconstant factors and small input sizesconstant factors and small input sizes O(O(gg((nn)): class of functions )): class of functions t t ((nn) that grow ) that grow

no fasterno faster than than gg((nn)) ΘΘ((gg((nn)): class of functions )): class of functions t t ((nn) that grow ) that grow

at same rateat same rate as as gg((nn)) ΩΩ((gg((nn)): class of functions )): class of functions t t ((nn) that grow ) that grow

at least as fastat least as fast as as gg((nn))see figures 2.1, 2.2, 2.3see figures 2.1, 2.2, 2.3

Big-ohBig-oh

Big-omegaBig-omega

Big-thetaBig-theta

Using LimitsUsing Limits

c

ng

ntn

0

)(

)(lim

if t(n) grows slower than g(n)

if t(n) grows faster than g(n)

if t(n) grows at the same rate as g(n)

L’Hopital’s RuleL’Hopital’s Rule

)('

)('

)(

)(lim

ng

nt

ng

ntn

Examples:Examples:

10n vs. 2n2 n(n+1)/2 vs. n2 logb n vs. logc n

DefinitionDefinition

f(n)f(n) = O( = O( g(n)g(n) ) if there exists ) if there exists a positive constant a positive constant cc and and a non-negative interger a non-negative interger nn00

such thatsuch thatf(n)f(n) cc·g(n)·g(n)

for every for every nn > > nn00

Examples:Examples: 10n is O(2n10n is O(2n22)) 5n+20 is O(10n)5n+20 is O(10n)

Basic Asymptotic Basic Asymptotic Efficiency classesEfficiency classes

11 constantconstant

log log nn logarithmiclogarithmic

nn linearlinear

n n log log nn n n log log nn

nn22 quadraticquadratic

nn33 cubiccubic

22nn exponentialexponential

n!n! factorialfactorial

Non-recursive Non-recursive algorithm analysisalgorithm analysisAnalysis Steps:Analysis Steps: Decide on parameter Decide on parameter nn indicating indicating input sizeinput size Identify algorithm’s Identify algorithm’s basic operationbasic operation Determine Determine worstworst, , averageaverage, and , and bestbest case for case for

input of size input of size nn Set up summation for Set up summation for C(n) C(n) reflecting reflecting

algorithm’s loop structurealgorithm’s loop structure Simplify summation using standard formulasSimplify summation using standard formulas

ExampleExample

for (x = 0; x < n; x++)for (x = 0; x < n; x++)a[x] = max(b[x],c[x]);a[x] = max(b[x],c[x]);

ExampleExample

for (x = 0; x < n; x++)for (x = 0; x < n; x++)

for (y = x; y < n; y++)for (y = x; y < n; y++)a[x][y] = max(b[x],c[y]);a[x][y] = max(b[x],c[y]);

ExampleExample

for (x = 0; x < n; x++)for (x = 0; x < n; x++)

for (y = 0; y < n/2; y++)for (y = 0; y < n/2; y++)

for (z = 0; z < n/3; z++)for (z = 0; z < n/3; z++)a[z] = max(a[x],c[y]);a[z] = max(a[x],c[y]);

ExampleExample

y = ny = n

while (y > 0)while (y > 0)

if (a[y--] == b[y--])if (a[y--] == b[y--])

break;break;


Go over the answer to hw2Go over the answer to hw2 Try electronic submissionTry electronic submission Do some problems on the boardDo some problems on the board Time permitting: Recursive Time permitting: Recursive

algorithm analysisalgorithm analysis

HomeworkHomework

Remember to electronically Remember to electronically submit hw3 before Tues. morningsubmit hw3 before Tues. morning

Read section 2.3 thoroughly!Read section 2.3 thoroughly!pages 61-67pages 61-67


First, what have we learned so far First, what have we learned so far about non-recursive algorithm about non-recursive algorithm analysisanalysis

Second, logSecond, logbbn and bn and bnn

The enigma is solved.The enigma is solved. Third, what is the deal with Third, what is the deal with

AbercrombieAbercrombie & & FitchFitch Fourth, recursive analysis tool kit.Fourth, recursive analysis tool kit.

http://www.abercrombie.com/



Homework 4 and Exam Homework 4 and Exam 11 Last homework before Exam 1Last homework before Exam 1 Due on Friday 2/6 (electronic?)Due on Friday 2/6 (electronic?) Will be returned on Monday 2/9Will be returned on Monday 2/9 All solutions will be posted next All solutions will be posted next

Monday 2/9Monday 2/9 Exam 1 Wed. 2/11Exam 1 Wed. 2/11

Homework 4Homework 4

Page 68 questions 4, 5 and 6Page 68 questions 4, 5 and 6 Pages 77 and 78 questions 8 and Pages 77 and 78 questions 8 and

99 We will do similar example We will do similar example

questions all day on Wed 2/4questions all day on Wed 2/4

What have we What have we learned?learned? Non-recursive algorithm analysisNon-recursive algorithm analysis First, Identify problem size.First, Identify problem size.

– Typically,Typically,– a loop countera loop counter– an array sizean array size– a set sizea set size– the size of a valuethe size of a value

Basic operationsBasic operations

Second, identify the basic operationSecond, identify the basic operation Usually a small block of code or even a Usually a small block of code or even a

single statement that is executed over single statement that is executed over and over.and over.

Sometimes the basic operation is a Sometimes the basic operation is a comparison that is hidden inside of the comparison that is hidden inside of the looploop

Example:Example: while (target != a[x])while (target != a[x])

x++; x++;

Single loopsSingle loops

One loop from 1 to nOne loop from 1 to n O(n)O(n) Be aware this is the same asBe aware this is the same as

– 2 independent loops from 1 to n2 independent loops from 1 to n– c independent loops from 1 to nc independent loops from 1 to n– A loop from 5 to n-1A loop from 5 to n-1– A loop from 0 to n/2A loop from 0 to n/2– A loop from 0 to n/cA loop from 0 to n/c

Nested loopsNested loops

for (x = 0; x < n; x++)for (x = 0; x < n; x++)for (y = 0; y < n; y++)for (y = 0; y < n; y++)

O(nO(n22))

for (x = 0; x < n; x++)for (x = 0; x < n; x++)for (y = 0; y < n; y++)for (y = 0; y < n; y++)

for (z = 0; z < n; z++)for (z = 0; z < n; z++) O(nO(n33))

But remember: We can have c independent But remember: We can have c independent nested loops, or the loops can be terminated nested loops, or the loops can be terminated early n/c.early n/c.

Most non-recursive Most non-recursive algorithms reduce to one algorithms reduce to one of these efficiency of these efficiency classesclasses

11 constantconstant

log log nn logarithmiclogarithmic

nn linearlinear

n n log log nn n n log log nn

nn22 quadraticquadratic

nn33 cubiccubic

22nn exponentialexponential

What else?What else?

Best cases often arise when loops Best cases often arise when loops terminate early for specific inputs.terminate early for specific inputs.

For worst cases, consider the For worst cases, consider the following: Is it possible that a loop will following: Is it possible that a loop will NOT terminate earlyNOT terminate early

Average cases are not the mid-pointer Average cases are not the mid-pointer or average of the best and worst caseor average of the best and worst case– Average cases require us to consider a set Average cases require us to consider a set

of typical inputof typical input– We won’t really worry too much about We won’t really worry too much about

average case until we hit more difficult average case until we hit more difficult problems.problems.

Anything else?Anything else?

Important questions you should Important questions you should be asking?be asking?

How does logHow does log22n arise in non-n arise in non-recursive algorithms?recursive algorithms?

How does 2How does 2nn arise? arise? Dr. B. Please, show me the actual Dr. B. Please, show me the actual

loops that cause this!loops that cause this!

LogLog22nn

for (x = 0; x < n; x++) {for (x = 0; x < n; x++) {

Basic operation;Basic operation;

x = x*2;x = x*2;

}}

for every item in the list dofor every item in the list do

Basic operation;Basic operation;

eliminate or discard half the items;eliminate or discard half the items;

Note: These types Note: These types of logof log22n loops can n loops can be nested inside of be nested inside of O(n), O(nO(n), O(n22), or O(n), or O(nkk) ) loops, which leads loops, which leads toto

•n log nn log n•nn22 log n log n•nnkk log n log n

LogLog22nn

Which, by the way, is pretty much Which, by the way, is pretty much equivalent to logequivalent to logbbn, ln, or the n, ln, or the magical lg(n)magical lg(n)

But, don’t be mistakenBut, don’t be mistaken O(n log n) is different than O(n) O(n log n) is different than O(n)

even though log n grows so slow.even though log n grows so slow.

Last thing: 2Last thing: 2nn

How does 2How does 2nn arise in real arise in real algorithms?algorithms?

Lets consider nLets consider nnn:: How can you program n nested How can you program n nested

loopsloops Its impossible right?Its impossible right? So, how the heck does it happen in So, how the heck does it happen in

practice?practice?

ThinkThink

Write an algorithm that prints “*” Write an algorithm that prints “*” 22nn times. times.

Is it hard?Is it hard? It depends on how you think.It depends on how you think.

Recursive wayRecursive way

fun(int x) {fun(int x) {

if (x > 0) {if (x > 0) {

print “*”;print “*”;

fun(x-1);fun(x-1);

fun(x-1);fun(x-1);

}}

}}

Then call the function fun(n);Then call the function fun(n);

Non-recursive wayNon-recursive way

for (x = 0; x < pow(2,n); x++)for (x = 0; x < pow(2,n); x++)

print “*”;print “*”;

WTF?WTF?

Be very wary of your input size.Be very wary of your input size.

If you input size is truly n, then this is If you input size is truly n, then this is truly O(2truly O(2nn) with just one loop.) with just one loop.

That’s it!That’s it!

That’s really it.That’s really it. That’s everything I want you do That’s everything I want you do

know about non-recursive know about non-recursive algorithm analysisalgorithm analysis

Well, for now.Well, for now.

EnigmaEnigma

We know that logWe know that log22n = n = (log(log33n)n) But what about 2But what about 2nn = = (3(3nn)) Here is how I will disprove it…Here is how I will disprove it… This gives you an idea of how I This gives you an idea of how I

like to see questions answered.like to see questions answered.

Algorithm AnalysisAlgorithm Analysis

Recursive Algorithm ToolkitRecursive Algorithm Toolkit

Example Recursive Example Recursive evaluation of evaluation of nn ! ! Definition:Definition: n n ! = 1*2! = 1*2*…*(n-*…*(n-1)*1)*nn Recursive definition of Recursive definition of nn!:!:

if if nn=0 then =0 then FF((nn) := 1) := 1

else else FF((nn) := ) := FF((nn-1) * -1) * nn

return return FF((nn) ) Recurrence for number of Recurrence for number of

multiplications to compute multiplications to compute nn!:!:

Important recurrence Important recurrence types:types: One (constant) operation reduces One (constant) operation reduces

problem size by one.problem size by one.T(T(nn) = T() = T(nn-1) + -1) + cc T(1) = T(1) = dd

Solution: T(Solution: T(nn) = () = (nn-1)-1)cc + + d d linearlinear

Important recurrence Important recurrence types:types: A pass through input reduces A pass through input reduces

problem size by one.problem size by one.T(T(nn) = T() = T(nn-1) + -1) + cncn T(1) = T(1) = dd

Solution: T(Solution: T(nn) = [) = [nn((n+n+1)/2 – 1] 1)/2 – 1] c c + + d d quadraticquadratic

Important recurrence Important recurrence types:types: One (constant) operation reduces One (constant) operation reduces

problem size by half. problem size by half. T(T(nn) = T() = T(nn/2) + /2) + cc T(1) = T(1) = dd

Solution: T(Solution: T(nn) = ) = cc lg lg n n + + d d logarithmiclogarithmic

Important recurrence Important recurrence types:types: A pass through input reduces A pass through input reduces

problem size by half.problem size by half.T(T(nn) = 2T() = 2T(nn/2) + /2) + cn cn T(1) = T(1) = dd

Solution: T(Solution: T(nn) = ) = cncn lg lg n n + + d n d n n n loglog n n

A general divide-and-A general divide-and-conquer recurrenceconquer recurrence

TT((nn) = ) = aTaT((n/bn/b) + ) + f f ((nn)) where where f f ((nn)) == ΘΘ((nnkk))

1.1. a < ba < bkk T T((nn) = ) = ΘΘ((nnkk))

2.2. a = ba = bkk T T((nn) = ) = ΘΘ((nnk k lg lg n n ))

3.3. a > ba > bkk T T((nn) = ) = Θ Θ ((nnlog log b b aa))

Note:Note: the same results hold with O instead of the same results hold with O instead of ΘΘ..

Recursive Algorithm Recursive Algorithm AnalysisAnalysisInput: an array of floats a[0…n-1] andInput: an array of floats a[0…n-1] anda integer counter xa integer counter xFun(int x, float a[]) {Fun(int x, float a[]) {

if (x == 0) if (x == 0) return a[0];return a[0];

if (a[0] > a[x])if (a[0] > a[x])swap(a[0], a[x]); swap(a[0], a[x]);

Fun(x-1, a);Fun(x-1, a);}}

ExampleExample

Fun(int x, float a[]) {Fun(int x, float a[]) {

if (x == 0) if (x == 0)

return a[0];return a[0];

if (a[0] > a[x])if (a[0] > a[x])

swap(a[0], a[x]); swap(a[0], a[x]);

Fun(x-1, a);Fun(x-1, a);

}}

9 5 1 6 3 89 5 1 6 3 8 Fun(5, a)Fun(5, a) 8 5 1 6 3 8 5 1 6 3 99 Fun(4, a)Fun(4, a) 3 5 1 6 3 5 1 6 88 99 Fun(3, a)Fun(3, a) 3 5 1 3 5 1 66 88 99 Fun(2, a)Fun(2, a) 1 5 1 5 33 66 88 99 Fun(1, a)Fun(1, a) 1 1 55 33 66 88 99 Fun(0, a)Fun(0, a) return a[0] return a[0] 1 1

AnalysisAnalysis

Fun(int Fun(int xx, float a[]) {, float a[]) {

if (if (x == 0x == 0) )


if (a[0] > a[x])if (a[0] > a[x])

swap(a[0], swap(a[0], a[x]); a[x]);

Fun(Fun(x-1x-1, a);, a);

}}

First, identify the First, identify the input sizeinput size

The running time The running time seems to depend seems to depend on the value of xon the value of x

AnalysisAnalysis


if (x == 0) if (x == 0)


if (a[0] > a[x])if (a[0] > a[x])



}}

Second, identify Second, identify what terminates what terminates the recursionthe recursion

Think of the running Think of the running time as function Fun(x)time as function Fun(x)

Fun(0) is the base caseFun(0) is the base case

AnalysisAnalysis


if (x == 0) if (x == 0)


if (a[0] > a[x])if (a[0] > a[x])



}}

Third, identify the Third, identify the basic operationbasic operation

The basic The basic operation could operation could be a constant be a constant operationoperation

Or it could be Or it could be embedded in a embedded in a loop that depends loop that depends on the input sizeon the input size

AnalysisAnalysis

Fun(int x, float a[]) {Fun(int x, float a[]) {if (x == 0) if (x == 0)

return a[0];return a[0];if (a[0] > a[x])if (a[0] > a[x])

swap(a[0], swap(a[0], a[x]); a[x]); Fun(x-1, a);Fun(x-1, a);

}}

Fourth, identify Fourth, identify the recursive call the recursive call and how the input and how the input size is changed.size is changed.

Warning the input Warning the input size reduction may size reduction may not be part of the not be part of the recursive call.recursive call.

AnalysisAnalysis

Fun(int x, float a[]) {Fun(int x, float a[]) {if (x == 0) if (x == 0)

return a[0];return a[0];if (a[0] > a[x])if (a[0] > a[x])

swap(a[0], swap(a[0], a[x]); a[x]); Fun(x-1, a);Fun(x-1, a);

}}

Finally, put all the Finally, put all the pieces togetherpieces together

Base case: Fun(0)Base case: Fun(0) Recursive structure:Recursive structure:

Fun(x) = Fun(x-1)Fun(x) = Fun(x-1) Interior complexityInterior complexity

Fun(x) = Fun(x-1) + O(1)Fun(x) = Fun(x-1) + O(1) Recursive algorithms Recursive algorithms

usually fit one of the 4 usually fit one of the 4 basic modelsbasic models

Recursive Algorithm Recursive Algorithm AnalysisAnalysis

Input: vector of integers v Input: vector of integers v Fun(v[1…n]) {Fun(v[1…n]) {

if size of v is 1 return 1if size of v is 1 return 1elseelse

q = Fun(v[1..n-1]);q = Fun(v[1..n-1]);if (v[q] > v[n])if (v[q] > v[n]) swap(v[q], v[n]);swap(v[q], v[n]);temp = 0;temp = 0;for x = 1 to nfor x = 1 to n if (v[x] > temp) if (v[x] > temp) temp = v[x];temp = v[x]; p = x;p = x;return p;return p;

}}

ExampleExample

Fun(3 1 4 2 5): Fun(3 1 4 2 5): temp = Fun(3 1 4 2)temp = Fun(3 1 4 2)

Fun(3 1 4 2):Fun(3 1 4 2):temp = Fun(3 1 4)temp = Fun(3 1 4)

Fun(3 1 4): Fun(3 1 4): temp = Fun(3 1)temp = Fun(3 1)

Fun(3 1): Fun(3 1): temp = Fun(3)temp = Fun(3)

Fun(3): Fun(3): return 1;return 1;

Fun(3 1): Fun(3 1): v[q] = 3; v[n] = 1; v: v[q] = 3; v[n] = 1; v: 1 31 3 4 2 5; return 2 4 2 5; return 2

Fun(3 1 4):Fun(3 1 4):v[q] = 3; v[n] = 4; v: 1 3 4 2 5; return 3v[q] = 3; v[n] = 4; v: 1 3 4 2 5; return 3

Fun(3 1 4 2):Fun(3 1 4 2):v[q] = 4; v[n] = 2; v: 1 3 v[q] = 4; v[n] = 2; v: 1 3 2 42 4 5; return 4 5; return 4

Fun(3 1 4 2 5):Fun(3 1 4 2 5):v[q] = 4; v[n] = 5; v: 1 3 2 4 5; return 5v[q] = 4; v[n] = 5; v: 1 3 2 4 5; return 5

Fun(v[1…n]) {Fun(v[1…n]) {if size of v is 1 return 1if size of v is 1 return 1elseelse


}}

ExampleExample

First, the First, the input size is input size is the vector the vector size.size.

Fun(v[Fun(v[1…n1…n]) {]) {if if size of v is 1size of v is 1 return 1 return 1elseelse

q = Fun(v[q = Fun(v[1..n-11..n-1]);]);if (v[q] > v[n])if (v[q] > v[n]) swap(v[q], v[n]);swap(v[q], v[n]);temp = 0;temp = 0;for x = 1 to nfor x = 1 to n if (v[x] > temp) if (v[x] > temp) temp = v[x];temp = v[x]; p = x;p = x;return p;return p;

}}

ExampleExample

Second, Second, when the when the size of the size of the vector is 1 vector is 1 the the recursion recursion terminatesterminates



}}

ExampleExample

Third, the Third, the basic basic operations are operations are not simplenot simple



}}

ExampleExample

There is O(1) There is O(1) compare and compare and swapswap



}}

ExampleExample

There is an There is an O(n) loop with O(n) loop with O(3) O(3) operations operations insideinside



}}

ExampleExample

Fourth, the Fourth, the recursive call recursive call decrease the decrease the vector size (n) vector size (n) by oneby one


q = q = Fun(v[1..n-1]);Fun(v[1..n-1]);if (v[q] > v[n])if (v[q] > v[n]) swap(v[q], v[n]);swap(v[q], v[n]);temp = 0;temp = 0;for x = 1 to nfor x = 1 to n if (v[x] > temp) if (v[x] > temp) temp = v[x];temp = v[x]; p = x;p = x;return p;return p;

}}

ExampleExample Finally, we can Finally, we can

describe the running describe the running time astime as

T(n) = T(n-1) + O(n)T(n) = T(n-1) + O(n) Which is the Which is the

quadratic basic formquadratic basic form..


q = q = Fun(v[1..n-Fun(v[1..n-1]);1]);

if (v[q] > v[n])if (v[q] > v[n]) swap(v[q], swap(v[q],

v[n]);v[n]);temp = 0;temp = 0;for x = 1 to nfor x = 1 to n if (v[x] > if (v[x] >

temp) temp) temp = v[x];temp = v[x]; p = x;p = x;return p;return p;

}}

Summary of simple Summary of simple recursive algorithmsrecursive algorithms How does the input size (i.e., How does the input size (i.e.,

terminating variable) change for terminating variable) change for each recursive function call.each recursive function call.

In practice there are to basic In practice there are to basic optionsoptions

1.1. n n n – 1 n – 1

2.2. n n n/2 n/2

nnn-1n-1

Case #1:Case #1:fun(n) {fun(n) {

if (n==1) quitif (n==1) quitO(1)O(1)fun(n-1)fun(n-1)

} } O(n) O(n)


if (n==1) quitif (n==1) quitO(n)O(n)fun(n-1)fun(n-1)

} } O(n O(n22))

This means the recursion is simply an O(n) This means the recursion is simply an O(n) loop. This leads to two common cases:loop. This leads to two common cases:

n n n-1 n-1

Generic caseGeneric case

fun(n) {fun(n) {

if (n==1) quitif (n==1) quit

O(nO(nkk))

fun(n-1)fun(n-1)

} } O(n O(nk+1k+1))

nnn/2n/2


if (n==1) quitif (n==1) quitO(1)O(1)fun(n/2)fun(n/2)

} } O(log n) O(log n)


if (n==1) quitif (n==1) quitO(n)O(n)fun(n/2)fun(n/2)fun(n/2)fun(n/2)

} } O(n log n) O(n log n)

This means the recursion is simply an This means the recursion is simply an O(log n) loop. This leads to three common O(log n) loop. This leads to three common cases:cases:


if (n==1) quitif (n==1) quitO(n)O(n)fun(n/2)fun(n/2)

} } O(n) O(n)

Generic Case:Generic Case:

TT((nn) = ) = aTaT((n/bn/b) + ) + f f ((nn)) where where f f ((nn)) == ΘΘ((nnkk))

1.1. a < ba < bkk T T((nn) = ) = ΘΘ((nnkk)) 2.2. a = ba = bkk T T((nn) = ) = ΘΘ((nnk k lg lg n n ))

3.3. a > ba > bkk T T((nn) = ) = Θ Θ ((nnlog log b b aa))

1/b is the reduction factor1/b is the reduction factora is the number of recursive callsa is the number of recursive callsf(n) is the interior complexity of the recursive f(n) is the interior complexity of the recursive

function.function.

Multiple Recursive Multiple Recursive CallsCallsO(2O(2nn) arises from multiple recursive ) arises from multiple recursive

function calls where the input size is function calls where the input size is not reduced by a factor of ½not reduced by a factor of ½

fun(n) {fun(n) {if (n==1) quitif (n==1) quitO(1)O(1)fun(n-1)fun(n-1)fun(n-1)fun(n-1)

} } O(2 O(2nn))

Multiple Recursive Multiple Recursive CallsCallsfun(n) {fun(n) {

if (n==1) quitif (n==1) quitO(1)O(1)fun(n-1)fun(n-1)fun(n-1)fun(n-1)

} } O(2 O(2nn)) Here the recursion acts as an O(n) loop, but for Here the recursion acts as an O(n) loop, but for

each loop the number of recursive calls is each loop the number of recursive calls is doubled.doubled.

Number of operations = Number of operations = 1+ 2+ 4 + 8 + 16 + … + 21+ 2+ 4 + 8 + 16 + … + 2nn = (2 = (2nn -1)+2 -1)+2nn = O(2 = O(2nn) )

Multiple Recursive Multiple Recursive CallsCalls

fun(n) {fun(n) {


O(n)O(n)

fun(n-1)fun(n-1)fun(n-1)fun(n-1)

} } O(???) O(???)

Multiple Recursive Multiple Recursive CallsCalls

fun(n) {fun(n) {


O(1)O(1)

fun(n-1)fun(n-1)fun(n-1)fun(n-1)fun(n-1)fun(n-1)

} } O(???) O(???)

Day 10 AgendaDay 10 Agenda(can you believe it’s day 10?)(can you believe it’s day 10?)

The last algorithm analysis The last algorithm analysis enigma.enigma.

Summary of enigmaSummary of enigma Can things like 1.5Can things like 1.5nn or 1.375 or 1.375nn

arise?arise? Homework 4 solutionsHomework 4 solutions Exam reviewExam review

Last EnigmaLast Enigma

fun1(n) {fun1(n) {


O(1)O(1)

fun1(n-1)fun1(n-1)fun1(n-1)fun1(n-1)

} } O(2 O(2nn))

fun2(n) {fun2(n) {

if (n==1) if (n==1) quitquit

O(n)O(n)


} } O(2 O(2nn))

fun3(n,m) {fun3(n,m) {


O(m)O(m)

fun3(n-1,m)fun3(n-1,m)fun3(n-1,m)fun3(n-1,m)

} } O(n2 O(n2nn) if we ) if we call fun(n,n)call fun(n,n)


1

1 1

1 1

1 1 1 1 1 1 1 1

1 1 4

8

…

2n

2

1

+

+

+

+

= 1 + 2 + 4 + … + 2n

=(2n - 1) + 2n

=2(2n) – 1

=O(2n)

fun1(n) {fun1(n) {


O(1)O(1)


} } O(2 O(2nn))


n

n-1 n-1

n-2 4(n-2)

8(n-3)…

2n(1)

2(n-1)

1(n)

+

+

+

+

= ??? =O(2n)

You have to see my program to believe it!

fun2(n) {fun2(n) {


O(n)O(n)


} } O(2 O(2nn))

n-2 n-2 n-2

1 1 1 1 1 1 1 1


n

n n

n n

n n n n n n n n

n n 4n

8n

…

2nn

2n

1n

+

+

+

+

= (1 + 2 + 4 + … + 2n)n

=((2n - 1) + 2n)n

=(2(2n) – 1)n

=O(n2n)

fun3(n,m) {fun3(n,m) {


O(m)O(m)

fun3(n-1,m)fun3(n-1,m)fun3(n-1,m)fun3(n-1,m)

} } O(n2 O(n2nn) if we ) if we call fun(n,n)call fun(n,n)

Summary of EnigmasSummary of Enigmas

log grows so small, the base log grows so small, the base makes no difference.makes no difference.

loglog33n = n = (log(log22n)n)

loglog100100n = n = (log(log22n)n)

loglogeen = n = (log(log22n)n)


Exponential grow so fast that the base Exponential grow so fast that the base makes a huge difference.makes a huge difference.

22nn (3(3nn)) (2+0.01)(2+0.01)nn (2(2nn)) However,However, 22nn = O(3 = O(3nn)) 33nn = = (2(2nn)) 22n+1n+1 = 2· 2 = 2· 2nn = = (2(2nn))


Cutting the input size in half recursively Cutting the input size in half recursively creates a log n loop that does NOT creates a log n loop that does NOT increase the efficiency class (except for increase the efficiency class (except for Case #3 O(1) internal complexity)Case #3 O(1) internal complexity)


if (n==1) quitif (n==1) quitO(1)O(1)fun(n/2)fun(n/2)

} } O(log n) O(log n)


if (n==1) quitif (n==1) quitO(n)O(n)fun(n/2)fun(n/2)

} } O(n) O(n)

Generic case:Generic case:fun(n) {fun(n) {

if (n==1) quitif (n==1) quitO(nO(nkk))fun(n/2)fun(n/2)

} } O(n O(nkk))


Here we cut the input size in half (log n), but Here we cut the input size in half (log n), but we spawn two recursive calls for each recursive we spawn two recursive calls for each recursive level (2level (2nn). This adds a factor of log n to the ). This adds a factor of log n to the internal complexity.internal complexity.

However, it adds a factor of n if the internal However, it adds a factor of n if the internal complexity is O(1)complexity is O(1)

Exception:Exception:

fun(n) {fun(n) {


O(1)O(1)

fun(n/2)fun(n/2)fun(n/2)fun(n/2)

} } O(n) O(n)

Case #4:Case #4:

fun(n) {fun(n) {


O(n)O(n)


} } O(n log n) O(n log n)

Generic caseGeneric case

fun(n) {fun(n) {


O(nO(nkk))


} } O(n O(nkk))

Fibonacci numbersFibonacci numbers(here is where things get difficult to (here is where things get difficult to analyze)analyze)

The Fibonacci sequence:The Fibonacci sequence:

0, 1, 1, 2, 3, 5, 8, 13, 21, … 0, 1, 1, 2, 3, 5, 8, 13, 21, … Fibonacci recurrence:Fibonacci recurrence:

F(F(nn) = F() = F(nn-1) + F(-1) + F(nn-2) -2)

F(0) = 0 F(0) = 0

F(1) = 1F(1) = 1 Another example:Another example:

A(A(nn) = 3A() = 3A(nn-1) - 2(-1) - 2(nn-2) A(0) = 1 A(1) = 3-2) A(0) = 1 A(1) = 3

2nd 2nd order linear homogeneous order linear homogeneous recurrence relation recurrence relation

with constant coefficientswith constant coefficients

How do we handle How do we handle things like this?things like this?

fun(n) {fun(n) {


O(1)O(1)

fun(n-1)fun(n-1)fun(n-2)fun(n-2)

} } O(???) O(???) The simple way is to draw a picture.The simple way is to draw a picture.

Exam 1Exam 1

Chapters 1 and 2 onlyChapters 1 and 2 only Review hw solutionsReview hw solutions Go through powerpoint slides and Go through powerpoint slides and

make your cheat sheet.make your cheat sheet.

Homework 4Homework 4

BTW, hw serves three purposes:BTW, hw serves three purposes:1.1. It helps me gauge if I’m going too It helps me gauge if I’m going too

fast.fast.2.2. It helps improve the grades of It helps improve the grades of

those who put forth effort but may those who put forth effort but may have test anxietyhave test anxiety

3.3. It helps prepare you for the examsIt helps prepare you for the exams hw1, hw2, hw3 & hw4 = 9 pointshw1, hw2, hw3 & hw4 = 9 points exam1 = 10 pointsexam1 = 10 points

Chapter 2 Analysis Framework 2.1 – 2.2. Homework Remember questions due Wed. Remember questions due Wed. Read sections 2.1 and 2.2 Read sections 2.1 and.

Documents