2008. 2. 12 : 1 Recurrences. Contents of Table Recurrence solution methods The substitution method The recursion-tree method 2 Relationship between asymptotic.

한양대학교 정보보호 및 알고리즘 연구실

2008. 2. 12이재준

담당교수님 : 박희진 교수님1

Recurrences

Contents of Table

Recurrence solution methodsThe substitution method

The recursion-tree method

2

Relationship between asymptotic notations

A. Growth of function

B. Recurrence

3

3. Relationship between this notations

• Analogy

f(n) = Θ(g(n)) ≈ f(n) = g(n).f(n) = O(g(n)) ≈ f(n) ≤ g(n).f(n) = Ω(g(n)) ≈ f(n) ≥ g(n).f(n) = o(g(n)) ≈ f(n) < g(n). f(n) = ω(g(n)) ≈ f(n) > g(n).

4


• Transitivity

• Transpose symmetry

f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)) ,f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) , f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)) ,f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)) ,f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)).

f(n) = O(g(n)) if and only if g(n) = Ω(f(n)),f(n) = o(g(n)) if and only if g(n) = ω(f(n)).

5


• Reflexivity

f(n) = Θ(f(n))f(n) = O(f(n))f(n) = Ω(f(n))

• Symmetry

f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)).

Recurrences

6

• Recurrence

When should we use the recurrence?

When algorithm contains a recursive call to itself, Its running time can often be described by a recurrence.

Recurrences

7

• Recurrence - Definition Equation or inequality that describes a function in terms

of its value on smaller inputs.

Recurrences

8

• Recurrence Example

Worst-case running time of T(n) could be described by the recurrence

Merge-Sort procedure

Show that the solution of T(n) = Θ(n log n)

Merge-Sort procedure

Show that the solution of T(n) = Θ(n log n)

)()2/(2

)1()(

nnTnT

if n=1

if n>1

Recurrences

9

• Recurrence Solution method to example

1) The substitution method

guess a bound and prove our guess correct.

2) The recursion-tree method

convert the recurrence into a tree

whose nodes represent the costs incurred at various level of recursion

3) The master method

determining asymptotic bounds

for many simple recurrences of the form)()/()( nfbnaTnT ( a ≥ 1, b ≥ 1, and f(n) is given function )

1. Technicalities for abbreviation

• Technicalities 1) assumption of integer

The running time T(n) of algorithm is only defined when

n is an integer 2) floors / ceilings

We often omit floors and ceilings3) boundary condition

The running time of an algorithm on a constant-sized input is

a constant that we typically ignore.

1. Technicalities for abbreviation

• Technicalities ExampleThe recurrence describing the worst-case running time of Merge-Sort

assumption of integer

Omit boundary condition

)()2/(2

)1()(

nnTnT

if n=1

if n>1

if n=1

if n>1

)()2/()2/(

)1()(

nnTnTnT

)()2/(2)( nnTnT

)()2/(2

)1()(

nnTnT

if n=1

if n>1

Omit floors and ceilings

2. Recurrence solution methods

12

• Substitution Method - Definition

- It can be used to establish either upper or lower bounds on a recurrence.

1. Guess the form of the solution

2. Use mathematical induction to find the constant and show that the solution works.

1. Guess the form of the solution

2. Use mathematical induction to find the constant and show that the solution works.


13

• Substitution Method Example

Let us determine an upper bound on the recurrence

Let us determine an upper bound on the recurrence

nnTnT )2/(2)(


14

• Substitution Method Solution to example

1. guess that the solution is 2. prove that ( c > 0 )

1. guess that the solution is 2. prove that ( c > 0 )

)lg()( nnnT

ncnnT lg)(

We need to find constant c and n0 to meet the requirement about assumption T(n) ≤ cn lg n ( c>0 )

We using the Mathematical induction to show that our assumption holds for the boundary condition.


15


Mathematical induction We can prove something stronger for a given value by assuming

something stronger for smaller values.

k means ⌊n/2⌋

k+1 means n

nnTnT )2/(2)(

→ T(1), T(2), T(3), … ….. T(⌊n/2⌋ ), T(n)

k k+1


16


Recurrence :

Guess : T(n)= O(n)Prove : T(n) ≤ cn

nnTnT )2/(2)(

k means k+1 means

n

-> 2/n

Substitution

T (⌊n/2 ) ≤ ⌋ c ⌊n/2 lg (⌋ ⌊n/2 )⌋

T(n) ≤ 2(c ⌊n/2 lg(⌋ ⌊n/2 )) + ⌋ n ≤ cn lg(n/2) + n ( because, ⌊n/2 < ⌋ n/2 )

= cn lg n - cn lg 2 + n = cn lg n -(c+1)n ( It’s our assumed fact )


17


Assumed Fact : cn lg n –(c+1)nProve : T(n) ≤ cn

T(n) ≤ cn lg n – (c+1)n ≤ cn lg n (as long as c ≥

1)

Assumed Fact ≤ Prove


18


- Find the constant n0 - we can take advantage of asymptotic notation,

we can replace base case

T(1) ≤ cn lg n T(1) = 1 but, c1 lg1 = 0

- So, we only have to prove T (n) = cn lg n for n ≥ n0 for n (n0 =2)

Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.

Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.

nnTnT )2/(2)(


19


- Find the constant c - choosing c large enough

T (2) = 4 and T (3) = 5.T (2) = 4 ≤ c ( 2 lg 2 ) T (3) = 5 ≤ c ( 3 lg 3 ).

Any choice of c ≥ 2 suffices for base case of n=2 and n =3 Any choice of c ≥ 2 suffices for base case of n=2 and n =3

nnTnT )2/(2)(


20

• Substitution Method Skill

- SubtletiesWe can prove something stronger for given value by assuming something stronger for a smaller values.

Overcome difficulty by subtracting a lower-order term from our previous guess


21

• Substitution Method Skill - Subtleties example

Recurrence :


1)2/()2/()( nTnTnT

Substitution

12/2/)( ncncnT

1cn

k means

k+1 means n

2/)2/( ncnT

2/)2/( ncnT

→ 2/n

2/n →

So, we now prove T(n) ≤ cn-bSo, we now prove T(n) ≤ cn-b


22


12/2/)( ncncnT

1cn

cn ( It’s Impossible. Then T(n)= O(n) is wrong? )

Subtracting a lower-order term from our previous guess T(n) ≤ cn


23


bcn

bcn

12

( b≥1, c must be chosen large enough )

1)2/()2/( bncbncT(n)

Recurrence :

Guess : T(n)= O(n)Prove : T(n) ≤ cn-b

1)2/()2/()( nTnTnT


24

• Substitution Method Skill

- Avoid pitfalls We need to prove the exact form of the inductive

hypothesis

T(n) ≤ cn

T(n) ≤ (c+1)n


25

• Substitution Method Skill - Avoid pitfalls example

nncnT )2/(2)(

)(

)1(2

2

nO

nc

nn

c

Recurrence :


nnTnT )2/(2)(

( Wrong !!! exact form :T(n) ≤ cn)


26

• Substitution Method Skill- Simplify this recurrence by changing variable

nnTnT lg)(2)(

mTT mm )2(2)2( 2/

mmSmS )2/(2)(

Renaming m = lg n

S(m) = T(2m)

This new recurrence has same solution : S(m) = O(m log m)


27

• Substitution Method Skill- Simplify this recurrence by changing variable

mmSmS )2/(2)(

This new recurrence has same solution : S(m) = O(m log m) Changing back from S(m) to T(n)

T(n) = T(2m) = S(m)= O(m lg m)= O( lg n lg(lg n) )


28

• Substitution Method Skill - Making a good guess

prove loose upper and lower bounds on the recurrence

and then reduce the range of uncertainty.

- Guessing a solution takes : experience, occasionally, creativity

We can make a good guess by using recursion tree !!We can make a good guess by using recursion tree !!

2. Recurrence solution methods• Recursion-tree - Definition

For come up with good guess,convert the recurrence into a tree that each node represents the cost of a single

subproblem somewhere in the set of recursive function

invocations.

We sum all the per-level costs to determine the total cost of all levels of the recursion

29

2. Recurrence solution methods• Recursion-tree Example

30

The recurrence isT (n) = 3T (⌊n/4 ) + Θ(⌋ n2)

= 3T (n/4) + cn2

Use the recursion tree to making a good guess for upper bound for the solution.

The recurrence isT (n) = 3T (⌊n/4 ) + Θ(⌋ n2)

= 3T (n/4) + cn2

Use the recursion tree to making a good guess for upper bound for the solution.

2. Recurrence solution methods• Recursion-tree Solution to example Draw recursion tree

31

cn2

)4

(n

T

T(n)

)4

(n

T )4

(n

T

)16

(n

T )16

(n

T )16

(n

T

2)4

(n

c

)16

(n

T )16

(n

T )16

(n

T

2)4

(n

c

)16

(n

T )16

(n

T )16

(n

T

2)4

(n

c

cn2

3T (n/4) + cn2

2. Recurrence solution methods• Recursion-tree Solution to example

32

cn2

c(n/4)2 c(n/4)2 c(n/4)2

2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c

T(1)T(1)T(1)T(1)T(1)T(1)T(1)T(1) T(1)T(1)T(1)T(1)…3log4n

log4n

When depth i then subprogram size is n/4i = 1 so, i = log4 n and it means tree’s height and tree has log4 n + 1 levels. The number of nodes at depth i is 3i and finally

When depth i then subprogram size is n/4i = 1 so, i = log4 n and it means tree’s height and tree has log4 n + 1 levels. The number of nodes at depth i is 3i and finally 3loglog 443 nn


33

• Recursion-tree Solution to exampleTotal Cost

)(1)16/3(

1)16/3(

)()16

3(

)()16

3(...)

16

3(

16

3)(

3log2log

3log21log

0

3log21log2222

4

4

4

4

44

ncn

ncn

ncncncncnnT

n

in

i

n

Sum of cost for root to i-1 depth

Sum of cost for i depth

)(

)(13

16

)()16/3(1

1

)()16

3(

)()16

3()(

2

3log2

3log2

3log2

0

3log21log

0

4

4

4

4

4

nO

ncn

ncn

ncn

ncnnT

i

i

in

i

xx

k

i

1

1

0


34

• Recursion-tree Solution to exampleTotal Cost is…

( When 0<x<1 then, )


35

• Recursion-tree Solution to exampleProve T (n) = O(n2) by the substitution method

Recurrence :

Guess : T (n) = O(n2) Prove : T (n) ≤ dn2 (for some d > 0 and for the same c > 0)

T (n) = 3T (⌊n/4 ) + Θ(⌋ n2) = 3T(⌊n/4 ) + ⌋ cn2


36

Recursion-tree Solution to exampleFind constant d

T(n) ≤ 3T(⌊n/4 ) + ⌋ cn2

≤ 3d⌊n/4⌋2 + cn2

≤ 3d(n/4)2 + cn2 = 3/16 dn2 + cn2

≤ dn2 (where the last step holds as long as d ≥ (16/13)c )

T (n) = 3T(⌊n/4 ) + ⌋ cn2

2. Recurrence solution methods• Recursion-tree Example

37

The recurrenceT (n) = 3T (n/3)+T(2n/3)+O(n)

Use the recursion tree to making a good guess for upper

bound for the solution.

The recurrenceT (n) = 3T (n/3)+T(2n/3)+O(n)

Use the recursion tree to making a good guess for upper

bound for the solution.


38

)9

(n

c )9

2(

nc

)3

(n

c

)9

2(

nc )

9

4(

nc

)3

2(

nc

cn

log3/2n

cn

cn

cn

Total : O(n log n)

T (n) = 3T (n/3)+T(2n/3)+ cn

This recursion tree is not a complete binary tree. Not all leaves contribute a cost of exactly cn


39

The total cost = Cost of each level x Height = cn x log3/2 n

= O(cnlog3/2n) = O(n lg n)

The longest path from root to a leaf is n→(2/3)n → (2/3)2n → … → 1. Since, (2/3)kn = 1 when k=log3/2 n and it means tree’s height is log3/2 n .

The longest path from root to a leaf is n→(2/3)n → (2/3)2n → … → 1. Since, (2/3)kn = 1 when k=log3/2 n and it means tree’s height is log3/2 n .

T (n) = 3T (n/3)+T(2n/3)+ cn

)9

(n

c )9

2(

nc

)3

(n

c

)9

2(

nc )

9

4(

nc

)3

2(

nc

cn

log3/2n

cn

cn

cn

2. Recurrence solution methods• Recursion-tree Solution to example Prove T (n) = O(n lg n) by the substitution

method

40

Recurrence :Guess : T (n) = O(n lg n)Prove : T (n) ≤ d n lg n (for some d > 0 and for the same c > 0)

T(n/3) + T(2n/3) + cn


41

Recursion-tree Solution to exampleFind constant d T(n) ≤ T(n/3) + T(2n/3) + cn

≤ d(n/3)lg(n/3) + d(2n/3)lg(2n/3) + cn = (d(n/3)lgn - d(n/3)lg 3) + (d(2n/3)lgn – d(2n/3)lg(3/2)) + cn = dnlgn - d((n/3)lg3 + (2n/3)lg(3/2)) + cn = dnlgn - d((n/3)lg3 + (2n/3)lg(3/2)) + cn = dnlgn - d((n/3)lg3 + (2n/3)lg3 - (2n/3)lg2) + cn = dnlgn - dn(lg3 - 2/3) + cn

≤ dnlgn (as long as d ≥ c/(lg 3 - (2/3)) )

T(n)=T(n/3) + T(2n/3) + cn

Content of next week

Recurrence solution methodsThe substitution method

The recursion-tree method

The master method

42

Ch4. Recurrence

2008. 2. 12 : 1 Recurrences. Contents of Table Recurrence solution methods The substitution method The recursion-tree method 2 Relationship between asymptotic.

Documents

recurrence slide

constant n

fn symmetry fn

solution of tn

substitution method

cn lg n c

ogn fn gn

cn lg n cn lg