한양대학교 정보보호 및 알고리즘 연구실 2008. 2. 12 이재준 담당교수님 : 박희진 교수님

1

한양대학교 정보보호 및 알고리즘 연구실

2008. 2. 12이재준담당교수님 : 박희진 교수님

Recurrences

Contents of Table

Recurrence solution methodsThe substitution methodThe recursion-tree method

2

Relationship between asymptotic nota-tions

A. Growth of function

B. Recurrence

3

3. Relationship between this notations

• Analogyf(n) = Θ(g(n)) ≈ f(n) = g(n).f(n) = O(g(n)) ≈ f(n) ≤ g(n).f(n) = Ω(g(n)) ≈ f(n) ≥ g(n).f(n) = o(g(n)) ≈ f(n) < g(n). f(n) = ω(g(n)) ≈ f(n) > g(n).

4


• Transitivity

• Transpose symmetry

f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)) ,f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) , f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)) ,f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)) ,f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)).

f(n) = O(g(n)) if and only if g(n) = Ω(f(n)),f(n) = o(g(n)) if and only if g(n) = ω(f(n)).

5


• Reflexivity

f(n) = Θ(f(n))f(n) = O(f(n))f(n) = Ω(f(n))

• Symmetry

f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)).

Recurrences

6

• Recurrence

When should we use the recurrence?

When algorithm contains a recursive call to it-self, Its running time can often be described by a re-currence.

Recurrences

7

• Recurrence - Definition Equation or inequality that describes a function in terms

of its value on smaller inputs.

Recurrences

8

• Recurrence Example

Worst-case running time of T(n) could be described by the recur-rence

Merge-Sort procedure

Show that the solution of T(n) = Θ(n log n)

)()2/(2

)1()(

nnTnT

if n=1

if n>1

Recurrences

9

• Recurrence Solution method to ex-ample

1) The substitution method guess a bound and prove our guess correct.

2) The recursion-tree method convert the recurrence into a tree whose nodes represent the costs incurred at various level of re-cursion

3) The master method determining asymptotic bounds

for many simple recurrences of the form)()/()( nfbnaTnT ( a ≥ 1, b ≥ 1, and f(n) is given function )

1. Technicalities for abbreviation

• Technicalities 1) assumption of integer

The running time T(n) of algorithm is only defined when

n is an integer 2) floors / ceilings

We often omit floors and ceilings3) boundary condition

The running time of an algorithm on a constant-sized input is

a constant that we typically ignore.

1. Technicalities for abbreviation

• Technicalities ExampleThe recurrence describing the worst-case running time of Merge-Sort

assumption of integer

Omit boundary condi-tion

)()2/(2

)1()(

nnTnT

if n=1if n>1

if n=1if n>1

)()2/()2/(

)1()(

nnTnTnT

)()2/(2)( nnTnT

)()2/(2

)1()(

nnTnT

if n=1if n>1

Omit floors and ceil-ings

2. Recurrence solution methods

12

• Substitution Method - Definition

- It can be used to establish either upper or lower bounds on a recur-rence.

1. Guess the form of the solution

2. Use mathematical induction to find the constant and show that the solution works.


13

• Substitution Method Example

Let us determine an upper bound on the recurrence

nnTnT )2/(2)(


14

• Substitution Method Solution to ex-ample

1. guess that the solution is 2. prove that ( c > 0 )

)lg()( nnnT

ncnnT lg)(

We need to find constant c and n0 to meet the requirement about assumption T(n) ≤ cn lg n ( c>0 )

We using the Mathematical induction to show that our assumption holds for the boundary condition.


15


Mathematical induction We can prove something stronger for a given value by assuming

something stronger for smaller values.

k means ⌊n/2⌋

k+1 means n

nnTnT )2/(2)(

→ T(1), T(2), T(3), … ….. T(⌊n/2⌋ ), T(n)

k k+1


16


Recurrence :

Guess : T(n)= O(n)Prove : T(n) ≤ cn

nnTnT )2/(2)(

k means k+1 means

n

-> 2/n

Substitution

T (⌊n/2 ) ≤ ⌋ c ⌊n/2 lg (⌋ ⌊n/2 )⌋

T(n) ≤ 2(c ⌊n/2 lg(⌋ ⌊n/2 )) + ⌋ n ≤ cn lg(n/2) + n ( because, ⌊n/2 < ⌋ n/2 )

= cn lg n - cn lg 2 + n = cn lg n -(c+1)n ( It’s our assumed fact )


17

• Substitution Method Solution to exam-ple

Assumed Fact : cn lg n –(c+1)nProve : T(n) ≤ cn

T(n) ≤ cn lg n – (c+1)n ≤ cn lg n (as long as c ≥

1)

Assumed Fact ≤ Prove


18


- Find the constant n0

- we can take advantage of asymptotic notation,we can replace base case

T(1) ≤ cn lg n

T(1) = 1 but, c1 lg1 = 0

- So, we only have to prove T (n) = cn lg n for n ≥ n0 for n (n0 =2)

Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.

nnTnT )2/(2)(


19

• Substitution Method Solution to example

- Find the constant c - choosing c large enough

T (2) = 4 and T (3) = 5.T (2) = 4 ≤ c ( 2 lg 2 ) T (3) = 5 ≤ c ( 3 lg 3 ).

Any choice of c ≥ 2 suffices for base case of n=2 and n =3

nnTnT )2/(2)(


20

• Substitution Method Skill - Subtleties

We can prove something stronger for given value by as-suming something stronger for a smaller values.

Overcome difficulty by subtracting a lower-order term from our previous guess


21

• Substitution Method Skill - Subtleties example

Recurrence :


1)2/()2/()( nTnTnT

Substitution

12/2/)( ncncnT1cn

k means

k+1 means n

2/)2/( ncnT

2/)2/( ncnT

→ 2/n

2/n →

So, we now prove T(n) ≤ cn-b


22


12/2/)( ncncnT

1cncn ( It’s Impossible. Then T(n)= O(n) is wrong? )

Subtracting a lower-order term from our previous guess T(n) ≤ cn


23


bcnbcn

12

( b≥1, c must be chosen large enough )

1)2/()2/( bncbncT(n)

Recurrence :

Guess : T(n)= O(n)Prove : T(n) ≤ cn-b

1)2/()2/()( nTnTnT


24

• Substitution Method Skill - Avoid pitfalls We need to prove the exact form of the inductive hypoth-

esis

T(n) ≤ cn

T(n) ≤ (c+1)n


25

• Substitution Method Skill - Avoid pitfalls example

nncnT )2/(2)(

)()1(

22

nOnc

nnc

Recurrence :


nnTnT )2/(2)(

( Wrong !!! exact form :T(n) ≤ cn)


26

• Substitution Method Skill- Simplify this recurrence by changing vari-able

nnTnT lg)(2)(

mTT mm )2(2)2( 2/

mmSmS )2/(2)(

Renaming m = lg n

S(m) = T(2m)

This new recurrence has same solution : S(m) = O(m log m)


27

• Substitution Method Skill- Simplify this recurrence by changing vari-able

mmSmS )2/(2)(This new recurrence has same solution : S(m) = O(m log m) Changing back from S(m) to T(n)

T(n) = T(2m) = S(m)= O(m lg m)= O( lg n lg(lg n) )


28

• Substitution Method Skill - Making a good guess

prove loose upper and lower bounds on the recurrence

and then reduce the range of uncertainty.

- Guessing a solution takes : experience, occasionally, creativity

We can make a good guess by using recursion tree !!


• Recursion-tree - Definition

For come up with good guess,convert the recurrence into a tree that each node represents the cost of a single sub-

problem somewhere in the set of recursive function invoca-

tions.

We sum all the per-level costs to determine the total cost of all levels of the recursion

29


• Recursion-tree Example

30

The recurrence isT (n) = 3T (⌊n/4 ) + Θ(⌋ n2)

= 3T (n/4) + cn2

Use the recursion tree to making a good guess for upper bound for the solution.


• Recursion-tree Solution to example Draw recursion tree

31

cn2

)4

(nT

T(n)

)4

(nT )4

(nT

)16

( nT )16

( nT )16

( nT

2)4

(nc

)16

( nT )16

( nT )16

( nT

2)4

(nc

)16

( nT )16

( nT )16

( nT

2)4

(nc

cn2

3T (n/4) + cn2


• Recursion-tree Solution to example

32

cn2

c(n/4)2 c(n/4)2 c(n/4)2

2)16

( nc 2)16

( nc 2)16

( nc 2)16

( nc 2)16

( nc 2)16

( nc 2)16

( nc 2)16

( nc 2)16

( nc

T(1)T(1)T(1)T(1)T(1)T(1)T(1)T(1) T(1)T(1)T(1)T(1)…3log4n

log4n

When depth i then subprogram size is n/4i = 1 so, i = log4 n and it means tree’s height and tree has log4 n + 1 levels. The number of nodes at depth i is 3i and finally 3loglog 443 nn


33

• Recursion-tree Solution to exampleTotal Cost

)(1)16/3(

1)16/3(

)()163(

)()163(...)

163(

163)(

3log2log

3log21log

0

3log21log2222

44

44

44

ncn

ncn

ncncncncnnT

n

in

i

n

Sum of cost for root to i-1 depth

Sum of cost for i depth

)(

)(1316

)()16/3(1

1

)()163(

)()163()(

2

3log2

3log2

3log2

0

3log21log

0

4

4

4

44

nO

ncn

ncn

ncn

ncnnT

i

i

in

i

xx

k

i

11

0


34

• Recursion-tree Solution to exampleTotal Cost is…

( When 0<x<1 then, )


35

• Recursion-tree Solution to exampleProve T (n) = O(n2) by the substitution method

Recurrence :

Guess : T (n) = O(n2) Prove : T (n) ≤ dn2 (for some d > 0 and for the same c > 0)

T (n) = 3T (⌊n/4 ) + Θ(⌋ n2) = 3T(⌊n/4 ) + ⌋ cn2


36

Recursion-tree Solution to exampleFind constant d

T(n) ≤ 3T(⌊n/4 ) + ⌋ cn2

≤ 3d⌊n/4⌋2 + cn2

≤ 3d(n/4)2 + cn2 = 3/16 dn2 + cn2

≤ dn2 (where the last step holds as long as d ≥ (16/13)c )

T (n) = 3T(⌊n/4 ) + ⌋ cn2


• Recursion-tree Example

37

The recurrenceT (n) = 3T (n/3)+T(2n/3)+O(n)

Use the recursion tree to making a good guess for upper bound for the

solution.



38

)9

(nc )9

2( nc

)3

(nc

)9

2( nc )9

4( nc

)3

2( nc

cn

log3/2n

cn

cn

cn

Total : O(n log n)

T (n) = 3T (n/3)+T(2n/3)+ cn

This recursion tree is not a complete binary tree. Not all leaves contribute a cost of exactly cn



39

The total cost = Cost of each level x Height = cn x log3/2 n

= O(cnlog3/2n) = O(n lg n)

The longest path from root to a leaf is n→(2/3)n → (2/3)2n → … → 1. Since, (2/3)kn = 1 when k=log3/2 n and it means tree’s height is log3/2 n .

T (n) = 3T (n/3)+T(2n/3)+ cn

)9

(nc )9

2( nc

)3

(nc

)92( nc )

94( nc

)3

2( nc

cn

log3/2n

cn

cn

cn


• Recursion-tree Solution to example Prove T (n) = O(n lg n) by the substitution

method

40

Recurrence :Guess : T (n) = O(n lg n)Prove : T (n) ≤ d n lg n (for some d > 0 and for the same c > 0)

T(n/3) + T(2n/3) + cn


41

Recursion-tree Solution to exampleFind constant d T(n) ≤ T(n/3) + T(2n/3) + cn

≤ d(n/3)lg(n/3) + d(2n/3)lg(2n/3) + cn = (d(n/3)lgn - d(n/3)lg 3) + (d(2n/3)lgn – d(2n/3)lg(3/2)) + cn = dnlgn - d((n/3)lg3 + (2n/3)lg(3/2)) + cn = dnlgn - d((n/3)lg3 + (2n/3)lg(3/2)) + cn = dnlgn - d((n/3)lg3 + (2n/3)lg3 - (2n/3)lg2) + cn = dnlgn - dn(lg3 - 2/3) + cn

≤ dnlgn (as long as d ≥ c/(lg 3 - (2/3)) )

T(n)=T(n/3) + T(2n/3) + cn

Content of next week

Recurrence solution methodsThe substitution methodThe recursion-tree methodThe master method

42

Ch4. Recurrence

한양대학교 정보보호 및 알고리즘 연구실 2008. 2. 12 이재준 담당교수님 : 박희진 교수님

Documents