한양대학교 정보보호 및 알고리즘 연구실 2008. 2. 12 이재준 담당교수님 : 박희진 교수님

1

한양대학교 정보보호 및 알고리즘 연구실

2008. 2. 12이재준담당교수님 : 박희진 교수님

Chapter 3. Growth of functionChapter 4. Recurrences

Contents of Table

1. Technicalities for abbreviation

2. Recurrence solution methodsThe substitution method

2

2. Notation of asymptotically larger/smaller

o-notation ω-notation

3. Relationship between this notations

A. Growth of function

B. Recurrence

f(n) is asymptotically smaller than g(n)

• o-notation Asymptotically smaller

- Definition for all n, n ≥ n0 , any positive constant c,

c>0, 0 ≤ f(n)< cg(n), the function f(n) is smaller to g(n) to within con-stant factor.

3

2. Notation of asymptotically larger/smaller

f(n)

o(g(n))

n0

4

2. Notation of asymptotically larger/smaller

• o-notation

- Definitiono(g(n)) = f(n) : for any positive constant c>0, there exist a constant

n0 >0 such that 0 ≤ f(n) < cg(n) for all n ≥ n0

0)()(lim

ngnf

n

this limit shows the function f(n) becomes in-significant relative to g(n) as n approaches in-finity

• o-notation Example

2. Notation of asymptotically larger/smaller

Use the definition of o-notation to prove the following prop-erty.

f(n) = o(n2)

nnf 2)(

6

2. Notation of asymptotically larger/smaller

• o-notation Solution to example

2n = o(n2)

If f(n) = o((g(n)) then

0)()(lim

ngnf

n

f(n) is asymptotically larger than g(n)

• ω -notation Asymptotically larger

- Definition for all n, n ≥ n0 , any positive constant c,

c>0, 0 ≤ cg(n) < f(n), the function f(n) is larger to g(n) to within constant factor.

7

2. Notation of asymptotically larger/smaller

f(n)

ω(g(n))

n0

8

2. Notation of asymptotically larger/smaller

• ω-notation

- Definition ω(g(n)) = f(n) : for any positive constant c>0, there exist a constant

n0 >0 such that 0 ≤ cg(n) < f(n) for all n ≥ n0

)()(lim ng

nfn

if the limit exists. That is, f(n) becomes arbitrarily large relative to g(n) as n approaches infinity.

• ω -notation

- We use ω-notation to denote a lower bound that is not asymptotically tight.

- We use o-notation to denote an upper bound that is not asymptotically tight.

- f(n) ω(∈ g(n)) if and only if g(n) ∈ o(f(n)).

9

2. Notation of asymptotically larger/smaller

10

2. Notation of asymptotically larger/smaller

• ω -notation Example

Use the definition of o-notation to prove the following prop-erty.

f(n) = ω(n)

2)(

2nnf

11

2. Notation of asymptotically larger/smaller

• ω -notation Solution to example

=ω(n)

If f(n) = ω((g(n)) then

2n

)(

)(lim ngnf

n

12

3. Relationship between this notations

f(n)

O(g(n))

o(g(n))

ω(g(n))

Ω(g(n))

θ(g(n))

O(g(n)) Ω(g(n))

ω(g(n)) o(g(n))

13

3. Relationship between this notations

f(n)

O(g(n))

o(g(n))

ω(g(n)) Ω(g(n))

• Transitivity

• Transpose symmetry

f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)) ,f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) , f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)) ,f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)) ,f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)).

f(n) = O(g(n)) if and only if g(n) = Ω(f(n)),f(n) = o(g(n)) if and only if g(n) = ω(f(n)).

14

3. Relationship between this notations

• Reflexivity

f(n) = Θ(f(n))f(n) = O(f(n))f(n) = Ω(f(n))

• Symmetry

f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)).

θ(g(n))

O(g(n)) Ω(g(n))

Recurrences

15

• Recurrence

When should we use the recurrence?

When algorithm contains a recursive call to it-self, Its running time can often be described by a re-currence.

Recurrences

16

• Recurrence

- Definition Equation or inequality that describes a function in terms

of its value on smaller inputs.

Recurrences

17

• Recurrence Example

Worst-case running time of T(n) could be described by the recur-rence

Merge-Sort procedure

Show that the solution of T(n) = Θ(n log n)

)()2/(2

)1()(

nnTnT

if n=1

if n>1

Recurrences

18

• Recurrence Solution method to example - The substitution method

guess a bound and prove our guess correct for small input.

- The recursion-tree method convert the recurrence into a tree whose nodes represent the costs incurred at various level of recursion

- The master method determining asymptotic bounds for many simple recurrences of

the form

)()/()( nfbnaTnT ( a ≥ 1, b ≥ 1, and f(n) is given function )

1. Technicalities for abbreviation

• Technicalities

- assumption of integerThe running time T(n) of algorithm is only defined when n is an integer

- floors / ceilingsWe often omit floors and ceilings

- boundary conditionThe running time of an algorithm on a constant-sized input is a constant that we typically ignore.

1. Technicalities for abbreviation

• Technicalities ExampleThe recurrence describing the worst-case running time of Merge-Sort

Omit assumption of in-teger

Omit boundary condi-tion

)()2/(2

)1()(

nnTnT

if n=1if n>1

if n=1if n>1

)()2/()2/(

)1()(

nnTnTnT

)()2/(2)( nnTnT

)()2/(2

)1()(

nnTnT

if n=1if n>1

Omit floors and ceil-ings

2. Recurrence solution methods

21

• Substitution Method

- DefinitionWhen recurrence is easy to guess form of the an-

swer withthe inductive hypothesis is applied to smaller

values,substitution of the guessed answer for the function

. - It can be used to establish either upper or lower bounds on a recur-rence.

2. Recurrence solution methods

22

• Substitution Method Steps

1. Guess the form of the solution

2. Use mathematical induction to find the constant and show that the solution works.

2. Recurrence solution methods

23

• Substitution Method - Making a good guess

prove loose upper and lower bounds on the recurrence

and then reduce the range of uncertainty.

- Guessing a solution takes : experience, occasionally, creativity

- we can use recursion tree

2. Recurrence solution methods

24

• Substitution Method - Subtleties

we can prove something stronger for given value by assuming something stronger for a smaller values.

1)2/()2/(2)( nTnTnTWe guess that the solution is O(n), and show that T(n) ≤ cn

12/2/)( ncncnT1cn

Overcome difficulty by subtracting a lower-order term from our previous guess

bcnnT )(

bcnbcn

12

( b≥1, c must be chosen large enough )

2. Recurrence solution methods

25

• Substitution Method - Avoid pitfalls

nncnT )2/(2)(

)(2

nOncn

)()2/(2)( nnTnT In recurrence we can falsely prove T(n)= O(n) by guessing T(n) ≤ cn

2. Recurrence solution methods

26

• Substitution Method Example

Let us determine an upper bound on the recurrence

nnTnT )2/(2)(

2. Recurrence solution methods

27

• Substitution Method Solution to ex-ample

1. guess that the solution is 2. prove that ( c > 0 )

)lg()( nnnT ncnnT lg)(

- Assume that this bound holds for ⌊n/2 , ⌋ that is, T (⌊n/2 ) ≤ ⌋ c ⌊n/2 lg(⌋ ⌊n/2 ).⌋

T(n) ≤ 2(c ⌊n/2 lg(⌋ ⌊n/2 )) + ⌋ n

≤ cn lg(n/2) + n ( because, ⌊n/2 < ⌋ n/2 ) = cn lg n - cn lg 2 + n = cn lg n - cn + n ≤ cn lg n (as long as c ≥ 1)

We need to find constant c and n0

2. Recurrence solution methods

28

• Substitution Method Solution to ex-ample

- Find the constant n0

- we can take advantage of asymptotic notation,we can replace base case

T(1) ≤ cn lg n T(1) = 1 but, c1 lg1 = 0

- So, we only have to prove T (n) = cn lg n for n ≥ n0 for n (n0 =2)

Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.

2. Recurrence solution methods

29

• Substitution Method Solution to ex-ample

- Find the constant c

- choosing c large enoughT (2) = 4 and T (3) = 5.

T (2) = 4 ≤ c ( 2 lg 2 ) T (3) = 5 ≤ c ( 3 lg 3 ).

Any choice of c ≥ 2 suffices for base case of n=2 and n =3

2. Recurrence solution methods

30

• Substitution Method Example

Let us determine an upper bound on the recurrence

nnTnT lg)(2)(

2. Recurrence solution methods

31

• Substitution Method Solution to ex-ample

- Simplify this recurrence by changing variable

nnTnT lg)(2)(

mTT mm )2(2)2( 2/

mmSmS )2/(2)(

Renaming m = lg n

S(m) = T(2m)

This new recurrence has same solution : S(m) = O(m log m)

2. Recurrence solution methods

32

• Substitution Method Solution to ex-ample

- Simplify this recurrence by changing variable mmSmS )2/(2)(

This new recurrence has same solution : S(m) = O(m log m)

Changing back from S(m) to T(n)

T(n) = T(2m) = S(m)= O(m lg m)= O( lg n lg(lg n) )