Lec 5 asymptotic notations and recurrences

Introduction to analysis

of algorithm

Module – 1 and 2

19-Jan-15Ankita R Karia, SFIT

1

Topics to cover

• Asymptotic Notations

• Recurrences

Substitution Method

Recursion Tree Method

Master Method

19-Jan-15Ankita R Karia, SFIT

2

ASYMPTOTIC NOTATION ( O, , )

• Are used to indicate the running times of an algorithm.

• It is a shorthand way to represent time complexity.

19-Jan-15 Ankita R Karia, SFIT 3

Given functions f(n) and g(n), we say that

f(n) = O(g(n)) if there are positive constants c and n0

such that

f(n) cg(n) for n n0

BIG-OH The big-Oh notation gives an upper bound

on the growth rate of a function.

Visualization of O(g(n))

n0

cg(n)

f(n)

n

For all values n, to the right of n0 , the value of function f(n) is ON or BELOW cg(n).



f(n) = (g(n)) if there are positive constants c and n0

such that

cg(n) f(n) for all n n0

BIG-OMEGA The big-Omega notation gives a lower

bound on the growth rate of a function.



f(n) = (g(n)) if there are positive constants c1, c2

and n0 such that

c1 g(n) f(n) c2 g(n) for all n n0

A function f(n) belongs to the set (g(n)) if there exists positive

constants c1 and c2 such that it can be “sandwiched” between c1

g(n) and c2 g(n) for large value of n.

BIG-THETA g(n) is ASYMPTOTICALLY TIGHT

BOUND for f(n)

Visualization of (g(n))

n0

c2g(n)

f(n)

c1g(n)

n

Recurrences

• A recurrence is an equation that describes a function in terms of its

value on smaller inputs.

• Recurrences go along with divide-and-conquer algorithm because

they provide a natural way to characterize the running time of these

algorithms.

• Recurrences take may forms.

T(n) = 2 T(n/2) + Ө(n)


Is the worst case running

time T(n) of QUICK SORT.

EXAMPLE

• A recursive algorithm if divides a problem into unequal sizes,

such as, 2/3 – to – 1/3 split.

• And if the divide and combine steps take linear time, Then its

recurrence is given as follows:-

T(n) = T(2n/3) + T(n/3) + Ө(n)


There are 3 methods to solve these recurrences and

obtain the running time of an algorithm in terms of Big-

Oh (O) notation.

Methods to solve recurrences

• Substitution Method

In this method, we guess a bound and then mathematical

induction is used to prove our guess is correct.

• Recursion Tree Method

• Master Method


SUBSTITUTION METHOD

• Two steps are to carried out:-

1. Guess the form of solution

2. Use mathematical induction to find the constants and

show that the solution works


MASTER METHOD• The master method provides a “cookbook” method for solving

recurrences of the form:-

T(n) = a T(𝑛

𝑏) + f(n)

where a ≥ 1 𝑎𝑛𝑑 𝑏 > 1 are constants

f(n) is an asymptotically positive function.


describes the Running Time of an algorithm that divides a

problem of size n into a sub-problem, each of size 𝒏

𝒃

is the cost of dividing the problem and

combining the results of the sub-problem.


CASE 1 CASE 2 CASE 3

T(n) = ( 𝒏logba)if

f(n) = O(𝒏logba−∈

)

For this case;

f(n) must be

polynomically

smaller than 𝑛logba

f(n) < 𝒏logba

T(n) = (𝒏logba 𝐥𝐨𝐠𝐧)if

f(n) = (𝒏logba )

For this case;

f(n) and 𝑛logba must be

of same size

f(n) = 𝒏logba

T(n) = (f 𝒏 )if

f(n) = (𝒏logba+∈)

For this case;

f(n) must be

polynomically

larger than 𝑛logba

f(n) > 𝒏logba

Lec 5 asymptotic notations and recurrences

Engineering