Design and Analysis Algorithm Pertemuan 04 · Pertemuan 04 . Contents ... Indikator efisiensi algoritma bedasar pada OoG ... Q(g(n)), Theta of g of n, the Asymptotic Tight Bound;

Design and Analysis Algorithm

Ahmad Afif Supianto, S.Si., M.Kom

Pertemuan 04

Contents

Brute Force Algorithm 1 2

Asymptotic Analysis 3 1

2

Asymptotic Analysis

Asymptotic Notation

Think of n as the number of records we wish to

sort with an algorithm that takes f(n) to run. How

long will it take to sort n records?

What if n is big?

We are interested in the range of a function as n

gets large.

Will f stay bounded?

Will f grow linearly?

Will f grow exponentially?

Our goal is to find out just how fast f grows with

respect to n.

Asymptotic Notation

Misalkan:

T(n) = 5n2 + 6n + 25

T(n) proporsional untuk ordo n2 untuk data yang sangat

besar.

5

Memperkirakan formula untuk run-time

Indikasi kinerja algoritma

(untuk jumlah data yang sangat besar)

Asymptotic Notation

Indikator efisiensi algoritma bedasar pada OoG

pada basic operation suatu algoritma.

Penggunaan notasi sebagai pembanding

urutan OoG:

O (big oh)

Ω (big omega)

Ө (big theta)

t(n) : algoritma running time (diindikasikan

dengan basic operation count (C(n))

g(n) : simple function to compare the count

6

Classifying functions by their

Asymptotic Growth Rates (1/2)

asymptotic growth rate, asymptotic order, or

order of functions

Comparing and classifying functions that ignores

constant factors and small inputs.

O(g(n)), Big-Oh of g of n, the Asymptotic Upper

Bound;

W(g(n)), Omega of g of n, the Asymptotic Lower

Bound.

Q(g(n)), Theta of g of n, the Asymptotic Tight

Bound; and

Example

Example: f(n) = n2 - 5n + 13.

The constant 13 doesn't change as n grows,

so it is not crucial. The low order term, -5n,

doesn't have much effect on f compared to

the quadratic term, n2.

We will show that f(n) = Q(n2) .

Q: What does it mean to say f(n) = Q(g(n)) ?

A: Intuitively, it means that function f is the

same order of magnitude as g.

Example (cont.)

Q: What does it mean to say f1(n) = Q(1)?

A: f1(n) = Q(1) means after a few n, f1 is

bounded above & below by a constant.

Q: What does it mean to say f2(n) = Q(n log n)?

A: f2(n) = Q(n log n) means that after a few n, f2

is bounded above and below by a constant

times n log n. In other words, f2 is the same

order of magnitude as n log n.

More generally, f(n) = Q(g(n)) means that f(n) is

a member of Q(g(n)) where Q(g(n)) is a set of

functions of the same order of magnitude.

Big-Oh

The O symbol was introduced in 1927 to

indicate relative growth of two functions based

on asymptotic behavior of the functions now

used to classify functions and families of

functions

Upper Bound Notation

We say Insertion Sort’s run time is O(n2) Properly we should say run time is in O(n2)

Read O as “Big-O” (you’ll also hear it as “order”)

In general a function f(n) is O(g(n)) if positive constants c and n0 such

that f(n) c g(n) n n0

e.g. if f(n)=1000n and g(n)=n2, n0 > 1000 and c = 1 then f(n0) < 1.g(n0) and we say that f(n) = O(g(n))

Asymptotic Upper Bound

f(n)

g(n)

c g(n) • f(n) c g(n) for all n n0

• g(n) is called an

asymptotic upper bound of f(n).

• We write f(n)=O(g(n))

• It reads f(n) is big oh of g(n).

n0

Big-Oh, the Asymptotic Upper Bound

This is the most popular notation for run time

since we're usually looking for worst case time.

If Running Time of Algorithm X is O(n2) , then

for any input the running time of algorithm X is

at most a quadratic function, for sufficiently

large n.

e.g. 2n2 = O(n3) .

From the definition using c = 1 and n0 = 2. O(n2)

is tighter than O(n3).

6

g(n)

f(n)

for all n>6, g(n) > 1 f(n).

Thus the function f is in the

big-O of g.

that is, f(n) in O(g(n)).

Example 1

g(n)

f(n)

5

There exists a n0 s.t. for all

n>n0, f(n) < 1 g(n).

Thus, f(n) is in O(g(n)).

Example 2

There exists a n0=5, c=3.5, s.t.

for all n>n0, f(n) < c h(n).

Thus, f(n) is in O(h(n)).

5

h(n)

f(n)

3.5 h(n)

Example 3

Example of Asymptotic Upper Bound

f(n)=3n2+5

g(n)=n2

4g(n)=4n2

4 g(n) = 4n2

= 3n2 + n2

3n2 + 9 for all n 3

> 3n2 + 5

= f(n)

Thus, f(n)=O(g(n)).

3

Exercise on O-notation

Show that 3n2+2n+5 = O(n2)

10 n2 = 3n2 + 2n2 + 5n2

3n2 + 2n + 5 for n 1

c = 10, n0 = 1

Exercise on O-notation

f1(n) = 10 n + 25 n2

f2(n) = 20 n log n + 5 n

f3(n) = 12 n log n + 0.05 n2

f4(n) = n1/2 + 3 n log n

• O(n2)

• O(n log n)

• O(n2)

• O(n log n)

Classification of Function : BIG

O (1/2)

A function f(n) is said to be of at most logarithmic

growth if f(n) = O(log n)

A function f(n) is said to be of at most quadratic

growth if f(n) = O(n2)

A function f(n) is said to be of at most polynomial

growth if f(n) = O(nk), for some natural number k > 1

A function f(n) is said to be of at most exponential

growth if there is a constant c, such that f(n) = O(cn),

and c > 1

A function f(n) is said to be of at most factorial growth

if f(n) = O(n!).

Classification of Function :

BIG O (2/2)

A function f(n) is said to have constant running

time if the size of the input n has no effect on

the running time of the algorithm (e.g.,

assignment of a value to a variable). The

equation for this algorithm is f(n) = c

Other logarithmic classifications:

f(n) = O(n log n)

f(n) = O(log log n)

Lower Bound Notation

We say InsertionSort’s run time is W(n)

In general a function

f(n) is W(g(n)) if positive constants c and n0 such

that 0 cg(n) f(n) n n0

Proof:

Suppose run time is an + b

• Assume a and b are positive (what if b is negative?)

an an + b

Big W Asymptotic Lower Bound

f(n)

c g(n)

• f(n) c g(n) for all n n0


asymptotic lower bound of f(n).

• We write f(n)=W(g(n))

• It reads f(n) is omega of g(n).

n0

Example of Asymptotic Lower Bound

f(n)=n2/2-7

c g(n)=n2/4

g(n)=n2

g(n)/4 = n2/4

= n2/2 – n2/4

n2/2 – 9 for all n 6

< n2/2 – 7

Thus, f(n)= W(g(n)).

6

g(n)=n2

Example: Big Omega

Example: n 1/2 = W( log n) .

Use the definition with c = 1 and n0 = 16.

Checks OK.

Let n > 16 : n 1/2 (1) log n if and only if n = ( log n )2

by squaring both sides.

This is an example of polynomial vs. log.

Big Theta Notation

Definition: Two functions f and g are said to be of equal growth, f = Big Theta(g) if and only if both

f=Q(g) and g = Q(f).

Definition: f(n) = O(g(n)) means positive constants c1, c2, and n0 such that

c1 g(n) f(n) c2 g(n) n n0

If f(n) = O(g(n)) and f(n) = W(g(n)) then f(n) = Q(g(n))

(e.g. f(n) = n2 and g(n) = 2n2)

Theta, the Asymptotic Tight

Bound

Theta means that f is bounded above and below

by g; BigTheta implies the "best fit".

f(n) does not have to be linear itself in order to

be of linear growth; it just has to be between two

linear functions,

Asymptotically Tight Bound

f(n)

c1 g(n)

• f(n) = O(g(n)) and f(n) = W(g(n))


asymptotically tight bound of f(n).

• We write f(n)=Q(g(n))

• It reads f(n) is theta of g(n).

n0

c2 g(n)

Other Asymptotic Notations

A function f(n) is o(g(n)) if positive constants c

and n0 such that

f(n) < c g(n) n n0

A function f(n) is (g(n)) if positive constants c

and n0 such that

c g(n) < f(n) n n0

Intuitively,

– o() is like <

– O() is like

– () is like >

– W() is like

– Q() is like =

Examples

1. 2n3 + 3n2 + n = 2n3 + 3n2 + O(n)

= 2n3 + O( n2 + n) = 2n3 + O( n2 )

= O(n3 ) = O(n4)

2. 2n3 + 3n2 + n = 2n3 + 3n2 + O(n)

= 2n3 + Q(n2 + n)

= 2n3 + Q(n2) = Q(n3)

Examples (cont.)

3. Suppose a program P is O(n3), and a program Q

is O(3n), and that currently both can solve

problems of size 50 in 1 hour. If the programs are

run on another system that executes exactly 729

times as fast as the original system, what size

problems will they be able to solve?

Example (cont.)

n3 = 503 * 729 3n = 350 * 729

n = n = log3 (729 * 350)

n = n = log3(729) + log3 350

n = 50 * 9 n = 6 + log3 350

n = 50 * 9 = 450 n = 6 + 50 = 56

Improvement: problem size increased by 9 times for n3

algorithm but only a slight improvement in problem size

(+6) for exponential algorithm.

3 3 729*5033 3 72950

More Examples

(a) 0.5n2 - 5n + 2 = W( n2).

Let c = 0.25 and n0 = 25.

0.5 n2 - 5n + 2 = 0.25( n2) for all n = 25

(b) 0.5 n2 - 5n + 2 = O( n2).

Let c = 0.5 and n0 = 1.

0.5( n2) = 0.5 n2 - 5n + 2 for all n = 1

(c) 0.5 n2 - 5n + 2 = Q( n2)

from (a) and (b) above.

Use n0 = 25, c1 = 0.25, c2 = 0.5 in the definition.

More Examples

(d) 6 * 2n + n2 = O(2n).

Let c = 7 and n0 = 4.

Note that 2n = n2 for n = 4. Not a tight upper bound, but

it's true.

(e) 10 n2 + 2 = O(n4).

There's nothing wrong with this, but usually we try to get

the closest g(n). Better is to use O(n2 ).

How to show a function f(n) is in/not

in O(g(n)). f(n) is in O(g(n)):

find a constant c and large n0 such that

for all n > n0, f(n) < c g(n).

f(n) is not in O(g(n)):

for any constant c and any large n0,

we can find a m such that

f(m) > c g(m).

Usually it is more difficult to proof that

a function f(n) is not in the big-O of another

function g(n).

c,n0 n>n0 s.t. f(n)< c g(n) c,n0 n>n0 s.t. f(n)>= c g(n)

Big O Again!!!!

O(1) The cost of applying the algorithm can be

bounded independently of the value of n. This is

called constant complexity.

O(log n) The cost of applying the algorithm to

problems of sufficiently large size n can be

bounded by a function of the form k log n,

where k is a fixed constant. This is called

logarithmic complexity.

O(n) linear complexity

O(n log n) n lg n complexity

O(n2) quadratic complexity

Big O Again!!!!

O(n3) cubic complexity

O(n4) quartic complexity

O(n32) polynomial complexity

O(cn) If constant c 1, then this is called

exponential complexity

O(2n) exponential complexity

O(en) exponential complexity

O(n!) factorial complexity

O(nn)

Practical Complexity t < 250

0

250

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

f(n) = n

f(n) = log(n)

f(n) = n log(n)

f(n) = n 2

f(n) = n 3

f(n) = 2 n


0

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

f(n) = n

f(n) = log(n)

f(n) = n log(n)

f(n) = n 2

f(n) = n 3

f(n) = 2 n


0

1000

1 3 5 7 9 11 13 15 17 19

f(n) = n

f(n) = log(n)

f(n) = n log(n)

f(n) = n 2

f(n) = n 3

f(n) = 2 n


0

1000

2000

3000

4000

5000

1 3 5 7 9 11 13 15 17 19

f(n) = n

f(n) = log(n)

f(n) = n log(n)

f(n) = n 2

f(n) = n 3

f(n) = 2 n

Algoritma Brute Force 1

Definisi Brute Force

Brute force : pendekatan straight forward untuk

memecahkan suatu masalah

Algoritma brute force memecahkan masalah

dengan sangat sederhana, langsung, danjelas

(obvious way)

Contoh-contoh

(Berdasarkan pernyataan masalah)

Mencari elemen terbesar (terkecil)

Persoalan: Diberikan sebuah array yang

beranggotakan n buah bilangan bulat (a1, a2, ...,

an).Carilah elemen terbesar di dalam array

tersebut.

Algoritma brute force: bandingkan setiap

elemen array untuk menemukan elemen

terbesar

Kompleksitas O(n)

Contoh-contoh


Mencari elemen terbesar (terkecil)

Contoh-contoh


Pencarian beruntun (Sequential Search)

Persoalan: Diberikan array yang berisi n buah

bilangan bulat (a1, a2, ..., an). Carilah nilai x di

dalam array tersebut. Jika x ditemukan, maka

keluarannya adalah indeks elemen array, jika x

tidak ditemukan, maka keluarannya adalah 0.

Algoritma brute force (sequential serach): setiap

elemen array dibandingkan dengan x.

Pencarian selesai jika x ditemukan atau elemen

array sudah habis diperiksa.

Kompleksitas O(n)

Contoh-contoh


Pencarian beruntun (sequential search)

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Menghitung an (a > 0, n adalah bilangan bulat

tak-negatif)

Definisi: an= a x a x ... x a (n kali) , jika n>0

= 1, jika n = 0

Algoritma brute force: kalikan 1 dengan a

sebanyak n kali

Kompleksitas O(n)

Contoh-contoh (Berdasarkan definisi

konsep yang terlibat)

Menghitung pangkat



Menghitung n! (n bilangan bulat tak-negatif)

Definisi: n!=1 ×2×3×...×n, jika n>0

= 1, jika n = 0

Algoritma brute force: kalikan n buah bilangan,

yaitu 1, 2, 3, ..., n, bersama-sama

Kompleksitas O(n)



Menghitung faktorial



Mengalikan dua buah matriks, A dan B

Definisi: Misalkan C = A × B dan elemen-

elemen matrik dinyatakan sebagai cij, aij, dan bij

Algoritma brute force: hitung setiap elemen hasil

perkalian satu per satu, dengan cara

mengalikan dua vektor yang panjangnya n.



Mengalikan dua buah matriks



Menemukan semua faktor dari bilangan bulat n

(selain dari 1 dan n itu sendiri).

Definisi: Bilangan bulat a adalah faktor dari

bilangan bulat b jika a habis membagi b.

Algoritma brute force: bagi n denga n setiap i =

2, 3, ..., n – 1. Jika n habis membagi i, maka I

adalah faktor dari n.



Menemukan faktor bilangan bulat n



Uji keprimaan Persoalan: Diberikan sebuah

bilangan bilangan bulat positif. Ujilah apakah

bilangan tersebut merupakan bilangan prima

atau bukan.

Definisi: bilangan prima adalah bilangan yang

hanya habis dibagi oleh 1 dan dirinya sendiri.

Algoritma brute force: bagi n dengan 2 sampai

n–1. Jika semuanya tidak habis membagi n,

maka n adalah bilangan prima

Perbaikan: cukup membagi dengan 2 sampai

√n saja



Uji bilangan prima



Algoritma Pengurutan Brute Force, Algoritma

apa yang paling lempang dalam memecahkan

masalah pengurutan?

Bubble sort dan selection sort!

Kedua algoritma ini memperlihatkan teknik brute

force dengan jelas sekali.



Bubble Sort

Mulai dari elemen ke-n:

1. Jika sn< sn-1, pertukarkan

2. Jika sn-1 < sn-2, pertukarkan ...

3. Jika s2< s1, pertukarkan

1 kali pass

Ulangi lagi untuk pass ke-i



Bubble sort



Selection Sort Pass ke –1:

1.Cari elemen terbesar mulai di dalam s[1..n]

2.Letakkan elemen terbesar pada posisi n

(pertukaran)

Pass ke-2:

1.Cari elemen terbesar mulai di dalam s[1..n - 1]

2.Letakkan elemen terbesar pada posisi n - 1

(pertukaran)

Ulangi sampai hanya tersisa 1 elemen



Selection sort



Mengevaluasi polinom

Persoalan: Hitung nilai polinom

p(x)=anxn +an-1x

n-1 +...+a1x +a0 untuk x = t.

Algoritma brute force: xi dihitung secara brute

force (seperti perhitungan an). Kalikan nilai xi

dengan ai, lalu jumlahkan dengan suku-suku

lainnya.



Analisa polinom



Perbaikan (improve)

Karakteristik Algoritma Brute Force

Algoritma brute force umumnya tidak “cerdas”

dan tidak cepat, karena ia membutuhkan jumlah

langkah yang besar dalam penyelesaiannya.

Kata “force” mengindikasikan “tenaga”

ketimbang “otak”

Kadang-kadang algoritma brute force disebut

juga algoritma naif (naïve algorithm).


Algoritma brute force lebih cocok untuk masalah

yang berukuran kecil.

Pertimbangannya:

sederhana,

Implementasinya mudah

Algoritma brute force sering digunakan sebagai

basis pembanding dengan algoritma yang lebih

cepat.


Meskipun bukan metode yang cepat, hampir

semua masalah dapat diselesaikan dengan

algoritma brute force.

Sukar menunjukkan masalah yang tidak dapat

diselesaikan dengan metode brute force.

Bahkan, ada masalah yang hanya dapat

diselesaikan dengan metode brute force.

Contoh: mencari elemen terbesar di dalam

array.

Tugas

Buatlah algoritma berikut dengan kompleksitasnya

Pencocokan String (String Matching)

Mencari pasangan titik yang jaraknya terdekat

(Closest pairs)

Travelling Salesman Problem (TSP)

Knapsack Problem

Click to edit subtitle style

Design and Analysis Algorithm Pertemuan 04 · Pertemuan 04 . Contents ... Indikator efisiensi algoritma bedasar pada OoG ... Q(g(n)), Theta of g of n, the Asymptotic Tight Bound;

Documents