Last lesson Arrays for implementing collection classes Performance analysis (review) Today Performance analysis Logarithm.

Last lesson Arrays for implementing collection classes Performance analysis (review)

Today Performance analysis Logarithm

Complexity Theory

Three different symbols will be discussed in class: O (called the Big Oh) (upper case Greek letter OMEGA) (upper case Greek letter THETA)

Weiss, chapter 5

O (Big Oh)

The Big Oh is the upper bound of a function. In the case of algorithm analysis, we use it to bound the

worst-case running time, or the longest running time possible for any input of size n.

We can say that the maximum running time of the algorithm is in the order of Big Oh.

T(n) = O(f(n)) if there are positive constants c and n0 such that T(n) c*f(n) for all n n0

is is the lower bound of a function. We can say that the minimum running time of the

algorithm is in the order of

T(n) = (f(n)) if there are positive constants c and n0 such that T(n) c*f(n) for all n n0

is composed of both O and . T(n) = (f(n)) if and only if T(n) is O(f(n)) and T(n)

is (f(n))

n

T

upper bound

lower bound

A Few Remarks

We only care what happens for large n (n n0) O, , "hide" the constant c from us. Find simple and small functions

E.g. (n2 + 3n + 4) is O(n3), since (n2 + 3n + 4) < n3 for all n > 3,

but it is also O(n2), since (n2 + 3n + 4) < 2n2 for all n > 10

The Logarithm

For any b, n>0: logbn=k if bk=n

We use log instead of log2 (as in Weiss) Be careful, often

lg instead of log2

log instead of log10

Properties of logarithms logb(x*y) = logbx + logby

logb(x/y) = logbx - logby

logbxa = a*logbx

logbx= (logax)/(logab)

For complexity analysis the base does not matter For any constant b>1: logbn = O(log n)

Repeated Doubling and Halving

Repeated doubling We can repeatedly double only logarithmically many

times until we reach n

Repeated halving Starting at n, we can halve only logarithmically many

times before we reach 1

Static Searching Problem

Given an integer num and an array arr, return the position of num in arr or an indication that it is not present. If num occurs more than once, return any occurrence. The array is never altered (is static).

Weiss, chapter 5.6

4

“7”“3” “12” “42” “56” “75” “77”

4

Linear (Sequential) Search

for (int i = 0; i <= last; i++) { if (arr[i] = num) {

return i; }

}

Complexity E.g. search for “56” O(n) (linear)

“7”“3” “12” “42” “56” “75” “77”

Linear (Sequential) Search

for (int i = 0; i <= last; i++) { if (arr[i] = num) {

return i; }

}

Complexity E.g. search for “98” O(n) (linear)

“7”“3” “12” “42” “56” “75” “77”

Binary Search (1)

Array must be sorted Routine to return index where element is equal to

argument Look in middle of occupied part of array If that is the element we are looking for, we found it Otherwise, decide whether the element must be in front

half or back half (do recursive call to look there) E. g. search for “56”

This is a divide-and-conquer algorithm

“7”“3” “12” “42” “56” “75” “77”

Binary Search (1)





“7”“3” “12” “42” “56” “75” “77”

Binary Search (1)





“7”“3” “12” “42” “56” “75” “77”

Binary Search (1)





“7”“3” “12” “42” “56” “75” “77”

Binary Search (1)





“7”“3” “12” “42” “56” “75” “77”

Binary Search (2)

“7”

“3” “12”

“42”

“56”

“75”

“77”

Binary Search (3)

It can also be written as a loop (non recursive) Warning: the idea (binary search) is simple, but

the code is tricky to get right Avoid infinite recursion or infinite loop Treat boundary cases properly Avoid looking at elements which are not in the sequence

Each time we deal with half as much of the array as the previous call (or iteration) So we make log n calls or iterations Each time we do const work besides the recursion So the total cost is O(log n) (much faster than simple

searching or searching in an unsorted array)

Code

public static int binarySearch(Comparable [ ] a, Comparable x) {

int low = 0; int high = a.length - 1; int mid;

while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)

low = mid + 1;else if(a[mid].compareTo(x) > 0)

high = mid - 1;else

return mid;}return NOT_FOUND; // NOT_FOUND = -1

}

Code





high = mid - 1;else


}

73 12 42 56 75 77

low high

find 56

Code





high = mid - 1;else


}

73 12 42 56 75 77

low high

find 56

Code





high = mid - 1;else


}

73 12 42 56 75 77

low highmid

find 56





high = mid - 1;else


}

Code

73 12 42 56 75 77

low

highmid

find 56





high = mid - 1;else


}

Code

73 12 42 56 75 77

low

highmid

find 56





high = mid - 1;else


}

Code

73 12 42 56 75 77

low

highmid

find 56





high = mid - 1;else


}

Code

73 12 42 56 75 77

low

high

mid

find 56





high = mid - 1;else


}

Code

73 12 42 56 75 77

low

high

mid

find 56





high = mid - 1;else


}

Code

73 12 42 56 75 77

low

high

mid

find 56





high = mid - 1;else


}

Code

73 12 42 56 75 77

low

high

mid

find 56

Code





high = mid - 1;else


}

73 12 42 56 75 77

low high

find 98

Code





high = mid - 1;else


}

73 12 42 56 75 77

low high

find 98

Code





high = mid - 1;else


}

73 12 42 56 75 77

low highmid

find 98





high = mid - 1;else


}

Code

73 12 42 56 75 77

low

highmid

find 98





high = mid - 1;else


}

Code

73 12 42 56 75 77

low

highmid

find 98





high = mid - 1;else


}

Code

73 12 42 56 75 77

low

highmid

find 98





high = mid - 1;else


}

Code

73 12 42 56 75 77

low

mid

find 98

high





high = mid - 1;else


}

Code

73 12 42 56 75 77

find 98

midhigh

low





high = mid - 1;else


}

Code

73 12 42 56 75 77

find 98

high

low

mid





high = mid - 1;else


}

Code

73 12 42 56 75 77

find 98

high

low

mid





high = mid - 1;else


}

Code

73 12 42 56 75 77

find 98

high

low

mid

Simplifying Rules

If f(n) is in O(g(n)) and g(n) is in O(h(n)), then f(n) is in O(h(n)).

If f(n) is in O(k*g(n)) for any constant k > 0, then f(n) is in O(g(n)).

If f1(n) is in O(g1(n)) and f2(n) is in O(g2(n)), then f1(n)+f2(n) is in max{O(g1(n)),

O(g2(n))}.

If f1(n) is in O(g1(n)) and f2(n) is in O(g2(n)), then f1(n)*f2(n) is in O(g1(n)*g2(n)).

Analysing Control Statements

While loop Analyse like a for loop

If statement Take greater complexity of then/else clauses

Switch statement Take complexity of most expensive case

Subroutine call Complexity of the subroutine

Limitations of Big Oh

Not for small amount of input Is designed for theoretical analysis

E.g. memory access is O(1), does not differentiate between main memory and disc

Constants may be to large for practical use Worst case is sometimes uncommon an can be ignored

Recursion

This is a review of the main ideas We expect some experience from first year Read Weiss Sections 7.1 - 7.3

Key Ideas

A method includes one or more calls to the same method The result of the nested calls is used to compute the

result of the outer call There is a base case The call may be indirectly

Understand the method by assuming the nested calls work correctly

First, let us look at a non-programming example

Recursive Definition

Define a list of number separated by commas Example: 23, 27, 31, 1

Like this LIST is number or LIST is number, LIST

Example

Try to see whether “23, 27, 31, 1” is a list Apply definition

LIST: number

or LIST: number, LIST23, 27, 31, 1

23, LIST

27, LIST

31, LIST

1

Erroneous Examples

4 . 54 , 23 D, 34, 2

LIST: number

or LIST: number, LIST

Base Case

Notice that we had one case (LIST: number) that does no recursion That is called the base case Otherwise, we would have infinite recursion

LIST: number

or LIST: number, LIST

31, LIST

1

…

What About Programs?

We can solve a lot of mathematical problems recursively

E.g. i=0 i = 1 + 2 + 3 + … + n

public static int sum (int n) {

if (n < 1)

return 0;

else

return(sum(n – 1) + n);

}

n

Example

public static int sum(int n) {

if(n < 1)

return 0;

return(sum(n – 1) + n);

}

sum(4)

sum(3)

sum(2)

sum(1)

sum(0)

0

0 + 1

1 + 2

3 + 3

6 + 4

Progress

Notice that we must make progress toward base case Sum(0) There is steady reduction in size of problem Otherwise, trouble

Iterative Solution

For this problem a recursive solution does not make sense Elegant, but slower than the following solution


int result = 0;

for (int i = 1; i <= n; i++)

result += i;

return(n);

} The best solution


return (n/2)*(n+1);

}

Last lesson Arrays for implementing collection classes Performance analysis (review) Today Performance analysis Logarithm.

Documents

log b n

large n n n

int high

log b x log b x

int mid whilelow

int low

log b x log b y log

olog n slide