Data Structures CS 310. Abstract Data Types (ADTs) An ADT is a formal description of a set of data values and a set of operations that manipulate the.

Data Structures

CS 310

Abstract Data Types (ADTs)

• An ADT is a formal description of a set of data values and a set of operations that manipulate the values.

• An ADT does not specify how the type should be implemented.

ADT: A concrete example

• Values– minimum– maximum– precision– Infinity– undefined

• Operators & Properties– add– subtract– multiply– divide– rounding mode

(4 modes supported)

IEEE floating point 754An ADT frequently implemented in hardware

ADTs – Why bother?

• Abstraction

• Encapsulation

• Reusability

• Ability to change the implementation without changing programs using the ADT.

ADTs and languages

• An ADT can be implemented in any language, even assembly.

• ADTs can be particularly elegant in object oriented languages. – Why?

• Is one algorithm better than another?

• How should we judge?

Comparing algorithms

• Maintainability

• Readability

• Elegance

• Complexity analysis

Complexity analysis

• How long does it take for an algorithm to run?– Is runtime a good indication?– How does the algorithm behave as the

number of items N increases?

• How much memory does the algorithm use?– Should we measure in bytes?– Behavior with respect to N?

Copyright © 2006 Pearson Addison-Wesley. All rights reserved



Sample analysis

• Maximum Contiguous subsequence

• Given a list of integers, return the subsequence that produces the maximum sum.

• Examples: 2, 3, 5, -33, 5, 6, 1, 3, 3 5, 6, 1, 3, 3-2, -5, -8, -2, -17, -33 empty3, -7, 13, -3, 10, 20, -50, 6, 3 13, -3, 10, 20

Brute force solution


How many operations?

• We estimated that O(N3) operations

• Could we make this a little more precise?

• We need a helping hand from combinatorics

If we have 3 items, how many ways can we choose two of them?

i

N

2

3

)!(!

!

iNi

N

i

N

combinatorics

• In general, we can determine how many possible ways i items can be chosen from N with:

3!1

3

)!23(!2

!3

2

3

Our first proof

How many times exactly?

• We know that

• 1 and N are constants, so we need to think about the number of ways to choose i, j, and k.

• They vary between 1 and N.

Njki

A first attempt

• Suppose we were to choose 3 integers from 1 to N.

• We know that this would be

• This is almost right, but why not?

3

N

A refinement

• k can be either equal to i or j

• Suppose we add two additional choices– low– high

giving us N+2 choices

• When we select low or high:– lowk=i– highk=j

What does it mean when wepick both low and high?

Number of operations contd

• Now, when we choose 3 from N+2 choices, we have the exact number of iterations through the loop.

!3

))(1)(2()!1(!3

)!1)()(1)(2()!1(!3

)!1)()(1)(2()!3)2((!3

)!2(

3

2

NNNN

NNNNN

NNNNN

NN

O(N2)


O(N) – Linear time

• For the quadratic algorithm, we simply changed the inner most loop of the cubic algorithm from linear O(N) time to constant O(1) time.

• To make a linear time algorithm, we will need to be a bit more clever…

Clever observation 1(Theorem 5.2)

• Let Ai,j denote the sequence from i to j.

• Let Si,j denote the sum of Ai,j.

If Ai,j has Si,j < 0 and q>j, then Ai,q is not a maximum subsequence.

Clever observation 2

• All contiguous subsequences that border the maximum contiguous subsequence must be negative or zero.

• Let’s prove it…

Clever observation 3Theorem 5.3

• For any i, let Ai,j be the first sequence with Si,j < 0. Then for any i≤p≤j and p≤q, Ap,q is either:– not a maximum contiguous subsequence– equal to a previously seen maximum

contiguous subsequence.


Remember– Ai,j is the first sequence whose sum Si,j < 0

– i≤p≤j and p≤q

• case 1: p=iThen Sp,q = Si,q but this is < 0


• case 2: i<p<q≤jAp,q cannot be a max

contiguous subsequence.


• case 3 i<p≤j<q

O(N) algorithm


Big-Oh, Big-Omega,Big-Theta, and Little-Oh


Note: We usually are finding the growth of the function with respect to Nwhich is Θ(F(N)), but we usually abuse the notation and write O(F(N))

Big Oh notation in style…

• Earlier, we showed that our cubic algorithm had operations.

• Nonetheless, we write that it is O(N3).

• The lower order terms are ignored as they are dominated by the N3 and we ignore constant scale factors.

!3

))(1)(2( NNN

NNN 312

613

61 3

Review - logarithms

• For any base B and N > 0,

• Note that logB BK = K.

• Which base should we use for complexity analysis?

NBKN KB if log

logs and complexity analysis

• Which base should we use for complexity analysis?

• It doesn’t matter! (See theorem 5.4, p. 181)

• For convenience, we always use base 2.

Fun with logarithms

• Repeated doubling– Given X = 1, how many times t must X be

doubled before X >= N?

inequality thesatisfies which smallest t theis log

log

log2log

2

N

Nt

N

Nt

t

Fun with logarithms

• Repeated halvingX = N

while X > 1 {

X = X / 2

}

• How many times will the loop execute?

Repeated halving

• Try to prove this at home…

division duringdown round weif log

division during up round weif log

N

N

Harmonic numbers

• The Nth harmonic number is defined as

• By theorem 5.5 (p. 183 Weiss)

N

iN iH

1

1

)(logNHN

log N tools

• We now have three tools at our disposal to recognize log N phenomenon in our programs:– repeated doubling– repeated halving– harmonic series

Static searching

• A static search occurs when we look for an element of an abstract data type and do not modify the data structure.

• For now, we will consider searching for integers in arrays.

Linear search

found = false;

done = false;

i = 0;

while (! done && ! found) {

if (a[i] == search_value){

found = true;

} else {

i = i+1;

if (i >= a.length) {

done = true;

}

}

Complexity analysis of our search

• Failure case – What does it cost if we don’t find anything?

• Worst case – What if we do find something, but it is the last item we try?

• Average case – What happens on average?

Binary search


Empirical runtime of binary searchor how to check your results…


Interpolation search

• Make a good guess about the next location based on the min and max values of the array.

• Assumes that numbers are uniformly distributed (they are spread randomly between the min and max values).

• Average case: O(log log N)

• Worst case: O(N)

Things to remember

• big-oh analysis is true for some N0 such that N>N0.

• big-oh analysis doesn’t take the physical devices into account:– e.g. disk and RAM access times are

dramatically different

• Worst case usually easier to determine than average case.

Common errors

• Total time in consecutive loops does not affect the big-oh estimate other than the the size of the biggest loop.

• Pay attention to how many times the loop iterates. A loop could be log N, N2, etc.

• Remember to drop the low order terms and constants:5N3+4N2+8N O(N3)

Common errors continued

• Remember, big-oh is an upper bound. You cannot say that T(N) > O(N3) as by definition O(N3) implies that there are less than order N3 operations. Write T(N)=O(N3)

Complexity recap

• O (big-oh) upper bound: order ≤ stated bound• Ω (big-omega) lower bound: order ≥ stated

bound• Θ (big-theta) is exactly of order: order = stated

bound• o (little-oh) is less than order: order < stated

bound• Remember, we usually compute big-theta and

write big-oh. Sigh…

Data Structures CS 310. Abstract Data Types (ADTs) An ADT is a formal description of a set of data values and a set of operations that manipulate the.

Documents

clever slide

combinatorics slide

hardware slide

proof slide

rights reserved slide

linear time algorithm

maximum subsequence

cubic algorithm