This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CSC401 – Analysis of AlgorithmsCSC401 – Analysis of Algorithms Chapter 1Chapter 1
Introduce algorithm and algorithm analysisIntroduce algorithm and algorithm analysisDiscuss algorithm analysis methodologiesDiscuss algorithm analysis methodologiesIntroduce pseudo code of algorithmsIntroduce pseudo code of algorithmsAsymptotic notion of algorithm efficiencyAsymptotic notion of algorithm efficiencyMathematics foundation for algorithm analysisMathematics foundation for algorithm analysisAmortization analysis techniquesAmortization analysis techniques
22
What is an algorithm ?What is an algorithm ?
AlgorithmInput Output
An algorithm is a step-by-step procedure for solving a problem in a finite amount of time.
What is algorithm analysis ?Two aspects:• Running time – How much time is taken to
complete the algorithm execution?• Storage requirement – How much memory
is required to execute the program?
33
Running Time Running Time Most algorithms Most algorithms transform input objects transform input objects into output objects.into output objects.The running time of an The running time of an algorithm typically grows algorithm typically grows with the input size.with the input size.Average case time is Average case time is often difficult to often difficult to determine.determine.We focus on the worst We focus on the worst case running time.case running time.– Easier to analyzeEasier to analyze– Crucial to applications such Crucial to applications such
as games, finance and as games, finance and roboticsrobotics
0
20
40
60
80
100
120
Runnin
g T
ime
1000 2000 3000 4000
Input Size
best caseaverage caseworst case
44
How Is An Algorithm Analyzed?How Is An Algorithm Analyzed?Two methodologiesTwo methodologies– Experimental analysisExperimental analysis
MethodMethod– Write a program implementing the Write a program implementing the
algorithmalgorithm– Run the program with inputs of varying Run the program with inputs of varying
size and compositionsize and composition– Use a method like Use a method like System.currentTimeMillis()System.currentTimeMillis()
to get an accurate measure of the to get an accurate measure of the actual running timeactual running time
– Plot the resultsPlot the results
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 50 100
Input Size
Tim
e (m
s)
LimitationsLimitations– It is necessary to implement the algorithm, which may It is necessary to implement the algorithm, which may
be difficultbe difficult– Results may not be indicative of the running time on Results may not be indicative of the running time on
other inputs not included in the experiment. other inputs not included in the experiment. – In order to compare two algorithms, the same In order to compare two algorithms, the same
hardware and software environments must be usedhardware and software environments must be used
55
How Is An Algorithm Analyzed?How Is An Algorithm Analyzed?– Theoretical Analysis Theoretical Analysis
MethodMethod– Uses a high-level description of the algorithm Uses a high-level description of the algorithm
instead of an implementationinstead of an implementation– Characterizes running time as a function of the Characterizes running time as a function of the
input size, input size, nn..– Takes into account all possible inputsTakes into account all possible inputs– Allows us to evaluate the speed of an algorithm Allows us to evaluate the speed of an algorithm
independent of the hardware/software independent of the hardware/software environmentenvironment
CharacteristicsCharacteristics– A description language -- Pseudo codeA description language -- Pseudo code– MathematicsMathematics
66
PseudocodePseudocode
High-level description High-level description of an algorithmof an algorithmMore structured than More structured than English proseEnglish proseLess detailed than a Less detailed than a programprogramPreferred notation for Preferred notation for describing algorithmsdescribing algorithmsHides program design Hides program design issuesissues
Algorithm arrayMax(A, n)Input array A of n integersOutput maximum element of A
The Random Access Machine (RAM) ModelThe Random Access Machine (RAM) Model
A A CPUCPU
An potentially An potentially unbounded bank of unbounded bank of memorymemory cells, each of cells, each of which can hold an which can hold an arbitrary number or arbitrary number or charactercharacter
01
2
Memory cells are numbered and accessing Memory cells are numbered and accessing any cell in memory takes unit time.any cell in memory takes unit time.
99
Primitive OperationsPrimitive OperationsBasic computations Basic computations performed by an algorithmperformed by an algorithm
Identifiable in pseudocodeIdentifiable in pseudocode
Largely independent from Largely independent from the programming languagethe programming language
Exact definition not Exact definition not important (we will see why important (we will see why later)later)
Assumed to take a constant Assumed to take a constant amount of time in the RAM amount of time in the RAM modelmodel
Examples:Examples:– Evaluating an Evaluating an
expressionexpression– Assigning a value to Assigning a value to
a variablea variable– Performing an Performing an
arithmetic operationarithmetic operation– Comparing two Comparing two
numbersnumbers– Indexing into an Indexing into an
arrayarray– Following an object Following an object
referencereference– Calling a methodCalling a method– Returning from a Returning from a
methodmethod
1010
Counting Primitive OperationsCounting Primitive OperationsBy inspecting the pseudocode, we can determine By inspecting the pseudocode, we can determine the maximum number of primitive operations the maximum number of primitive operations executed by an algorithm, as a function of the executed by an algorithm, as a function of the input sizeinput size
AlgorithmAlgorithm arrayMaxarrayMax((AA, , nn))
# operations# operations
currentMaxcurrentMax AA[0][0] 22forfor ii 11 toto nn 1 1 dodo 11 nn
Estimating Running TimeEstimating Running TimeAlgorithm Algorithm arrayMaxarrayMax executes executes 77nn 2 2 primitive primitive operations in the worst case. operations in the worst case.
Define:Define:aa = Time taken by the fastest primitive operation= Time taken by the fastest primitive operation
bb = Time taken by the slowest primitive operation= Time taken by the slowest primitive operation
Let Let TT((nn)) be the worst-case time of be the worst-case time of arrayMax.arrayMax. ThenThen
a a (7(7nn 1) 1) TT((nn)) bb(7(7nn 1) 1)
Hence, the running time Hence, the running time TT((nn)) is bounded by is bounded by two linear functionstwo linear functions
1212
Growth Rate of Running TimeGrowth Rate of Running TimeChanging the hardware/ software environment Changing the hardware/ software environment – Affects Affects TT((nn)) by a constant factor, but by a constant factor, but– Does not alter the growth rate of Does not alter the growth rate of TT((nn))
The linear growth rate of the running time The linear growth rate of the running time TT((nn)) is is an intrinsic property of algorithm an intrinsic property of algorithm arrayMaxarrayMax
Growth rates of Growth rates of functions:functions:– Linear Linear nn– Quadratic Quadratic nn22
– Cubic Cubic nn33
In a log-log chart, the In a log-log chart, the slope of the line slope of the line corresponds to the corresponds to the growth rate of the growth rate of the functionfunction
Constant FactorsConstant FactorsThe growth rate is The growth rate is not affected bynot affected by– constant factors or constant factors or – lower-order termslower-order terms
ExamplesExamples– 101022nn 101055 is a linear is a linear
Big-Oh NotationBig-Oh NotationGiven functions Given functions ff((nn) ) and and gg((nn)), , we say that we say that ff((nn) ) is is OO((gg((nn)))) if if there are positive constantsthere are positive constantscc and and nn00 such that such that
ff((nn)) cgcg((nn) ) for for n n nn00
Example: Example: 22nn 1010 is is OO((nn))– 22nn 1010 cncn
– ((cc 2) 2) n n 1010
– n n 1010((cc 2) 2)
– Pick Pick c c 3 3 and and nn0 0 1010
Example: the function Example: the function nn22 is is not not OO((nn))– nn22 cncn
– n n cc– The above inequality cannot The above inequality cannot
be satisfied since be satisfied since cc must be must be a constant a constant
1
10
100
1,000
10,000
1 10 100 1,000n
3n
2n+10
n
1
10
100
1,000
10,000
100,000
1,000,000
1 10 100 1,000n
n̂ 2
100n
10n
n
1515
More Big-Oh ExamplesMore Big-Oh Examples7n-27n-2
7n-2 is O(n)7n-2 is O(n)need c > 0 and nneed c > 0 and n00 1 such that 7n-2 1 such that 7n-2 c c•n for n •n for n n n00
this is true for c = 7 and nthis is true for c = 7 and n00 = 1 = 1 3n3n33 + 20n + 20n22 + 5 + 5
3n3n33 + 20n + 20n22 + 5 is O(n + 5 is O(n33)) need c > 0 and nneed c > 0 and n00 1 such that 3n 1 such that 3n33 + 20n + 20n22 + 5 + 5
cc•n•n33 for n for n n n00 this is true for c = 4 and nthis is true for c = 4 and n00 = 21 = 21
3 log n + log log n3 log n + log log n 3 log n + log log n is O(log n)3 log n + log log n is O(log n) need c > 0 and nneed c > 0 and n00 1 such that 3 log n + log log 1 such that 3 log n + log log
n n c c•log n for n •log n for n n n00 this is true for c = 4 and nthis is true for c = 4 and n00 = 2 = 2
1616
Big-Oh and Growth RateBig-Oh and Growth RateThe big-Oh notation gives an upper bound on The big-Oh notation gives an upper bound on the growth rate of a functionthe growth rate of a function
The statement “The statement “ff((nn) ) is is OO((gg((nn))))” means that the ” means that the growth rate of growth rate of ff((nn) ) is no more than the growth is no more than the growth rate of rate of gg((nn))
We can use the big-Oh notation to rank We can use the big-Oh notation to rank functions according to their growth ratefunctions according to their growth rate
ff((nn) ) is is OO((gg((nn)))) gg((nn) ) is is OO((ff((nn))))
gg((nn) ) grows moregrows more YesYes NoNo
ff((nn) ) grows moregrows more NoNo YesYes
Same growthSame growth YesYes YesYes
1717
Big-Oh RulesBig-Oh RulesIf is If is ff((nn)) a polynomial of degree a polynomial of degree dd, then , then ff((nn)) is is OO((nndd)), i.e.,, i.e.,
1.1.Drop lower-order termsDrop lower-order terms
2.2.Drop constant factorsDrop constant factors
Use the smallest possible class of Use the smallest possible class of functionsfunctions
– Say “Say “22nn is is OO((nn))”” instead of “instead of “22nn is is OO((nn22))””
Use the simplest expression of the classUse the simplest expression of the class– Say “Say “33nn 55 is is OO((nn))”” instead of “instead of “33nn 55 is is
The asymptotic analysis of an algorithm The asymptotic analysis of an algorithm determines the running time in big-Oh notationdetermines the running time in big-Oh notationTo perform the asymptotic analysisTo perform the asymptotic analysis
– We find the worst-case number of primitive operations We find the worst-case number of primitive operations executed as a function of the input sizeexecuted as a function of the input size
– We express this function with big-Oh notationWe express this function with big-Oh notation
Example:Example:– We determine that algorithm We determine that algorithm arrayMaxarrayMax executes at executes at
most most 77nn 2 2 primitive operationsprimitive operations– We say that algorithm We say that algorithm arrayMaxarrayMax “runs in “runs in OO((nn) ) time”time”
Since constant factors and lower-order terms are Since constant factors and lower-order terms are eventually dropped anyhow, we can disregard eventually dropped anyhow, we can disregard them when counting primitive operationsthem when counting primitive operations
1919
Summation:Summation:
Geometric summation:Geometric summation:
Natural summation:Natural summation:
Harmonic number:Harmonic number:
Split summation:Split summation:
SummationsSummations
a
aaaaa
nn
n
i
i
1
11
12
0
2
)1(321
0
nnni
n
i
)()2()1()()( bfafafafifb
ai
nnii
ln1
3
1
2
11
1
0
n
ki
k
i
n
i
ififif110
)()()(
aaaaa n
i
i
1
11 2
0
2020
LogarithmsLogarithms– properties of logarithms:properties of logarithms:
ExponentsExponents– properties of exponentialsproperties of exponentials::
aa(b+c)(b+c) = a = abba a cc
aabcbc = (a = (abb))cc
aabb /a /acc = a = a(b-c)(b-c)
b = a b = a loglogaabb
bbcc = a = a c*logc*logaabb
Floor and ceiling functionsFloor and ceiling functions– the largest integer less than or equal to x– the smallest integer greater than or equal to x
Logarithms and Exponents Logarithms and Exponents
x x
2121
By Example -- counterexampleBy Example -- counterexampleContrapositiveContrapositive– Principle: To justify “if p is true, then q is true”, show “if Principle: To justify “if p is true, then q is true”, show “if
q is not true, then p is not true”.q is not true, then p is not true”.– Example: To justify “if ab is odd, a is odd or b is even”, Example: To justify “if ab is odd, a is odd or b is even”,
assume “’a is odd or b is odd’ is not true”, then “a is assume “’a is odd or b is odd’ is not true”, then “a is even and b is odd”, then “ab is even”, then “’ab is odd’ even and b is odd”, then “ab is even”, then “’ab is odd’ is not true”is not true”
ContradictionContradiction– Principle: To justify “p is true”, show “if p is not true, Principle: To justify “p is true”, show “if p is not true,
then there exists a contradiction”.then there exists a contradiction”.– Example: To justify “if ab is odd, then a is odd or b is Example: To justify “if ab is odd, then a is odd or b is
even”, let “ab is odd”, and assume “’a is odd or b is even”, let “ab is odd”, and assume “’a is odd or b is even’ is not true”, then “a is even and b is odd”, thus even’ is not true”, then “a is even and b is odd”, thus “ab is even” which is a contradiction to “ab is odd”.“ab is even” which is a contradiction to “ab is odd”.
Proof Techniques Proof Techniques
2222
Induction -- To justify S(n) for n>=n0Induction -- To justify S(n) for n>=n0– PrinciplePrinciple
Base cases: Justify S(n) is true for n0<=n<=n1Base cases: Justify S(n) is true for n0<=n<=n1Assumption: Assume S(n) is true for n=N>=n1;Assumption: Assume S(n) is true for n=N>=n1;
or Assume S(n) is true for or Assume S(n) is true for n1<=n<=Nn1<=n<=NInduction: Justify S(n) is true for n=N+1Induction: Justify S(n) is true for n=N+1
– Example: Example: Question: Fibonacci sequence is defined as F(1)=1, Question: Fibonacci sequence is defined as F(1)=1, F(2)=1, and F(n)=F(n-1)+F(n-2) for n>2. Justify F(2)=1, and F(n)=F(n-1)+F(n-2) for n>2. Justify F(n)<2^n for all n>=1F(n)<2^n for all n>=1Proof by induction where n0=1. Let n1=2.Proof by induction where n0=1. Let n1=2.
– Base cases: For n0<=n<=n1, n=1 or n=2. If n=1, Base cases: For n0<=n<=n1, n=1 or n=2. If n=1, F(1)=1<2=2^1. If n=2, F(2)=2<4=2^2. It holds.F(1)=1<2=2^1. If n=2, F(2)=2<4=2^2. It holds.
– Assumption: Assume F(n)<2^n for n=N>=2.Assumption: Assume F(n)<2^n for n=N>=2.– Induction: For n=N+1, F(N+1)=F(N)+F(N-Induction: For n=N+1, F(N+1)=F(N)+F(N-
1)<2^N+2^(N-1)<2 2^N=2^(N+1). It holds.1)<2^N+2^(N-1)<2 2^N=2^(N+1). It holds.
Proof by Induction Proof by Induction
2323
Principle:Principle:– To prove a statement S about a loop is correct, define S in To prove a statement S about a loop is correct, define S in
terms of a series of smaller statements S0, S1, …, Sk, terms of a series of smaller statements S0, S1, …, Sk, wherewhere
The initial claim S0 is true before the loop beginsThe initial claim S0 is true before the loop beginsIf Si-1 is true before iteration i begins, then show that Si If Si-1 is true before iteration i begins, then show that Si is true after iteration i is overis true after iteration i is overThe final statement Sk is trueThe final statement Sk is true
Example: Consider the algorithm Example: Consider the algorithm arrayMaxarrayMax– Statement S: max is the maximum number when finishedStatement S: max is the maximum number when finished– A series of smaller statements:A series of smaller statements:
Si: max is the maximum Si: max is the maximum in the first i+1 elements in the first i+1 elements of the arrayof the array
S0 is true before the loopS0 is true before the loopIf Si is true, then easy to If Si is true, then easy to
show Si+1 is trueshow Si+1 is trueS=Sn-1 is also trueS=Sn-1 is also true
Proof by Loop Invariants Proof by Loop Invariants
AlgorithmAlgorithm arrayMaxarrayMax((AA, , nn))maxmax AA[0] [0] forfor ii 11 toto nn 1 1 dodo
ifif AA[[ii] ] maxmax thenthenmaxmax AA[[ii]]
returnreturn maxmax
2424
Sample space: Sample space: the set of all possible outcomes from the set of all possible outcomes from some experimentsome experiment
Probability space: Probability space: a sample space S together with a a sample space S together with a probability function that maps subsets of S to real numbers probability function that maps subsets of S to real numbers between 0 and 1between 0 and 1
Event: Each subset of A of S called an Event: Each subset of A of S called an eventeventProperties of the probability function PrProperties of the probability function Pr– Pr(Ø)=0Pr(Ø)=0– Pr(S)=1Pr(S)=1– 0<=Pr(A)<=1 for any subset A of S0<=Pr(A)<=1 for any subset A of S– If A and B are subsets of S and AIf A and B are subsets of S and AB=B=, then , then
Pr(APr(AB)=Pr(A)+Pr(B)B)=Pr(A)+Pr(B)
IndependenceIndependence– A and B are independent if Pr(AA and B are independent if Pr(AB)=Pr(A)Pr(B)B)=Pr(A)Pr(B)– AA11, A, A22, …, A, …, Ann are mutually independent if Pr(A are mutually independent if Pr(A11AA22… …
AAnn)=Pr(A)=Pr(A11)Pr(A)Pr(A22)…Pr(A)…Pr(Ann) )
Basic Probability Basic Probability
2525
Conditional probabilityConditional probability– The conditional probability that A occurs, given B, is The conditional probability that A occurs, given B, is
defined as: Pr(A|B)=Pr(Adefined as: Pr(A|B)=Pr(AB)/Pr(B), assuming Pr(B)>0B)/Pr(B), assuming Pr(B)>0
Random variablesRandom variables– Intuitively, Variables whose values depend on the Intuitively, Variables whose values depend on the
outcomes of some experimentoutcomes of some experiment– Formally, a function X that maps outcomes from some Formally, a function X that maps outcomes from some
sample space S to real numberssample space S to real numbers– Indicator random variable: a variable that maps outcomes Indicator random variable: a variable that maps outcomes
to either 0 or 1to either 0 or 1
Expectation:a random variable has random valuesExpectation:a random variable has random values– Intuitively, average value of a random variableIntuitively, average value of a random variable– Formally, expected value of a random variable X is Formally, expected value of a random variable X is
defined as E(X)=defined as E(X)=xxxPr(X=x)xPr(X=x)– Properties:Properties:
Linearity: E(X+Y) = E(X) + E(Y)Linearity: E(X+Y) = E(X) + E(Y)Independence: If X and Y are independent, that is, Independence: If X and Y are independent, that is, Pr(X=x|Y=y)=Pr(X=x), then E(XY)=E(X)E(Y)Pr(X=x|Y=y)=Pr(X=x), then E(XY)=E(X)E(Y)
Basic Probability Basic Probability
2626
Computing Prefix AveragesComputing Prefix AveragesWe further illustrate We further illustrate asymptotic analysis with asymptotic analysis with two algorithms for prefix two algorithms for prefix averagesaveragesThe The ii-th prefix average of -th prefix average of an array an array XX is average of is average of the first the first ((ii 1) 1) elements of elements of XX::AA[[ii]] XX[0] [0] XX[1] [1] … …
XX[[ii])/(])/(ii+1)+1)
Computing the array Computing the array AA of of prefix averages of prefix averages of another array another array XX has has applications to financial applications to financial analysisanalysis
0
5
10
15
20
25
30
35
1 2 3 4 5 6 7
X
A
2727
Prefix Averages (Quadratic)Prefix Averages (Quadratic)The following algorithm computes prefix The following algorithm computes prefix averages in quadratic time by applying the averages in quadratic time by applying the definitiondefinition
AlgorithmAlgorithm prefixAverages1prefixAverages1((X, nX, n))InputInput array array XX of of nn integers integersOutputOutput array array AA of prefix averages of of prefix averages of XX #operations#operations
AA new array of new array of nn integers integers nnforfor ii 00 toto nn 1 1 dodo nn
ss XX[0] [0] nnforfor jj 11 toto ii dodo 1 1 2 2 …… ( (nn 1) 1)
ss ss XX[[jj]] 1 1 2 2 …… ( (nn 1) 1)AA[[ii]] ss ( (ii 1) 1) nn
returnreturn A A 11
2828
Arithmetic ProgressionArithmetic ProgressionThe running time of The running time of prefixAverages1 prefixAverages1 isisOO(1 (1 2 2 ……nn))
The sum of the first The sum of the first nn integers is integers is nn((nn 1) 1) 22– There is a simple visual There is a simple visual
proof of this factproof of this fact
Thus, algorithm Thus, algorithm prefixAverages1 prefixAverages1 runs in runs in OO((nn22) ) time time 0
1
2
3
4
5
6
7
1 2 3 4 5 6
2929
Prefix Averages (Linear)Prefix Averages (Linear)The following algorithm computes prefix The following algorithm computes prefix averages in linear time by keeping a running averages in linear time by keeping a running sumsum
AlgorithmAlgorithm prefixAverages2prefixAverages2((X, nX, n))InputInput array array XX of of nn integers integersOutputOutput array array AA of prefix averages of of prefix averages of XX #operations#operations
AA new array of new array of nn integers integers nnss 0 0 11forfor ii 00 toto nn 1 1 dodo nn
ss ss XX[[ii]] nnAA[[ii]] ss ( (ii 1) 1) nn
returnreturn A A 11Algorithm Algorithm prefixAverages2 prefixAverages2 runs in runs in OO((nn) ) time time
3030
AmortizationAmortizationAmortizationAmortization– Typical data structure supports a wide variety of Typical data structure supports a wide variety of
operations for accessing and updating the elementsoperations for accessing and updating the elements– Each operation takes a varying amount of running timeEach operation takes a varying amount of running time– Rather than focusing on each operationRather than focusing on each operation– Consider the interactions between all the operations by Consider the interactions between all the operations by
studying the running time of a series of these operationsstudying the running time of a series of these operations
– Average the operations’ running timeAverage the operations’ running time Amortized running timeAmortized running time– The amortized running time of an operation within a series The amortized running time of an operation within a series
of operations is defined as the worst-case running time of of operations is defined as the worst-case running time of the series of operations divided by the number of the series of operations divided by the number of operationsoperations
– Some operations may have much higher actual running Some operations may have much higher actual running time than its amortized running time, while some others time than its amortized running time, while some others have much lowerhave much lower
3131
The Clearable Table Data StructureThe Clearable Table Data Structure The clearable table The clearable table – An ADT An ADT
Storing a table of elements Storing a table of elements
Being accessing by their index in the tableBeing accessing by their index in the table
– Two methods:Two methods:add(e)add(e) -- add an element -- add an element ee to the next available cell to the next available cell in the tablein the table
clear()clear() -- empty the table by removing all elements -- empty the table by removing all elements
Consider a series of operations (Consider a series of operations (addadd and and clearclear) performed on a clearable table ) performed on a clearable table SS– Each Each addadd takes O(1) takes O(1)– Each Each clearclear takes O(n) takes O(n)– Thus, a series of operations takes O(nThus, a series of operations takes O(n22), ),
because it may consist of only because it may consist of only clearclearss
3232
Amortization AnalysisAmortization AnalysisTheorem: Theorem: – A series of n operations on an initially empty clearable A series of n operations on an initially empty clearable
table implemented with an array takes O(n) timetable implemented with an array takes O(n) time
Proof:Proof:– Let MLet M00, M, M11, …, M, …, Mn-1n-1 be the series of operations performed be the series of operations performed
on S, where k operations are on S, where k operations are clearclear
– Let MLet Mi0i0, M, Mi1i1, …, M, …, Miik-1k-1 be the k be the k clearclear operations within the operations within the
series, and others be the (n-k) series, and others be the (n-k) addadd operations operations – Define iDefine i-1-1=-1,M=-1,Mijij takes i takes ijj-i-ij-1j-1, because at most i, because at most ijj-i-ij-1j-1-1 -1
elements are added by add operations between Melements are added by add operations between M iij-1j-1 and M and Miijj
– The total time of the series is: The total time of the series is:
Accounting MethodAccounting MethodThe methodThe method– Use a scheme of credits and debits: each Use a scheme of credits and debits: each
operation pays an amount of operation pays an amount of cyber-dollarcyber-dollar– Some operations overpay --> creditsSome operations overpay --> credits– Some operations underpay --> debitsSome operations underpay --> debits– Keep the balance at any time at least 0Keep the balance at any time at least 0Example: the clearable tableExample: the clearable table– Each operation pays two cyber-dollarsEach operation pays two cyber-dollars– addadd always overpays one dollar -- one credit always overpays one dollar -- one credit– clearclear may underpay a variety of dollars may underpay a variety of dollars
the underpaid amount equals the number of the underpaid amount equals the number of addadd operations since last operations since last clear clear - 2 - 2
– Thus, the balance is at least 0Thus, the balance is at least 0– So the total cost is 2n -- may have creditsSo the total cost is 2n -- may have credits
3434
Potential FunctionsPotential FunctionsBased on energy modelBased on energy model– associate a value with the structure, Ø, associate a value with the structure, Ø,
representing the current energy staterepresenting the current energy state– each operation contributes to Ø a certain amount each operation contributes to Ø a certain amount
of energy t’ and consumes a varying amount of of energy t’ and consumes a varying amount of energy tenergy t
– ØØ00 -- the initial energy, Ø -- the initial energy, Øii -- the energy after the -- the energy after the i-th operationi-th operation
– for the i-th operationfor the i-th operationttii -- the actual running time -- the actual running time
t’t’ii -- the amortized running time -- the amortized running time
ttii = t’ = t’ii + Ø + Øi-1i-1 - Ø - Øii – Overall: T=∑(tOverall: T=∑(tii), T’=∑(t’), T’=∑(t’ii))– The total actual time T = T’ + ØThe total actual time T = T’ + Ø00 - Ø - Ønn – As long as ØAs long as Ø00 <= Ø <= Ønn, T<=T’ , T<=T’
3535
Potential FunctionsPotential FunctionsExample -- The clearable tableExample -- The clearable table– the current energy state Ø is defined as the the current energy state Ø is defined as the
number of elements in the table, thus Ø>=0number of elements in the table, thus Ø>=0– each operation contributes to Ø t’=2each operation contributes to Ø t’=2
– ØØ00 = 0 = 0
– ØØii = Ø = Øi-1i-1 + 1, if the i-th operation is + 1, if the i-th operation is addaddadd: t = t’+ Øadd: t = t’+ Øi-1i-1 - Ø - Øii = 2 - 1 = 1 = 2 - 1 = 1
– ØØii = 0, if the i-th operation is = 0, if the i-th operation is clearclearclear: t =t’+ Øclear: t =t’+ Øi-1i-1 = 2 + Ø = 2 + Øi-1i-1
– The total actual time T = T’ + ØThe total actual time T = T’ + Ø00 - Ø - Ønn – Because ØBecause Ø00 = 0 <= Ø = 0 <= Ønn, T<=T’ =2n, T<=T’ =2n
3636
Extendable ArrayExtendable ArrayADT: ADT: Extendable array is an array with extendable Extendable array is an array with extendable size. One of its methods is size. One of its methods is addadd to add an element to to add an element to the array. If the array is not full, the element is the array. If the array is not full, the element is added to the first available cell. When the array is added to the first available cell. When the array is full, the full, the addadd method performs the following method performs the following– Allocate a new array with double sizeAllocate a new array with double size– Copy elements from the old array to the new arrayCopy elements from the old array to the new array– Add the element to the first available cell in the new arrayAdd the element to the first available cell in the new array– Replace the old array with the new arrayReplace the old array with the new array
Question: Question: What is the amortization time of What is the amortization time of addadd??– Two situations of Two situations of addadd: :
add -- O(1)add -- O(1)Extend and add -- (n)Extend and add -- (n)
– Amortization analysis:Amortization analysis:Each Each add add deposits 3 dollars and spends 1 dollardeposits 3 dollars and spends 1 dollarEach extension spends k dollars from the size k to 2k Each extension spends k dollars from the size k to 2k (copy)(copy)Amortization time of Amortization time of addadd is O(1) is O(1)
3737
Relatives of Big-OhRelatives of Big-Ohbig-Omegabig-Omega– f(n) is f(n) is (g(n)) if there is a constant c > 0 (g(n)) if there is a constant c > 0
and an integer constant nand an integer constant n00 1 such that 1 such that
f(n) f(n) c c••g(n) for n g(n) for n n n00
big-Thetabig-Theta– f(n) is f(n) is (g(n)) if there are constants c’ > 0 and c’’ > 0 and (g(n)) if there are constants c’ > 0 and c’’ > 0 and
an integer constant nan integer constant n00 1 such that c’ 1 such that c’••g(n) g(n) f(n) f(n) c’’c’’••g(n) for n g(n) for n n n00
little-ohlittle-oh– f(n) is o(g(n)) if, for any constant c > 0, there is an integer f(n) is o(g(n)) if, for any constant c > 0, there is an integer
constant nconstant n00 0 such that f(n) 0 such that f(n) c c••g(n) for n g(n) for n n n00
little-omegalittle-omega– f(n) is f(n) is (g(n)) if, for any constant c > 0, there is an integer (g(n)) if, for any constant c > 0, there is an integer
constant nconstant n00 0 such that f(n) 0 such that f(n) c c••g(n) for n g(n) for n n n00
3838
Intuition for Asymptotic NotationIntuition for Asymptotic NotationBig-OhBig-Oh– f(n) is O(g(n)) if f(n) is asymptotically f(n) is O(g(n)) if f(n) is asymptotically less than or less than or
equalequal to g(n) to g(n)
big-Omegabig-Omega– f(n) is f(n) is (g(n)) if f(n) is asymptotically (g(n)) if f(n) is asymptotically greater than greater than
or equalor equal to g(n) to g(n)big-Thetabig-Theta– f(n) is f(n) is (g(n)) if f(n) is asymptotically (g(n)) if f(n) is asymptotically equalequal to g(n) to g(n)
little-ohlittle-oh– f(n) is o(g(n)) if f(n) is asymptotically f(n) is o(g(n)) if f(n) is asymptotically strictly lessstrictly less
than g(n)than g(n)little-omegalittle-omega– f(n) is f(n) is (g(n)) if is asymptotically (g(n)) if is asymptotically strictly greaterstrictly greater
than g(n)than g(n)
3939
Example Uses of the Relatives of Big-OhExample Uses of the Relatives of Big-Oh 5n2 is (n2)
f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0 1 such that f(n) c•g(n) for n n0 let c = 5 and n0 = 1
5n2 is (n)f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0 1 such that f(n) c•g(n) for n n0 let c = 1 and n0 = 1
5n2 is (n)f(n) is (g(n)) if, for any constant c > 0, there is an integer constant n0 0 such that f(n) c•g(n) for n n0 need 5n0
2 c•n0 given c, the n0 that satisfies this is n0 c/5 0