O-Notation April 23, 2003 Prepared by Doug Hogan CSE 260
Dec 14, 2015
O-Notation
April 23, 2003Prepared by Doug Hogan
CSE 260
O-notation: The Idea
Big-O notation is a way of ranking about how much time it takes for an algorithm to executeHow many operations will be done when the program is executed? Find a bound on the running time, i.e. functions that are on the same order. We care about what happens for large
amounts of data asymptotic order.
O-notation: The Idea
Use mathematical tools to find asymptotic order. Real functions to approximate integer
functions. Depends on some variable, like n or
X, which is usually the size of an array or how much data is going to be processed
O-notationThe complicated math behind it all…
Given f and g, real functions of variable x…First form: g provides an upper bound for f ≡
graph of f lies closer to the x axis than g
More general form: g provides an upper bound for f ≡
graph of f lies closer to the x axis than some positive multiple (M) of g after some minimum value of x (x0).
O-notation: A graphical view
So what does “closer to the x-axis” MEAN?
-M ∙ g(x) ≤ f(x) ≤ M ∙ g(x)
But that’s absolute value…
|f(x)| ≤ M ∙ |g(x)|
Another graphical view
x0
g
M ∙g
f
x
y
Before x0, nothing claimed about f’s growth
After x0, |f(x)| ≤ M ∙ |g(x)|
Formal Definition
Let f and g be real-valued functions defined on the same set of reals. f is of order g, written f(x) = O(g(x)), iff there exists a positive real number M (multiple) a real number x0 (starting point)
such that for all x in the domain of f and g, |f(x)| ≤ M ∙ |g(x)| when x > x0
Example
Use the definition of O-notation to express|17x6 – 3x3 + 2x + 8| ≤ 30|x6| for all x > 1
M = 30x0 = 117x6 – 3x3 + 2x + 8 is O(x6)
Graphically…17x6 – 3x3 + 2x + 8 is O(x6); M = 30; x0 = 1
30x6
x6
17x6 – 3x3 + 2x + 8
Problem
Use the definition of O-notation to express for all x > 6
M = 45x0 = 6
x
x
xx45
1
9215
xOxx
xx is 45
1
9215
Graphically…M = 45; x0 = 6 xOx
x
xx is 45
1
9215
1
9215
x
xx
x45
x
Another graphical example
f(x) = 7x3 - 2x + 3
12x3
x3
M = 12, x0 = 1 7x3 - 2x + 3 is O(x3)
Using O-notation…
Order of Power Functions:For any rational numbers r and s, if r < s,
xr is O(xs)
Order of Polynomial Functions: If a0, a1,…, an are real numbers and an ≠ 0
anxn + an-1xn-1 +… + a1x + a0 is O(xm)
for all m ≥ n
Examples
Example: Find an order for
f(x) = 7x5 + 5x3 – x + 4 (all reals x)
O(x5)
Is that the only answer?No…But it’s the “best”
Showing that a function is NOT Big-O of another…
Show that x2 is not O(x).[Arguing by contradiction.] Suppose not, that x2 is O(x). By definition of O(…), then there exista positive real number Ma real number x0
such that |x2| ≤ M ∙ |x| for all x > x0 (1)
Showing that a function is NOT Big-O of another…,
ctd.Let x be a positive real number greater than both M and x0, i.e. x>M and x>x0.Then by multiplying both sides of x>M by x, x∙x>M∙x.Since x is positive, |x2|>M∙|x|. So there is a real number x>x0 s.t. |x2|>M∙|x|. This contradicts (1) above. So, the supposition is false and thus x2 is not O(x). □
Generalization
If a0, a1,…, an are real numbers and an ≠ 0
anxn + an-1xn-1 +… + a1x + a0 is NOT
O(xm) for all m < n
Best approximation
Definition Suppose S is a set of functions from a subset of R to R and f is a R->R function.Function g is a best big-O approximation for f in S iff f(x) is O(g(x)) for any h in S, if f(x) is O(h(x)), then g(x) is
O(h(x)).
Problem
Find best big-O approx for f(x) = 5x3 – 2x + 1By thm. on polynomial orders,f(x) is O(xn) for all n ≥ 3
By previous property, f(x) is NOT O(xm) for all m < 3
So O(x3) is the best approximation.
O-Arithmetic
Let f and g be functions and k be a constant.
1. O(k*f) = O(f)2. O(f *g) = O(f) * O(g)3. O(f/g) = O(f) / O(g)4. O(f) ≥ O(g) iff f dominates* g5. O(f + g) = Max[O(f), O(g)]
(Headington 546)
DominanceLet f and g be functions and a,b,m,n be
constants. 1. xx dominates x!2. x! dominates ax
3. ax dominates bx if a > b4. ax dominates xn if n > m5. x dominates logax if a > 1
6. logax dominates logbx if b > a > 1
7. logax dominates 1 if a > 1(Headington 547)
Works Cited
Epp, Susanna. Discrete Mathematics with Applications. 2nd Ed. Belmont, CA: Brooks, 1995.
Headington, Mark A., and David Riley. Data Abstraction and Structures using C++. Lexington, MA: Heath, 1994.