61A Lecture 13
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
61A Lecture 13
Announcements
Measuring Efficiency
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(3)
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)fib(3)
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Recursive Computation of the Fibonacci Sequence
Our first example of tree recursion:
4
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
(Demo)
http://en.wikipedia.org/wiki/File:Fibonacci.jpg
def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)
Memoization
Memoization
Idea: Remember the results that have been computed before
6
Memoization
Idea: Remember the results that have been computed before
def memo(f):
6
Memoization
Idea: Remember the results that have been computed before
def memo(f):
cache = {}
6
Memoization
Idea: Remember the results that have been computed before
def memo(f):
cache = {} def memoized(n):
6
Memoization
Idea: Remember the results that have been computed before
def memo(f):
cache = {} def memoized(n):
if n not in cache:
6
Memoization
Idea: Remember the results that have been computed before
def memo(f):
cache = {} def memoized(n):
if n not in cache: cache[n] = f(n)
6
Memoization
Idea: Remember the results that have been computed before
def memo(f):
cache = {} def memoized(n):
if n not in cache: cache[n] = f(n)
return cache[n]
6
Memoization
Idea: Remember the results that have been computed before
def memo(f):
cache = {} def memoized(n):
if n not in cache: cache[n] = f(n)
return cache[n] return memoized
6
Memoization
Idea: Remember the results that have been computed before
def memo(f):
cache = {} def memoized(n):
if n not in cache: cache[n] = f(n)
return cache[n] return memoized
Keys are arguments that map to return values
6
Memoization
Idea: Remember the results that have been computed before
def memo(f):
cache = {} def memoized(n):
if n not in cache: cache[n] = f(n)
return cache[n] return memoized
Keys are arguments that map to return values
Same behavior as f, if f is a pure function
6
Memoization
Idea: Remember the results that have been computed before
def memo(f):
cache = {} def memoized(n):
if n not in cache: cache[n] = f(n)
return cache[n] return memoized
Keys are arguments that map to return values
Same behavior as f, if f is a pure function
6
(Demo)
Memoized Tree Recursion
7
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Memoized Tree Recursion
7
Call to fibfib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Memoized Tree Recursion
7
Call to fib
Found in cachefib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Skipped
Space
The Consumption of Space
9
The Consumption of Space
Which environment frames do we need to keep during evaluation?
9
The Consumption of Space
Which environment frames do we need to keep during evaluation?
At any moment there is a set of active environments
9
The Consumption of Space
Which environment frames do we need to keep during evaluation?
At any moment there is a set of active environments
Values and frames in active environments consume memory
9
The Consumption of Space
Which environment frames do we need to keep during evaluation?
At any moment there is a set of active environments
Values and frames in active environments consume memory
Memory that is used for other values and frames can be recycled
9
The Consumption of Space
Which environment frames do we need to keep during evaluation?
At any moment there is a set of active environments
Values and frames in active environments consume memory
Memory that is used for other values and frames can be recycled
9
Active environments:
The Consumption of Space
Which environment frames do we need to keep during evaluation?
At any moment there is a set of active environments
Values and frames in active environments consume memory
Memory that is used for other values and frames can be recycled
9
Active environments:
• Environments for any function calls currently being evaluated
The Consumption of Space
Which environment frames do we need to keep during evaluation?
At any moment there is a set of active environments
Values and frames in active environments consume memory
Memory that is used for other values and frames can be recycled
9
Active environments:
• Environments for any function calls currently being evaluated
• Parent environments of functions named in active environments
The Consumption of Space
Which environment frames do we need to keep during evaluation?
At any moment there is a set of active environments
Values and frames in active environments consume memory
Memory that is used for other values and frames can be recycled
9
Active environments:
• Environments for any function calls currently being evaluated
• Parent environments of functions named in active environments
(Demo)
Interactive Diagram
Fibonacci Space Consumption
10
Fibonacci Space Consumption
10
fib(5)
Fibonacci Space Consumption
10
fib(5)
fib(3)
Fibonacci Space Consumption
10
fib(5)
fib(4)fib(3)
Fibonacci Space Consumption
10
fib(5)
fib(4)fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Fibonacci Space Consumption
10
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Fibonacci Space Consumption
10
Assume we have reached this step
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Fibonacci Space Consumption
11
Assume we have reached this step
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Fibonacci Space Consumption
11
Assume we have reached this step
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Has an active environment
Fibonacci Space Consumption
11
Assume we have reached this step
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Has an active environmentCan be reclaimed
Fibonacci Space Consumption
11
Assume we have reached this step
fib(5)
fib(4)
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
fib(2)
fib(0) fib(1)
0 1
fib(3)
fib(1)
1
fib(2)
fib(0) fib(1)
0 1
Has an active environmentCan be reclaimedHasn't yet been created
Time
Comparing Implementations
13
Comparing Implementations
Implementations of the same functional abstraction can require different resources
13
Comparing Implementations
Implementations of the same functional abstraction can require different resources
13
Problem: How many factors does a positive integer n have?
Comparing Implementations
Implementations of the same functional abstraction can require different resources
13
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Comparing Implementations
Implementations of the same functional abstraction can require different resources
13
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
def factors(n):
Comparing Implementations
Implementations of the same functional abstraction can require different resources
13
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
def factors(n):
Comparing Implementations
Implementations of the same functional abstraction can require different resources
13
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
Fast: Test each k from 1 to square root n For every k, n/k is also a factor!
def factors(n):
Comparing Implementations
Implementations of the same functional abstraction can require different resources
13
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
Fast: Test each k from 1 to square root n For every k, n/k is also a factor!
def factors(n):
Question: How many time does each implementation use division? (Demo)
Comparing Implementations
Implementations of the same functional abstraction can require different resources
13
Time (number of divisions)
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
Fast: Test each k from 1 to square root n For every k, n/k is also a factor!
def factors(n):
Question: How many time does each implementation use division? (Demo)
n
Comparing Implementations
Implementations of the same functional abstraction can require different resources
13
Time (number of divisions)
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
Fast: Test each k from 1 to square root n For every k, n/k is also a factor!
def factors(n):
Question: How many time does each implementation use division? (Demo)
n
Comparing Implementations
Implementations of the same functional abstraction can require different resources
13
Time (number of divisions)
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
Fast: Test each k from 1 to square root n For every k, n/k is also a factor!
def factors(n):
Greatest integer less thanpn
Question: How many time does each implementation use division? (Demo)
Orders of Growth
Order of Growth
15
Order of Growth
A method for bounding the resources used by a function by the "size" of a problem
15
Order of Growth
A method for bounding the resources used by a function by the "size" of a problem
15
n: size of the problem
Order of Growth
A method for bounding the resources used by a function by the "size" of a problem
15
n: size of the problem
R(n): measurement of some resource used (time or space)
R(n) = �(f(n))
Order of Growth
A method for bounding the resources used by a function by the "size" of a problem
15
n: size of the problem
R(n): measurement of some resource used (time or space)
R(n) = �(f(n))
Order of Growth
A method for bounding the resources used by a function by the "size" of a problem
15
n: size of the problem
R(n): measurement of some resource used (time or space)
means that there are positive constants k1 and k2 such that
R(n) = �(f(n))
k1 · f(n) � R(n) � k2 · f(n)
Order of Growth
A method for bounding the resources used by a function by the "size" of a problem
15
n: size of the problem
R(n): measurement of some resource used (time or space)
means that there are positive constants k1 and k2 such that
R(n) = �(f(n))
k1 · f(n) � R(n) � k2 · f(n)
Order of Growth
A method for bounding the resources used by a function by the "size" of a problem
15
n: size of the problem
R(n): measurement of some resource used (time or space)
means that there are positive constants k1 and k2 such that
for all n larger than some minimum m
R(n) = �(f(n))
k1 · f(n) � R(n) � k2 · f(n)
Order of Growth
A method for bounding the resources used by a function by the "size" of a problem
15
n: size of the problem
R(n): measurement of some resource used (time or space)
means that there are positive constants k1 and k2 such that
for all n larger than some minimum m
R(n) = �(f(n))
k1 · f(n) � R(n) � k2 · f(n)
Order of Growth
A method for bounding the resources used by a function by the "size" of a problem
15
n: size of the problem
R(n): measurement of some resource used (time or space)
means that there are positive constants k1 and k2 such that
for all n larger than some minimum m
Order of Growth of Counting Factors
Implementations of the same functional abstraction can require different amounts of time
16
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
Fast: Test each k from 1 to square root n For every k, n/k is also a factor!
def factors(n):
Order of Growth of Counting Factors
Implementations of the same functional abstraction can require different amounts of time
16
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
Fast: Test each k from 1 to square root n For every k, n/k is also a factor!
def factors(n): Time Space
Order of Growth of Counting Factors
Implementations of the same functional abstraction can require different amounts of time
16
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
Fast: Test each k from 1 to square root n For every k, n/k is also a factor!
def factors(n): Time Space
�(n) �(1)
Order of Growth of Counting Factors
Implementations of the same functional abstraction can require different amounts of time
16
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
Fast: Test each k from 1 to square root n For every k, n/k is also a factor!
def factors(n): Time Space
�(n) �(1)
⇥(pn) �(1)
Order of Growth of Counting Factors
Implementations of the same functional abstraction can require different amounts of time
16
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
Fast: Test each k from 1 to square root n For every k, n/k is also a factor!
def factors(n): Time Space
�(n) �(1)
⇥(pn) �(1)
Assumption: integers occupy a fixed amount of
space
Order of Growth of Counting Factors
Implementations of the same functional abstraction can require different amounts of time
16
Problem: How many factors does a positive integer n have?
A factor k of n is a positive integer that evenly divides n
Slow: Test each k from 1 through n
Fast: Test each k from 1 to square root n For every k, n/k is also a factor!
def factors(n): Time Space
�(n) �(1)
⇥(pn) �(1)
Assumption: integers occupy a fixed amount of
space
(Demo)
Exponentiation
Exponentiation
18
Exponentiation
Goal: one more multiplication lets us double the problem size
18
Exponentiation
Goal: one more multiplication lets us double the problem size
18
def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)
bn =
�1 if n = 0
b · bn�1 otherwise
Exponentiation
Goal: one more multiplication lets us double the problem size
18
def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)
bn =
�1 if n = 0
b · bn�1 otherwise
bn =
���
��
1 if n = 0
(b12 n)2 if n is even
b · bn�1 if n is odd
Exponentiation
Goal: one more multiplication lets us double the problem size
18
def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)
bn =
�1 if n = 0
b · bn�1 otherwise
bn =
���
��
1 if n = 0
(b12 n)2 if n is even
b · bn�1 if n is odd
Exponentiation
Goal: one more multiplication lets us double the problem size
18
def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)
def square(x): return x*x
def exp_fast(b, n): if n == 0: return 1 elif n % 2 == 0: return square(exp_fast(b, n//2)) else: return b * exp_fast(b, n-1)
bn =
�1 if n = 0
b · bn�1 otherwise
bn =
���
��
1 if n = 0
(b12 n)2 if n is even
b · bn�1 if n is odd
Exponentiation
Goal: one more multiplication lets us double the problem size
18
def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)
def square(x): return x*x
def exp_fast(b, n): if n == 0: return 1 elif n % 2 == 0: return square(exp_fast(b, n//2)) else: return b * exp_fast(b, n-1)
(Demo)
Exponentiation
19
Time Space
Goal: one more multiplication lets us double the problem size
def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)
def square(x): return x*x
def exp_fast(b, n): if n == 0: return 1 elif n % 2 == 0: return square(exp_fast(b, n//2)) else: return b * exp_fast(b, n-1)
Exponentiation
19
Time Space
�(n) �(n)
Goal: one more multiplication lets us double the problem size
def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)
def square(x): return x*x
def exp_fast(b, n): if n == 0: return 1 elif n % 2 == 0: return square(exp_fast(b, n//2)) else: return b * exp_fast(b, n-1)
Exponentiation
19
Time Space
�(n) �(n)
�(log n) �(log n)
Goal: one more multiplication lets us double the problem size
def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)
def square(x): return x*x
def exp_fast(b, n): if n == 0: return 1 elif n % 2 == 0: return square(exp_fast(b, n//2)) else: return b * exp_fast(b, n-1)
Comparing Orders of Growth
Properties of Orders of Growth
21
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
⇥(n)
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
⇥(n) ⇥(500 · n)
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
⇥(n) ⇥(500 · n) ⇥(1
500· n)
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
⇥(n) ⇥(500 · n) ⇥(1
500· n)
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n)
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n) ⇥(log10 n)
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n) ⇥(log10 n) ⇥(lnn)
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n) ⇥(log10 n) ⇥(lnn)
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n) ⇥(log10 n) ⇥(lnn)
def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n) ⇥(log10 n) ⇥(lnn)
def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count
Outer: length of a
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n) ⇥(log10 n) ⇥(lnn)
def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count
Outer: length of a
Inner: length of b
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n) ⇥(log10 n) ⇥(lnn)
def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count
Outer: length of a
Inner: length of b
If a and b are both length n, then overlap takes steps⇥(n2)
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n) ⇥(log10 n) ⇥(lnn)
def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count
Outer: length of a
Inner: length of b
If a and b are both length n, then overlap takes steps⇥(n2)
Lower-order terms: The fastest-growing part of the computation dominates the total
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n) ⇥(log10 n) ⇥(lnn)
def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count
Outer: length of a
Inner: length of b
If a and b are both length n, then overlap takes steps⇥(n2)
Lower-order terms: The fastest-growing part of the computation dominates the total
⇥(n2)
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n) ⇥(log10 n) ⇥(lnn)
def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count
Outer: length of a
Inner: length of b
If a and b are both length n, then overlap takes steps⇥(n2)
Lower-order terms: The fastest-growing part of the computation dominates the total
⇥(n2 + n)⇥(n2)
Properties of Orders of Growth
Constants: Constant terms do not affect the order of growth of a process
21
Logarithms: The base of a logarithm does not affect the order of growth of a process
Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps
⇥(n) ⇥(500 · n) ⇥(1
500· n)
⇥(log2 n) ⇥(log10 n) ⇥(lnn)
def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count
Outer: length of a
Inner: length of b
If a and b are both length n, then overlap takes steps⇥(n2)
Lower-order terms: The fastest-growing part of the computation dominates the total
⇥(n2 + n)⇥(n2) ⇥(n2+ 500 · n+ log2 n+ 1000)
Comparing orders of growth (n is the problem size)
22
Comparing orders of growth (n is the problem size)
22
�(bn)
Comparing orders of growth (n is the problem size)
22
�(bn) Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Comparing orders of growth (n is the problem size)
22
�(bn) Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Comparing orders of growth (n is the problem size)
22
�(bn)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Comparing orders of growth (n is the problem size)
22
�(bn)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Quadratic growth. E.g., overlap
Comparing orders of growth (n is the problem size)
22
�(bn)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
Comparing orders of growth (n is the problem size)
22
�(bn)
�(n)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
Comparing orders of growth (n is the problem size)
22
�(bn)
�(n)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Linear growth. E.g., slow factors or exp
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
Comparing orders of growth (n is the problem size)
22
�(bn)
�(n)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Linear growth. E.g., slow factors or exp
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
⇥(pn)
Comparing orders of growth (n is the problem size)
22
�(bn)
�(n)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Linear growth. E.g., slow factors or exp
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
⇥(pn) Square root growth. E.g., factors_fast
Comparing orders of growth (n is the problem size)
22
�(bn)
�(n)
�(log n)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Linear growth. E.g., slow factors or exp
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
⇥(pn) Square root growth. E.g., factors_fast
Comparing orders of growth (n is the problem size)
22
�(bn)
�(n)
�(log n)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Linear growth. E.g., slow factors or exp
Logarithmic growth. E.g., exp_fast
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
⇥(pn) Square root growth. E.g., factors_fast
Comparing orders of growth (n is the problem size)
22
�(bn)
�(n)
�(log n)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Linear growth. E.g., slow factors or exp
Logarithmic growth. E.g., exp_fastDoubling the problem only increments R(n).
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
⇥(pn) Square root growth. E.g., factors_fast
Comparing orders of growth (n is the problem size)
22
�(bn)
�(n)
�(log n)
�(1)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Linear growth. E.g., slow factors or exp
Logarithmic growth. E.g., exp_fastDoubling the problem only increments R(n).
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
⇥(pn) Square root growth. E.g., factors_fast
Comparing orders of growth (n is the problem size)
22
�(bn)
�(n)
�(log n)
�(1)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Linear growth. E.g., slow factors or exp
Logarithmic growth. E.g., exp_fastDoubling the problem only increments R(n).
Constant. The problem size doesn't matter
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
⇥(pn) Square root growth. E.g., factors_fast
Comparing orders of growth (n is the problem size)
22
�(bn)
�(n)
�(log n)
�(1)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Linear growth. E.g., slow factors or exp
Logarithmic growth. E.g., exp_fastDoubling the problem only increments R(n).
Constant. The problem size doesn't matter
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
⇥(pn) Square root growth. E.g., factors_fast
Comparing orders of growth (n is the problem size)
22
�(bn)
�(n)
�(log n)
�(1)
⇥(n2)
Exponential growth. Recursive fib takes
�(�n) � =1 +
�5
2� 1.61828steps, where
Incrementing the problem scales R(n) by a factor
Linear growth. E.g., slow factors or exp
Logarithmic growth. E.g., exp_fastDoubling the problem only increments R(n).
Constant. The problem size doesn't matter
Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n
⇥(pn) Square root growth. E.g., factors_fast
Related Documents
