Sid Chi-Kin Chau COMP2100/6442 Software Design Methodologies / Software Construction Introduction to Computational Complexity
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Sid Chi-Kin Chau
COMP2100/6442
Software Design Methodologies / Software Construction
Introduction to
Computational Complexity
Last time
• Karatsuba Integer Multiplication
▪ By divide-and-conquer
• How did we measure the speed of an
algorithm?
▪ Count the number of operations
▪ How the number scales w.r.t. the input size
▪ Time complexity of computation
▪ Karatsuba multiplication scales as n1.6
2
Goals
• How to formally measure the runtime of an
algorithm?
▪ Big-O notations
• Runtime with recursive algorithms
▪ The Master theorem
3
Big-O notation
• What do we mean when we measure runtime?
▪ How long does it take to solve the problem, in
seconds or minutes or hours?
• This is heavily dependent on the programming
language, system architecture, etc.
• But the most factor is the input size (e.g. the
number of bits that used in encoding input)
• We want a way to talk about the running time of an
algorithm, w.r.t. the input size
4
Main idea:
Focus on how the runtime scales with n (the input size).
5
Asymptotic AnalysisHow does the running time scale as n gets large?
• Abstracts away from
hardware- and language-
specific issues
• Makes algorithm analysis
much more tractable
• Only makes sense if n
is large (compared to
the constant factors)
Pros: Cons:
One algorithm is “faster” than another if its runtime
scales better with the size of the input
2100000000000000 n
is “better” than n2 ?!?!
6
O(…) means an upper bound
• Let T(n), g(n) be functions of positive integers
– Think of T(n) as being a runtime: positive and increasing in n
• We say “T(n) is O(g(n))” if g(n) grows at least as fast as T(n) as n gets large
• Formally,
𝑇 𝑛 = 𝑂 𝑔 𝑛
⟺
∃𝑐, 𝑛0 > 0 𝑠. 𝑡. ∀𝑛 ≥ 𝑛0,
0 ≤ 𝑇 𝑛 ≤ 𝑐 ⋅ 𝑔(𝑛)
7
Formally:• Choose c = 3
• Choose n0 = 4
• Then:
∀𝑛 ≥ 4,
0 ≤ 2𝑛2 + 10 ≤ 3 ⋅ 𝑛2
3n2
n2
Example 2𝑛2 + 10 = 𝑂 𝑛2𝑇 𝑛 = 𝑂 𝑔 𝑛
⟺∃𝑐, 𝑛0 > 0 𝑠. 𝑡. ∀𝑛 ≥ 𝑛0,
0 ≤ 𝑇 𝑛 ≤ 𝑐 ⋅ 𝑔(𝑛)
8
Example 2𝑛2 + 10 = 𝑂 𝑛2
Formally:• Choose c = 7
• Choose n0 = 2
• Then:
∀𝑛 ≥ 2,
0 ≤ 2𝑛2 + 10 ≤ 7 ⋅ 𝑛2
7n2
n2
𝑇 𝑛 = 𝑂 𝑔 𝑛
⟺∃𝑐, 𝑛0 > 0 𝑠. 𝑡. ∀𝑛 ≥ 𝑛0,
0 ≤ 𝑇 𝑛 ≤ 𝑐 ⋅ 𝑔(𝑛)
9
Another example: 𝑛 = 𝑂(𝑛2)
• Choose c = 1
• Choose n0 = 1
• Then
∀𝑛 ≥ 1,
0 ≤ 𝑛 ≤ 𝑛2
g(n) = n2
T(n) = n
10
Lower bound and tight bound
• Lower bound
– 𝑇 𝑛 = Ω 𝑔 𝑛
• Tight bound
– 𝑇 𝑛 = Θ 𝑔 𝑛
11
Example 1
• What is the running time of the following procedure?
• public void method(int n) {
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
for (int k = 0; k < n; k++) {
for (int l = 0; l < n; l++) {
c();
}
}
}
}
12
Requires constant
number of operations
Example 1
• What is the running time of the following procedure?
• public void method(int n) {
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
for (int k = 0; k < n; k++) {
for (int l = 0; l < n; l++) {
c();
}
}
}
}
13
T(n) = O(n4)
Example 2
• What is the running time of the following procedure?
• public void method(int n) {
h=1;
while (h <= n)
{
c();
h = 2*h;
}
14
Example 2
• What is the running time of the following procedure?
• public void method(int n) {
h=1;
while (h <= n)
{
c();
h = 2*h;
}
15
h = 1, 2, 4, … 2log n
T(n) = O(log n)
Example 3
• What is the running time of the following procedure?
• public void method(int n) {
for (int j = 0; j < n; j++) {
for (int i = 0; i < j; i++) {
c();
}
}
16
Example 3
• What is the running time of the following procedure?
• public void method(int n) {
for (int j = 0; j < n; j++) {
for (int i = 0; i < j; i++) {
c();
}
}
17
Each inner for-loop (i) gets j times
T(n) = 1+2+…+n = O(n2)
What have we learned?
• Asymptotic notation is useful because
• It has a precise mathematical definition
• while we don’t have to pay close attention to all
those pesky constant factors
• But we should always be careful not to abuse it
• 𝑛 ≥ 𝑛0 = 210000000
18
The Master theorem
• The master theorem applies to recurrence form:
T(n) = aT(n/b) + f(n),
where a 1, b > 1, and f is asymptotically positive
• Common results:
▪ T(n) = T(n-1) + O(n) T(n) = O(n2)
▪ T(n) = T(n/2) + O(1) T(n) = O(log n)
▪ T(n) = 2T(n/2) + O(1) T(n) = O(n)
▪ T(n) = 2T(n/2) + O(n) T(n) = O(n log n)19
20
• Let T(n) be the runtime of method()
• T(n) = {, if n > 0, otherwise
and is O(__________)
public void method(int n) {
c();
if (n > 0) method(n-1);
}
Example 4
21
• Let T(n) be the runtime of method()
• T(n) = {1 + T n − 1 , if n > 0
1, otherwiseand is O(n)
public void method(int n) {
c();
if (n > 0) method(n-1);
}
Example 4
22
• Let T(n) be the runtime of method()
• T(n) = {, if n > 0, otherwise
and is O(__________)
public void method(int n) {
c();
if (n > 0) method(n/2);
}
Example 5
23
• Let T(n) be the runtime of method()
• T(n) = {1 + T 𝑛/2 , if n > 0
1, otherwiseand is O(log n)
public void method(int n) {
c();
if (n > 0) method(n/2);
}
Example 5
Recap
• Learnt Big-O notation
• Used the Master Theorem for recursive
procedures
• Post your questions on wattle
Related Documents
