1 G64ADS Advanced Data Structures Guoping Qiu Room C34, CS Building
1 G64ADS Advanced Data Structures Guoping Qiu Room C34, CS Building.
Jan 02, 2016
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
G64ADSAdvanced Data Structures
Guoping Qiu
Room C34 CS Building
2
About this course
o Study advanced data structures algorithm design analysis and implementation techniques
o To obtain advanced knowledge and practical skills in the efficient implementation of algorithms on modern computers
3
About this course
o Pre-requisites
o Good programming experience (this course does not teach programming)
o Java C++ or C
4
About this course
o Timetable
o LectureFriday 1000 - 1200 C60
o LabsTuesday 900-1100 B52
5
About this course
o Assessments
o Final Exam 50
o Continuous assessment 50
o Homework assignments o Programming projects
6
About this courseo References and learning materials
o Textbooks
1 Mark Allen Weiss Data Stuctures and Algorithm Analysis in Java 2nd Edition Addison Wesley 2007
2 Mark Allen Weiss Data Stuctures and Problem Solving using Java 3rd Edition Addison Wesley 2006
3 Robert Sedgewick Algorithms in C 3rd Edition Addison Wesley 19984 Robert Sedgewick Algorithms in C++ 3rd Edition Addison Wesley 19985 Robert Sedgewick Algorithms in Java 3rd Edition Addison Wesley 2003
o Course web page
httpwwwcsnottacuk~qiuTeachingG64ADS
Slides coursework other materials
7
Course Overview
o Advanced data structures1048708
o Trees hash tables heaps disjoint sets graphs
1048708o Algorithm development and analysis1048708
o Insert delete search sort
o Applications
o Implementation in Java C++ C
8
Advanced Data Structures
o ldquoWhy not just use a big arrayrdquo
o Example problemSearch for a number k in a set of N numbers
o SolutionStore numbers in an array of size NIterate through array until find k
Number of checksBest case 1 (k=29)Worst case N (k=25)Average case N2
29 20 65 392180 25
9
Advanced Data Structures
o Solution 2
o Store numbers in a binary search treeo Search tree until find k
o Number of checks
Best case 1 (k=29)
Worst case log2N (k=25)
Average case (log2N) 2
29
20 65
258021 39
10
Analysis
o Does it matter
N vs (log2N)
05
101520253035404550
0 10 20 30 40 50
N
Num
ber o
f Che
cks
11
Analysis
o Does it matter
Assume o N = 1000000000o 1 billion (Walmart transactions in 100 days)o 1 Ghz processor = 109 cycles per second
Solution 1 (10 cycles per check)
o Worst case 1 billion checks = 10 seconds
Solution 2 (100 cycles per check)
o Worst case 30 checks = 0000003 seconds
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
2
About this course
o Study advanced data structures algorithm design analysis and implementation techniques
o To obtain advanced knowledge and practical skills in the efficient implementation of algorithms on modern computers
3
About this course
o Pre-requisites
o Good programming experience (this course does not teach programming)
o Java C++ or C
4
About this course
o Timetable
o LectureFriday 1000 - 1200 C60
o LabsTuesday 900-1100 B52
5
About this course
o Assessments
o Final Exam 50
o Continuous assessment 50
o Homework assignments o Programming projects
6
About this courseo References and learning materials
o Textbooks
1 Mark Allen Weiss Data Stuctures and Algorithm Analysis in Java 2nd Edition Addison Wesley 2007
2 Mark Allen Weiss Data Stuctures and Problem Solving using Java 3rd Edition Addison Wesley 2006
3 Robert Sedgewick Algorithms in C 3rd Edition Addison Wesley 19984 Robert Sedgewick Algorithms in C++ 3rd Edition Addison Wesley 19985 Robert Sedgewick Algorithms in Java 3rd Edition Addison Wesley 2003
o Course web page
httpwwwcsnottacuk~qiuTeachingG64ADS
Slides coursework other materials
7
Course Overview
o Advanced data structures1048708
o Trees hash tables heaps disjoint sets graphs
1048708o Algorithm development and analysis1048708
o Insert delete search sort
o Applications
o Implementation in Java C++ C
8
Advanced Data Structures
o ldquoWhy not just use a big arrayrdquo
o Example problemSearch for a number k in a set of N numbers
o SolutionStore numbers in an array of size NIterate through array until find k
Number of checksBest case 1 (k=29)Worst case N (k=25)Average case N2
29 20 65 392180 25
9
Advanced Data Structures
o Solution 2
o Store numbers in a binary search treeo Search tree until find k
o Number of checks
Best case 1 (k=29)
Worst case log2N (k=25)
Average case (log2N) 2
29
20 65
258021 39
10
Analysis
o Does it matter
N vs (log2N)
05
101520253035404550
0 10 20 30 40 50
N
Num
ber o
f Che
cks
11
Analysis
o Does it matter
Assume o N = 1000000000o 1 billion (Walmart transactions in 100 days)o 1 Ghz processor = 109 cycles per second
Solution 1 (10 cycles per check)
o Worst case 1 billion checks = 10 seconds
Solution 2 (100 cycles per check)
o Worst case 30 checks = 0000003 seconds
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
3
About this course
o Pre-requisites
o Good programming experience (this course does not teach programming)
o Java C++ or C
4
About this course
o Timetable
o LectureFriday 1000 - 1200 C60
o LabsTuesday 900-1100 B52
5
About this course
o Assessments
o Final Exam 50
o Continuous assessment 50
o Homework assignments o Programming projects
6
About this courseo References and learning materials
o Textbooks
1 Mark Allen Weiss Data Stuctures and Algorithm Analysis in Java 2nd Edition Addison Wesley 2007
2 Mark Allen Weiss Data Stuctures and Problem Solving using Java 3rd Edition Addison Wesley 2006
3 Robert Sedgewick Algorithms in C 3rd Edition Addison Wesley 19984 Robert Sedgewick Algorithms in C++ 3rd Edition Addison Wesley 19985 Robert Sedgewick Algorithms in Java 3rd Edition Addison Wesley 2003
o Course web page
httpwwwcsnottacuk~qiuTeachingG64ADS
Slides coursework other materials
7
Course Overview
o Advanced data structures1048708
o Trees hash tables heaps disjoint sets graphs
1048708o Algorithm development and analysis1048708
o Insert delete search sort
o Applications
o Implementation in Java C++ C
8
Advanced Data Structures
o ldquoWhy not just use a big arrayrdquo
o Example problemSearch for a number k in a set of N numbers
o SolutionStore numbers in an array of size NIterate through array until find k
Number of checksBest case 1 (k=29)Worst case N (k=25)Average case N2
29 20 65 392180 25
9
Advanced Data Structures
o Solution 2
o Store numbers in a binary search treeo Search tree until find k
o Number of checks
Best case 1 (k=29)
Worst case log2N (k=25)
Average case (log2N) 2
29
20 65
258021 39
10
Analysis
o Does it matter
N vs (log2N)
05
101520253035404550
0 10 20 30 40 50
N
Num
ber o
f Che
cks
11
Analysis
o Does it matter
Assume o N = 1000000000o 1 billion (Walmart transactions in 100 days)o 1 Ghz processor = 109 cycles per second
Solution 1 (10 cycles per check)
o Worst case 1 billion checks = 10 seconds
Solution 2 (100 cycles per check)
o Worst case 30 checks = 0000003 seconds
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
4
About this course
o Timetable
o LectureFriday 1000 - 1200 C60
o LabsTuesday 900-1100 B52
5
About this course
o Assessments
o Final Exam 50
o Continuous assessment 50
o Homework assignments o Programming projects
6
About this courseo References and learning materials
o Textbooks
1 Mark Allen Weiss Data Stuctures and Algorithm Analysis in Java 2nd Edition Addison Wesley 2007
2 Mark Allen Weiss Data Stuctures and Problem Solving using Java 3rd Edition Addison Wesley 2006
3 Robert Sedgewick Algorithms in C 3rd Edition Addison Wesley 19984 Robert Sedgewick Algorithms in C++ 3rd Edition Addison Wesley 19985 Robert Sedgewick Algorithms in Java 3rd Edition Addison Wesley 2003
o Course web page
httpwwwcsnottacuk~qiuTeachingG64ADS
Slides coursework other materials
7
Course Overview
o Advanced data structures1048708
o Trees hash tables heaps disjoint sets graphs
1048708o Algorithm development and analysis1048708
o Insert delete search sort
o Applications
o Implementation in Java C++ C
8
Advanced Data Structures
o ldquoWhy not just use a big arrayrdquo
o Example problemSearch for a number k in a set of N numbers
o SolutionStore numbers in an array of size NIterate through array until find k
Number of checksBest case 1 (k=29)Worst case N (k=25)Average case N2
29 20 65 392180 25
9
Advanced Data Structures
o Solution 2
o Store numbers in a binary search treeo Search tree until find k
o Number of checks
Best case 1 (k=29)
Worst case log2N (k=25)
Average case (log2N) 2
29
20 65
258021 39
10
Analysis
o Does it matter
N vs (log2N)
05
101520253035404550
0 10 20 30 40 50
N
Num
ber o
f Che
cks
11
Analysis
o Does it matter
Assume o N = 1000000000o 1 billion (Walmart transactions in 100 days)o 1 Ghz processor = 109 cycles per second
Solution 1 (10 cycles per check)
o Worst case 1 billion checks = 10 seconds
Solution 2 (100 cycles per check)
o Worst case 30 checks = 0000003 seconds
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
5
About this course
o Assessments
o Final Exam 50
o Continuous assessment 50
o Homework assignments o Programming projects
6
About this courseo References and learning materials
o Textbooks
1 Mark Allen Weiss Data Stuctures and Algorithm Analysis in Java 2nd Edition Addison Wesley 2007
2 Mark Allen Weiss Data Stuctures and Problem Solving using Java 3rd Edition Addison Wesley 2006
3 Robert Sedgewick Algorithms in C 3rd Edition Addison Wesley 19984 Robert Sedgewick Algorithms in C++ 3rd Edition Addison Wesley 19985 Robert Sedgewick Algorithms in Java 3rd Edition Addison Wesley 2003
o Course web page
httpwwwcsnottacuk~qiuTeachingG64ADS
Slides coursework other materials
7
Course Overview
o Advanced data structures1048708
o Trees hash tables heaps disjoint sets graphs
1048708o Algorithm development and analysis1048708
o Insert delete search sort
o Applications
o Implementation in Java C++ C
8
Advanced Data Structures
o ldquoWhy not just use a big arrayrdquo
o Example problemSearch for a number k in a set of N numbers
o SolutionStore numbers in an array of size NIterate through array until find k
Number of checksBest case 1 (k=29)Worst case N (k=25)Average case N2
29 20 65 392180 25
9
Advanced Data Structures
o Solution 2
o Store numbers in a binary search treeo Search tree until find k
o Number of checks
Best case 1 (k=29)
Worst case log2N (k=25)
Average case (log2N) 2
29
20 65
258021 39
10
Analysis
o Does it matter
N vs (log2N)
05
101520253035404550
0 10 20 30 40 50
N
Num
ber o
f Che
cks
11
Analysis
o Does it matter
Assume o N = 1000000000o 1 billion (Walmart transactions in 100 days)o 1 Ghz processor = 109 cycles per second
Solution 1 (10 cycles per check)
o Worst case 1 billion checks = 10 seconds
Solution 2 (100 cycles per check)
o Worst case 30 checks = 0000003 seconds
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
6
About this courseo References and learning materials
o Textbooks
1 Mark Allen Weiss Data Stuctures and Algorithm Analysis in Java 2nd Edition Addison Wesley 2007
2 Mark Allen Weiss Data Stuctures and Problem Solving using Java 3rd Edition Addison Wesley 2006
3 Robert Sedgewick Algorithms in C 3rd Edition Addison Wesley 19984 Robert Sedgewick Algorithms in C++ 3rd Edition Addison Wesley 19985 Robert Sedgewick Algorithms in Java 3rd Edition Addison Wesley 2003
o Course web page
httpwwwcsnottacuk~qiuTeachingG64ADS
Slides coursework other materials
7
Course Overview
o Advanced data structures1048708
o Trees hash tables heaps disjoint sets graphs
1048708o Algorithm development and analysis1048708
o Insert delete search sort
o Applications
o Implementation in Java C++ C
8
Advanced Data Structures
o ldquoWhy not just use a big arrayrdquo
o Example problemSearch for a number k in a set of N numbers
o SolutionStore numbers in an array of size NIterate through array until find k
Number of checksBest case 1 (k=29)Worst case N (k=25)Average case N2
29 20 65 392180 25
9
Advanced Data Structures
o Solution 2
o Store numbers in a binary search treeo Search tree until find k
o Number of checks
Best case 1 (k=29)
Worst case log2N (k=25)
Average case (log2N) 2
29
20 65
258021 39
10
Analysis
o Does it matter
N vs (log2N)
05
101520253035404550
0 10 20 30 40 50
N
Num
ber o
f Che
cks
11
Analysis
o Does it matter
Assume o N = 1000000000o 1 billion (Walmart transactions in 100 days)o 1 Ghz processor = 109 cycles per second
Solution 1 (10 cycles per check)
o Worst case 1 billion checks = 10 seconds
Solution 2 (100 cycles per check)
o Worst case 30 checks = 0000003 seconds
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
7
Course Overview
o Advanced data structures1048708
o Trees hash tables heaps disjoint sets graphs
1048708o Algorithm development and analysis1048708
o Insert delete search sort
o Applications
o Implementation in Java C++ C
8
Advanced Data Structures
o ldquoWhy not just use a big arrayrdquo
o Example problemSearch for a number k in a set of N numbers
o SolutionStore numbers in an array of size NIterate through array until find k
Number of checksBest case 1 (k=29)Worst case N (k=25)Average case N2
29 20 65 392180 25
9
Advanced Data Structures
o Solution 2
o Store numbers in a binary search treeo Search tree until find k
o Number of checks
Best case 1 (k=29)
Worst case log2N (k=25)
Average case (log2N) 2
29
20 65
258021 39
10
Analysis
o Does it matter
N vs (log2N)
05
101520253035404550
0 10 20 30 40 50
N
Num
ber o
f Che
cks
11
Analysis
o Does it matter
Assume o N = 1000000000o 1 billion (Walmart transactions in 100 days)o 1 Ghz processor = 109 cycles per second
Solution 1 (10 cycles per check)
o Worst case 1 billion checks = 10 seconds
Solution 2 (100 cycles per check)
o Worst case 30 checks = 0000003 seconds
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
8
Advanced Data Structures
o ldquoWhy not just use a big arrayrdquo
o Example problemSearch for a number k in a set of N numbers
o SolutionStore numbers in an array of size NIterate through array until find k
Number of checksBest case 1 (k=29)Worst case N (k=25)Average case N2
29 20 65 392180 25
9
Advanced Data Structures
o Solution 2
o Store numbers in a binary search treeo Search tree until find k
o Number of checks
Best case 1 (k=29)
Worst case log2N (k=25)
Average case (log2N) 2
29
20 65
258021 39
10
Analysis
o Does it matter
N vs (log2N)
05
101520253035404550
0 10 20 30 40 50
N
Num
ber o
f Che
cks
11
Analysis
o Does it matter
Assume o N = 1000000000o 1 billion (Walmart transactions in 100 days)o 1 Ghz processor = 109 cycles per second
Solution 1 (10 cycles per check)
o Worst case 1 billion checks = 10 seconds
Solution 2 (100 cycles per check)
o Worst case 30 checks = 0000003 seconds
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
9
Advanced Data Structures
o Solution 2
o Store numbers in a binary search treeo Search tree until find k
o Number of checks
Best case 1 (k=29)
Worst case log2N (k=25)
Average case (log2N) 2
29
20 65
258021 39
10
Analysis
o Does it matter
N vs (log2N)
05
101520253035404550
0 10 20 30 40 50
N
Num
ber o
f Che
cks
11
Analysis
o Does it matter
Assume o N = 1000000000o 1 billion (Walmart transactions in 100 days)o 1 Ghz processor = 109 cycles per second
Solution 1 (10 cycles per check)
o Worst case 1 billion checks = 10 seconds
Solution 2 (100 cycles per check)
o Worst case 30 checks = 0000003 seconds
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
10
Analysis
o Does it matter
N vs (log2N)
05
101520253035404550
0 10 20 30 40 50
N
Num
ber o
f Che
cks
11
Analysis
o Does it matter
Assume o N = 1000000000o 1 billion (Walmart transactions in 100 days)o 1 Ghz processor = 109 cycles per second
Solution 1 (10 cycles per check)
o Worst case 1 billion checks = 10 seconds
Solution 2 (100 cycles per check)
o Worst case 30 checks = 0000003 seconds
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
11
Analysis
o Does it matter
Assume o N = 1000000000o 1 billion (Walmart transactions in 100 days)o 1 Ghz processor = 109 cycles per second
Solution 1 (10 cycles per check)
o Worst case 1 billion checks = 10 seconds
Solution 2 (100 cycles per check)
o Worst case 30 checks = 0000003 seconds
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
12
Advanced Data Structures
o Does it matter
o The Message
o Appropriate data structures ease design and improve performance
o The Challenge
o Design appropriate data structure and associated algorithms for a problem
o Analyze to show improved performance
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
13
Algorithm Analysis
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
14
Purpose
o Why bother analyzing code isnrsquot getting it to work enough
o Estimate time and memory in the average case and worst case
o Identify bottlenecks ie where to reduce time
o Speed up critical algorithms
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
15
Algorithm
o Problem
o Specifies the desired input-output relationship
o Algorithm
o Well-defined computational procedure for transforming inputs to outputs
o Correct algorithm
o Produces the correct output for every possible input in finite timeo Solves the problem
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
16
Algorithm Analysis
o Predict resource utilization of an algorithm
o Running timeo Memory
o Dependent on architecture
o Serialo Parallelo Quantum
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
17
What to Analyze
o Main focus is on running time
o Memorytime tradeoff
o Simple serial computing model
o Single processor infinite memory
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
18
What to Analyze
o Running time T(N)
o N is typically the size of the inputo Sortingo Multiplying two integerso Multiplying two matriceso Traversing a graph
o T(N) measures number of primitive operations performed
eg addition multiplication comparison assignment
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
19
Example
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
20
Example
o General Rules
o Rule 1 ndash for loop
o The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
21
Example
o General Rules
o Rule 2 ndashnested loop
for (i = 0 iltni++)
for(j=0 jltn j++)
k++
This fragment is O(N2)
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
22
Example
o General Rules
o Rule 3 ndash Consecutive statements
for (i = 0 iltni++)
a[i]=0
for (i = 0 iltni++)
for(j=0 jltn j++)
a[i]+=[a[j]+i+j
This fragment is O(N) work followed by O(N2) work -gt O(N2)
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
23
Example
o General Rules
o Rule 4 ndash ifelse
If (condition)
S1
else
S2
No more than the running time of the test plus max S1S2)
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
24
What to Analyze
o Worst-case running time Tworst(N)
o Average-case running time Tavg(N)
o Tavg(N) lt= Tworst(N)
o Typically analyze worst-case behavior
o Average case hard to compute
o Worst-case gives guaranteed upper bound
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
25
Rate of Growth
o Exact expressions for T(N) meaningless and hard to compare
o Rate of growth
o Asymptotic behavior of T(N) as N gets big
o Usually expressed as fastest growing term in T(N) dropping constant coefficients
eg T(N) = 3N2+ N + 1 rarr Θ(N2)
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
26
Rate of Growth
o T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) lecf(N) when N gen0
o Asymptotic upper boundo ldquoBig-Ohrdquo notation
o T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) gecg(N) when N ge n0
o Asymptotic lower boundo ldquoBig-Omegardquo notation
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
27
Rate of Growth
o T(N) = Θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N))
o Asymptotic tight bound
o T(N) = o(p(N)) if for all constants c there exists an n0 such that T(N) lt cp(N) when Ngtn0
o ie T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) neΘ(p(N))
o ldquoLittle-ohrdquo notation
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
28
Rate of Growth
o N2= O(N2) = O(N3) = O(2N)
o N2= Ω(1) = Ω(N) = Ω(N2)
o N2= Θ(N2)
o N2= o(N3)
o 2N2+ 1 = Θ()
o N2+ N = Θ()
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
29
Rate of Growth
o Rule 1 If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
o T1(N) + T2(N) = O(f(N) + g(N))o T1(N) T2(N) = O(f(N) g(N))
o Rule 2 If T(N) is a polynomial of degree k then T(N) = Θ(Nk)
o Rule 3 logkN = O(N) for any constant k
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
30
Rate of Growth
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
31
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
32
Example
o Maximum Subsequence Sum problem
o Given (possibly negative) integers A1 A2 hellip AN find the maximum value of
o eg for input -2 11 -4 13 -5 -2 the answer is 20 (A2 through A4)
j
ikkA
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
33
Example
o Algorithm 1
o Compute each possible subsequence independently
MaxSubSum1 (A)maxSum = 0
for i = 1 to Nfor j = i to Nsum = 0
for k = i to jsum = sum + A[k]
if (sum gt maxSum)then maxSum = sum
return maxSum
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
34
Example
o Algorithm 11 2 Cubic maximum contiguous subsequence sum algorithm 3 4 public static int maxSubSum1( int [ ] a ) 5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ ) 8 for( int j = i j lt alength j++ ) 9
10 int thisSum = 0
11 for( int k = i k lt= j k++ ) 12 thisSum += a[ k ]13 14 if( thisSum gt maxSum ) 15 maxSum = thisSum 16
17 return maxSum 18
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
35
Example
o Algorithm 1 Analysis
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
36
Example
o Algorithm 2
o Note that
o No reason to re-compute sum each time
MaxSubSum2 (A)maxSum = 0for i = 1 to Nsum = 0
for j = i to Nsum = sum + A[j]
if (sum gt maxSum)then maxSum = sum
return maxSum
1j
ikkj
j
ikk AAA
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
37
ExampleAlgorithm 2
1 2 Quadratic maximum contiguous subsequence sum algorithm3 4 public static int maxSubSum2( int [ ] a )5 6 int maxSum = 0
7 for( int i = 0 i lt alength i++ )8 9 int thisSum = 010 for( int j = i j lt alength j++ )11 12 thisSum += a[ j ]
13 if( thisSum gt maxSum )14 maxSum = thisSum15 16
17 return maxSum18
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
38
Example
o Algorithm 2 Analysis
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
39
Example
o Algorithm 3
o Recursive divide and conquero Divide sequence in halfo A(1 center) and A(center+1 N)o Recursively compute MaxSubSum of left halfo Recursively compute MaxSubSum of right halfo Compute MaxSubSum of sequence constrained to
use A(center) and A(center+1)
o eg lt4 -3 5 -2 -1 2 6 -2gt
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
40
Example
o Algorithm 3
MaxSubSum3 (A i j)maxSum = 0if (i = j)then if A[i] gt 0
then maxSum = A[i]else k = floor((i+j)2)
maxSumLeft = MaxSubSum3(Aik)maxSumRight = MaxSubSum3(Ak+1j) compute maxSumThruCentermaxSum = maximum(maxSumLeftmaxSumRightmaxSumThruCenter)
return maxSum
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
41
Example
o Algorithm 312 Recursive maximum contiguous subsequence sum
algorithm3 Finds maximum sum in subarray spanning a[leftright]4 Does not attempt to maintain actual best sequence5 6 private static int maxSumRec( int [ ] a int left int right )7 8 if( left == right ) Base case9 if( a[ left ] gt 0 )10 return a[ left ]11 else12 return 0
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
42
Example 13 14 int center = ( left + right ) 215 int maxLeftSum = maxSumRec( a left center )16 int maxRightSum = maxSumRec( a center + 1 right )17 18 int maxLeftBorderSum = 0 leftBorderSum = 019 for( int i = center i gt= left i-- )20 21 leftBorderSum += a[ i ]22 if( leftBorderSum gt maxLeftBorderSum )23 maxLeftBorderSum = leftBorderSum24 25 26 int maxRightBorderSum = 0 rightBorderSum = 027 for( int i = center + 1 i lt= right i++ )28 29 rightBorderSum += a[ i ]30 if( rightBorderSum gt maxRightBorderSum )31 maxRightBorderSum = rightBorderSum32 33 34 return max3( maxLeftSum maxRightSum35 maxLeftBorderSum + maxRightBorderSum )36
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
43
Example
o Algorithm 3 Analysis
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
44
Example
o Algorithm 4
o Observationo Any negative subsequence cannot be a prefix to the
maximum sequenceo Or a positive contiguous subsequence is always worth
adding
o T(N) = MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum gt maxSum) then maxSum = sum else if (sum lt 0) then sum = 0return maxSum
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
45
Example
o Algorithm 4
1 Linear-time maximum contiguous subsequence sum algorithm
1 public static int maxSubSum4( int [ ] a )2 3 int maxSum = 0 thisSum = 0
4 for( int j = 0 j lt alength j++ )5 6 thisSum += a[ j ]
7 if( thisSum gt maxSum )8 maxSum = thisSum9 else if( thisSum lt 0 )10 thisSum = 011
12 return maxSum13
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
46
MaxSubSum Running Times
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
47
MaxSubSum Running Times
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
48
MaxSubSum Running Times
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
49
Logarithmic Behaviour
o T(N) = O(log2 N) usually occurs when
o Problem can be halved in constant timeo Solutions to sub-problems combined in constant
time
o Examples
o Binary searcho Euclidrsquos algorithmo Exponentiation
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
50
Binary Search
o Given an integer X and integers A0A1hellipAN-1 which are presorted and already in memory find i such that
Ai=X or return i = -1 if X is not in the input
o T(N) = O(log2 N)
o T(N) = Θ(log2 N)
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
51
Binary Search
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
52
Euclidrsquos Algorithm
o Compute the greatest common divisor gcd(MN) between the integers M and N
o ie largest integer that divides botho Used in encryption
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
53
Euclidrsquos Algorithm
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
54
Euclidrsquos Algorithm Analysis
o Note After two iterations remainder is at most half its original value
o Theorem 21 If M gt N then M mod N lt M2
o T(N) = 2 log2 N = O(log2 N) o log2225 = 78 T(225) = 16 ()
o Better worst case T(N) = 144 log2 No T(225) = 11o Average case T(N) = 12ln2ln(N)pi2+147o T(225) = 6
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
55
Exponentiation
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
56
Exponentiation
- G64ADS Advanced Data Structures
- About this course
- Slide 3
- Slide 4
- Slide 5
- Slide 6
- Course Overview
- Advanced Data Structures
- Slide 9
- Analysis
- Slide 11
- Slide 12
- Algorithm Analysis
- Purpose
- Algorithm
- Slide 16
- What to Analyze
- Slide 18
- Example
- Slide 20
- Slide 21
- Slide 22
- Slide 23
- Slide 24
- Rate of Growth
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Slide 37
- Slide 38
- Slide 39
- Slide 40
- Slide 41
- Slide 42
- Slide 43
- Slide 44
- Slide 45
- MaxSubSum Running Times
- Slide 47
- Slide 48
- Logarithmic Behaviour
- Binary Search
- Slide 51
- Euclidrsquos Algorithm
- Slide 53
- Euclidrsquos Algorithm Analysis
- Exponentiation
- Slide 56
-
Related Documents
