1 Week 3.3, Friday, Sept 6 Announcement: Homework 1 Solutions released on Piazza Homework 2 Due: September 16 th , 2019 @ 11:59PM on Gradescope
1
Week 3.3, Friday, Sept 6
Announcement: Homework 1 Solutions released on PiazzaHomework 2 Due: September 16th, 2019 @ 11:59PM on Gradescope
Recap: Master Theorem
β’ Derived by analyzing recursion tree
ππ ππ = οΏ½ππππππππππππππ ππππ ππ β€ ππππππ
ππ Γ ππππππ
+ ππ + ππππ πππππππππππππππππ
Obtain Geometric Series: ππ ππ = Ξ ππππ βππ=1logππ ππ ππ
ππππππ
β’ Key Ratio: a/bc ππππππππ ππ β₯ ππ β ππππππβ₯ ππ
β’ Case 1: ππππππ
< 1 ππ ππ = Ξ ππππ
β’ Case 2: ππππππ
= 1 ππ ππ = Ξ ππππ logππ
β’ Case 3: ππππππ
> 1 ππ ππ = Ξ ππlogππ ππ
2
Other Form of Master Theorem
β’ What if MergeCost is not exactly f(n)=nc? β’ f(n) = n log n? or β’ f n = ππ
110+logππ ππ/log(ππ)?
3
Case 3: ππ ππ = ππ ππlogππ ππβΞ΅ ππ ππ = Ξ ππlogππ ππ
ππ ππ = οΏ½πΆπΆ ππ ππππ ππ β€ ππππππ
ππ Γ ππππππ
+ ππππ + ππ(ππ) πππππππππππππππππ
Case 1: ππ ππ = Ξ© ππΞ΅+logππ ππ ππ ππ = Ξ ππ(ππ)
Case 2: ππ ππ = Ξ ππlogππ ππ logππ ππ ππ ππ = Ξ ππlogππ ππ logππ+1 ππ(assumes ππ β₯ 0)
Assumeππ(ππ) β₯ 0
Clicker Question
Suppose that f n = ππ110+logππ ππ/log(ππ) above what is ππ ππ ?
A. Ξ ππ(ππ) B. Ξ ππlogππ ππ logππ+1 ππ
C. ππ ππ = Ξ ππlogππ ππ D. More info required
4
Case 3: ππ ππ = ππ ππlogππ ππβΞ΅ ππ ππ = Ξ ππlogππ ππ
ππ ππ β€ οΏ½ππππππππππππππ ππππ ππ β€ ππππππ
ππ Γ ππππππ
+ ππππ + ππ(ππ) πππππππππππππππππ
Case 1: ππ ππ = Ξ© ππΞ΅+logππ ππ ππ ππ = Ξ ππ(ππ)
Case 2: ππ ππ = Ξ ππlogππ ππ logππ ππ ππ ππ = Ξ ππlogππ ππ logππ+1 ππ(assumes ππ β₯ 0)
Assumeππ(ππ) β₯ 0
Clicker Question
Suppose that f n = ππ110+logππ ππ/log(ππ) above what is ππ ππ ?
A. Ξ ππ(ππ) B. Ξ ππlogππ ππ logππ+1 ππ
C. ππ ππ = Ξ ππlogππ ππ D. More info required
5
Case 3: ππ ππ = ππ ππlogππ ππβΞ΅ ππ ππ = Ξ ππlogππ ππ
ππ ππ β€ οΏ½ππππππππππππππ ππππ ππ β€ ππππππ
ππ Γ ππππππ
+ ππππ + ππ(ππ) πππππππππππππππππ
Case 1: ππ ππ = Ξ© ππΞ΅+logππ ππ ππ ππ = Ξ ππ(ππ)
Case 2: ππ ππ = Ξ ππlogππ ππ logππ ππ ππ ππ = Ξ ππlogππ ππ logππ+1 ππ(assumes ππ β₯ 0)
Assumeππ(ππ) β₯ 0
Other Types of Recurrences
β’ T n = T n β 1 + 1 (Unroll: T n = Ξ(n))β’ T n = T n β 1 + 1 = T n β 2 + 1 + 1
= T n β 3 + 1 + 1 + 1 = β― = T n β k + k= T 1 + n β 1
β’ T n = 2 Γ T n β 10 (Exponential)
T n = 2T n β 10 = 2 2ππ ππ β 20 = 4T n β 20
= 8T n β 30 = β― = 2ππT n β 10i
= 2ππ10β1T 10 = Ξ 2
ππ10
6
Only constant reduction in input sizeTwo branches
Other Recurrences
β’ T n = T n β 1 + T(ππ β 3) (Exponential)
T n = Ξ cn
β’ Unrolling gets messy fast! How to find c? [Trick]Assume ππ ππ = ππππ for some cππππ = ππ ππ = T n β 1 + T ππ β 3 = ππππβ1 + ππππβ3
β ππππ = ππππβ1 + ππππβ3 β c3 = ππ2 + 1
β c β 1.46577(Root of Characteristic Equation)
Must verify solution by induction7
Only constant reduction in input sizeTwo branches
Other Recurrences
β’ T n = T n β 1 + T(ππ β 3) (Exponential)
β ππππ = ππππβ1 + ππππβ3 β c3 = ππ2 + 1
β c β 1.46577(Root of Characteristic Equation)
Claim: T n β€ ππππππ (pick k s.t. T(0) < k)
Inductive Step: T n = T n β 1 + T ππ β 3β€ ππ ππππβ1 + ππππβ3 (IH)= ππ ππππ (Choice of c)
8
Only constant reduction in input sizeTwo branches
Other Recurrences
β’ MergeSort with Uneven Split: Split L into A, B of sizes n/4 and 3n/4.
T n = Tn4
+ ππ3ππ4
+ ππ
A bit harder to Analyze with recursion tree
9
n
οΏ½ππ 16 οΏ½9ππ16οΏ½3ππ
16
οΏ½3ππ4οΏ½ππ 4
οΏ½3ππ16
O(log n)
n
n
n
ππ ππ = π£π£ ππ π₯π₯π₯π₯π₯π₯ππβ¦ β¦
Another Unbalanced Recurrence
β’ Geometric Series
T n = Tn5
+ ππ3ππ4
+ ππ
β’ Claim: T n = Ξ ππ
10
n
οΏ½ππ 25 οΏ½9ππ16 οΏ½3ππ
20
οΏ½3ππ4οΏ½ππ 5
οΏ½3ππ20
O(log n)
n
(19/20)n
(19/20)2n
T n β€ nοΏ½ππ=0
β1920
ππ
11
Mergesort Quicksort Binary Search Linear-time selection Skyline Problem Maximum Subarray Counting inversions
12
Maximum Subarray ProblemGiven an array A of n (positive and negative) numbers, find the contiguous subarray whose sum has the largest value.
10 5 -20 5 12 -6 33 6 2 -52 6 45 3 -4
Brute Force For every pair i and j, iβ€j, compute the sum from A(i) to A(j).
Remember the pair resulting in the maximum. How many pairs? O(n2) Using previously computed values, the total running time is
O(n2) (n3 is excessive brute force)
13
Aim for O(n) or O(n log n)? Divide and conquer? Split the problem into two halves and solve each recursively Combine two solutions to produce the final answer
14
10 5 -20 21 12 -18 12 5 15 -22 6 5 3 -4 -12 55
10 5 -20 21 12 -18 12 5 15 -22 6 5 3 -4 -12 55
10 5 -20 21 12 -18 12 5 15 -22 6 5 3 -4 -12 55
32 46
15
Combining the answers from the two subproblems
The maximum subarray can be in one the two halves (easy case) or it can cross the midpoint
To determine the maximum subarray crossing the midpoint, we can compute the two maximum subarrays βanchoredβ at the midpoint. Running Sum: sweep left (resp. right) from midpoint keeping track of max Requires time O(n) for merge step
Results in T(n) = 2T(n/2) + cnGives an O(n log n) time algorithm
16
A Faster O(n) time solutionIdea: Each recursive call returns additional information to make merge step easier
Optimal Solution: (i,j) and value ππ = βπ₯π₯=ππππ π΄π΄[π₯π₯]
Total Sum: ππ = βπ₯π₯=1ππ π΄π΄[π₯π₯] ππππππππ maximizing value ππππππππ = βπ₯π₯=ππππππππ
ππ π΄π΄[π₯π₯]
ππππππππππππ maximizing value ππππππππππππ = βπ₯π₯=1ππππππππππππ π΄π΄[π₯π₯]
10 5 -20 21 12 -18 12 5 -50 2 5
ππππ , ππππππππππππ ππππππππ
ππππππππ = ππππππππππππππ = ππππ ππ = ππππ
17
A Faster O(n) time solution
Merge in Constant Time: Suppose A was split into L and R
Three possibilities for (i,j): πππΏπΏ , πππΏπΏ , πππ π , πππ π , πππππππππΏπΏ , πππππππππππππΏπΏ
Case 1: Opt in L Case 2: Opt in R Case 3: Opt crosses L and R
10 5 -20 21 12 -18 12 5 -50 2 5 -5 50 1 -2 -80 44 1 1 6
π£π£πΏπΏ π£π£π π π£π£πππππππΏπΏ + π£π£πππππππππππ π
π―π―ππ = πππππ―π―ππππππππ = ππππ
π―π―πππππ₯π₯ππππππ = πππππ―π―ππππππππ = πππ£π£πΏπΏ = 32πππππππππππππ³π³ = ππππ
π£π£ = max π£π£πΏπΏ,π£π£π π , π£π£πππππππΏπΏ + π£π£πππππππππππ π = π£π£πππππππΏπΏ + π£π£πππππππππππ π = 53β ππ, ππ = πππππππππΏπΏ , πππππππππππππΏπΏ
18
A Faster O(n) time solution
Merge in Constant Time: Suppose A was split into L and R
Still Needs to Compute Extra Values Update Total: ππ = πππΏπΏ + πππ π
ππππππππ maximizing value ππππππππ = βπ₯π₯=ππππππππππ π΄π΄[π₯π₯]
Case 1: ππππππππ = πππππππππ π (interval in R)
Case 2: ππππππππ = πππππππππΏπΏ (interval crosses L)
10 5 -20 21 12 -18 12 5 -50 2 5 -5 50 1 -2 -80 44 1 1 6π―π―ππππππππ = πππππ―π―ππππππππ = ππ
π―π―ππππππ = max{π―π―ππππππππ ,πππΉπΉ + π―π―ππππππππ } = π―π―ππππππππ = ππππ ππππππππ = πππππππππ π
For any merges higher in the recursion tree!
19
A Faster O(n) time solution
Merge in Constant Time: Suppose A was split into L and R
Merge Needs to Compute Extra Values ππ = πππΏπΏ + πππ π
ππππππππππππ maximizing value ππππππππππππ = βπ₯π₯=1ππππππππππππ π΄π΄[π₯π₯]
Similar to computing ππππππππ Case 1: ππππππππππππ = πππππππππππππΏπΏ (interval in L)
Case 2: ππππππππππππ = πππππππππππππ π (interval crosses R)
10 5 -20 21 12 -18 12 5 -50 2 5 -5 50 1 -2 -80 44 1 1 6
π―π―πππππ₯π₯ππππππ = πππππππππππππππππ³π³ = ππππ
π―π―πππππ₯π₯ππππ = max{π―π―ππππππππππππ ,πππ³π³ + π―π―πππππ₯π₯πππππΉπΉ } = πππ³π³ + π―π―πππππ₯π₯πππππΉπΉ = βππππ + ππππ = ππππ ππππππππππππ = πππππππππππππ π
20
Summary: A Faster O(n) time solution
Idea: Each recursive call returns additional information to make merge step easier
Constant Time Merge:
ππ ππ = 2ππππ2
+ 1
Master Theorem (a=2, b=2, c=0): ππ ππ = Ξ ππ
21
Summary: Maximum Subarray problemGiven an array A of n (positive and negative) numbers, find the contiguous subarray whose sum has the largest value.
10 5 -20 5 12 -6 33 6 2 -52 6 45 3 -4
All pairs: O(n2)
D&C with simple combine step: O(n log n)
D&C with extra information: O(n)
Simple non-D&C solution: O(n)