Chapter 4: Divide and Conquer - Intro • A recurrence equation (recurrence) is an equation or inequality that describes a function in terms of its values on smaller inputs • There are various forms: 1. Recurse on subproblems of equal size (typical of divide and conquer) T (n)= cT n c ! + f (n) 2. Recurse on subproblems of unequal size (typical of divide and conquer) T (n)= T c 1 n c 3 ! + T (c 3 - c 1 )n c 3 + f (n) 3. Recurse on a single smaller problem size (typical of decrease and conquer) T (n)= T (n - c)+ f (n) • Issues: 1. Sometimes recurrences are expressed as inequalities – If T (n) ≤ some recurrence function, solution will be in terms of big oh notation – If T (n) ≥ some recurrence function, solution will be in terms of big omega notation 2. For exact solutions, must usually consider floor and ceiling functions – For example, when n is not even T (n)= T & n 2 ’! + T $ n 2 %! + f (n) 3. Boundary conditions are generalized – Rather than specify an exact function, usually represent the aspects of minor importance using asymptotic notation – Unless an exact solution is required, the base case need not be repre- sented by an exact function 1
22
Embed
Chapter 4: Divide and Conquer - Introdjmoon/algs-g/algs-notes/c4... · 2019. 2. 18. · Chapter 4: Divide and Conquer - The Max Subarray Problem Consider a vector A[n] of numeric
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Chapter 4: Divide and Conquer - Intro
• A recurrence equation (recurrence) is an equation or inequality that describesa function in terms of its values on smaller inputs
• There are various forms:
1. Recurse on subproblems of equal size (typical of divide and conquer)
T (n) = cT
(n
c
)+ f(n)
2. Recurse on subproblems of unequal size (typical of divide and conquer)
T (n) = T
(c1n
c3
)+ T
(c3 − c1)nc3
+ f(n)
3. Recurse on a single smaller problem size (typical of decrease and conquer)
T (n) = T (n− c) + f(n)
• Issues:
1. Sometimes recurrences are expressed as inequalities
– If T (n) ≤ some recurrence function, solution will be in terms of big ohnotation
– If T (n) ≥ some recurrence function, solution will be in terms of bigomega notation
2. For exact solutions, must usually consider floor and ceiling functions
– For example, when n is not even
T (n) = T
(⌈n
2
⌉)+ T
(⌊n
2
⌋)+ f(n)
3. Boundary conditions are generalized
– Rather than specify an exact function, usually represent the aspects ofminor importance using asymptotic notation
– Unless an exact solution is required, the base case need not be repre-sented by an exact function
1
Chapter 4: Divide and Conquer - The Max Subarray Problem
• Consider a vector A[n] of numeric values
• The maximum subarray problem is to find i0, j0 such that i0 ≤ j0 ≤ n and∑j0k=i0
A[k] is greater or equal to the sum over any other assignments of i and j
• The brute force approach would be to id all pairs < i, j > that meet the abovecriteria and find the largest sum over those pairs
– The number of pairs for (i0, j0) is
n2
– Value of each subarray can be computed in constant time using values of
previously computed subarrays, so overall ∈ Θ(n2)
• Divide and conquer approach
– Strategy:
∗ Divide array in half
∗ The max subarray is either
1. Entirely in the left half
2. Entirely in the right half
3. Spanning the middle
∗ The alg will
1. Recurse on the left and right halves
2. Find the largest subarray that spans the middle
3. Return the largest of the three
2
Chapter 4: Divide and Conquer - The Max Subarray Problem (2)
Chapter 4: Divide and Conquer - Matrix Multiplication Recursive Approach(2)
• Analysis
– The partitioning (line 5) can be done in Θ(1) time if index calculationsare used instead of copying the elements into new matrices (which wouldrequire Θ(n2) time)
– Base case when n = 1 ∈ Θ(1) time
– Recurse eight times on matrices of size n/2× n/2 (lines 7 - 10)
– Four additions performed on matrices that hold n2/4 elements (Θ(n2) lines7 - 10)
– The recursion is then
T (n) =
Θ(1) n = 18T (n2 ) + Θ(n2) n > 1
7
Chapter 4: Divide and Conquer - Strassen’s Algorithm
• Strassen’s algorithm improves on this by making the recursion tree slightly lessbushy at the expense of increased additions and subtractions
– But these are negligible wrt asymptotic growth, being subsumed by themultiplications
• As in the recursive algorithm, n × n matrices A, B, C are partitioned inton/2× n/2 sub matrices: C11 C12
C21 C22
=
A11 A12
A21 A22
∗ B11 B12
B21 B22
• The four subarrays of C are computed in terms of n/2× n/2 matrices mi:
Chapter 4: Divide and Conquer - Analysis Techniques (2)
• Backward substitution
– Start at the top level (n elements) and work towards the base case
– Substitute the value from the next recursion into the recursion equation;i.e.,Express T (di) in terms of T (di−1) and substitute in original recurrencerelation
– Repeat with successively smaller terms until a formula that expresses thesequence can be determined
– For example, consider
T (n) = cT
(n
d
)+ f(n)
Let n = dk with base case T (1) = d0 = b (step i = 0)
Then,
i = 1: T (dk) = cT ((dk)/d) + f(dk) = cT (dk−i) + f(dk)Since T (dk−1) = cT ((dk−1)/d) + f(dk−1) = cT (dk−2) + f(dk−1) we canrewrite T (dk) asi = 2: T (dk) = c[cT (dk−i)+f(dk−i+1)]+f(dk) = c2T (dk−i)+cf(dk−i+1)+f(dk)etc.
– Will ultimately express i in terms of n
12
Chapter 4: Divide and Conquer - Analysis Techniques, The Substitution Method
• This technique involves
1. Guess a form for the equation (i.e., g(n))
2. Use induction to find asymptotic constant(s) and show that the guess works
• The guess might be based on
1. A previous analysis that had a similar equation
2. Result of another proof technique
• The proof itself consists of
1. Substituting the guess for the recurrence in the equation
2. Find a constant (could be two) that satisfies the relation in the asymptoticdefinition
3. Demonstrate that the guess holds for the base case too
• Example: Consider (pp 83 - 84)
T (n) = 2T
(⌊n
2
⌋)+ n
– The guess is T (n) = O(nlg n)
– To prove guess correct, must show that T (n) ≤ cnlg n for some c > 0
– Assume that this holds for all m < n (i.e.,⌊n2
⌋)
– Substitute cnlg n in the right-hand side of the inequality, using⌊n2
⌋for n
– Then,T (n) ≤ 2
(c⌊n2
⌋lg
(⌊n2
⌋))+ n
≤ cnlg(n2
)+ n
= cnlg n− cnlg 2 + n= cnlg n− cn+ n
≤ cnlg n
which holds for c ≥ 1
13
Chapter 4: Divide and Conquer - Analysis Techniques, The SubstitutionMethod (2)
– Next, need to demonstrate that the guess holds for the base case
∗ This step can sometimes require some ingenuity
∗ Assume that T (1) = 1 for the example above; then
T (1) = 1 ≤ c ∗ 1 ∗ lg 1 = 0
∗ This contradicts the hypothesis
∗ What we must do is distinguish between the base case of the equation,and the base case of the proof
∗ While the base case of the recursion is n = 1, for the proof we can uselarger values (because we only have to show that it holds for all n ≥some n0, which we essentially get to choose)
∗ Since n = 1 is problemmatic, judicious choices for base cases are n = 2and n = 3
· These are the two values that result in recursion with n = 1
∗ Using these values as base cases,
· T (2) = 4 (i.e., 2T (1) + 2 = 2 + 2 ≤ 2lg 2)
· T (3) = 5 (i.e., 2T (1) + 3 = 2 + 3 ≤ 3lg 3)
• Heuristics for making a good guess:
1. If know bounds for a similar recursion, try it
2. Prove weak upper and lower bounds, then iteratively try to improve themby tightening them
• Issues
1. May have correct bound but induction doesn’t work
– Reason might be because induction isn’t strong enough to prove thedetailed (exact) bound
– Can sometimes subtract a lower-order term to make it work
Clearly this isn’t less than cn for any value of n
– The problem is the constant term 1
– To remedy the situation, subtract a constant d ≥ 0
– The revised guess is T (n) ≤ cn− d
T (n) ≤(c
⌊n
2
⌋− d
)+
(c
⌈n
2
⌉− d
)+ 1 = cn− 2d+ 1 ≤ cn− d
15
Chapter 4: Divide and Conquer - Analysis Techniques, The SubstitutionMethod (4)
2. You must prove the exact form of the inductive hypothesis
– For example, if you want to prove T (n) ∈ O(n), you must showT (n) ≤ cn
– Consider the following common example of incorrect logic
∗ Want to show
T (n) = 2T
(⌊n
2
⌋)+ n ∈ O(n)
∗ ThenT (n) ≤ 2
(c⌊n2
⌋)+ n
≤ cn+ n ∈ O(n)
∗ But this reasoning is fallacious because have not shown thatT (n) ≤ cn
3. Sometimes changing variables makes proofs easier
– ConsiderT (n) = 2T (b
√nc) + lg n
– Let m = lg n⇒ n = 2m
– ThenT (2m) = 2T (2
m2 ) +m
– Then, let S(m) = T (2m), which lets us rewrite the above as
S(m) = 2S
(m
2
)+m
– This has the same form as
T (n) = 2T
(⌊n
2
⌋)+ n ∈ O(n)
– So we can guess a solution of
S(m) = O(mlg m) = O((lg n)(lg lg n))
16
Chapter 4: Divide and Conquer - Analysis Techniques, Recursion Trees
• This approach can be used to find a good guess to OoG to be used in thesubstitution method, or - if done in an exacting manor - it can be a proof initself
• The approach:
1. Create a recursion tree
2. Calculate the cost of each level
3. Calculate the cost over all levels
• Consider
T (n) = 3T
(⌊n
4
⌋)+O(n2)
– Consider the following examples of ’sloppiness’ (i.e., laxness) that is allow-able since we are looking for a guesstimate
∗ Let n = 4i, eliminating the need for the floor function in the above
∗ Replace O(n2) by cn2
– This generates the following recursion tree:
17
Chapter 4: Divide and Conquer - Analysis Techniques, Recursion Trees(2)
– Let i represent the level of the tree; then
Level Cost
i = 0 c30(n40
)2= cn2
i = 1 c31(n41
)2=(316
)cn2
i = 2 c32(n42
)2=(316
)2cn2
. . . . . . . . .
– The leaf level corresponds to 4i = n⇒ log4(4i) = log4 n
Thus i = log4n, which corresponds to the height of the tree
– Cost of the leaf level is 3log4nT (1) = cnlog43
– The total cost of the tree is
T (n) =∑log4n−1i=0
(316
)icn2 + cnlog43
<∑∞i=0
(316
)icn2 + cnlog43
= 11− 3
16
cn2 + cnlog43
= 1613cn
2 + cnlog43
∈ O(n2)
– The root of the tree contributes the greatest amount to the total cost (cout of 16
13c, with 313c contributed from the lower levels of the tree)
18
Chapter 4: Divide and Conquer - Analysis Techniques, Recursion Trees (3)
• To prove that T (n) ∈ O(n2) (via substitution), must show that T (n) ≤ dn2 forsome d > 0
T (n) ≤ 3T(⌊n4
⌋)+ cn2
≤ 3d⌊n4
⌋2+ cn2
≤ 3d(n4
)2+ cn2
= 316dn
2 + cn2
≤ dn2
when d ≥ 1613c
19
Chapter 4: Divide and Conquer - Analysis Techniques, The Master Theorem
• The Master Theorem allows easy solutions to recursions of the form
T (n) = aT
(n
b
)+ f(n)
– f(n) represents the cost of dividing and combining
– Omitting floors and ceilings has no effect on the results
• The Master Theorem:
Let
1. a ≥ 1, b > 1 be constants
2. f(n) be a function that is asymptotically positive
3. T (n) is defined for integers n ≥ 0 as
T (n) = aT
(n
b
)+ f(n)
where nb is either dn/be or bn/bc
Then T (n) has the following asymptotic bounds:
T (n) =
Θ(nlogba) when f(n) ∈ O(nlogba−ε) for constant ε > 0
Θ(nlogbalg n) when f(n) ∈ Θ(nlogba)
Θ(f(n)) when f(n) ∈ Ω(nlogba+ε) for constant ε > 0,af(nb ) ≤ cf(n) for some constant c < 1, and
for all sufficiently large n
• Points re the theorem:
1. Form of solution based on comparison of f(n) with nlogba, the larger beingthe determining factor
– In the first case, f(n) can’t be just smaller, but must be so by a factorof nε
– In the third case, f(n) can’t be just larger, but must be so by a factorof nε AND must satisfy af(nb ) ≤ cf(n)
2. Not all situations are covered
– For example, when f(n) > nlogba but is not polynomially smaller (andsimilarly for < case)
20
Chapter 4: Divide and Conquer - Analysis Techniques, The Master Theorem(2)
• Examples:
1. T (n) = 9T (n/3) + na = 9, b = 3, f(n) = nnlogba = nlog39 = n2
Since f(n) ∈ O(n2−ε), where ε = 1, the Master Theorem appliesT (n) ∈ Θ(n2)
2. T (n) = T (2n/3) + 1a = 1, b = 3/2, f(n) = 1nlogba = nlog3/21 = n0 = 1Since f(n) ∈ Θ(1),T (n) ∈ Θ(lg n)
3. T (n) = 3T (n/4) + nlg n
a = 3, b = 4, f(n) = nlg n
nlogba = nlog43 = n0.793
Since f(n) ∈ Ω(nlog43+ε), where ε ∼ 0.2, andcf(n/b) = 3(n/4)lg(n/4) ≤ 3
4nlg n = cf(n) for c = 34 , the Master Theorem
appliesT (n) ∈ Θ(nlg n)
4. T (n) = 2T (n/2) + nlg n
a = 2, b = 2, f(n) = nlg n
nlogba = nlog22 = nnlg n is asymptotically greater than n, but not polynomially larger:
f(n)
nlogba=nlg n
n= lg n
which is asymptotically smaller then n2 for any ε > 0The Master Theorem does not apply
21
Chapter 4: Divide and Conquer - Analysis Techniques, Proof of the MasterTheorem