Design and Analysis of Algorithms (XI) LP based Approximation Guoqiang Li School of Software, Shanghai Jiao Tong University
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Design and Analysis ofAlgorithms (XI)
LP based Approximation
Guoqiang Li
School of Software, Shanghai Jiao Tong University
Rounding
Set Cover of LP-Rounding
We will design two approximation algorithms for set cover problemusing the method of LP-rounding.
The first one is a simple rounding algorithm achieving a guarantee off .
The second one is based on randomized rounding and achieves aguarantee of O(log n).
The Set Cover LP-Relaxation
min∑S∈S
c(S)xS∑S:e∈S
xS ≥ 1, e ∈ U
xS ≥ 0, S ∈ S
A Simple Rounding Algorithm
Algorithm
1 Find an optimal solution to the LP-relaxation.
2 Pick all sets S for which xS ≥ 1/f in this solution.
Analysis
TheoremThis algorithm achieves an approximation factor of f for the set coverproblem.
Proof.Let C be the collection of picked sets. We first show that C is indeed aset cover.• Consider an element e. Since e is in at most f sets, one of these
sets must be picked to the extent of at least 1/f in the fractionalcover, due to the pigeonhole principle.• Thus, e is covered by C, and hence C is a valid set cover.
Analysis
• The rounding process increases xS, for each set S ∈ C, by a factorof at most f .• Therefore, the cost of C is at most f times the cost of the
fractional cover, thereby proving the desired approximationguarantee.
Randomized Rounding
Why Randomization
The performance guarantee of a random approximation algorithmholds with high probability.
In most cases a randomized approximation algorithms can bederandomized, and
the randomization gains simplicity in the algorithm design andanalysis.
Randomized Rounding for Set Cover
Intuitively, a set with larger value is more likely to be chosen in theoptimal solution.
This motivates us to view the fractions as probability and roundaccordingly.
Analysis (I)
• Let x = p be an optimal solution to the linear program.• For each set S ∈ S, our algorithm pick S with probability pS.• The expected cost of C is
E[cost(C)] =∑S∈S
Pr[S is picked] · cS =∑S∈S
pS · cS = OPTf .
• Next, let us compute the probability that an element a ∈ U iscovered by C.
Analysis (II)
• Suppose that a occurs in k sets of S .• Let the probabilities associated with these sets be p1, . . . , pk .
Then p1 + p2 + · · ·+ pk ≥ 1.• Then we have
Pr[a is covered by C] ≥ 1− (1− 1k
)k ≥ 1− 1e
• To get a complete set cover, independently pick c log n suchsubcollections, and let C′ be their union, where c is a constantsuch that
(1e
)c log n ≤ 14n
Analysis (III)
• Now we have
Pr[a is not covered by C′] ≤ (1e
)c log n ≤ 14n
• Summing over all elements a ∈ U, we get
Pr[C′ is not a valid set cover ] ≤ n · 14n≤ 1
4.
Analysis (IV)
• Clearly E[C′] ≤ OPTf · c log n.• Applying Markov’s Inequality, we get
Pr[cost(C′) ≥ OPTf · 4c log n] ≤ 14
• Thus
Pr[C′ is a valid set cover and has cost ≤ OPTf · 4c log n] ≥ 12
MAX-SAT
MAX-SAT
Given n boolean variables x1, . . . , xn, a CNF
ϕ(x1, . . . , xn) =
m∧j=1
Cj
and a nonnegative weight wj for each Cj.Find an assignment to the xis that maximizes the weight of thesatisfied clauses.
Flipping a Coin
A very straightforward randomized approximation algorithm is to seteach xi to true independently with probability 1/2.
Setting each xi to true with probability 1/2 independently gives arandomized 1
2 -approximation algorithm for weighted MAX-SAT.
Proof
Proof.Let W be a random variable that is equal to the total weight of thesatisfied clauses. Define an indicator random variable Yj for eachclause Cj such that Yj = 1 if and only if Cj is satisfied. Then
W =
m∑j=1
wjYj
We use OPT to denote value of optimum solution, then
E[W ] =
m∑j=1
wjE[Yj] =
m∑j=1
wj · Pr[clause Cj satisfied]
Proof (cont’d)
Since each variable is set to true independently, we have
Pr[clause Cj satisfied] =
(1−
(12
)lj)≥ 1
2
where lj is the number of literals in clause Cj. Hence,
E[W ] ≥ 12
m∑j=1
wj ≥12
OPT .
A Finer Analysis
Observe that if lj ≥ k for each clause j, then the analysis above showsthat the algorithm is a (1− ( 1
2)k)-approximation algorithm for suchinstances. For instance, the performance guarantee of MAX E3SAT is7/8.
From the analysis, we can see that the performance of the algorithm isbetter on instances consisting of long clauses.
TheoremIf there is an (7
8 + ε)-approximation algorithm for MAX E3SAT forany constant ε > 0, then P = NP.
Derandomization
Derandomization
The previous randomized algorithm can be derandomized. Note that
E[W ] = E[W | x1 ← true] · Pr[x1 ← true]
+ E[W | x1 ← false] · Pr[x1 ← false]
=12
(E[W | x1 ← true] + E[W | x1 ← false])
We set b1 true if E[W | x1 ← true] ≥ E[W | x1 ← false] and setb1 false otherwise. Let the value of x1 be b1.
Continue this process until all bi are found, i.e., all n variables havebeen set.
An Example
x3 ∨ x5 ∨ x7
• Pr[clause satisfied | x1 ← true, x2 ← false, x3 ← true] = 1• Pr[clause satisfied | x1 ← true, x2 ← false, x3 ← false] = 1−( 1
2 )2 = 3
4
Derandomization
This is a deterministic 12 -approximation algorithm because of the
following two facts:
1 E[W | x1 ← b1, . . . , xi ← bi] can be computed in polynomialtime for fixed b1, . . . , bi.
2 E[W | x1 ← b1, . . . , xi ← bi, xi+1 ← bi+1] ≥ E[W | x1 ←b1, . . . , xi ← bi] for all i, and by induction,E[W | x1 ← b1, . . . , xi ← bi, xi+1 ← bi+1] ≥ E[W ].
Flipping Biased Coins
Flipping Biased Coins
• Previously, we set each xi true or false with probability 12
independently. 12 is nothing special here.
• In the following, we set each xi true with probability p ≥ 12 .
• We first consider the case that no clause is of the form Cj = xi.
LemmaIf each xi is set to true with probability p ≥ 1/2 independently, thenthe probability that any given clause is satisfied is at leastmin(p, 1− p2) for instances with no negated unit clauses.
Proof
Proof.• If the clause is a unit clause, then the probability the clause is
satisfied is p.• If the clause has length at least two, then the probability that the
clause is satisfied is 1− pa(1− p)b, where a is the number ofnegated variables and b is the number of unnegated variables.Since p > 1
2 > 1− p, this probability is at least 1− p2.
Flipping Biased Coins
Armed with previous lemma, we then maximize min(p, 1− p2),which is achieved when p = 1− p2, namely p = 1
2(√
5− 1) ≈ 0.618.
We need more effort to deal with negated unit clauses, i.e., Cj = xi forsome j.
We distinguish between two cases:
1. Assume Cj = xi and there is no clause such that C = xi. In thiscase, we can introduce a new variable y and replace theappearance of xi in ϕ by y and the appearance of xi by y.
Flipping Biased Coins2. Cj = xi and some clause Ck = xi. W.L.O.G we assume
w(Cj) ≤ w(Ck). Note that for any assignment, Cj and Ck cannotbe satisfied simultaneously. Let vi be the weight of the unit clausexi if it exists in the instance, and let vi be zero otherwise, we have
OPT ≤m∑
j=1
wj −n∑
i=1
vi
We set each xi true with probability p = 12(√
5− 1), then
E[W ] =
m∑j=1
wjE[Yj]
≥ p ·
m∑j=1
wj −n∑
i=1
vi
≥ p · OPT
Rounding by Linear Programming
The Use of Linear Program
Integer Program Characterization: Linear Program Relaxation:
max∑m
j=1 wjzj∑i∈Pj
yi +∑
i∈Nj(1− yi) ≥ zj, ∀Cj =
∨i∈Pj
xi ∨∨
i∈Njxi,
yi ∈ {0, 1}0 ≤ yi ≤ 1, i = 1, . . . , n,zj ∈ {0, 1}0 ≤ zj ≤ 1, j = 1, . . . ,m.
where yi indicate the assignment of variable xi and zj indicateswhether clause Cj is satisfied.
Flipping Different Coins
• Let (y∗, z∗) be an optimal solution of the linear program.• We set xi to true with probability y∗i .• This can be viewed as flipping different coins for every variable.
Randomized rounding gives a randomized (1− 1e )-approximation
algorithm for MAX-SAT.
Analysis
Pr[clause Cj not satisfied]
=∏i∈Pj
(1− y∗i )∏i∈Nj
y∗i
≤[
1lj
(∑i∈Pj
(1− y∗i ) +∑
i∈Njy∗i)]lj Arithmetic-Geometric
Mean Inequality
=
1− 1lj
∑i∈Pj
y∗i +∑i∈Nj
(1− y∗i )
lj
≤
(1−
z∗jlj
lj)
Analysis
Pr[clause Cj satisfied]
≥ 1−(
1−z∗jlj
)lj
≥[
1−(
1− 1lj
)lj]
z∗j Jensen’s Inequality
Therefore, we haveE[W ] =
m∑j=1
wj Pr[clause Cj satisfied]
≥m∑
j=1
wjz∗j
[1−
(1− 1
lj
)lj]
≥(
1− 1e
)· OPT
The Combined Algorithm
Choosing the Better of Two
• The randomized rounding algorithm performs better when lj-sare small. (
(1− 1
k
)k is nondecreasing)• The unbiased randomized algorithm performs better when lj-s
are large.• We will combine them together.
Choosing the better of the two solutions given by the two algorithmsyields a randomized 3
4 -approximation algorithm for MAX SAT.
Analysis
Let W1 and W2 be the r.v. of value of solution of randomized roundingalgorithm and unbiased randomized algorithm respectively. Then
E[max(W1,W2)] ≥ E[12
W1 +12
W2]
≥ 12
m∑j=1
wjz∗j
[1−
(1− 1
lj
)lj]
+12
m∑j=1
wj(1− 2−lj
)≥
m∑j=1
wjz∗j
[12
(1−
(1− 1
lj
)lj)
+12(1− 2−lj
)]
≥ 34· OPT
Primal-Dual Schema
Vertex Cover, Revisited Again
Cardinality Vertex Cover
AlgorithmFind a maximal matching in G and output the set of matchedvertices.
(b) (c)
The algorithm
Consider the following algorithm:
1. M ← ∅2. S ← ∅3. while G is not empty do
3.1 Choose an edge e = {u, v} ∈ E(G) and let M ← M ∪ {e}3.2 S ← S ∪ {u, v}3.2 G← G[V \ {u, v}]
4. return S
Cardinality Vertex Cover, Revisit
min∑v∈V
xv
xu + xv ≥ 1, (u, v) ∈ E
xv ≥ 0, v ∈ V
max∑e∈E
ye∑e=(u,v)∈E
ye ≤ 1, u ∈ V
ye ≥ 0, e ∈ E
Vertex Cover
• Vertex Cover Problem is the special case of set cover problemwhen f = 2.• The dual of vertex cover problem is Maximum Matching
Problem.• The duality theorem implies
maximum matching ≤ minimum vertex cover
The algorithm
Consider the following algorithm:
1. M ← ∅2. S ← ∅3. while G is not empty do
3.1 Choose an edge e = {u, v} ∈ E(G) and let M ← M ∪ {e}3.2 S ← S ∪ {u, v}3.2 G← G[V \ {u, v}]
4. return S
This is the combinatorial interpretation of a primal-dual algorithm.
Overview of the Schema
Primal and Dual
min
n∑j=1
cjxj
n∑j=1
aijxj ≥ bi, i = 1, 2, . . . ,m
xj ≥ 0, j = 1, 2, . . . , n
max
m∑i=1
biyi
m∑i=1
aijyi ≤ ci, j = 1, 2, . . . , n
yi ≥ 0, j = 1, 2, . . . ,m
Complementary Slackness
Let x and y be primal and dual feasible solutions, respectively. Then, xand y are both optimal iff all of the following conditions are satisfied:• Primal complementary slackness conditions:
For each 1 ≤ j ≤ n: either xj = 0 or∑m
i=1 aijyi = cj; and• Dual complementary slackness conditions:
For each 1 ≤ i ≤ m: either yi = 0 or∑n
j=1 aijxj = bi.
DiscussionThe theorem ensures
xj > 0 =⇒m∑
i=1
aijyi = cj
In general, we cannot hope
yi > 0 =⇒n∑
j=1
aijxj = bi
We want to show it is not too slack, i.e.
yi > 0 =⇒n∑
j=1
aijxj ≤ β · bi
Complementary SlacknessRelaxation
Let x and y be primal and dual feasible solutions, respectively. Then, xand y are both optimal iff all of the following conditions are satisfied:• Primal complementary slackness conditions:
let α ≥ 1For each 1 ≤ j ≤ n: either xj = 0 or cj/α ≤
∑mi=1 aijyi ≤ cj; and
• Dual complementary slackness conditions:Let β ≥ 1For each 1 ≤ i ≤ m: either yi = 0 or bi ≤
∑nj=1 aijxj ≤ β · bi.
Primal-Dual Schema
Theorem If x and y are primal and dual feasible solutions satisfyingthe conditions stated in the previous page, then
n∑j=1
cjxj ≤ α · βm∑
i=1
biyi
Proof.
n∑j=1
cjxj ≤ αn∑
j=1
(
m∑i=1
aijyi)xj = α
m∑i=1
(
n∑j=1
aijxj)yi ≤ αβm∑
i=1
biyi
• If primal conditions are ensured, we set α = 1• If dual conditions are ensured, we set β = 1.
Primal-Dual ApproximationAlgorithms
1. Formulate a given problem as an IP. Relax the variableconstraints to obtain the primal LP P , then find the dual D.
2. Starts with a primal infeasible solution x and a dual feasiblesolution y, usually x = 0 and y = 0.
3. Until x is feasible, do3.1. Increase the value yi in some fashion until some dual constraints
go tight.3.2. Select some subset of tight dual constraints and increase the
values of primal variables corresponding to these constraints by anintegral amounts. This ensures that the final solution x is integral.
4. The cost of the dual solution is used as a lower bound on OPT .Note that the approximation guarantee of the algorithm is αβ.
Applied to Set Cover
Primal-Dual to Set Cover
min∑S∈S
c(S)xS∑S:e∈S
xS ≥ 1, e ∈ U
xS ≥ 0, S ∈ S
max∑e∈U
ye∑e:e∈S
ye ≤ c(S), S ∈ S
ye ≥ 0, e ∈ U
α = 1 and β = f.
Primal Condition
Primal conditions
∀S ∈ S : xS 6= 0⇒∑e:e∈S
ye = c(S)
• Set S will be said to be tight if∑
e:e∈S ye = c(S).• Since we will increment the primal variables integrally, we can
state the conditions as:• Pick only tight sets in the cover.
Dual Condition
Dual conditions
∀e ∈ U : ye 6= 0⇒∑
S:e∈S
xS ≤ f
• Since we will find a 0/1 solution for x, these conditions areequivalent to:• Each element having a nonzero dual value can be covered at
most f times.
The Algorithm
1. Initialization: x ← 0, y← 02. Until all elements are covered, do:
2.1 Pick an uncovered element, say e, and raise ye until some set goestight.
2.2 Pick all tight sets in the cover and update x.2.3 Declare all the elements occurring in these sets as “covered”.
3. Output the set cover x.
The Guarantee Factor
TheoremThe algorithm achieves an approximation factor of f .
Proof.Clearly there will be no uncovered elements and no overpacked sets atthe end of the algorithm. Thus, the primal and dual solutions will bothbe feasible. Since they satisfy the relaxed complementary slacknessconditions with α = 1 and β = f .
Referred Materials
Content of this lecture comes from Chapter 14 and 15 Chapter 16 in[Vaz04], and Section 5.1-5.5 7.1-7.2 in [WS11].
Related Documents
