Approximation Algorithms
2301681 Approximation Algorithms 2
OutlinesWhy approximation algorithm?Approximation ratioApproximation vertex cover problemApproximation traveling salesman
problem (TSP)Other interesting problems
2301681 Approximation Algorithms 3
3 approaches for NP-complete Problems
For small inputs, an exponential algorithm is OK.
Try to solve important special cases in polynomial time.
Find near-optimal solutions in polynomial time (worst case or on average).
An algorithm that returns near-optimal solutions is called an approximation algorithm.
2301681 Approximation Algorithms 4
Approximation Ratio
If, for any input of size n for a problem, the cost C of the solution produced by an algorithm is within a factor of ρ(n) of the cost C∗ of an optimal solution (where max(C/C∗ , C∗/C) ≤ ρ(n)), we say that,
the algorithm has an approximation ratio of ρ(n) or
the algorithm is a ρ(n)-approximation algorithm.
2301681 Approximation Algorithms 5
On Approximation Ratio maximization problem
0 < C ≤ C ∗
C ∗ /C gives the factor by which the cost of an optimal solution is larger than the cost of the approximate solution.
minimization problem 0 < C ∗ ≤ C C/C ∗ gives the factor by which the cost of the
approximate solution is larger than the cost of an optimal solution.
A 1-approximation algorithm produces an optimal solution.
large approximation ratio => a solution can be much worse than optimal.
2301681 Approximation Algorithms 6
Polynomial-time approximation algorithms
may:have small constant approximation
ratios.have approximation ratios that grow as
functions of the input size n.achieve increasingly smaller
approximation ratios by using more and more computation time.
2301681 Approximation Algorithms 7
Approximation schemeAn approximation algorithm that takes
as input both an instance of the problem, and a value > 0 such that for any fixed , the
scheme is a (1 + )-approximation algorithm.
2301681 Approximation Algorithms 8
Polynomial-time Approximation Scheme
An approximation scheme is a polynomial-time approximation scheme
if for any fixed > 0, the scheme runs in time polynomial in the size n of its input instance.
a fully polynomial-time approximation scheme if it is an approximation scheme and its running
time is polynomial both in 1/ and in the size n of the input instance.
Ex: running time of O((1/)2 n3).
2301681 Approximation Algorithms 10
Vertex Cover A vertex cover of an undirected graph G = (V, E) i
s a subset V’ ⊆V such that if (u, v) is an edge of G, then either u∈V or v∈V.
The size of a vertex cover is the number of vertices in it.
An optimal vertex cover is a vertex cover of minimum size in an undirected graph.
The vertex-cover problem is to find an optimal vertex cover in a given undirected graph.
2301681 Approximation Algorithms 12
Approximate Vertex CoverAVC(G)C ← E ← E[G]while E
do Choose an arbitrary edge (u,v) from EC ← C ∪ {u, v}E=E-{(x, y)| x=u or x=v or y=u or y=v}
return C
2301681 Approximation Algorithms 13
Find Approximted Vertex Covers (1)C ← E ← E[G]while E
do Choose an arbitrary
edge (u,v) from EC ← C ∪ {u, v}E = E - {(x, y)|
x=u or x=v or y=u or y=v} return C
A B
C ED
F
2301681 Approximation Algorithms 14
Find Approximted Vertex Covers (2)C ← E ← set of edges in Gwhile E
do Choose an arbitrary
edge (u,v) from EC ← C ∪ {u, v}E = E - {(x, y)|
x=u or x=v or y=u or y=v} return C
c d
e gf
b
a
c d
e gf
b
a
2301681 Approximation Algorithms 15
Approximation ratio of AVC (1)AVC is a polynomial-time 2-approximation algorithm.Proof
From the algorithm, AVC runs in O(V+E), which is polynomial time.
Let C* be a set of optimal vertex cover.Let C be the set of vertices returned by AVC.
C is a vertex cover because AVC loops until all edges in the graph is removed.
Next, we will show that |C|/|C*| ≤ 2.
2301681 Approximation Algorithms 16
Approximation ratio of AVC (2)
Let A be the set of all edges chosen in the loop.
Since all edges incident on the chosen nodes in C are removed, no two edges in A share an endpoint.
An edge is chosen in the loop when neither of its endpoints are already in C.
Thus, |C|=2|A|.
2301681 Approximation Algorithms 17
Approximation ratio of AVC (3)
For any vertex v in C*, there is at least one edge in A with v as an endpoint (because C* is a cover).
And, no two edges in A share an endpoint.
Thus, |A| ≤ |C*|.
Thus, |C|=2|A|≤ 2|C*|.That is, |C|/|C*| ≤ 2.
Thus, AVC has approximation ratio of 2.
2301681 Approximation Algorithms 19
Traveling salesman problem (TSP) Given a complete undirected graph G=(V, E)
that has a nonnegative integer cost c(u, v) associated with each edge (u, v) ∈ E, find a hamiltonian cycle (a tour) of G with minimum cost.
C(A) = c(u, v) (u,v)∈A
TSP is an NP-complete problem.
2301681 Approximation Algorithms 20
TSP with triangle inequality The cost function c satisfies the triangle inequalit
y if for all vertices u, v,w ∈ V,
c(u, w) ≤ c(u, v) + c(v, w) .
Example of cost functions that satisfy the triangle i
nequality
Euclidean distance in the plane.
Even with the triangle inequality, TSP is NP-compl
ete.
2301681 Approximation Algorithms 21
Approximation TSP w triangle inequality
ATSP(G, c)
Select a vertex r∈V [G] to be a “root” vertex
Compute a minimum spanning tree T for G from root r using MST-PRIM(G, c, r).
Let L be the list of vertices visited in a preorder tree walk of T.
Return the hamiltonian cycle H that visits the vertices in the order L.
2301681 Approximation Algorithms 22
Example: Approximate TSP (1)
From
: C
orm
en, R
ivest
, Le
isers
on a
nd S
tein
, In
trodu
ctio
n t
o A
lgori
thm
s, M
IT P
ress
, 2001
.
Graph whose cost is the euclidean distance
Minimum spanning tree with root a.
2301681 Approximation Algorithms 23
Example: Approximate TSP (2)
From
: C
orm
en, R
ivest
, Le
isers
on a
nd S
tein
, In
trodu
ctio
n t
o A
lgori
thm
s, M
IT P
ress
, 2001
.
Preorder traversal and preorder walk
Tour obtained from preorder walk
2301681 Approximation Algorithms 24
Example: Approximate TSP (3)
From
: C
orm
en, R
ivest
, Le
isers
on a
nd S
tein
, In
trodu
ctio
n t
o A
lgori
thm
s, M
IT P
ress
, 2001
.
Approximate tour Shortest tour
2301681 Approximation Algorithms 25
ATSP is 2-approximation algorithm (1)APPROX-TSP-TOUR is a 2-approximation algorithm
for the traveling-salesman problem with the triangle inequality.
Proof Let H* denote an optimal tour for the given set of vertices. If an edge is removed from H*, the result is a spanning tree, say Ts. Thus, c(Ts) ≤ c(H*). Let T be a minimum spanning tree. Then, c(T) ≤ c(Ts). Thus, c(T) ≤ c(H*).
2301681 Approximation Algorithms 26
ATSP is 2-approximation algorithm (2)
Let W be a full walk of T.
Then, W lists the vertices when they are first visited and also whenever they are returned to after a visit to a subtree.
Since the full walk traverses every edge of T exactly twice, c(W) = 2c(T ).
From c(T) ≤ c(H*) and c(W) = 2c(T ),
we get c(W) ≤ 2c(H*).
2301681 Approximation Algorithms 27
ATSP is 2-approximation algorithm (3)
However, W is not a tour, since it visits some vertices more than once.
By the triangle inequality, we can delete a visit to any vertex from W and the cost does not increase. (If a vertex v is deleted from W between visits to u and w, the resulting ordering specifies going directly from u to w.)
By repeatedly applying this operation, we can remove from W all but the first visit to each vertex.
2301681 Approximation Algorithms 28
ATSP is 2-approximation algorithm (4)
Let H be the cycle corresponding to this preorder walk.
H is a hamiltonian cycle, because every vertex is visited exactly once.
Since H is obtained by deleting vertices from the full walk W, c(H) ≤ c(W).
c(H) ≤ 2c(H*) since c(H) ≤ c(W) & c(W) ≤ 2c(H*).
That is, APPROX-TSP-TOUR is a 2-approximation algorithm.
2301681 Approximation Algorithms 30
MAX-3CNF-Satisfiability Given a 3-CNF expression , satisfiability, find an
assignment of the variables that maximizes the number of clauses evaluating to 1.
According to the definition of 3-CNF satisfiability, we require each clause to consist of exactly three distinct literals.
We further assume that no clause contains both a variable and its negation.
We now show that randomly setting each variable to 1 with probability 1/2 and to 0 with probability 1/2 is a randomized 8/7-approximation algorithm.
2301681 Approximation Algorithms 31
Randomized MAX-3-CNF-SAT is 8/7-approximate (1)
Given an instance of MAX-3-CNF satisfiability with m clauses and variables x1, x2, . . . , xn, the randomized algorithm that independently sets each variable to 0/1 with prob. 1/2 is a randomized 8/7-approximation algorithm.
Proof
Suppose that we have independently set each variable to 1 with probability 1/2 and to 0 with probability 1/2.
2301681 Approximation Algorithms 32
Randomized MAX-3-CNF-SAT is 8/7-approximate (2)
For i = 1, 2, . . . , n, we define the indicator random variable
Yi = I {clause i is satisfied} ,so that Yi = 1 as long as at least one of the literals in the ith clause has been set to 1. Since no literal appears more than once in the same clause, and since we have assumed that no variable and its negation appear in the same clause, the settings of the three literals in each clause are independent.
2301681 Approximation Algorithms 33
Randomized MAX-3-CNF-SAT is 8/7-approximate (3)
A clause is not satisfied only if all three of its literals are set to 0, and so Pr {clause i is not satisfied} = (1/ 2)3 = 1/ 8.
Thus, Pr {clause i is satisfied} = 1 − 1/ 8 = 7/ 8.
Therefore, E [Yi ] = 7/ 8.
Let Y be the number of satisfied clauses overall, so that Y = Y1 + Y2 +· · · +Ym. Then, we have
m
E [Y ] = E [ ∑ Yi ] i=1
2301681 Approximation Algorithms 34
Randomized MAX-3-CNF-SAT is 8/7-approximate (4)
m
E [Y ] = E [ ∑ Yi ]
m i =1
= ∑ E [Yi ] (by linearity of expectation)
i=1m
= ∑ 7/8 i=1
= 7m/8 .Since m is an upper bound on the number of satisfied clauses, the approximation ratio is at most m/(7m/ 8) = 8/ 7.
2301681 Approximation Algorithms 36
Minimum-weight vertex-cover problem For any vertex cover V′ ⊆ V, we define the weight of the
vertex cover w(V′) = ∑ w(v). vV ′
Given an undirected graph G = (V, E) in which each vertex vV has an associated positive weight w(v), find a vertex cover of minimum weight.
We cannot apply the algorithm used for unweighted vertex cover, nor random solution.
Solution: Use linear programming to compute a lower bound on the weight of
the minimum-weight vertex cover. Round this solution and use it to obtain a vertex cover.
2301681 Approximation Algorithms 37
Objective/Constraint for Linear Programming
Minimize w(V′) = ∑ w(v) x(v).
vV ′
subject tox(u) + x(v) ≥ 1 for each v V
(every edge must be covered)
x(v) ≤ 1 for each v V
x(v) ≥ 0 for each v V.
(a vertex is either in or not in the cover)
2301681 Approximation Algorithms 38
AMINVC
AMINVC(G,w)
C = Ø
computex, an optimal solution to the linear program
for each v V
do ifx(v) ≥ 1/2
then C = C {v}
return C
2301681 Approximation Algorithms 39
The approx.-ratio of AMINVC is 2 (1)
AMINVC is 2-approximation algorithm for the minimum-weight vertex-cover problem.
Proof
Let C* be an optimal solution to the minimum-weight vertex-cover problem.
Let z* be the value of an optimal solution to the linear program.
Since an optimal vertex cover is a feasible solution to the linear program,
z* ≤ w(C*).
2301681 Approximation Algorithms 40
The approx.-ratio of AMINVC is 2 (2)
Next, we claim that by rounding the fractional values of the variablesx(v), we produce a set C that is a vertex cover and satisfies w(C) ≤ 2z*.
To see that C is a vertex cover, consider any edge (u, v) E.
Because x(u) + x(v) ≥ 1, at least one ofx(u) andx(v) is at least 1/2.
Then, at least one of u and v will be included in the vertex cover, and so every edge will be covered.
2301681 Approximation Algorithms 41
The approx.-ratio of AMINVC is 2 (3)
Now we consider the weight of the cover. We have
z* = ∑ w(v) ·x(v) vV
≥ ∑ w(v)·x(v) ≥ ∑ w(v)·(1/2) vV:x(v)≥1/2 vV:x(v)≥1/2
= ∑ w(v)·(1/2) = (1/2)·∑ w(v) vC vC
= w(C)/2
Thus, w(C) ≤ 2z* ≤ 2w(C*)
AMINVC is a 2-approximation algorithm.
2301681 Approximation Algorithms 43
Exponential-time Exact Subset SumEXACT-SUBSET-SUM(S, t)
n ← |S|
L0 ← 0 for i ← 1 to n
do Li ← MERGE-LISTS(Li−1, Li−1 + xi )
remove from Li every element a, a>t
return the largest element in Ln
O(2n)
Max. size of Li is 2n.
2301681 Approximation Algorithms 44
ExampleLet S = {1, 4, 5} and t =8.
L0 = 0
L1 = 0, 1 ,
L2 = 0, 1, 4, 5 ,
L3 = 0, 1, 4, 5, 6, 9, 10 .
n ← |S|
L0 ← 0
for i ← 1 to n
do Li ← MERGE(Li−1,Li−1+xi )
Li+1= Li – {a|a Li, a>t}
return the largest element in Ln
2301681 Approximation Algorithms 45
fully polynomial-time approximation scheme
Approximation by trimming each list Li after it is created.
If two values in L are close to each other, then one of them can be removed.
Let δ, 0 < δ < 1, be a trimming parameter. x approximates y with trimming parameter δ if y/(1
+ δ) ≤ x ≤ y . To trim a list L by δ
remove an element z from L if there is an element x in L′ which approximates y.
2301681 Approximation Algorithms 46
Trimming a list: ExampleLet δ = 0.1 and
L = 10, 11, 12, 15, 20, 21, 22, 23, 24, 29.
To trim L, we obtain
L′ = 10, 12, 15, 20, 23, 29.
11 is represented by 10
21 and 22 are represented by 20
24 is represented by 23.