Top Banner
Approximation Algorithms
47

Approximation Algorithms. 2301681Approximation Algorithms2 Outlines Why approximation algorithm? Approximation ratio Approximation vertex cover.

Dec 19, 2015

Download

Documents

Philip Jordan
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

Approximation Algorithms

Page 2: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 2

OutlinesWhy approximation algorithm?Approximation ratioApproximation vertex cover problemApproximation traveling salesman

problem (TSP)Other interesting problems

Page 3: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 3

3 approaches for NP-complete Problems

For small inputs, an exponential algorithm is OK.

Try to solve important special cases in polynomial time.

Find near-optimal solutions in polynomial time (worst case or on average).

An algorithm that returns near-optimal solutions is called an approximation algorithm.

Page 4: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 4

Approximation Ratio

If, for any input of size n for a problem, the cost C of the solution produced by an algorithm is within a factor of ρ(n) of the cost C∗ of an optimal solution (where max(C/C∗ , C∗/C) ≤ ρ(n)), we say that,

the algorithm has an approximation ratio of ρ(n) or

the algorithm is a ρ(n)-approximation algorithm.

Page 5: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 5

On Approximation Ratio maximization problem

0 < C ≤ C ∗

C ∗ /C gives the factor by which the cost of an optimal solution is larger than the cost of the approximate solution.

minimization problem 0 < C ∗ ≤ C C/C ∗ gives the factor by which the cost of the

approximate solution is larger than the cost of an optimal solution.

A 1-approximation algorithm produces an optimal solution.

large approximation ratio => a solution can be much worse than optimal.

Page 6: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 6

Polynomial-time approximation algorithms

may:have small constant approximation

ratios.have approximation ratios that grow as

functions of the input size n.achieve increasingly smaller

approximation ratios by using more and more computation time.

Page 7: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 7

Approximation schemeAn approximation algorithm that takes

as input both an instance of the problem, and a value > 0 such that for any fixed , the

scheme is a (1 + )-approximation algorithm.

Page 8: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 8

Polynomial-time Approximation Scheme

An approximation scheme is a polynomial-time approximation scheme

if for any fixed > 0, the scheme runs in time polynomial in the size n of its input instance.

a fully polynomial-time approximation scheme if it is an approximation scheme and its running

time is polynomial both in 1/ and in the size n of the input instance.

Ex: running time of O((1/)2 n3).

Page 9: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

Approximate Vertex Cover

Page 10: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 10

Vertex Cover A vertex cover of an undirected graph G = (V, E) i

s a subset V’ ⊆V such that if (u, v) is an edge of G, then either u∈V or v∈V.

The size of a vertex cover is the number of vertices in it.

An optimal vertex cover is a vertex cover of minimum size in an undirected graph.

The vertex-cover problem is to find an optimal vertex cover in a given undirected graph.

Page 11: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 11

Example of Vertex Covers

A B

C ED

F

A B

C ED

F

Page 12: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 12

Approximate Vertex CoverAVC(G)C ← E ← E[G]while E

do Choose an arbitrary edge (u,v) from EC ← C ∪ {u, v}E=E-{(x, y)| x=u or x=v or y=u or y=v}

return C

Page 13: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 13

Find Approximted Vertex Covers (1)C ← E ← E[G]while E

do Choose an arbitrary

edge (u,v) from EC ← C ∪ {u, v}E = E - {(x, y)|

x=u or x=v or y=u or y=v} return C

A B

C ED

F

Page 14: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 14

Find Approximted Vertex Covers (2)C ← E ← set of edges in Gwhile E

do Choose an arbitrary

edge (u,v) from EC ← C ∪ {u, v}E = E - {(x, y)|

x=u or x=v or y=u or y=v} return C

c d

e gf

b

a

c d

e gf

b

a

Page 15: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 15

Approximation ratio of AVC (1)AVC is a polynomial-time 2-approximation algorithm.Proof

From the algorithm, AVC runs in O(V+E), which is polynomial time.

Let C* be a set of optimal vertex cover.Let C be the set of vertices returned by AVC.

C is a vertex cover because AVC loops until all edges in the graph is removed.

Next, we will show that |C|/|C*| ≤ 2.

Page 16: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 16

Approximation ratio of AVC (2)

Let A be the set of all edges chosen in the loop.

Since all edges incident on the chosen nodes in C are removed, no two edges in A share an endpoint.

An edge is chosen in the loop when neither of its endpoints are already in C.

Thus, |C|=2|A|.

Page 17: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 17

Approximation ratio of AVC (3)

For any vertex v in C*, there is at least one edge in A with v as an endpoint (because C* is a cover).

And, no two edges in A share an endpoint.

Thus, |A| ≤ |C*|.

Thus, |C|=2|A|≤ 2|C*|.That is, |C|/|C*| ≤ 2.

Thus, AVC has approximation ratio of 2.

Page 18: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

Traveling Salesman Problem

Page 19: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 19

Traveling salesman problem (TSP) Given a complete undirected graph G=(V, E)

that has a nonnegative integer cost c(u, v) associated with each edge (u, v) ∈ E, find a hamiltonian cycle (a tour) of G with minimum cost.

C(A) = c(u, v) (u,v)∈A

TSP is an NP-complete problem.

Page 20: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 20

TSP with triangle inequality The cost function c satisfies the triangle inequalit

y if for all vertices u, v,w ∈ V,

c(u, w) ≤ c(u, v) + c(v, w) .

Example of cost functions that satisfy the triangle i

nequality

Euclidean distance in the plane.

Even with the triangle inequality, TSP is NP-compl

ete.

Page 21: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 21

Approximation TSP w triangle inequality

ATSP(G, c)

Select a vertex r∈V [G] to be a “root” vertex

Compute a minimum spanning tree T for G from root r using MST-PRIM(G, c, r).

Let L be the list of vertices visited in a preorder tree walk of T.

Return the hamiltonian cycle H that visits the vertices in the order L.

Page 22: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 22

Example: Approximate TSP (1)

From

: C

orm

en, R

ivest

, Le

isers

on a

nd S

tein

, In

trodu

ctio

n t

o A

lgori

thm

s, M

IT P

ress

, 2001

.

Graph whose cost is the euclidean distance

Minimum spanning tree with root a.

Page 23: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 23

Example: Approximate TSP (2)

From

: C

orm

en, R

ivest

, Le

isers

on a

nd S

tein

, In

trodu

ctio

n t

o A

lgori

thm

s, M

IT P

ress

, 2001

.

Preorder traversal and preorder walk

Tour obtained from preorder walk

Page 24: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 24

Example: Approximate TSP (3)

From

: C

orm

en, R

ivest

, Le

isers

on a

nd S

tein

, In

trodu

ctio

n t

o A

lgori

thm

s, M

IT P

ress

, 2001

.

Approximate tour Shortest tour

Page 25: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 25

ATSP is 2-approximation algorithm (1)APPROX-TSP-TOUR is a 2-approximation algorithm

for the traveling-salesman problem with the triangle inequality.

Proof Let H* denote an optimal tour for the given set of vertices. If an edge is removed from H*, the result is a spanning tree, say Ts. Thus, c(Ts) ≤ c(H*). Let T be a minimum spanning tree. Then, c(T) ≤ c(Ts). Thus, c(T) ≤ c(H*).

Page 26: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 26

ATSP is 2-approximation algorithm (2)

Let W be a full walk of T.

Then, W lists the vertices when they are first visited and also whenever they are returned to after a visit to a subtree.

Since the full walk traverses every edge of T exactly twice, c(W) = 2c(T ).

From c(T) ≤ c(H*) and c(W) = 2c(T ),

we get c(W) ≤ 2c(H*).

Page 27: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 27

ATSP is 2-approximation algorithm (3)

However, W is not a tour, since it visits some vertices more than once.

By the triangle inequality, we can delete a visit to any vertex from W and the cost does not increase. (If a vertex v is deleted from W between visits to u and w, the resulting ordering specifies going directly from u to w.)

By repeatedly applying this operation, we can remove from W all but the first visit to each vertex.

Page 28: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 28

ATSP is 2-approximation algorithm (4)

Let H be the cycle corresponding to this preorder walk.

H is a hamiltonian cycle, because every vertex is visited exactly once.

Since H is obtained by deleting vertices from the full walk W, c(H) ≤ c(W).

c(H) ≤ 2c(H*) since c(H) ≤ c(W) & c(W) ≤ 2c(H*).

That is, APPROX-TSP-TOUR is a 2-approximation algorithm.

Page 29: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

MAX-3-CNF satisfiability

A randomized approximation algorithm

Page 30: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 30

MAX-3CNF-Satisfiability Given a 3-CNF expression , satisfiability, find an

assignment of the variables that maximizes the number of clauses evaluating to 1.

According to the definition of 3-CNF satisfiability, we require each clause to consist of exactly three distinct literals.

We further assume that no clause contains both a variable and its negation.

We now show that randomly setting each variable to 1 with probability 1/2 and to 0 with probability 1/2 is a randomized 8/7-approximation algorithm.

Page 31: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 31

Randomized MAX-3-CNF-SAT is 8/7-approximate (1)

Given an instance of MAX-3-CNF satisfiability with m clauses and variables x1, x2, . . . , xn, the randomized algorithm that independently sets each variable to 0/1 with prob. 1/2 is a randomized 8/7-approximation algorithm.

Proof

Suppose that we have independently set each variable to 1 with probability 1/2 and to 0 with probability 1/2.

Page 32: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 32

Randomized MAX-3-CNF-SAT is 8/7-approximate (2)

For i = 1, 2, . . . , n, we define the indicator random variable

Yi = I {clause i is satisfied} ,so that Yi = 1 as long as at least one of the literals in the ith clause has been set to 1. Since no literal appears more than once in the same clause, and since we have assumed that no variable and its negation appear in the same clause, the settings of the three literals in each clause are independent.

Page 33: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 33

Randomized MAX-3-CNF-SAT is 8/7-approximate (3)

A clause is not satisfied only if all three of its literals are set to 0, and so Pr {clause i is not satisfied} = (1/ 2)3 = 1/ 8.

Thus, Pr {clause i is satisfied} = 1 − 1/ 8 = 7/ 8.

Therefore, E [Yi ] = 7/ 8.

Let Y be the number of satisfied clauses overall, so that Y = Y1 + Y2 +· · · +Ym. Then, we have

m

E [Y ] = E [ ∑ Yi ] i=1

Page 34: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 34

Randomized MAX-3-CNF-SAT is 8/7-approximate (4)

m

E [Y ] = E [ ∑ Yi ]

m i =1

= ∑ E [Yi ] (by linearity of expectation)

i=1m

= ∑ 7/8 i=1

= 7m/8 .Since m is an upper bound on the number of satisfied clauses, the approximation ratio is at most m/(7m/ 8) = 8/ 7.

Page 35: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

Approximating weighted vertex

cover

Using linear programming

Page 36: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 36

Minimum-weight vertex-cover problem For any vertex cover V′ ⊆ V, we define the weight of the

vertex cover w(V′) = ∑ w(v). vV ′

Given an undirected graph G = (V, E) in which each vertex vV has an associated positive weight w(v), find a vertex cover of minimum weight.

We cannot apply the algorithm used for unweighted vertex cover, nor random solution.

Solution: Use linear programming to compute a lower bound on the weight of

the minimum-weight vertex cover. Round this solution and use it to obtain a vertex cover.

Page 37: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 37

Objective/Constraint for Linear Programming

Minimize w(V′) = ∑ w(v) x(v).

vV ′

subject tox(u) + x(v) ≥ 1 for each v V

(every edge must be covered)

x(v) ≤ 1 for each v V

x(v) ≥ 0 for each v V.

(a vertex is either in or not in the cover)

Page 38: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 38

AMINVC

AMINVC(G,w)

C = Ø

computex, an optimal solution to the linear program

for each v V

do ifx(v) ≥ 1/2

then C = C {v}

return C

Page 39: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 39

The approx.-ratio of AMINVC is 2 (1)

AMINVC is 2-approximation algorithm for the minimum-weight vertex-cover problem.

Proof

Let C* be an optimal solution to the minimum-weight vertex-cover problem.

Let z* be the value of an optimal solution to the linear program.

Since an optimal vertex cover is a feasible solution to the linear program,

z* ≤ w(C*).

Page 40: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 40

The approx.-ratio of AMINVC is 2 (2)

Next, we claim that by rounding the fractional values of the variablesx(v), we produce a set C that is a vertex cover and satisfies w(C) ≤ 2z*.

To see that C is a vertex cover, consider any edge (u, v) E.

Because x(u) + x(v) ≥ 1, at least one ofx(u) andx(v) is at least 1/2.

Then, at least one of u and v will be included in the vertex cover, and so every edge will be covered.

Page 41: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 41

The approx.-ratio of AMINVC is 2 (3)

Now we consider the weight of the cover. We have

z* = ∑ w(v) ·x(v) vV

≥ ∑ w(v)·x(v) ≥ ∑ w(v)·(1/2) vV:x(v)≥1/2 vV:x(v)≥1/2

= ∑ w(v)·(1/2) = (1/2)·∑ w(v) vC vC

= w(C)/2

Thus, w(C) ≤ 2z* ≤ 2w(C*)

AMINVC is a 2-approximation algorithm.

Page 42: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

Subset Sum

Fully polynomial-time approximate scheme

Page 43: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 43

Exponential-time Exact Subset SumEXACT-SUBSET-SUM(S, t)

n ← |S|

L0 ← 0 for i ← 1 to n

do Li ← MERGE-LISTS(Li−1, Li−1 + xi )

remove from Li every element a, a>t

return the largest element in Ln

O(2n)

Max. size of Li is 2n.

Page 44: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 44

ExampleLet S = {1, 4, 5} and t =8.

L0 = 0

L1 = 0, 1 ,

L2 = 0, 1, 4, 5 ,

L3 = 0, 1, 4, 5, 6, 9, 10 .

n ← |S|

L0 ← 0

for i ← 1 to n

do Li ← MERGE(Li−1,Li−1+xi )

Li+1= Li – {a|a Li, a>t}

return the largest element in Ln

Page 45: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 45

fully polynomial-time approximation scheme

Approximation by trimming each list Li after it is created.

If two values in L are close to each other, then one of them can be removed.

Let δ, 0 < δ < 1, be a trimming parameter. x approximates y with trimming parameter δ if y/(1

+ δ) ≤ x ≤ y . To trim a list L by δ

remove an element z from L if there is an element x in L′ which approximates y.

Page 46: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 46

Trimming a list: ExampleLet δ = 0.1 and

L = 10, 11, 12, 15, 20, 21, 22, 23, 24, 29.

To trim L, we obtain

L′ = 10, 12, 15, 20, 23, 29.

11 is represented by 10

21 and 22 are represented by 20

24 is represented by 23.

Page 47: Approximation Algorithms. 2301681Approximation Algorithms2 Outlines  Why approximation algorithm?  Approximation ratio  Approximation vertex cover.

2301681 Approximation Algorithms 47

APPROX-SUBSET-SUMAPPROX-SUBSET-SUM(S, t, )n = |S|

L0 = 0

for i = 1 to n

do Li = MERGE-LISTS(Li−1, Li−1 + xi )

Li = TRIM(Li, /2n)

Li+1= Li – {a| a Li and a>t }

Find the largest value, z*, in Ln

return z*