Approximation Algorithms: Éva Tardos Cornell University problems, techniques, and their use in game theory
Approximation Algorithms:
Éva TardosCornell University
problems, techniques, and their use in game theory
FOCS 2002 2
What is approximation?
Find solution for an optimization problem guaranteed to have value close to the best possible.
How close?
• additive error: (rare)– E.g., 3-coloring planar graphs is
NP-complete, but 4-coloring always possible
• multiplicative error: -approximation: finds solution
for an optimization problem within an factor to the best possible.
FOCS 2002 3
Why approximate?
• NP-hard to find the true optimum
• Just too slow to do it exactly
• Decisions made on-line
• Decisions made by selfish players
FOCS 2002 4
Outline of talk
Techniques:
• Greedy
• Local search
• LP techniques:• rounding• Primal-dual
Problems:
• Disjoint paths
• Multi-way cut and labeling
•network design, facility location
Relation to Games– local search price of anarchy– primal dual cost sharing
FOCS 2002 5
Max disjoint paths problem
Given graph G, n nodes, m edges, and source-sink pairs.
Connect as many as possible via edge-disjoint path.
t
s t
s s
t
t
s
FOCS 2002 6
Greedy Algorithm
Greedily connect s-t pairs via disjoint paths, if there is a free path using at most m½ edges:
m½ 4
s t
s s
t
t
st
If there is no short path at all, take a single long one.
FOCS 2002 7
Greedy Algorithm
Theorem: m½ –approximation.
Kleinberg’96Proof: One path used can block
m½ better paths
m½ 4
s t
s s
t
t
st
Essentially best possible:
m½- lower bound unless P=NP by [Guruswami, Khanna, Rajaraman, Shepherd, Yannakakis’99]
FOCS 2002 8
Disjoint paths:open problem
Connect as many as pairs possible via paths where 2 paths may share any edge
t
s t
s s
t
t
s
• Same practical motivation
• Best greedy algorithm: n½ - (and also m1/3 -) approximation: Awerbuch, Azar, Plotkin’93.
• No lower bound …
FOCS 2002 9
Outline of talk
Techniques:
• Greedy
• Local search
• LP techniques:• rounding• Primal-dual
Problems:
• Disjoint paths
• Multi-way cut and labeling
•network design, facility location
Relation to Games– local search Price of anarchy– primal dual Cost sharing
FOCS 2002 10
Multi-way Cut Problem
Given: – a graph G = (V,E) ;– k terminals {s1, …, sk}
– cost we for each edge e
Goal: Find a partition that separates terminals, and minimizes the cost
{e separated} we
Separated edgess1
s2
s3
s4
FOCS 2002 11
Greedy Algorithm
For each terminal in turn– Find min cut separating si
from other terminals The first cut
The next cut
s2
s1
s4
s3
s2
s1
s4
s3
FOCS 2002 12
Theorem: Greedy is a2-approximation
Proof: Each cut costs at most the optimum’s cut [Dahlhaus, Johnson, Papadimitriou, Seymour, and Yannakakis’94]
Cuts found by algorithm:
Optimum partition
Selected cuts, cheaper than optimum’s cut, but
each edge in optimum is counted twice.
s4
s3
s2
s1
FOCS 2002 13
Multi-way cuts extension
Given: – graph G = (V,E), we0 for e
E– Labels L={1,…,k} – Lv L for each node v
Objective: Find a labeling of nodes such that each node v assigned to a label in Lv and it minimizes cost {e separated} we
Separated edges
part 1
part 2
part 3
part 4
FOCS 2002 14
Example
Does greedy work? For each terminal in turn
– Find min cut separating si
from other terminals
Blu
e or green
Red o
r gre
en
Red or blue
cheapmediumexpensive
s3
s1
s2
FOCS 2002 15
Greedy doesn’t work
GreedyFor each terminal in turn
– Find min cut separating si
from other terminals
The first two cuts:
Remaining part not valid!
Blu
e or green
Red o
r gre
en
Red or blues2
s1
s3
FOCS 2002 16
Local search
[Boykov Veksler Zabih CVPR’98] 2-approximation
1. Start with any valid labeling.
2. Repeat (until we are tired):
a. Choose a color c.
b. Find the optimal move where a subset of the vertices can be recolored, but only with the color c.
(We will call this a c-move.)
FOCS 2002 17
A possible -move
Thm [Boykov, Vekler, Zabih] The best -move can be found via an (s,t) min-cut
FOCS 2002 18
Idea of the flow networkfor finding a -move
s = all other terminals: retain current color
sc = change color to c =
G
FOCS 2002 19
Theorem: local optimum is a 2-approximation
Partition found by algorithm:
Cuts used by optimum
The parts in optimum each give a possible local move:
FOCS 2002 20
Theorem: local optimum is a 2-approximation
Partition found by algorithm:
Possible move using the optimum
Changing partition does not help current cut cheaper
Sum over all colors:Each edge in optimum counted twice
FOCS 2002 21
Metric labeling classification
open problemGiven:
– graph G = (V,E); we0 for e E– k labels L– subsets of allowed labels Lv
– a metric d(.,.) on the labels.
Objective: Find labeling f(v)Lv for each node v to minimize
e=(v,w) we d(f(v),f(w))
Best approximation known: O(ln k ln ln k) Kleinberg-T’99
FOCS 2002 22
Outline of talk
Techniques:
• Greedy
• Local search
• LP techniques:• rounding• Primal-dual
Problems:
• Disjoint paths
• Multi-way cut and labeling
•network design, facility location
Relation to Games– local search Price of anarchy– primal dual Cost sharing
FOCS 2002 23
Using Linear Programs for multi-way cuts
Using a linear program = fractional cut probabilistic assignment of
nodes to parts
?
Idea: Find “optimal” fractional labeling via linear programming
Label ? as : ½ + ½
FOCS 2002 24
Fractional Labeling
Variables:0 xva 1 p=node, a=label in Lv
– xva fraction of label a
used on node v
Constraints:
xva = 1aLv
for all nodes v V
– each node is assigned to a label
cost as a linear function of x: we ½ |xua - xva |
e=(u,v) aL
FOCS 2002 25
From Fractional x to multi-way cut
The Algorithm (Calinescu, Karloff, Rabani, ’98, Kleinberg-T,’99)
While there are unassigned nodes• select a label a at randomxva
1
u v Unassigned nodes
FOCS 2002 26
The Algorithm (Cont.)
While there are unassigned nodes– select a label a at random
xva
1
u vUnassigned nodes
select 0 1 at randomassign all unassigned nodes v to selected label a if xva
FOCS 2002 27
Why Is This Choice Good?
select 0 1 at randomassign all unassigned nodes v to
selected label a if xva
Note:• Probability of assigning node v to
label a is xva
• Probability of separating nodes u and v in this iteration is |xua – xva |
xpa
1
p qUnassigned nodes
FOCS 2002 28
From Fractional x to Multi-way cut (Cont.)
Theorem: Given a fractional x, we find multi-way cut with expected
separation cost 2 (LP cost of x)
Corollary: if x is LP optimum . 2-approximation
Calinescu, Karloff, Rabani, ’98 1.5 approximation for multi-way cut
(does not work for labeling)
Karger, Klein, Stein, Thorup, Young’99 improved bound 1.3438..
FOCS 2002 29
Outline of talk
Techniques:
• Greedy
• Local search
• LP techniques:• rounding• Primal-dual
Problems:
• Disjoint paths
• Multi-way cut and labeling
•network design, facility location
Relation to Games– local search Price of anarchy– primal dual Cost sharing
FOCS 2002 30
Metric Facility Location
F is a set of facilities (servers).D is a set of clients.
cij is the distance between any i and j in D F.
Facility i in F has cost fi.
client
facility54
2
3
FOCS 2002 31
Problem Statement
We need to:
1) Pick a set S of facilities to open.
2) Assign every client to an open facility (a facility in S).
Goal: Minimize cost of S + p dist(p,S).
client
facility54
2
3
openedfacility
FOCS 2002 32
What is known?
All techniques can be used:• Clever greedy [Jain, Mahdian,
Saberi ’02]
• Local search [starting with Korupolu, Plaxton, and Rajaraman ’98], can handle capacities
• LP and rounding: [starting with Shmoys, T, Aardal ’97]
Here: primal-dual [starting with Jain-Vazirani’99]
FOCS 2002 33
What is the primal-dual method?
• Uses economic intuition from cost sharing– For each requirement, like
aLv xva = 1, someone has to pay to make it true…
• Uses ideas from linear programming:– dual LP and weak
duality– But does not solve linear
programs
FOCS 2002 34
Dual Problem: Collect Fees
Client p has a fee αp (cost-share)
Goal: collect as much as possible max p αp
Fairness: Do no overcharge: for any subset A of clients and any possible facility i we must have
p A [αp – dist(p,i)] fi
amount client p would contribute to building facility i.
FOCS 2002 35
Exact cost-sharing
• All clients connected to a facility
• Cost share αp covers connection costs for each client p
• Costs are “fair”
• Cost fi of selecting a facility i is covered by clients using it
p αp = f(S)+ p dist(p,S) , and
both facilities are fees are optimal
FOCS 2002 36
Approximate cost-sharing
Idea 1: each client starts unconnected, and with fee αp=0
Then it starts raising what it is willing to pay to get connected
• Raise all shares evenly α
Example:
= client
= possible facility with its cost
4 4
4
FOCS 2002 37
Primal-Dual Algorithm (1)
• Each client raises his fee α evenly what it is willing to pay
α = 1
Its α =1 share could be used towards building a connection to either facility
4 4
FOCS 2002 38
Primal-Dual Algorithm (2)
• Each client raises evenly what it is willing to pay
Starts contributing towards facility cost
α = 2
4 4
FOCS 2002 39
Primal-Dual Algorithm (3)
• Each client raises evenly what it is willing to pay
Three clients contributing
α = 3
4 4
FOCS 2002 40
Primal-Dual Algorithm (4)
Open facility, when cost is covered by contributions
4
clients connected to open facility
Open facility
α = 3
4
FOCS 2002 41
Primal-Dual Algorithm: Trouble
Trouble: – one client p connected to
facility i, but contributes to also to facility j
4
Open facility
α = 3
4i j
p
FOCS 2002 42
Primal-Dual Algorithm (5)
Close facility j: will not open this facility.
Will this cause trouble?• Client p is close to both i and j
facilities i and j are at most 2α from each other.
4
Open facility
α = 3
4i j
p
ghost
FOCS 2002 43
4
Primal-Dual Algorithm (6)
Not yet connected clients raise their fee evenly
Until all clients get connected
4
no not need to pay more than 3
Open facility
α =6
α =3
α =3 α =3
ghost
FOCS 2002 44
Feasibility + fairness ??
All clients connected to a facility
Cost share αp covers connection costs of client p
Cost fi of opening a facility i is covered by clients connected to it
• ?? Are costs “fair” ??
FOCS 2002 45
a set of clients A, and any possible facility i we have
p A [αp – dist(p,i)] fi
– Why? we open facility i if there is enough contribution, and do not raise fees any further
But closed facilities are ignored! and may violate fairness
Are costs “fair”??
44
open facility
closed facility, ignored
FOCS 2002 46
Fair till it reaches a “ghost” facility.
Let α’q αq be the fee till a ghost facility is reached
Are costs “fair”??
44
open facility
Closed facility, ignored
cause of closing
ji
p
α’q=4
FOCS 2002 47
Feasibility + fairness ??
All clients connected to a facility
Cost share αp covers connection costs for client p
Cost αp also covers cost of selected a facilities
Costs α’p are “fair”
How much smaller is α’ α ??44
p
FOCS 2002 48
How much smaller is α’ α?
q client met ghost facility j j became a ghost due to client p
qi
p stopped raising its share first
αp α’q αq
Recall dist(i,j) 2 αp, so
αq α’q +2 αp 3α’q
44
p
j
FOCS 2002 49
Primal-dual approximation
The algorithm is a 3-approximation algorithm for the facility location problem
[Jain-Vazirani’99, Mettu-Plaxton’00]
Proof: Fairness of the α’p fees
p α’p min cost [max min]
cost-recovery:f(S) + p dist(p,S) = p αp
α 3α’q
3-approximation algorithm
FOCS 2002 50
Outline of talk
Techniques:
• Greedy
• Local search
• LP techniques:• rounding• Primal-dual
Problems:
• Disjoint paths
• Multi-way cut and labeling
•network design, facility location
Relation to Games– primal dual Cost sharing– local search Price of anarchy
FOCS 2002 51
primal dual Cost sharing
Dual variables αp are natural cost-shares:
Recall: fair = no set is overcharged
= core allocation
p Aαp – dist(p,i) fi for all A and i.
[Chardaire’98; Goemans-Skutella’00] strong connection between core cost-allocation and linear programming dual solutions
See also Shapley’67, Bondareva’63 for other games
FOCS 2002 52
Primal-Dual Cost-sharing
Primal dual = for each requirement someone willing to pay to make it true
Cost-sharing: only players can have shares.
• Not all requirements are naturally associated with individual players.
• Real players need to share the cost.
FOCS 2002 53
primal dual Cost sharing
Fair no subset is overcharged
Stronger desirable property: population monotone (cross-monotone):
Extra clients do not increase cost-shares.
• Spanning-tree game: [Kent and Skorin-Kapov’96 and Jain Vazirani’01]
• Facility location, single source rent-or-buy [Pal-T’02]
FOCS 2002 54
Local search (for facility location)
Local search: simple search steps to improve objective:
• add(s) adds new facility s• delete(t) closes open facility t• swap(s,t) replaces open
facility s by a new facility t
Key to approximation bound:How bad can be a local optima?3-approximation [Charikar, Guha’00]
FOCS 2002 55
Local search Price of anarchy in games
Price of anarchy: facilities are operated by separate selfish agents
Agents open/close facilities when it benefits their own objective.
Agent’s “best response” dynamic:
• Simple local steps analogous to local search.
Price of anarchy: • How bad can be a stable state?• 2-approximation in a related
maximization game: [Vetta’02]
FOCS 2002 56
Conclusions for approximation
Greedy, Local search• clever greedy/local steps can
lead to great results
Primal-dual algorithms• Elegant combinatorial methods• Based on linear programming
ideas, but fast, avoids explicitly solving large linear programs
Linear programming• very powerful tool, but slow to
solve
Interesting connections to issues in game theory