On the Unique Games Conjecture Subhash Khot Georgia Inst. Of Technology. At FOCS 2005
Feb 02, 2016
On the Unique Games Conjecture
Subhash Khot Georgia Inst. Of Technology.
At FOCS 2005
NP-hard Problems
• Vertex Cover• MAX-3SAT• Bin-Packing • Set Cover • Clique • MAX-CUT • ……………..• ……………..
Approximability : Algorithms
A C-approximation algorithm computes (C > 1), for problem instance I , solution A(I) s.t.
Minimization problems :
A(I) C OPT(I)
Maximization problems :
A(I) OPT(I) / C
Some Known Approximation Algorithms
• Vertex Cover 2 - approx.
• MAX-3SAT 8/7 - approx. Random assignment. • Packing/Scheduling (1+) – approx. > 0
(PTAS)
• Set Cover ln n approx.
• Clique n/log n [Boppana Halldorsson’92] • Many more , ref. [Vazirani’01]
PCP Theorem
[B’85, GMR’89, BFL’91, LFKN’92, S’92,……] [PY’91] [FGLSS’91, AS’92 ALMSS’92]
Theorem : It is NP-hard to tell whether a MAX-3SAT instance is * satisfiable (i.e. OPT = 1) or * no assignment satisfies more than 99%
clauses (i.e. OPT 0.99).
i.e. MAX-3SAT is 1/0.99 = 1.01 hard to approximate.
i.e. MAX-3SAT and MAX-SNP-complete problems [PY’91] have no PTAS.
Approximability : Towards Tight Hardness Results
• [Hastad’96] Clique n1-
• [Hastad’97] MAX-3SAT 8/7 -
• [Feige’98] Set Cover (1- ) ln n
[Dinur’05] Combinatorial Proof of PCP Theorem !
Open Problems in Approximability
– Vertex Cover (1.36 vs. 2) [DinurSafra’02]
– Coloring 3-colorable graphs (5 vs. n3/14) [KhannaLinialSafra’93, BlumKarger’97]– Sparsest Cut (1 vs. (logn)1/2) [AroraRaoVazirani’04]– Max Cut (17/16 vs 1/0.878… )
[Håstad’97, GoemansWilliamson’94] ………………………..
Unique Games Conjecture [Khot’02]
Implies these hardness results : • Vertex Cover 2- [KR’03]
• Coloring 3-colorable (1) [DMR’05]
graphs (variant of UGC)
• MAX-CUT 1/0.878.. - [KKMO’04]
• Sparsest Cut, Multi-cut [KV’05,
(1) CKKRS’04]
Min-2SAT-Deletion [K’02, CKKRS’04]
Unique Games Conjecture
Led to …
[MOO’05] Majority Is Stablest Theorem
[KV’05] “Negative type” metrics do not embed into L1 with O(1) “distortion”.
Optimal “integrality gap” for MAX-CUT
SDP with “Triangle Inequality”.
Integrality Gap : Definition Given : Maximization Problem + Specific SDP relaxation.
• For every problem instance G,
SDP(G) OPT(G)
• Integrality Gap = Max G SDP(G) / OPT(G)
• Constructing gap instance = negative result.
Overview of the talk
• The UGC • Hardness of Approximation Results• I hope UGC is true • Attempts to Disprove : Algorithms
Connections/applications : • Fourier Analysis • Integrality Gaps• Metric Embeddings
Unique Games Conjecture
• A maximization problem called “Unique Game” is hard to approximate.
• “Gap-preserving” reductions from Unique Game Hardness results for Vertex Cover,
MAX-CUT, Graph-Coloring, …..
Example of Unique GameOPT = max fraction of equations that can be satisfied by any assignment. x1 + x3 = 2 (mod k)
3 x5 - x2 = -1 (mod k)
x2 + 5 x1 = 0 (mod k)
UGC For large k, it is NP-hard to tell whether OPT 99% or OPT 1%
2-Prover-1-Round Game (Constraint Satisfaction
Problem )
variables
constraints
2-Prover-1-Round Game (Constraint Satisfaction
Problem )
variables
k labelsHere k=4
constraints
2-Prover-1-Round Game (Constraint Satisfaction
Problem )
variables
k labelsHere k=4
Constraints = Bipartite graphsor Relations [k] [k]
2-Prover-1-Round Game (Constraint Satisfaction
Problem )
variables
k labelsHere k=4
OPT(G) = 7/7
Find a labeling that satisfies max # constraints
Hardness of Finding OPT(G)
• Given a 2P1R game G, how hard is it to find OPT(G) ? • PCP Theorem + Raz’s Parallel Repetition Theorem
:
For every , there is integer k(), s.t. it is NP-hard to tell whether a 2P1R game with k = k() labels has OPT = 1 or OPT
In fact k = 1/poly()
Reductions from 2P1R Game
• Almost all known hardness results (e.g. Clique, MAX-3SAT, Set Cover, SVP, …. ) are reductions from 2P1R games.
• Many special cases of 2P1R games are known to be hard, e.g. Multipartite graphs,
Expander graphs, Smoothness property, ….
What about unique games ?
Unique Game = 2P1R Game with
Permutationsvariable
k labelsHere k=4
Unique Game = 2P1R Game with Permutations
variable
k labelsHere k=4
Permutations or matchings : [k] [k]
OPT(G) = 6/7
Find a labeling that satisfies max # constraints
Unique Game = 2P1R Game with Permutations
Unique Games Considered before …… [Feige Lovasz’92] Parallel Repetition of UG reduces OPT(G).
How hard is approximating OPT(G) for a unique game G ?
Observation : Easy to decide whether OPT(G) = 1.
MAX-CUT is Special Case of Unique Game
• Vertices : Binary variables x, y, z, w, …….
• Edges : Equations x + y = 1 (mod 2)
• [Hastad’97] NP-hard to tell whether OPT(MAX-CUT) 17/21 or OPT(MAX-CUT) 16/21
Unique Games Conjecture
For any , , there is integer k(, ), s.t. it is NP-hard to tell whether a UniqueGame with k = k(, ) labels has OPT 1- or OPT
i.e. Gap-Unique Game (1- , ) is NP-hard.
Overview of the talk
• The UGC • Hardness of Approximation Results• I hope UGC is true • Attempts to Disprove : Algorithms
Connections/applications : • Fourier Analysis • Integrality Gaps• Metric Embeddings
Case Study : MAX-CUT
• Given a graph, find a cut that maximizes fraction of edges cut.
• Random cut : 2-approximation.
• [GW’94] SDP-relaxation and rounding. min 0 < < 1 / (arccos (1-2) / ) = 1/0.878 … approximation.
• [KKMO’04] Assuming UGC, MAX-CUT is 1/0.878… - hard to approximate.
Reduction to MAX-CUT Unique Game Graph H
• Completeness : OPT(UG) > 1-o(1) - o(1) cut.
• Soundness : OPT(UG) < o(1) No cut with size arccos (1-2) / + o(1)
• Hardness factor = / (arccos (1-2) / ) - o(1)
• Choose best to get 1/0.878 … (= [GW’94])
Reduction from Unique Game
Gadget constructed via Fourier theorem + Connecting gadgets via Unique Game instance
[DMR’05] “UGC reduces the analysis of the entire construction to the analysis of the gadget”.
Gadget = Basic gadget ---> Bipartite gadget ---> Bipartite gadget with permutation
Basic Gadget
A graph on {0,1} k with specific properties
(e.g. cuts, vertex covers, colorability)
{0,1} k
k = # labelsx = 011
Y = 110
Basic Gadget : MAX-CUT Weighted graph, total edge weight = 1. Picking random edge : x R {0,1} k
y <-- flip every co-ordinate of x with
probability ( 0.8)
x
{0,1} k
y
MAX-CUT Gadget : Co-ordinate Cut Along Dimension i
Fraction of edges cut = Pr(x,y) [xi yi ]
=
Observation : These are the maximum cuts.
xi = 0 xi = 1
Bipartite Gadget
A graph on {0,1} k {0,1} k (double cover of basic
gadget)x = 011
y’ = 110
Cuts in Bipartite Gadget {0,1} k {0,1} k
Matching co-ordinate cuts have size =
Bipartite Gadget with Permutation : [k] -> [k] Co-ordinates in second hypercube permuted via
.
x = 011
Y ’ = 110
(y’) = 011
Example : = reversal of co-ordinates.
Reduction from Unique Game
Variables
k labels
OPT 1 – o(1)or OPT o(1)
Permutations : [k] [k]
Instance H of MAX-CUT
{0,1} k
Vertices
Edges
Bipartite Gadgetvia
Proving Completeness
Unique Game Graph H
(Completeness) : OPT(UG) > 1-o(1) H has - o(1) cut.
Completeness : OPT(UG) 1-o(1)
label = 2
label = 1
label = 3
label = 1label = 1
label = 3
label = 2Labels = [1,2,3]
Completeness : OPT(UG) 1-o(1)
{0,1} k
Vertices
Edges
Hypercubes are cut along dimensions = labels.
MAX-CUT - o(1)
Proving Soundness
Unique Game Graph H
(Soundness) : OPT(UG) < o(1) H has no cut of size arccos (1-2) / + o(1)
MAX-CUT Gadget
Cuts = Boolean functions f : {0,1} k {0,1}
Compare boolean functions * that depend only on single co-ordinate
vs * where every co-ordinate has negligible “influence” (i.e. “non-junta” functions)
{0,1} k
x
y
f(x1 x2 …….. xk) = xi
f(x1 x2 …….. xk) = MAJORITY Influence (i, f) = Prx [ f(x) f(x+ei) ]
Gadget : “Non-junta” Cuts
How large can non-junta cuts be ? i.e. cuts with all influences negligible ? Random Cut : ½ Majority Cut : arccos (1-2) / > ½
• [MOO’05] Majority Is Stablest (Best) Any cut slightly better than Majority Cut must have “influential” co-ordinate.
Non-junta Cuts in Bipartite Gadget
[MOO’05] Any “special” cut with value arccos (1-2) / + must define a matching pair of influential co-ordinates.
{0,1} k {0,1} k
Non-junta Cuts in Bipartite Gadget
{0,1} k {0,1} k
f : {0,1} k --> {0, 1}
g : {0,1} k --> {0, 1}
i Infl (i, f), Infl (i, g) > (1)
cut > arccos (1-2) / +
Instance H of MAX-CUT
{0,1} k
Vertices
Edges
Bipartite Gadgetvia
Proving Soundness
• Assume arccos (1-2) / + cut exists.
• On /2 fraction of constraints, the bipartite gadget has arccos (1-2) / + /2 cut.
matching pair of labels on this constraint.
This is impossible since OPT(UG) = o(1).
Done !
Other Hardness Results• Vertex Cover Friedgut’s Theorem Every boolean function with low “average sensitivity” is a junta.
• Sparsest Cut, Min-2SAT Deletion KahnKalaiLinial Every balanced boolean function has a
co-ordinate with influence log n/n.
Bourgain’s Theorem (inspired by Hastad-Sudan’s 2-bit Long Code test)
Every boolean function with low “noise sensitivity” is a junta.
• Coloring 3-Colorable [MOO’05] inspired. Graphs
Basic Paradigm by [BGS’95,
Hastad’97] Hardness results for Clique, MAX-3SAT, ……. • Instead of Unique Games, use reduction from general 2P1R Games (PCP Theorem + Raz).
• Hypercube = Bits in the Long Code [Bellare
Goldreich Sudan’95]
• PCPs with 3 or more queries (testing Long Code).
• Not enough to construct 2-query PCPs.
Why UGC and not 2P1R Games?
Power in simplicity. “Obvious” way of encoding a
permutation constraint. Basic Gadget ----> Bipartite Gadget with permutation.
Overview of the talk
• The UGC • Hardness of Approximation Results• I hope UGC is true • Attempts to Disprove : Algorithms
Connections/applications : • Fourier Analysis • Integrality Gaps• Metric Embeddings
I Hope UGC is True• Implies all the “right” hardness results in a unifying way.
• Neat applications of Fourier theorems [Bourgain’02, KKL’88, Friedgut’98, MOO’05]
• Surprising application to theory of metric embeddings and SDP-relaxations [KV’05].
• Mere coincidence ?
Supporting Evidence
[Feige Reichman’04] Gap-Unique Game (C, ) is NP-hard.
i.e. For every constant C, there is s.t. it is NP-hard to tell if a UG has OPT > C or OPT < .
However C --> 0 as --> 0.
Supporting Evidence
[Khot Vishnoi’05]
SDP relaxation for Unique Game
has integrality gap (1- , ).
Overview of the talk
• The UGC • Hardness of Approximation Results• I hope UGC is true • Attempts to Disprove : Algorithms
Connections/applications : • Fourier Analysis • Integrality Gaps• Metric Embeddings
Disproving UGC means ..
For small enough (constant) , given a UG with optimum 1- , algorithm that finds a labeling
satisfying (say) 50% constraints.
Algorithmic Results
Algorithm that finds a labeling satisfying f(, k, n) fraction of constraints.
[Khot’02] 1- 1/5 k2 [Trevisan’05] 1- 1/3 log1/3 n [Gupta Talwar’05] 1- log n [CMM’05] 1/k , 1- 1/2 log1/2 k
None of these disproves UGC.
Quadratic Integer Program For Unique Game [Feige
Lovasz’92] variable
k labels
: [k] [k]
u1 , u2 , … , uk {0,1}
v1 , v2 , … , vk {0,1}
u
v
vi = 1 if Label(v) = i = 0 otherwise
Quadratic Program for Unique Games
Constraints on edge-set E.
• Maximize ui vπ(i)
(u, v) E i=1,2,..,k
• u i [k], ui {0,1}
• u ui2 = 1
i
• u i ≠ j , ui uj = 0
SDP Relaxation for Unique Games
• Maximize ui, vπ(i)
(u, v) E i=1,2,..,k
• u i [k], ui is a vector.
• u || ui ||2 = 1 i=1,2,..,k
• u i≠j [k], ui, uj = 0
[Feige Lovasz’92]
• OPT(G) SDP(G) 1.
• If OPT(G) < 1, then SDP(G) < 1.
• SDP(Gm) = (SDP(G))m
• Parallel Repetition Theorem for UG : OPT(G) < 1 OPT(Gm) 0
[Khot’02] Rounding Algorithm
u1
uk
u2
vk
v2
v1
r r
Label(u) = 2, Label(v) = 2
Pr [ Label(u) = Label(v) ] > 1 - 1/5 k2
Labeling satisfies 1 - 1/5 k2 fraction of constraints in expected sense.
Random ru v
[CMM’05] Algorithm• Labeling that satisfies 1/k fraction
of constraints. (Optimal [KV’05]) vk
v2
v1
r
u1
uk
u2
r
All i s.t. ui is “close” to r are taken as candidate labels to u.
Pick one of them at random.
[Trevisan’05] Algorithm
• Given a unique game with optimum 1- 1/log n, algorithm finds a labeling that satisfies 50% of constraints.
• Limit on hardness factors achievable via UGC (e.g. loglog n for Sparsest
Cut).
[Trevisan’05] Algorithm
[Leighton Rao’88] Delete a few constraints and
remaining graph has connected
components of low diameter.
Variables and constraints
[Trevisan’05] Algorithm
A good algorithm for graphs with low
diameter.
Overview of the talk
• The UGC • Hardness of Approximation Results• I hope UGC is true • Attempts to Disprove : Algorithms
Connections/applications : • Fourier Analysis • Integrality Gaps• Metric Embeddings
Already Covered
Let’s move on ….
Overview of the talk
• The UGC • Hardness of Approximation Results• I hope UGC is true • Attempts to Disprove : Algorithms
Connections/applications : • Fourier Analysis • Integrality Gaps• Metric Embeddings
[KV’05] Integrality Gaps for
SDP-relaxations • MAX-CUT • Sparsest Cut • Unique Game
Gaps hold for SDPs with “Triangle Inequality”.
Integer Program for MAX-CUT
Given G(V,E)
• Maximize ¼ |vi - vj |2
(i, j) E
• i, vi {-1,1}
• Triangle Inequality (Optional) : i, j , k, |vi - vj |2
+ |vj - vk |2 |vi - v k|2
Goemans-Williamson’s SDP Relaxation for MAX-CUT
• Maximize ¼ || vi - vj ||2
(i, j) E
• i, vi Rn, || vi || = 1
• Triangle Inequality (Optional) : i, j , k, || vi - vj ||2
+ || vj - vk ||2 || vi - v k||2
Integrality Gap for MAX-CUT• [Goemans Williamson’94]
Integrality gap 1/0.878..
• [Karloff’99] [Feige Schetchman ’01]
Integrality gap 1/0.878.. -
SDP solution does not satisfy Triangle Inequality.
Does Triangle Inequality make the SDP tighter ? NO if Unique Games Conj. is true !
Integrality Gap for Unique Games SDP
Unique Game G with
OPT(G) = o(1)
SDP(G) = 1-o(1)
OrthonormalBases for Rk
u1 , u2 ,
… , uk
v1 , v2 ,
… , vkvariables
k labels
Matchings [k] [k]u
v
Integrality Gap for MAX-CUT with
Triangle Inequality
{-1,1}k
u1 , u2 ,
… , uk
u1 u2 u3 ……… uk-1
uk
PCP Reduction
OPT(G) = o(1)
No large cut
Good SDP solution
Overview of the talk
• The UGC • Hardness of Approximation Results• I hope UGC is true • Attempts to Disprove : Algorithms
Connections/applications : • Fourier Analysis • Integrality Gaps• Metric Embeddings
Metrics and Embeddings
• Metric is a distance function on [n] such that
d(i, j) + d(j, k) d(i, k).
• Metric d embeds into metric with distortion 1 if i, j d(i, j) (i, j) d(i, j).
Negative Type Metrics
Given a set of vectors satisfying Triangle Inequality : i, j , k, || vi - vj ||2
+ || vj - vk ||2 || vi - v k||2
d(i, j) = || vi - vj ||2 defines a metric.
These are called “negative type metrics”.
L1 NEG METRICS
NEG vs L1 Question [Goemans, Linial’ 95] Conjecture : NEG metrics embed into
L1
with O(1) distortion.
Sparsest Cut
O(1) Integrality Gap O(1) Approximation
[Linial London Rabinovich’94][Aumann Rabani’98]
Unique Games Conjecture
[Chawla Krauthgamer Kumar Rabani Sivakumar ’05][KV’05]
(1) hardness result
NEG vs L1 Lower Bound
(loglog n) integrality gap for Sparsest
Cut SDP. [KhotVishnoi’05, KrauthgamerRabani’05]
A negative type metric that needs distortion (loglog n) to embed into L1.
Open Problems
• (Dis)Prove Unique Games Conjecture.
• Prove hardness results bypassing UGC.
• NEG vs L1 , Close the gap.
(log log n) vs (log n loglog n) [Arora Lee Naor’04]
Open Problems
• Prove hardness of Min-Deletion version of Unique Games. (log n approx. [GT’05])
• Integrality gaps with “k-gonal” inequalities.
• Is hypercube (Long Code) necessary ?
Open Problems More hardness results, integrality gaps, embedding lower bounds, Fourier Analysis,
……
[Samorodnitsky Trevisan’05] “Gowers Uniformity, Influence of Variables, and PCPs”. UGC Boolean k-CSP is hard to approximate within 2k- log k
Independent Set on degree D graphs is hard to approximate within D/poly(log D).
Open Problems in Approximability Traveling Salesperson
Steiner Tree Max Acyclic Subgraph, Feedback Arc Set Bin-packing (additive approximation) ……………………
Recent progress on Edge Disjoint Paths Network Congestion Shortest Vector Problem Asymmetric k-center (log* n) Group Steiner Tree (log2 n) Hypergraph Vertex Cover ………………
Linear Unique Games System of linear equations mod k. x1 + x3 = 2
3 x5 - x2 = -1
x2 + 5 x1 = 0
[KKMO’04] UGC UGC in the special case of linear equations mod k.
Variations of Conjecture• 2-to-1 Conjecture [K’02]
-Conjecture [DMR’05]
NP-hard to color 3-colorable graphs with O(1) colors.
[k] [k]
[k] [k]