How to Round Any CSP Prasad Raghavendra University of Washington, Seattle David Steurer, Princeton University (In Principle)

How to Round Any CSP

Prasad RaghavendraUniversity of Washington, Seattle

David Steurer,Princeton University

(In Principle)

Constraint Satisfaction ProblemA Classic Example : Max-3-SAT

Given a 3-SAT formula,Find an assignment to the variables that satisfies the maximum number of clauses.

))()()(( 145532532321 xxxxxxxxxxxx Equivalently the

largest fraction of clauses

Variables : {x1 , x2 , x3 ,x4 , x5} Constraints : 4 clauses

Constraint Satisfaction Problem

Instance :• Set of variables.• Predicates Pi applied on variables

Find an assignment that satisfies the largest fraction of constraints.

Problem :

Domain : {0,1,.. q-1}Predicates : {P1, P2 , P3 … Pr}

Pi : [q]k -> {0,1}

Max-3-SAT

Domain : {0,1}Predicates :

P1(x,y,z) = x ѵ y ѵ z

))()()(( 145532532321 xxxxxxxxxxxx

Theorem: [Raghavendra 08]Assuming Unique Games Conjecture, For every CSP, “a simple semidefinite program (SDP1) gives the best approximation computable efficiently.”

[Raghavendra08]A generic rounding scheme for (SDP1) that is optimal for every CSP under UGC.

Independent of UGC, for 2CSPs, the generic rounding scheme for (SDP1) achieves an

Approximation Ratio ≥ (1-²) Integrality Gap of SDP.

Rounding Algorithm

minimumover all instances

=

value of rounded solution

value of SDP solution

rounding – ratioA ( ¦ )(approximation ratio)

≥ (1-²) integrality gap ( ¦ )

=

value of optimal solution

value of SDP solution

minimumover all instances

For any CSP ¦ and any ²>0, there exists an efficient algorithm A,

Unconditionally, the algorithm A as good as all known algorithms for CSPs

Very Simple : No Invariance Principle, Dictatorship Tests, Unique Games.

Drawbacks•Running Time(A) On CSP over alphabet size q, arity k

•No explicit approximation ratio)(2

)/1,,(2 npolyqkpoly

Computing Integrality Gaps

Theorem:

For any CSP ¦ and any ²>0, there exists an algorithm A to compute integrality gap (¦) within an accuracy ²

Running Time(A) On CSP over alphabet size q, arity k

)/1,,(22qkpoly

Previous Work SDP ALGORITHMS[Charikar-Makarychev-Makarychev 06]

MaxCut [Goemans-Williamson] [Charikar-Wirth]

[Lewin-Livnat-Zwick][Charikar-Makarychev-Makarychev 07]

[Hast] [Charikar-Makarychev-Makarychev 07]

[Frieze-Jerrum][Karloff-Zwick]

[Zwick SODA 98][Zwick STOC 98]

[Zwick 99][Halperin-Zwick 01]

[Goemans-Williamson 01][Goemans 01]

[Feige-Goemans][Matuura-Matsui]

[Trevisan-Sudan-Sorkin-Williamson]

[O’Donnell-Wu] Optimal rounding schemes for MaxCut

ALGORITHM OUTLINERounding Any Constraint Satisfaction Problem

Max Cut

10

15

3

7

11

Max CUTInput : A weighted graph G

Find :A cut with maximum fraction of crossing edges

Eji

jiij vvw),(

2||4

1

Semidefinite Program

Variables : v1 , v2 … vn

| vi |2 = 1

Maximize

Max Cut SDP

10

15

3

7

11

1

1

1

-1

-1

-1

-1-1

-1

v1

v2

v3

v4

v5

MaxCut Rounding Problem

Given a graph on the n - dimensional unit ball,Find the maximum cut of the graph.

Approximation using Finite Models

¦-CSP Instance =

¦-CSP Instance =finite

variablefolding

(identifyingvariables)

optimal solution for

=finite

approximate solution

for =

unfolding ofthe assignment

constant time

Challenge: ensure = finite has a good solution

10

15

3

7

11

1

1

-1

-1

-1

-1-1

-1

-1

1

1

11

Approximation using Finite Models

[Frieze-Kannan]For a dense instance =, it is possible to construct finite model

=finite

OPT(=finite) ≥ (1-ε) OPT(=)

General Method for CSPs

What we will do :

SDP value (=finite) > (1-ε)SDP value (=)

PTAS for dense instances

Analysis of Rounding Scheme

¦-CSP Instance =

¦-CSP Instance =finite

SDP value ®

SDP value > ® - ²

OPT value¯

rounded value¯

010001001010001001

Hence: rounding-ratio for = < (1+²) integrality-ratio for = finite

unfolding

CONSTRUCTING FINITE MODELS (MAXCUT)

Rounding Any Constraint Satisfaction Problem

v1

v2

v3

v4

v5

STEP 1 : Dimension Reduction

• Pick d = 1/ Є4 random Gaussian vectors {G1 , G2 , .. Gd} • Project the SDP solution along these directions.Map vector V

V → V’ = (V G∙ 1 , V G∙ 2 , … V G∙ d)v

1

v3v

4 v5

Constant dimensions

STEP 2 : SurgeryScale every vector V’ to unit length

STEP 3 : Discretization•Pick an Є –net for the

d dimensional sphere• Move every vertex to the nearest point in the Є –net

v2v

2

FINITE MODEL Graph on Є –net points

To Show:

SDP value (=finite) > (1-ε)SDP value (=)

Lemma : “Inner Products are almost preserved under random

projections”

If V’,U’ are random projections of U, V on 1/ ε4 directions,

Pr [ |V U – V’ U’| > ∙ ∙ ε] < ε2

STEP 1 : Dimension Reduction•Project the SDP solution along 1/ Є4 random directions.

STEP 2 : SurgeryScale every vector V’ to unit length

STEP 3 : Discretization•Pick an Є –net for the

d dimensional sphere• Move every vertex to the nearest point in the Є –net

For SDP value (=)Contribution of an edge e = (U,V)

|U-V|2 = 2-2 V U ∙

To Show:

SDP value (=finite) > (1-ε)SDP value (=)SDP Vectors for =finite = Corresponding vectors in Є –net

STEP 1With probability > 1- Є2 ,

| |U-V|2 - |U’-V’|2 | < 2Є

STEP 2With probability > 1- 2Є2 ,

1+ Є < |V’|2 ,|U’|2 < 1- Є, Normalization changes distance by at most 2Є

STEP 3Changes edge length by at most 2Є

For SDP value (=)Contribution of an edge e = (U,V)

|U-V|2 = 2-2 V U ∙

To Show:

SDP value (=finite) > (1-ε)SDP value (=)SDP Vectors for =finite = Corresponding vectors in Є –net

STEP 1With probability > 1- Є2 ,

| |U-V|2 - |U’-V’|2 | < 2Є

STEP 2With probability > 1- 2Є2 ,

1+ Є < |V’|2 ,|U’|2 < 1- Є, Normalization changes distance by at most 2Є

STEP 3Changes edge length by at most 2Є

ANALYSISWith probability 1-3Є2,The contribution of edge e changes by < 6Є

In expectation,For (1-3Є2) edges, the contribution of edge e changes by < 6Є

SDP value (=finite) > SDP value (=) - 6Є – 3Є2

FINITE MODELS FOR GENERAL CSPRounding Any Constraint Satisfaction Problem

Semidefinite Program for CSPs

Variables :For each variable Xa

Vectors {V(a,0) , V(a,1)}

For each clause P = (xa ν xb ν xc),Scalar variables

μ(P,000) , μ(P,001) , μ(P,010) , μ(P,100) , μ(P,011) , μ(P,110) , μ(P,101) , μ(P,111)

))()()(( 145532532321 xxxxxxxxxxxx

Xa = 1 V(a,0) = 0 V(a,1) = 1

Xa = 0 V(a,0) = 1 V(a,1) = 0

If Xa = 0, Xb = 1, Xc = 1

μ(P,000) = 0 μ(P,011) = 1μ(P,001) = 0 μ(P,110) = 0μ(P,010) = 0 μ(P,101) = 0μ(P,100) = 0 μ(P,111) = 0

Objective Function :

PClauses sassignment

PP

3}1,0{

),()(

Constraints : For each clause P,

0 ≤μ(P,α) ≤ 1

For each clause P (xa ν xb ν xc), For each pair Xa , Xb in P,

consitency between vector and LP variables.

V(a,0) V∙ (b,0) = μ(P,000) + μ(P,001) V(a,0) V∙ (b,1) = μ(P,010) + μ(P,011) V(a,1) V∙ (b,0) = μ(P,100) + μ(P,101) V(a,1) V∙ (b,1) = μ(P,100) + μ(P,101)

1),(

P

Semidefinite Relaxation for CSPSDP solution for =:

SDP objective:

for every constraint Á in =- local distributions ¹Á over

assignments to the variables of Á

Example of local distr.: Á = 3XOR(x3, x4, x7)

x3 x4 x7 ¹Á0 0 0 0.10 0 1 0.010 1 0 0 …1 1 1 0.6for every variable xi in =

- vectors vi,1 , … , vi,q

constraints

(also for first moments)

Explanation of constraints:first and second moments of distributions are consistent and form PSD matrix

maximize

Strong and WeakSTRENGTHFor every clause Á in =- local distributions ¹Á over assignments to the variables of Á

Vector variables vi,a within a clause Á satisfy all valid constraints (like triangle inequality)

– the inner products are in the integral hull.WEAKNESS

The above hard constraint is only for variables that participate together in a clause

Throwing away constraints

{vi,a } { μ …}

-Infeasible SDP solution for a instance = , it does not satisfy the consistency for a clause P.

Consider instance =‘ = = - {P}

Now {vi,a } { μ … } is a good SDP solution for =‘

Throw away clauses from CSP

Throw away constraints from SDP relaxation

v1

v2

v3

v4

v5v

1

v3v

4 v5

Constant dimensions

v2v

2

FINITE MODEL CSP on Є –net points

STEP 1 : Dimension Reduction•Project the SDP solution along d =1/ Є4 random directions.

STEP 3 : Discretization•Pick an Є –net for the

d dimensional sphere• Move every variable to the nearest point in the Є –net =finite = discretized instance

STEP 2 : Throw away Discard clauses for which the corresponding inner products are not preserved within Є.

=‘ = New instance

To Show:

SDP value (=finite) > (1-ε)SDP value (=)

SDP Vectors for =finite = Corresponding vectors in Є –net

LP variables { μ …}?

Problem :

The inner products of vectors corresponding to a clause P might not be in the integral hull.( For example : 3 arbitrary vectors in a Є –net are not guaranteed to satisfy triangle inequality )

The initial SDP solution satisfied all the constraints

STEP 1 : Dimension Reduction•Project the SDP solution along d =1/ Є4 random directions.

STEP 3 : Discretization•Pick an Є –net for the

d dimensional sphere• Move every variable to the nearest point in the Є –net =finite = discretized instance

STEP 2 : Throw away Discard clauses for which the corresponding inner products are not preserved within Є.

=‘ = New instance

From STEP 2,

We have discarded clauses for which inner products are not preserved within Є

Discarding a clause P

Forget about constraints corresponding to P

Discretization changes inner product by Є

For every remaining clause, all inner products are within 2Є of what it was.

Smoothing Operation

Canonical SDP SolutionUniform Distribution over all Integral solutions.

Example:Va,0 V∙ a,0 = Va,1 V∙ a,1 = ½Va,0 V∙ b,0 = Va,0 V∙ b,1 = Va,1 V∙ b,0 = Va,1 V∙ b,1 = 1/4

Є –net SolutionSDP Vectors for =finite =

Corresponding vectors in Є –net (1-Є) X

+

=Final SDP solution

IntegralHull

Є X

Є

Consider the inner products corresponding to a single clause P

SDP Objective value remains roughly the same.

Conclusions

• Rounding stronger SDPs.

• More efficient rounding? Can this SDP be solved in constant dimensional space directly?

• Integrality gaps for stronger SDP relaxation of Unique Games

Thank You

Good finite Models from SDP solutions – Dimension Reduction & Discretization

¦-CSP Instance =

¦-CSP Instance =finite

SDP solution for =

compute

Dimension Reduction

Project on randomlow dimensional

subspace

almostSDP solution

for =

Discretize

Move vectors to closest point

on ²-net

almostSDP solution

for =

Rn Rd

identify variableswith same vectors

Theorem: SDP value (=finite) > SDP value (=)

Idea: use almostSDP solution and

do surgery

finite number ofdifferent vectrs

Constraint Satisfaction Problems (CSP)CSP ¦

finite set of allowed types of constraints Á : [q]k {0,1} (alphabet [q], arity k)e.g. ¦ = { 3XOR, 3SAT, 3NAE}

¦-CSP Instance =- variables x1,…,xn

- list of constraints Á of type ¦ on subsets of variables

Goal: Find assignment x 2 [q]n so as to maximize fraction of satisfied constraints opt(=)

Examples: Max-Cut, Max-3SAT,…

PCP Theorem: NP-hard to distinguish opt(=)=1 and opt(=)<0.9 (even for constant k and q)

Approximation Algorithms: Goemans-Williamson, Zwick, CMM, …