Transcript
8/12/2019 Discrepancy Berlin
1/64
1/30
Discrepancy and SDPs
Nikhil Bansal (TU Eindhoven)
8/12/2019 Discrepancy Berlin
2/64
Outline
Discrepancy: definitions and applications
Basic results: upper/lower bounds
Partial Coloring method (non-constructive)
SDPs: basic method
Algorithmic Spencers ResultLovett-Meka result
Lower bounds via SDP duality (Matousek)2/30
8/12/2019 Discrepancy Berlin
3/64
Material
Classic: Geometric Discrepancy by J. Matousek
Papers:
Bansal. Constructive algorithms for discrepancy minimization,
FOCS 2010
Matousek. The determinant lower bound is almost tight
Lovett, Meka. Discrepancy minimization by walking on the
edges
Survey with fewer technical details:
Bansal. 3/30
8/12/2019 Discrepancy Berlin
4/64
4/30
Discrepancy: What is it?
Study of gaps in approximating the continuous by the discrete.
Original motivation: Numerical Integration/ Sampling
Problem: How wellcan you approximate a region by discrete points
Discrepancy:Max over intervals I
|(# points in I)(length of I)|
8/12/2019 Discrepancy Berlin
5/64
5/30
Discrepancy: What is it?
Study of gaps in approximating the continuous by the discrete.
Problem: How uniformlycan you distribute points in a grid.
Uniform: For every axis-parallel rectangle R
| (# points in R) - (Area of R) | should be low.
n1/2
n1/2
Discrepancy:
Max over rectangles R|(# points in R)(Area of R)|
8/12/2019 Discrepancy Berlin
6/64
6/30
Distributing points in a grid
Problem: How uniformlycan you distribute points in a grid.
Uniform: For every axis-parallel rectangle R
| (# points in R) - (Area of R) | should be low.
Uniform Random Van der Corput Set
n= 64points
n1/2discrepancy n1/2(loglog n)1/2O(log n)discrepancy!
8/12/2019 Discrepancy Berlin
7/64
Quasi-Monte Carlo Methods
With N random samples: Error \prop 1/\sqrt{n}
Quasi-Monte Carlo Methods: \prop Disc/n
Can discrepancy be O(1) for 2d grid?
No. \Omega(log n) [Schmidt ]
d-dimensions: O(log^{d-1} n) [Halton-Hammersely ]
\Omega(log^{(d-1)/2} n) [Roth ]
\Omega(log^{(d-1)/2 + \eta} n
[Bilyk,Lacey,Vagharshakyan08]
7/30
8/12/2019 Discrepancy Berlin
8/64
8/30
Discrepancy: Example 2
Input: n points placed arbitrarilyin a grid.Color them red/bluesuch that each rectangle is colored as
evenly as possible
Discrepancy: max over rect. R ( | # redin R - #bluein R | )
Continuous: Color each element
1/2 red and 1/2 blue (0 discrepancy)
Discrete:
Random has about O(n1/2log1/2n)
Can achieve O(log2.5n)
8/12/2019 Discrepancy Berlin
9/64
9/30
Combinatorial Discrepancy
Universe:U= [1,,n]
Subsets:S1,S2,,Sm
Color elements red/blue so eachset is colored as evenlyas possible.
Find : [n] !{-1,+1} to
Minimize |(S)|1= maxS| i 2S(i) |
If A is m \times n incidence matrix.
Disc(A) = min_{x \in {-1,1}^n} |Ax|_\infty
S1
S2
S3
S4
8/12/2019 Discrepancy Berlin
10/64
10/30
Applications
CS: Computational Geometry, Comb. Optimization, Monte-Carlosimulation, Machine learning, Complexity, Pseudo-Randomness,
Math: Dynamical Systems, Combinatorics, Mathematical Finance,
Number Theory, Ramsey Theory, Algebra, Measure Theory,
8/12/2019 Discrepancy Berlin
11/64
Hereditary Discrepancy
11/30
8/12/2019 Discrepancy Berlin
12/64
Rounding
Lovasz-Spencer-Vesztermgombi86
Given any matrix A, and x \in R^n
can round x to \tilde{x} \in Z^n s.t.
|AxA\tilde{x}|_\infty < Herdisc(A)
Proof: Round the bits one by one.
12/30
8/12/2019 Discrepancy Berlin
13/64
Can we find it efficiently?
Nothing known until recently.
Thm [B10]. Can efficiently round so thatError \leq O(\sqrt{log m log n})
Herdisc(A)
13/30
8/12/2019 Discrepancy Berlin
14/64
More rounding approaches
Bin Packing
Refined further by Rothvoss (Entropy roundingmethod)
14/30
8/12/2019 Discrepancy Berlin
15/64
Dynamic Data Structures
N points in a 2-d region.
Weights update over time.
Query: Given an axis-parallel rectangle R,
determine the total weight on points in R.
Preprocess:1) Low query time
2) Low update time (upon weight change)
15/30
8/12/2019 Discrepancy Berlin
16/64
Example
Line:
Query = O(n) Update = 1
Query = 1 Update = O(n^2)Query = 2 Update = O(n)
Query = O(log n) Update = O(log n)
Recursively can get for 2-d.
16/30
8/12/2019 Discrepancy Berlin
17/64
What about other objects?
Query
Circles arbitrary rectangles aligned triangle
Turns out t_q t_u \geq n^{1/2}/log^2 n ?
Larsen-Green: t_q t_u \geq disc(S)^n/log^2 n
17/30
8/12/2019 Discrepancy Berlin
18/64
Sketch of idea
A good data structure impliesD = A P
A = row sparse P = Column sparse
(low query time) (low update time)18/30
8/12/2019 Discrepancy Berlin
19/64
Outline again
19/30
8/12/2019 Discrepancy Berlin
20/64
Basic Results
20/30
8/12/2019 Discrepancy Berlin
21/64
21/30
Best Known Algorithm
Random: Color each element i independently as
x(i) = +1 or -1 with probability each.
Thm: Discrepancy = O (n log n)1/2
Pf: For each set, expect O(n1/2)discrepancy
Standard tail bounds: Pr[ | i 2Sx(i) | cn1/2] e-c
2
Union bound + Choose c (log n)1/2
Analysis tight:Random actually incurs ((n log n)1/2
).
8/12/2019 Discrepancy Berlin
22/64
22/30
Better Colorings Exist!
[Spencer 85]: (Six standard deviations suffice)Always exists coloring with discrepancy6n1/2
(In general for arbitrary m, discrepancy = O(n1/2log(m/n)1/2)
Tight: For m=n, cannot beat 0.5 n1/2 (Hadamard Matrix, orthogonal sets)
Inherentlynon-constructiveproof(pigeonhole principle on exponentially large universe)
Challenge:Can we find italgorithmically?
Certain algorithms do not work [Spencer]
Conjecture[Alon-Spencer]:May not be possible.
8/12/2019 Discrepancy Berlin
23/64
23/30
Beck Fiala Thm
U = [1,,n] Sets: S1,S2,,Sm
Suppose each element lies in at most tsets (t
8/12/2019 Discrepancy Berlin
24/64
24/30
Approximating Discrepancy
Question:If a set system has low discrepancy (say
8/12/2019 Discrepancy Berlin
25/64
25/30
Our Results
Thm 1: Can get Spencers bound constructively.
That is, O(n1/2)discrepancy for m=n sets.
Thm 2: If each element lies in at mosttsets, get bound of
O(t1/2log n) constructively (Srinivasans bound)
Thm 3: For any set system, can find
Discrepancy O(log (mn))Hereditary discrepancy.
Other Problems:Constructive bounds (matching current best)
k-permutation problem [Spencer, Srinivasan,Tetali]
Geometric problems ,
8/12/2019 Discrepancy Berlin
26/64
26/30
Relaxations: LPs and SDPs
Not clear how to use.
Linear Programis useless. Can color each element redandblue. Discrepancy of each set = 0!
SDPs (LP on vivj, cannot control dimension of vs)
| i 2Svi|2 n 8S
|vi|2= 1
Intended solution vi= (+1,0,,0) or (-1,0,,0).
Trivially feasible: vi = ei (all vis orthogonal)
Yet, SDPs will be a major tool.
8/12/2019 Discrepancy Berlin
27/64
27/30
Punch line
SDP very helpful if tighter bounds needed for some sets.
|i 2Svi|2 2 n
| i 2S
vi|2 n/log n
|vi|21
Not apriori clear why one can do this.
Entropy Method.
Algorithm will construct coloring over time and
use several SDPs in the process.
Tighter bound for S
8/12/2019 Discrepancy Berlin
28/64
28/30
Talk Outline
Introduction
The Method
Low Hereditary discrepancy -> Good coloring
Additional Ideas
Spencers O(n1/2
) bound
8/12/2019 Discrepancy Berlin
29/64
Partial Coloring Method
29/30
8/12/2019 Discrepancy Berlin
30/64
A Question
30/30
-n n
8/12/2019 Discrepancy Berlin
31/64
Slight improvement
Can be improved to O(\sqrt{n})/2^n
If you pick a random {-1,1} coloring sw.p. say >= |a \cdot s| \leq c \sqrt{n}
2^{n-1} colorings s, with |a\cdot s| \leq c\sqrt{n}
31/30
8/12/2019 Discrepancy Berlin
32/64
Algorithmically
Easy: 1/poly(n) (How?)
Answer: Pick any poly(n) colorings.
[Karmarkar-Karp81]: \approx 1/n^log n
Huge gap: Major open question
Remark: {-1,+1} not enough. Really need
color 0 also. 32/30
8/12/2019 Discrepancy Berlin
33/64
Yet another enhancement
There is a {-1,0,1} coloring with at least
n/2 {-1,1}s s.t. \sum_i a_i s_i \leq n/2^{n/5}
Make buckets of size 2n/2^{n/5}At least 2^{4n/5} sums fall in same bucket
Claim: Some two s and s in same bucket and differ in at
least n/2 coordinates
Again consider s = (s-s)/2
33/30
8/12/2019 Discrepancy Berlin
34/64
Proof of Claim
Claim: Any set of 2^{4n/5} vertices of the
boolean cube has
[Kleitman66] Isoperimetry for cube.
Hamming ball B(v,r) has the smallest
diameter for a given number of vertices.
|B(v,n/4)| < 2^{4n/5}34/30
8/12/2019 Discrepancy Berlin
35/64
Spencers proof
35/30
8/12/2019 Discrepancy Berlin
36/64
36/30
Our Approach
8/12/2019 Discrepancy Berlin
37/64
37/30
Algorithm (at high level)
Cube: {-1,+1}n
Analysis: Few steps to reach a vertex (walk has high variance)
Disc(Si) does a random walk (with low variance)
start
finish
Algorithm: Sticky random walk
Each step generated by rounding a suitable SDP
Move in various dimensions correlated, e.g. t
1+ t
2 0
Each dimension: An Element
Eachvertex: A Coloring
8/12/2019 Discrepancy Berlin
38/64
38/30
An SDP
Hereditary disc. ) the following SDP is feasible
SDP:
Low discrepancy: |i 2Sj
vi|2 2
|vi|2 = 1
Rounding:Pick random Gaussian g =(g1,g2,,gn)
each coordinate giis iid N(0,1)
For each i, consider i = gvi
Obtain vi2Rn
8/12/2019 Discrepancy Berlin
39/64
39/30
Properties of Rounding
Lemma:If g 2Rnis random Gaussian. For any v 2Rn,
g v is distributed as N(0, |v|2)
Pf: N(0,a2) + N(0,b2) = N(0,a2+b2) gv = i v(i) gi N(0, iv(i)2)
1. Each i N(0,1)
2. For each set S,
i 2Si = g (i2Svi) N(0, 2)(std deviation )
SDP:
|vi|2 = 1
|i2Svi|
22
Recall: i = g vi
s mimics a low discrepancy coloring (but is not {-1,+1})
8/12/2019 Discrepancy Berlin
40/64
40/30
Algorithm Overview
Construct coloring iteratively.
Initially: Start with coloring x0= (0,0,0, ,0) at t = 0.
At Time t: Update coloring as xt= xt-1+ (t1,,
tn)
(tiny: 1/n suffices)
x(i)
xt(i) = (1i+ 2i+ + ti)
Color of element i: Does random walk
over time with step size N(0,1)
Fixedif reaches -1 or +1.
time
-1
+1
Set S: xt(S) = i 2S xt(i) does a random walk w/ step N(0,2)
8/12/2019 Discrepancy Berlin
41/64
41/30
Analysis
Consider time T = O(1/2)
Claim 1:With prob. , at least n/2 elementsreach -1 or +1.Pf: Each element doing random walk with size .
Recall: Random walk with step 1, is O(t1/2) away in t steps.
A Trouble:Various element updates are correlated
Consider basic walk x(t+1) = x(t) 1 with prob
Define Energy (t)= x(t)2
E[(t+1)] = (x(t)+1)2+ (x(t)-1)2 = x(t)2+ 1 = (t)+1
Expected energy = n at t= n.
Claim 2: Each set has O() discrepancyin expectation.Pf: For each S, x
t(S) doing random walk with step size
8/12/2019 Discrepancy Berlin
42/64
42/30
Analysis
Consider time T = O(1/2)
Claim 1:With prob. , at least n/2 variablesreach -1 or +1.
) Everything colored in O(log n)rounds.
Claim 2: Each set has O() discrepancyin expectation per round.
) Expected discrepancy of a set at end = O(log n)
Thm: Obtain a coloring with discrepancy O(log (mn))Pf: By Chernoff, Prob. that disc(S) >= 2 Expectation + O(log m)
= O(log (mn))is tiny (poly(1/m)).
8/12/2019 Discrepancy Berlin
43/64
43/30
Recap
At each step of walk, formulate SDP on unfixed variables.
Use some (existential) property to argue SDP is feasible
Rounding SDP solution -> Step of walk
Properties of walk:
HighVariance -> Quick convergence
Lowvariance for discrepancy on sets -> Low discrepancy
8/12/2019 Discrepancy Berlin
44/64
44/30
Refinements
Spencers six std deviations result:
Goal: Obtain O(n1/2)discrepancy for any set system on m = O(n) sets.
Random coloring has n1/2(log n)1/2 discrepancy
Previous approach seems useless:
Expected discrepancy for a set O(n1/2),
but some random walks willdeviateby up to (log n)1/2factor
Need an additional idea to prevent this.
8/12/2019 Discrepancy Berlin
45/64
45/30
Spencers O(n1/2) result
Partial Coloring Lemma:For any system with m sets, there exists acoloring on n/2elements with discrepancy O(n1/2log1/2(2m/n))[For m=n, disc = O(n1/2)]
Algorithm for total coloring:
Repeatedly apply partial coloring lemma
Total discrepancy
O( n1/2 log1/22 ) [Phase 1]
+ O( (n/2)1/2log1/24 ) [Phase 2]
+ O((n/4)1/2log1/2 8 ) [Phase 3]
+ = O(n1/2)
8/12/2019 Discrepancy Berlin
46/64
46/30
Proving Partial Coloring Lemma
Beautiful Counting argument (entropy method + pigeonhole)Idea:Too many colorings (2n), but few discrepancy profiles
Key Lemma: There existk=24n/5colorings X1,,Xksuch that
everytwo Xi, Xj are similar foreveryset S1,,Sn.
Some X1,X2 differ on n/2positions
Consider X = (X1X2)/2
Pf: X(S) = (X1(S)X2(S))/2 2 [-10 n1/2, 10 n1/2]
X1= ( 1,-1, 1 , ,1,-1,-1)
X2= (-1,-1,-1, ,1, 1, 1)X = ( 1, 0, 1 , ,0,-1,-1)
8/12/2019 Discrepancy Berlin
47/64
47/30
A useful generalization
There exists a partial coloring with non-uniformdiscrepancy bound Sfor set S
Even if S = ( n1/2) in some average sense
8/12/2019 Discrepancy Berlin
48/64
48/30
An SDP
Suppose there existspartialcoloring X:
1. On n/2 elements2. Each set S has |X(S)| S
SDP:
Low discrepancy: |i 2Sj vi|
2 S2
Many colors: i |vi|2n/2
|vi|2 1
Pick random Gaussian g =(g1,g2,,gn)
each coordinate giis iid N(0,1)
For each i, consider i = g vi
Obtain vi2Rn
Al i h
8/12/2019 Discrepancy Berlin
49/64
49/30
Algorithm
Initially write SDP with S= c n1/2
Each set S does random walk and expectsto reach
discrepancy of O(S) = O(n1/2)
Some sets will becomeproblematic.
Reducetheir Son the fly.
Not many problematic sets, and entropy penalty low.
0 20n1/2 30n1/2 35n1/2
Danger 1 Danger 2 Danger 3
8/12/2019 Discrepancy Berlin
50/64
50/30
Concluding Remarks
Construct coloring over timeby solving sequence of SDPs(guided by existence results)
Works quite generally
Can be derandomized [Bansal-Spencer]
(use entropy method itself for derandomizing + usual tech.)
E.g. Deterministic six standard deviations can be viewed as a way to
derandomize something stronger than Chernoff bounds.
8/12/2019 Discrepancy Berlin
51/64
51/30
Thank You!
8/12/2019 Discrepancy Berlin
52/64
52/30
8/12/2019 Discrepancy Berlin
53/64
53/30
8/12/2019 Discrepancy Berlin
54/64
54/30
Rest of the talk
1. How to generate i with required properties.
2. How to update Sover time.
Show n1/2(log log log n)1/2bound.
8/12/2019 Discrepancy Berlin
55/64
55/30
Why so few algorithms?
Often algorithms rely on continuous relaxations.
Linear Program is useless. Can color each element
redand blue.
Improved results of Spencer, Beck, Srinivasan,
based on clever counting(entropy method).
Pigeonhole Principle on exponentially large systems(seems inherently non-constructive)
P ti l C l i L
8/12/2019 Discrepancy Berlin
56/64
56/30
Partial Coloring Lemma
Suppose we have discrepancy bound Sfor set S.
Consider 2n possible colorings
Signatureof a coloring X: (b(S1), b(S2),, b(Sm))
Want partial coloring with signature (0,0,0,,0)
8/12/2019 Discrepancy Berlin
57/64
57/30
Progress Condition
Energy increases at each step:
E(t) = \sum_i x_i(t)^2
Initially energy =0, can be at most n.
Expected value of E(t) = E(t-1) + \sum_i
\gamma_i(t)^2
Markovs inequality.
8/12/2019 Discrepancy Berlin
58/64
58/30
Missing Steps
1. How to generate the \eta_i
2. How to update \Delta_S over time
P ti l C l i
8/12/2019 Discrepancy Berlin
59/64
59/30
Partial Coloring
If exist two colorings X1,X2
1. Samesignature (b1,b2,,bm)
2. Differ in at least n/2positions.
Consider X = (X1X2)/21. -1 or 1 on at least n/2positions, i.e.partial coloring
2. Has signature (0,0,0,,0)
X(S) = (X1(S)X2(S)) / 2, so |X(S)| S for all S.
Can show that there are 24n/5colorings with same signature.
So, some twowill differ on > n/2positions. (Pigeon Hole)
X1= (1,-1, 1 , , 1,-1,-1)
X2= (-1,-1,-1, , 1,1, 1)
8/12/2019 Discrepancy Berlin
60/64
60/30
S O( 1/2) lt
8/12/2019 Discrepancy Berlin
61/64
61/30
Spencers O(n1/2) result
Partial Coloring Lemma:For any system with m sets,
there exists a coloring on n/2elements withdiscrepancy O(n1/2log1/2(2m/n))
[For m=n, disc = O(n1/2)]
Algorithm for total coloring:
Repeatedly apply partial coloring lemma
Total discrepancy
O( n1/2
log1/2
2 ) [Phase 1]+ O( (n/2)1/2log1/24 ) [Phase 2]
+ O((n/4)1/2log1/2 8 ) [Phase 3]
+ = O(n1/2)Let us prove the lemma for m = n
P i P ti l C l i L
8/12/2019 Discrepancy Berlin
62/64
62/30
Proving Partial Coloring Lemma
Pf: Associate with coloring X, signature = (b1,b2,,bn)
(bi= bucket in which X(Si) lies )
Wish to show: There exist 24n/5colorings with same signature
Choose X randomly: Induces distribution on signatures.
Entropy () n/5 implies some signature hasprob. 2-n/5.
Entropy () iEntropy( bi) [Subadditivity of Entropy]
bi = 0 w.p. 1- 2 e-50,
= 1 w.p. e-50
= 2 w.p. e-450
-10 n1/2-30 n1/2 10 n1/2 30 n1/2
0 21-1-2
Ent(b1) 1/5
8/12/2019 Discrepancy Berlin
63/64
63/30
A useful generalization
Partial coloring with non-uniformdiscrepancy Sfor set S
S-3S -S 3S 5S
0 1 2-1-2
For each set S, consider the bucketing
Suffices to have s Ent (bs) n/5
Or, if S= sn1/2, then s g(s) n/5
g() e-2/2 > 1
ln(1/) < 1
Bucket of n1/2/100has penalty ln(100)
Recap
8/12/2019 Discrepancy Berlin
64/64
Recap
Partial Coloring: S 10 n1/2 gives low entropy) 24n/5 colorings exist with same signature.
) some X1,X2with large hamming distance.
(X1X2) /2 gives the desired partial coloring.
Trouble: 24n/5/2n is an exponentially small fraction.
Only if we could find the partial coloring
efficiently
top related