Top Banner
Spatial decay of correlations and efficient methods for computing partition functions. David Gamarnik Joint work with Antar Bandyopadhyay (U of Chalmers), Dmitriy Rogozhnikov-Katz (MIT) June, 2006
41

Spatial decay of correlations and efficient methods for computing partition functions.

Feb 11, 2016

Download

Documents

Tyrell

Spatial decay of correlations and efficient methods for computing partition functions. David Gamarnik Joint work with Antar Bandyopadhyay ( U of Chalmers ), Dmitriy Rogozhnikov-Katz ( MIT ) June, 2006. Talk Outline. Partition functions. Where do we see them ? - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Spatial decay of correlations and efficient methods for computing partition functions.

Spatial decay of correlations and efficient methods for computing partition functions.

David Gamarnik

Joint work with

Antar Bandyopadhyay (U of Chalmers), Dmitriy Rogozhnikov-Katz (MIT)

June, 2006

Page 2: Spatial decay of correlations and efficient methods for computing partition functions.

Talk Outline

• Partition functions. Where do we see them ?

• Computing partition functions. Monte Carlo method.

• Correlation decay.

• Our results: computation tree, correlation decay and • Deterministic algorithm for approximate computation of partition functions for matchings and colorings.• Structural results and large deviations.

• Conclusions

Page 3: Spatial decay of correlations and efficient methods for computing partition functions.

Partition functions - feature in

• statistical mechanics Gibbs measure and Ising models

• computer science and combinatorics counting problems

• queueing theory product form loss networks

• electrical engineeringcoding theory

• statistics bayesian networks

Page 4: Spatial decay of correlations and efficient methods for computing partition functions.

• Calls arrive as and request communication link

• Call is accepted only if no other link attached to is occupied

• Unaccepted call is lost

• Call duration is

Queueing Example: loss system with shared resources

Page 5: Spatial decay of correlations and efficient methods for computing partition functions.

• At any moment the set of occupied links is a matching

• The steady-state distribution is product form:

- partition function.

Page 6: Spatial decay of correlations and efficient methods for computing partition functions.

• Calls arrive as and occupy a node

• Call is accepted only if no neighbor is occupied

• Unaccepted call is lost

• Call duration is

Example II: multicasting in a communication network

Page 7: Spatial decay of correlations and efficient methods for computing partition functions.

• At any moment the set of occupied nodes is an independent set

• The steady-state distribution is product form:

- partition function.

Page 8: Spatial decay of correlations and efficient methods for computing partition functions.

• Calls arrive as and occupy a node and use frequency

• Call is accepted only if no neighbor is occupied and uses the same fr.

• Unaccepted call is lost

• Call duration is

Example III: multicasting with many frequencies

Page 9: Spatial decay of correlations and efficient methods for computing partition functions.

• At any moment the set of occupied nodes is a partial coloring

• The steady-state distribution is product form:

- partition function.

Page 10: Spatial decay of correlations and efficient methods for computing partition functions.

• Communication (matching) problem with

From queueing to statistical physics

- Gibbs distribution on Ising type models. Important object in stat mechanics.

- inverse temperature

- Monomer-dimer model.

Page 11: Spatial decay of correlations and efficient methods for computing partition functions.

• Matching problem with

From statistical physics to computer science

total number of matchings in the graph (counting)

Page 12: Spatial decay of correlations and efficient methods for computing partition functions.

Can we compute partition function?...

… easily when the underlying graph is a tree.

Example (independent sets)

This leads to

Page 13: Spatial decay of correlations and efficient methods for computing partition functions.

Theorem. Spitzer [75], Zachary [83,85], Kelly [85]. In -ary tree

Is independent from the boundary condition (correlation decay) if and only if

Ramanan, Sengupta, Zeidins, Mitra [2002] Related work on unicasting and multicasting on trees

Implication: if the graph is locally-tree like, then computing marginals is possible in the regime

Page 14: Spatial decay of correlations and efficient methods for computing partition functions.

Computing partition function in general

• Valiant [1979] -- #P complexity class. Exact counting is hard for most of the counting problems (matchings, independent sets, colorings, etc. )

• Focus – approximate counting.

Our contribution: - use of correlation decay for

- Deterministic (non-simulation based) algorithms for computing approximately partition functions for

• Matchings in low degree graphs

• Colorings in low degree graphs

- Structural properties of partition functions in special classes of graphs

Page 15: Spatial decay of correlations and efficient methods for computing partition functions.

Existing approaches for computing partition function

• Main approximation method: Markov Chain Monte Carlo (MCMC)

• The MCMC is based on

- computing the marginal distribution via simulation.

- reducing partition function to marginals (cavity method).Jerrum, Valiant & Vazirani [86]

• Technical challenge: establishing rapid mixing

Page 16: Spatial decay of correlations and efficient methods for computing partition functions.

Computing partition functions using MCMC

Jerrum [95]. Coloring

Vigoda [2000]. Coloring Coloring

Jerrum & Sinclair [89] Matchings

Dyer, Frieze & Kannan [91] Volume of a convex body.

Jerrum, Sinclair & Vigoda [2004]. Permanents

Page 17: Spatial decay of correlations and efficient methods for computing partition functions.

(Temporal) Decay of correlations in Markov chains

A Markov chain with transition matrix satisfies decay of correlation (mixes)

if and only if it is aperiodic

(Spatial) Decay of correlations

Same thing, but time is replaced by a “spatial” distance

Page 18: Spatial decay of correlations and efficient methods for computing partition functions.

Correlation DecayA sequence of spatially (graph) related random variables exhibits a decay of correlation (long-range independence),if when is large

Principle motivation - statistical phyisics. Uniqueness of Gibbs measures on infinite lattices, Dobrushin [60s].

Page 19: Spatial decay of correlations and efficient methods for computing partition functions.

What is known about correlation decay ?

Spitzer [75], Zachary [83,85], Kelly [85]. Independent sets -ary tree

J. van den Berg [98] Matchings

Goldberg, Martin & Paterson [05]. Coloring. General graphs

Jonasson [01]. Coloring. Regular trees

Link between Correlation Decay and rapid mixing of MC:

• CD implies rapid mixing in subexp. growing graphs. • Converse not true Kenyon, Mossel & Peres [01].

Page 20: Spatial decay of correlations and efficient methods for computing partition functions.

Our results:

Theorem I. There exists a deterministic algorithm for computing approximately the partition function corresponding to matchings in graphs with constant degree (deterministic FPTAS), for arbitrary

Related work: Weitz [2005]. Self-avoiding walk based algorithm for counting independent sets when

Page 21: Spatial decay of correlations and efficient methods for computing partition functions.

Algorithm and proof:

Step I. Reduce computing partition function to computing marginals (cavity method)

Thus computing marginals implies computing the partition function

Page 22: Spatial decay of correlations and efficient methods for computing partition functions.

Step II. Cavity recursion

Page 23: Spatial decay of correlations and efficient methods for computing partition functions.

Step II. Cavity recursion

Page 24: Spatial decay of correlations and efficient methods for computing partition functions.

Step II. Cavity recursion

Page 25: Spatial decay of correlations and efficient methods for computing partition functions.

Algorithm: repeat the recursion times.

Initialize at the bottom arbitrarily.

Compute recursively.

- Computation tree

Page 26: Spatial decay of correlations and efficient methods for computing partition functions.

Proposition. The computation tree satisfies the decay of correlation property

Proof: look at the recursion function:

Introduce change of variables:

Page 27: Spatial decay of correlations and efficient methods for computing partition functions.

Mean Value Theorem:

- contraction

Page 28: Spatial decay of correlations and efficient methods for computing partition functions.
Page 29: Spatial decay of correlations and efficient methods for computing partition functions.
Page 30: Spatial decay of correlations and efficient methods for computing partition functions.

Theorem II. There exists a deterministic algorithm for computing approximately the number of list colorings in triangle-free graphs when the size of each list is constant and

for all nodes

Page 31: Spatial decay of correlations and efficient methods for computing partition functions.

Cavity recursion

Page 32: Spatial decay of correlations and efficient methods for computing partition functions.

Cavity recursion

x

x

x

Page 33: Spatial decay of correlations and efficient methods for computing partition functions.

Cavity recursion

We establish correlation decay for this recursion

x

x

x

Page 34: Spatial decay of correlations and efficient methods for computing partition functions.

Why can’t we use conventional decay of correlation directly for counting by computingmarginals locally for small (constant) ?

Problem:

We need accuracy in order to have accuracy

Page 35: Spatial decay of correlations and efficient methods for computing partition functions.

But:

Page 36: Spatial decay of correlations and efficient methods for computing partition functions.

Theorem III. The partition function of independent sets in every r-regular locally tree-like graphs satisfies

when

Structural results

The decay of correlation property implies the following large deviations results:

Page 37: Spatial decay of correlations and efficient methods for computing partition functions.

Queueing/large deviations interpretation

1. In a multicasting model (independent sets) the probability that nobody is transmitting a signal is

2. The probability that the set of active nodes is is given as

Page 38: Spatial decay of correlations and efficient methods for computing partition functions.

These results are not “provable” using MCMC technique

Structural results

Theorem IV. The partition function of the number of q-colorings in every r-regular graph with large girth satisfies

Page 39: Spatial decay of correlations and efficient methods for computing partition functions.

Note: removing a node when computing marginals destroys regularity

A fix comes from a rewiring trick Mezard-Parisi [05].

Lemma. The rewiring operation can be performed on pairs of nodes without creating small cycles.

Page 40: Spatial decay of correlations and efficient methods for computing partition functions.

Final thoughts and goals

• Queueing and stationarity.

Consider a queueing version of the “matching” problem. Assume FIFO.

Does the loss of stationarity occur before or after onset of long-range dependence?

Page 41: Spatial decay of correlations and efficient methods for computing partition functions.

Final thoughts and goals

• Create an implementable version of our algorithm (aka Belief Propagation). Our algorithm is only nominally efficient.

• Combining algorithm with importance sampling to handle large degree instances.

• Other counting problems: permanent, volume of a polyhedron.

• What other structures have the underlying computation tree satisfy the correlation decay property? Markov random fields?