Exploring the connection between sampling problems in Bayesian inference and statistical mechanics

Exploring the connection between sampling problems in Bayesian inference and

statistical mechanics

Andrew Pohorille

NASA-Ames Research Center

Outline

• Enhanced sampling of pdfs

• Dynamical systems

• Stochastic kinetics

flat histogramsmulticanonical methodWang-Landautransition probability methodparallel tempering

Enhanced sampling techniques

Preliminariesdefine: variables x, , N a function U(x,,N) a probability:

marginalize x

define “free energy” or “thermodynamic potential”

partition function Q(x,,N)

energies areBoltzmann-distributed = 1/kT

The problem:

What to do if

is difficult to estimate because we can’t get sufficient statistics for all of interest.

Flat histogram approach

pdf sampled uniformly for all , N

weight

Example:

original pdf

weighted pdf

marginalization

“canonical”partition function

1. get 2. get Q

General MC sampling schemeinsertion

deletion

insertion

deletion

adjust weights

adjust free energy

free energy

Multicanonical method

bin count shift

normalization of

Berg and Neuhaus, Phys. Rev. Lett. 68, 9 (1992)

The algorithm• Start with any weights (e.g. 1(N) = 0)

• Perform a short simulation and measure P(N; 1) as histogram

• Update weights according to

• Iterate until P(N; 1) is flat

or better

Typical example

Wang-Landau sampling

entropy

acceptance criterion

Wang and Landau, Phys. Rev. Lett. 86, 2050 (2001), Phys. Rev. E 64, 056101 (2001)

update constant

Example: estimate entropies for (discrete) states

The algorithm

• Set entropies of all states to zero; set initial g

• Accept/reject according to the criterion:

• Always update the entropy estimate for the end state

• When the pdf is flat reduce g

Transition probability method

i j

K

IJ

Wang, Tay, Swendsen, Phys. Rev. Lett., 82 476 (1999)Fitzgerald et al. J. Stat. Phys. 98, 321 (1999)

detailed balance

macroscopicdetailed balance

Parallel tempering

Dynamical systems

The idea: the system evolves according to equations of motion (possibly Hamiltonian)

we need to assign masses to variables

Assumption -ergodicity

Advantages• No need to design sampling techniques

• Specialized methods for efficient sampling are available (Laio-Parrinello, Adaptive Biasing Force)

• No probabilistic sampling

• Possibly complications with assignment of masses

Disadvantages

Two formulations:

• Hamiltonian

• Lagrangian

Numerical, iterative solution of equations of motion (a trajectory)

Assignment of masses

• Masses too large - slow motions

• Masses too small - difficult integration of equations of motion

• Large separation of masses - adiabatic separation

Thermostats are availableLagrangian - e.g. Nose-HooverHamiltonian - Leimkuhler

Energy equipartition needs to be addressed

Adaptive Biasing Force

A = a

b ∂H()/∂ *d

*

Darve and Pohorile, J. Chem. Phys. 115:9169-9183 (2001).

force A

Summary

• A variety of techniques are available to sample efficiently rarely visited states.

• Adaptive methods are based on modifying sampling while building the solution.

• One can construct dynamical systems to seek the solution and efficient adaptive techniques are available. But one needs to do it carefully.

Stochastic kinetics

The problem• {Xi} objects, i = 1,…N

• ni copies of each objects

• undergo r transformations

• With rates {k}, = 1,…r

• {k} are constant

• The process is Markovian (well-stirred reactor)

Assumptions

Example

7 objects5 transformations

Deterministic solution

concentrations

kinetics (differential equations)

steady state (algebraic equations)

Works well for large {ni} (fluctuations suppressed)

A statistical alternative

generate trajectories

• which reaction occurs next?• when does it occur?

next reaction is at time

next reaction is at any time

any reaction at time

Direct method - Algorithm

• Initialization

• Calculate the propensities {ai}

• Choose (r.n.)

• Choose (r.n.)

• Update no. of molecules and reset tt+ • Go to step 2

Gillespie, J. Chem. Phys. 81, 2340 (1977)

First reaction method -Algorithm• Initialization

• Calculate the propensities {ai}

• For each generate according to (r.n.)

• Choose reaction for which is the shortest

• Set =

• Update no. of molecules and reset tt+ • Go to step 2

Gillespie, J. Chem. Phys. 81, 2340 (1977)

Next reaction method

Complexity - O(log r)

Gibson and Bruck,J. Phys. Chem. A 1041876 (2000)

Extensions

• k = k(t) (GB)

• Non-Markovian processes (GB)

• Stiff reactions (Eric van den Eijden)

• Enzymatic reactions (A.P.)