Biips software: Bayesian inference with interacting particle systems Rencontres AppliBUGS Adrien Todeschini † , Franc ¸ois Caron * , Pierrick Legrand † , Pierre Del Moral ‡ and Marc Fuentes † † Inria Bordeaux, * Univ. Oxford, ‡ UNSW Sydney Montpellier, Novembre 2014 A. Todeschini 1 / 39
72
Embed
[width=2cm]biipslogosmooth Biips software: …genome.jouy.inra.fr/applibugs/applibugs.14_11_28...SMC algorithm I A.k.a. interacting MCMC, particle filtering, sequential Monte Carlo
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Biips software: Bayesian inference with interactingparticle systems
Rencontres AppliBUGS
Adrien Todeschini†, Francois Caron∗, Pierrick Legrand†, Pierre DelMoral‡ and Marc Fuentes†
†Inria Bordeaux, ∗Univ. Oxford, ‡UNSW Sydney
Montpellier, Novembre 2014
A. Todeschini 1 / 39
Outline
Context
Graphical models and BUGS language
SMC
Biips software
Particle MCMC
A. Todeschini 2 / 39
Summary
Context
Graphical models and BUGS language
SMC
Biips software
Particle MCMC
A. Todeschini 3 / 39
Context
Biips = Bayesian inference with interacting particle systems
Bayesian inferenceI Sample from a posterior distribution p(X|Y ) = p(X,Y )
p(Y )I High dimensional, arbitrary complexityI Simulation methods: MCMC, SMC...
MotivationI Last 20 years: success of SMC in many applicationsI No general and easy-to-use software for SMC
A. Todeschini 4 / 39
Context
Biips = Bayesian inference with interacting particle systems
Bayesian inferenceI Sample from a posterior distribution p(X|Y ) = p(X,Y )
p(Y )I High dimensional, arbitrary complexityI Simulation methods: MCMC, SMC...
MotivationI Last 20 years: success of SMC in many applicationsI No general and easy-to-use software for SMC
A. Todeschini 4 / 39
Context
Biips = Bayesian inference with interacting particle systems
ObjectivesI BUGS language compatibleI Extensibility: custom functions/samplersI Black-box SMC inference engineI Interfaces with popular software: Matlab/Octave, RI Post-processing tools
A. Todeschini 5 / 39
Summary
Context
Graphical models and BUGS language
SMC
Biips software
Particle MCMC
A. Todeschini 6 / 39
Graphical models
X1
X2 X3
Y1 Y2
Directed acyclic graph
The graph displays a factorization of thejoint distribution:
p(x1:3, y1:2)
A. Todeschini 7 / 39
Graphical models
X1
X2 X3
Y1 Y2
Directed acyclic graph
The graph displays a factorization of thejoint distribution:
rate <- c(c1*x[1], c2*x[1]*x[2], c3*x[2])sum_rate <- sum(rate);# Sample the next event from an exponential distributiont <- t - log(runif (1))/sum_rateif (t>dt)
break# Sample the type of eventind <- which (( sum_rate*runif (1)) <= cumsum (rate))[1]x <- x + z[,ind]
Recent algorithms that use SMC algorithms within a MCMC algorithm
[Andrieu et al., 2010]
I Particle Independant Metropolis-Hastings (PIMH)I Particle Marginal Metropolis-Hastings (PMMH)
A. Todeschini 30 / 39
Static parameter estimation
. . . Xt−1 Xt Xt+1 . . .
YtYt−1 Yt+1
θ
Due to the successive resamplings, SMC estimations of p(θ|y1:n) mightbe poor.
The PMMH splits the variables in the graphical model into two sets:I a set of variables X that will be sampled using a SMC algorithmI a set θ = (θ1, . . . , θp) sampled with a MH proposal
A. Todeschini 31 / 39
Standard PMMH algorithmSet Z(0) = 0 and initialize θ(0)For k = 1, . . . ,niter,
I Sample θ? ∼ ν(.|θ(k−1))I Run a SMC to approximate p(x1:n|y1:n, θ
?) with output(X?(i)
1:n ,W ?(i)n )i=1,...,N and Z? ≈ p(y1:n|θ?)
I With probability
min(
1,ν(θ?|θ(k − 1))p(θ?)Z?
ν(θ(k − 1)|θ?)p(θ(k − 1))Z(k − 1)
)
set X1:n(k) = X?(`)1:n , θ(k) = θ? and Z(k − 1) = Z?, where
I BUGS language compatibleI Extensibility: custom functions/samplersI Black-box SMC inference engineI Interfaces with popular software: Matlab/Octave, RI Post-processing toolsI And more: backward smoothing algorithm, particle independent
Bibliography IAndrieu, C., Doucet, A., and Holenstein, R. (2010).Particle markov chain monte carlo methods.Journal of the Royal Statistical Society B, 72:269–342.
Boys, R. J., Wilkinson, D. J., and Kirkwood, T. B. L. (2008).Bayesian inference for a discretely observed stochastic kinetic model.Statistics and Computing, 18(2):125–135.
Del Moral, P. (2004).Feynman-Kac Formulae. Genealogical and Interacting Particle Systems with Application.Springer.
Doucet, A., de Freitas, N., and Gordon, N., editors (2001).Sequential Monte Carlo Methods in Practice.Springer-Verlag.
Doucet, A. and Johansen, A. (2010).A tutorial on particle filtering and smoothing: Fifteen years later.In Crisan, D. and Rozovsky, B., editors, Oxford Handbook of Nonlinear Filtering. OxfordUniversity Press.
Gillespie, D. T. (1977).Exact stochastic simulation of coupled chemical reactions.The journal of physical chemistry, 81(25):2340–2361.
A. Todeschini 37 / 39
Bibliography II
Golightly, A. and Gillespie, C. S. (2013).Simulation of stochastic kinetic models.In In Silico Systems Biology, pages 169–187. Springer.
Lunn, D., Jackson, C., Best, N., Thomas, A., and Spiegelhalter, D. (2012).The BUGS Book: A Practical Introduction to Bayesian Analysis.CRC Press/ Chapman and Hall.
Plummer, M. (2012).JAGS Version 3.3.0 user manual.