Trans-dimensional MCMC Peter Greendellaert/pub/Dellaert05RJMCMC.pdf•Peter Green, Reversible jump Markov chain Monte Carlo computation and Bayesian model determination, Biometrika,

1

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Trans-dimensional MCMC

Frank Dellaert

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

References

• Peter Green, Reversible jump Markov chain MonteCarlo computation and Bayesian model determination,Biometrika, 1995

• Peter Green, Trans-dimensional Markov chain MonteCarlo, in “Highly Structured Stochastic Systems”, 2003

• David Hastie, Ph.D. Thesis, 2005• Paskin & Thrun, Robotic Mapping with Polygonal

Random Fields, UAI 2005

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Outline

• Model Selection• Reversible Jump MCMC• Regression Example• Rao-Blackwellized Sampling• Polygonal Random Fields

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Model Selection

• Most common case: inference on p(x|z), xcontinuous parameters

• What if there are competing models ?

!

p(x1 | z),x1 " R3

!

p(x2 | z),x2 " R5

!

p(x3 | z),x3 " R7

2

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Ex. 1: Regression

• Degree of polynomial ?

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Ex.2: 3D Curve Fitting

• How many control points ?• How many sharp corners ?

Image courtesy of Michael Kaess

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Ex.3: Tracking Multiple Targets

• With an unknown number of targetsImage courtesy of Zia Khan

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Union Space

• Model indicator k• Parameter vector• State space = union space

!

"k# R

nk

!

X = Uk"K ({k}# R

nk )

!

{1}" R3

!

{2}" R5

!

{3}" R7

!

"

!

"

!

"K

3

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Graphical Model

k

θk

z• Joint density factors as

p(z,θk,k)= p(z|θk,k) p(θk|k) p(k)

Model indicatorp(k)

Datap(z|θk,k)

Model parametersp(θk |k)

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Outline

• Model Selection• Reversible Jump MCMC• Regression Example• Rao-Blackwellized Sampling• Polygonal Random Fields

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Trans-Dimensional MCMC

• set up Markov chain in union space X• allow trans-dimensional jumps

!

{1}" R3

!

{2}" R5

!

{3}" R7

!

"

!

"

!

"K

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Regression Example

• Live Demo !

4

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Explicit Jump Random Variables

• Re-state MCMC to avoid measure theory• Explicit jump random variable u• Proposal density q(x,x’) replaced by

– draw u ~ g(u)– calculate x’=h(x,u)

Pictorial representationinspired by Peter Green

g(u)

u

xx’=h(x,u)

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Example (single dimension)

• Random walk on SO(2)

!

q(x,x ') = N(x';x," 2)

#

u ~ N(0," 2)

x'= h(x,u) = x + u

g(u)

u

x

x’=x+u

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

“Jacobian”

A New Proposal Ratio

g(u)

u

xx’=h(x,u)

g(u’)

u’

=h’(x’,u’)

!

a =" (x')q(x',x)

" (x)q(x,x ')

!

a =" (x')g(u')

" (x)g(u)

#(x',u')

#(x,u)

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Jacobians• Jacobian arises from change of variable• Review from probability 101:

!

x ~ g(x)

y = t(x)

f (y) ="t(x)

"x

#1

g(t#1(y)) =

"x

"yg(x)

!

x ~ g(x)

y = t(x) = x /2

f (y) ="t(x)

"x

#1

g(t#1(y)) = 2g(2y)

g(x)

f(y)

Simple example:

5

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Detailed Balance in RJMCMC

!

f (y)dy = g(x)dx and dx ="x

"ydy hence : f (y)dy =

"x

"yg(x)dy

• Change of variables as differential equality

!

"(x,x ')# (x)g(u)dxdu ="(x',x)g(u)# (x ')g(u')dx 'du'

dx'du'=$(x ',u')

$(x,u)dxdu

"(x,x ') =min 1,# (x ')g(u')

# (x)g(u)

$(x ',u')

$(x,u)

% & '

( ) *

Shows up in detailed balance equation:

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Reversible -> Diffeomorphism

• We need to calculate both x’ and u’:(x’,u’) = t (x,u)

• Required that t is a diffeomorphism:– invertible– differentiable

• Jacobian =

!

"(x ',u')

"(x,u)="t(x,u)

"(x,u)=

"tx '(x,u)

"x

"tx'(x,u)

"u"t

u'(x,u)

"x

"tu'(x,u)

"u

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

A Constructive MCMC Recipe (Green)

• (Conventional) MCMC with jump variables:– draw u ~ g(u)– calculate proposal x’=h(x,u)– calculate reverse jump variable u’ s.t. x=h’(x’,u’)– calculate acceptance ratio:

!

a =min 1," (x ')g(u')

" (x)g(u)

#(x ',u')

#(x,u)

$ % &

' ( )

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

1D Example (continued)

• Random walk on SO(2)

!

u ~ N(0," 2)

x'= x + u

!

"(x + u,#u)

"(x,u)=

"(x + u)

"x

"(x + u)

"u"(#u)

"x

"(#u)

"u

=1 1

0 #1=1 g(u)

u

x

x’=x+u

!

u'= x " x '= "u

# t(x,u) = (x + u,"u)

6

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Reversible-Jump MCMC

g(u)

u

xx’=h(x,u)

g(u’)

u’

=h’(x’,u’)

!

a =" (x')g(u')

" (x)g(u)

#(x',u')

#(x,u)

• Story holds if x and x’have different dimensions !

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

5D5D

2D 3D

Dimension Matching Constraint• Requirement that (x’,u’)=t(x,u)

remains a diffeomorphism⇒ dim(x’)+dim(u’)= dim(x)+dim(u)

u

xx’=h(x,u)

u’

=h’(x’,u’)2D

3D

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Move Types

• In each space, proposal is mixture of differentmove types

• Each move has probability jm(x)

m=1m=2

m=3

x!

" a =# (x') jm (x ')gm (u')

# (x) jm (x)gm (u)

$(x ',u')

$(x,u)

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Outline

• Model Selection• Reversible Jump MCMC• Regression Example• Rao-Blackwellized Sampling• Polygonal Random Fields

7

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Regression Example

• k = degree of polynomial

P(k=0)=0.5 P(k=1)=0.5

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Regression Example

• k = degree of polynomial

!

{0}" R1

!

{1}" R2

!

{2}" R3!

a =" (x') jm (x ')gm (u')

" (x) jm (x)gm (u)

#(x ',u')

#(x,u)

m=01,10 m=12,21m=0 m=1 m=2 m=23,32

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Regression Example

• k = degree of polynomial

!

{0}" R1

!

{1}" R2

!

{2}" R3!

a =" (x') jm (x ')gm (u')

" (x) jm (x)gm (u)

#(x ',u')

#(x,u)

!

"(c '0,c'

1)

"(c0,u)

=1 0

0 1=1

!

x = c0[ ]

!

x'=c0

u

"

# $ %

& '

g(u)

u

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Regression Example

• Ambivalent data• 10000 Samples

8

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Regression Example

• Less Noise

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Regression Example

• More Data

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Regression Example

• No ambivalence

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Regression Example

• No ambivalencefor k=0 model

9

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Careful Move Design

• E.g., move from constantto line such that reachessame height at mean(x)

!

{0}" R1

!

{1}" R2

!

{2}" R3

!

"(c '0,c'

1)

"(c0,u)

=1 0

1/ x #1/ x =1/ x !

x = c0[ ]

!

x'

!

x'=c0

(c0" u) / x

#

$ %

&

' (

!

x

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Tracking Multiple Targets

• Showing meannumber of targets

Khan et al PAMI October

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Outline

• Model Selection• Reversible Jump MCMC• Regression Example• Rao-Blackwellized Sampling• Polygonal Random Fields

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Rao-Blackwellized Sampling

• Second strategy:– integrate out model parameters θk

– sample over marginal posterior

!

{1}" R3

!

{2}" R5

!

{3}" R7

!

"

!

"

!

"K!

p(k | z)" p(k) p(z |#k,k)p(#k | k)# k$

!

"k

#!

p(k | z)

1 2 3

10

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Rao-Blackwell Theorem

• Variance of sample approximation isreduced when part of state space isintegrated out

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Curve Fitting Results

Images courtesy of Michael Kaess

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Probabilistic Topological Maps

Sampling over the space of topologies

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

The space of topologies

Topologies ⇔ Space of set Partitions, very big !!

11

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Experiments

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Experiments (2)

With appearance measurements

99.5% 0.25% 0.14% 0.12% 0.01%

Only odometry

43% 14% 7% 3% 2%

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Outline

• Model Selection• Reversible Jump MCMC• Regression Example• Rao-Blackwellized Sampling• Polygonal Random Fields

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

Images by Mark Paskin

• From field of spatial statistics• Space = colorings of R2 in window D• Uses measure theory to build a (non-

RJMCMC) sampler that satisfies detailedbalance

12

ICCV05 Tutorial: MCMC for Vision. Zhu / Dellaert / Tu October 2005

PRF in Robotics

Images by Mark Paskin