CS B553: ALGORITHMS FOR OPTIMIZATION AND LEARNING Monte Carlo Methods for Probabilistic Inference
Feb 23, 2016
CS B553: ALGORITHMS FOR OPTIMIZATION AND LEARNINGMonte Carlo Methods for Probabilistic Inference
AGENDA Monte Carlo methods
O(1/sqrt(N)) standard deviation For Bayesian inference
Likelihood weighting Gibbs sampling
MONTE CARLO INTEGRATION Estimate large integrals/sums:
I = f(x)p(x) dx I = f(x)p(x)
Using a sample of N i.i.d. samples from p(x) I 1/N f(x(i))
Examples: [a,b] f(x) dx (b-a)/N f(x(i)) E[X] = x p(x) dx 1/N x(i)
Volume of a set in Rn
MEAN & VARIANCE OF ESTIMATE Let IN be the random variable denoting the
estimate of the integral with N samples What is the bias (mean error) E[I-IN]?
MEAN & VARIANCE OF ESTIMATE Let IN be the random variable denoting the
estimate of the integral with N samples What is the bias (mean error) E[I-IN]?
E[I-IN]=I-E[IN] (linearity of expectation)
MEAN & VARIANCE OF ESTIMATE Let IN be the random variable denoting the
estimate of the integral with N samples What is the bias (mean error) E[I-IN]?
E[I-IN]=I-E[IN] (linearity of expectation)= E[f(x)] - 1/N E[f(x(i))] (definition of I
and IN)
MEAN & VARIANCE OF ESTIMATE Let IN be the random variable denoting the
estimate of the integral with N samples What is the bias (mean error) E[I-IN]?
E[I-IN]=I-E[IN] (linearity of expectation)= E[f(x)] - 1/N E[f(x(i))] (definition of I
and IN)= 1/N (E[f(x)]-E[f(x(i))]) = 1/N 0 (x and x(i) are distributed
w.r.t. p(x))= 0
MEAN & VARIANCE OF ESTIMATE Let IN be the random variable denoting the
estimate of the integral with N samples What is the bias (mean error) E[I-IN]?
Unbiased estimator What is the variance Var[IN]?
MEAN & VARIANCE OF ESTIMATE Let IN be the random variable denoting the
estimate of the integral with N samples What is the bias (mean error) E[I-IN]?
Unbiased estimator What is the variance Var[IN]?
Var[IN] = Var[1/N f(x(i))] (definition)
MEAN & VARIANCE OF ESTIMATE Let IN be the random variable denoting the
estimate of the integral with N samples What is the bias (mean error) E[I-IN]?
Unbiased estimator What is the variance Var[IN]?
Var[IN] = Var[1/N f(x(i))] (definition)= 1/N2 Var[ f(x(i))] (scaling of
variance)
MEAN & VARIANCE OF ESTIMATE Let IN be the random variable denoting the
estimate of the integral with N samples What is the bias (mean error) E[I-IN]?
Unbiased estimator What is the variance Var[IN]?
Var[IN] = Var[1/N f(x(i))] (definition)= 1/N2 Var[ f(x(i))] (scaling of
variance)= 1/N2 Var[f(x(i))] (variance of a sum of
independent variables)
MEAN & VARIANCE OF ESTIMATE Let IN be the random variable denoting the
estimate of the integral with N samples What is the bias (mean error) E[I-IN]?
Unbiased estimator What is the variance Var[IN]?
Var[IN] = Var[1/N f(x(i))] (definition)= 1/N2 Var[ f(x(i))] (scaling of
variance)= 1/N2 Var[f(x(i))]= 1/N Var[f(x)] (i.i.d. sample)
MEAN & VARIANCE OF ESTIMATE Let IN be the random variable denoting the
estimate of the integral with N samples What is the bias (mean error) E[I-IN]?
Unbiased estimator What is the variance Var[IN]?
1/N Var[f(x)] Standard deviation: O(1/sqrt(N))
APPROXIMATE INFERENCE THROUGH SAMPLING Unconditional simulation:
To estimate the probability of a coin flipping heads, I can flip it a huge number of times and count the fraction of heads observed
APPROXIMATE INFERENCE THROUGH SAMPLING Unconditional simulation:
To estimate the probability of a coin flipping heads, I can flip it a huge number of times and count the fraction of heads observed
Conditional simulation: To estimate the probability P(H) that a coin
picked out of bucket B flips heads: Repeat for i=1,…,N:1. Pick a coin C out of a random bucket b(i) chosen
with probability P(B)2. h(i) = flip C according to probability P(H|b(i))3. Sample (h(i),b(i)) comes from distribution P(H,B)
Result approximates P(H,B)
MONTE CARLO INFERENCE IN BAYES NETS BN over variables X Repeat for i=1,…,N
In top-down order, generate x(i) as follows: Sample xj
(i) ~ P(Xj |paXj(i))
(RHS is taken by putting parent values in sample into the CPT for Xj)
Sample x(1)… x(N) approximates the
distribution over X
APPROXIMATE INFERENCE: MONTE-CARLO SIMULATION Sample from the joint distribution
B E P(A|…)
TTFF
TFTF
0.950.940.290.001
Burglary Earthquake
Alarm
MaryCallsJohnCalls
P(B)0.001
P(E)0.002
A P(J|…)TF
0.900.05
A P(M|…)
TF
0.700.01
B=0E=0A=0J=1M=0
APPROXIMATE INFERENCE: MONTE-CARLO SIMULATION As more samples are generated, the
distribution of the samples approaches the joint distribution
B=0E=0A=0J=1M=0
B=0E=0A=0J=0M=0
B=0E=0A=0J=0M=0
B=1E=0A=1J=1M=0
BASIC METHOD FOR HANDLING EVIDENCE Inference: given evidence E=e (e.g., J=1),
approximate P(X/E|E=e) Remove the samples that conflict
B=0E=0A=0J=1M=0
B=0E=0A=0J=0M=0
B=0E=0A=0J=0M=0
B=1E=0A=1J=1M=0
Distribution of remaining samples approximates the conditional distribution
RARE EVENT PROBLEM: What if some events are really rare (e.g.,
burglary & earthquake ?) # of samples must be huge to get a
reasonable estimate Solution: likelihood weighting
Enforce that each sample agrees with evidence While generating a sample, keep track of the
ratio of(how likely the sampled value is to occur in the real world)
(how likely you were to generate the sampled value)
LIKELIHOOD WEIGHTING Suppose evidence Alarm & MaryCalls Sample B,E with P=0.5
B E P(A|…)
TTFF
TFTF
0.950.940.290.001
Burglary Earthquake
Alarm
MaryCallsJohnCalls
P(B)0.001
P(E)0.002
A P(J|…)TF
0.900.05
A P(M|…)
TF
0.700.01
w=1
LIKELIHOOD WEIGHTING Suppose evidence Alarm & MaryCalls Sample B,E with P=0.5
B E P(A|…)
TTFF
TFTF
0.950.940.290.001
Burglary Earthquake
Alarm
MaryCallsJohnCalls
P(B)0.001
P(E)0.002
A P(J|…)TF
0.900.05
A P(M|…)
TF
0.700.01
B=0E=1
w=0.008
LIKELIHOOD WEIGHTING Suppose evidence Alarm & MaryCalls Sample B,E with P=0.5
B E P(A|…)
TTFF
TFTF
0.950.940.290.001
Burglary Earthquake
Alarm
MaryCallsJohnCalls
P(B)0.001
P(E)0.002
A P(J|…)TF
0.900.05
A P(M|…)
TF
0.700.01
B=0E=1A=1
w=0.0023
A=1 is enforced, and the weight updated to reflect the likelihood that this occurs
LIKELIHOOD WEIGHTING Suppose evidence Alarm & MaryCalls Sample B,E with P=0.5
B E P(A|…)
TTFF
TFTF
0.950.940.290.001
Burglary Earthquake
Alarm
MaryCallsJohnCalls
P(B)0.001
P(E)0.002
A P(J|…)TF
0.900.05
A P(M|…)
TF
0.700.01
B=0E=1A=1M=1J=1
w=0.0016
LIKELIHOOD WEIGHTING Suppose evidence Alarm & MaryCalls Sample B,E with P=0.5
B E P(A|…)
TTFF
TFTF
0.950.940.290.001
Burglary Earthquake
Alarm
MaryCallsJohnCalls
P(B)0.001
P(E)0.002
A P(J|…)TF
0.900.05
A P(M|…)
TF
0.700.01
B=0E=0
w=3.988
LIKELIHOOD WEIGHTING Suppose evidence Alarm & MaryCalls Sample B,E with P=0.5
B E P(A|…)
TTFF
TFTF
0.950.940.290.001
Burglary Earthquake
Alarm
MaryCallsJohnCalls
P(B)0.001
P(E)0.002
A P(J|…)TF
0.900.05
A P(M|…)
TF
0.700.01
B=0E=0A=1
w=0.004
LIKELIHOOD WEIGHTING Suppose evidence Alarm & MaryCalls Sample B,E with P=0.5
B E P(A|…)
TTFF
TFTF
0.950.940.290.001
Burglary Earthquake
Alarm
MaryCallsJohnCalls
P(B)0.001
P(E)0.002
A P(J|…)TF
0.900.05
A P(M|…)
TF
0.700.01
B=0E=0A=1M=1J=1
w=0.0028
LIKELIHOOD WEIGHTING Suppose evidence Alarm & MaryCalls Sample B,E with P=0.5
B E P(A|…)
TTFF
TFTF
0.950.940.290.001
Burglary Earthquake
Alarm
MaryCallsJohnCalls
P(B)0.001
P(E)0.002
A P(J|…)TF
0.900.05
A P(M|…)
TF
0.700.01
B=1E=0A=1
w=0.00375
LIKELIHOOD WEIGHTING Suppose evidence Alarm & MaryCalls Sample B,E with P=0.5
B E P(A|…)
TTFF
TFTF
0.950.940.290.001
Burglary Earthquake
Alarm
MaryCallsJohnCalls
P(B)0.001
P(E)0.002
A P(J|…)TF
0.900.05
A P(M|…)
TF
0.700.01
B=1E=0A=1M=1J=1
w=0.0026
LIKELIHOOD WEIGHTING Suppose evidence Alarm & MaryCalls Sample B,E with P=0.5
B E P(A|…)
TTFF
TFTF
0.950.940.290.001
Burglary Earthquake
Alarm
MaryCallsJohnCalls
P(B)0.001
P(E)0.002
A P(J|…)TF
0.900.05
A P(M|…)
TF
0.700.01
B=1E=1A=1M=1J=1
w=5e-7
LIKELIHOOD WEIGHTING Suppose evidence Alarm & MaryCalls Sample B,E with P=0.5
N=4 gives P(B|A,M)~=0.371 Exact inference gives P(B|A,M) = 0.375
B=0E=1A=1M=1J=1
w=0.0016
B=0E=0A=1M=1J=1
w=0.0028
B=1E=0A=1M=1J=1
w=0.0026
B=1E=1A=1M=1J=1
w~=0
ANOTHER RARE-EVENT PROBLEM B=b given as evidence Probability each bi is rare given all but one
setting of Ai (say, Ai=1)
Chance of sampling all 1’s is very low => most likelihood weights will be too low
Problem: evidence is not being used to sample A’s effectively (i.e., near P(Ai|b))
A1 A2 A10
B1 B2 B10
GIBBS SAMPLING Idea: reduce the computational burden of
sampling from a multidimensional distribution P(x)=P(x1,…,xn) by doing repeated draws of individual attributes Cycle through j=1,…,n Sample xj ~ P(xj | x[1…j-1,j+1,…n])
Over the long run, the random walk taken by x approaches the true distribution P(x)
GIBBS SAMPLING IN BNS Each Gibbs sampling step: 1) pick a variable
Xi, 2) sample xi ~ P(Xi|X/Xi) Look at values of “Markov blanket” of Xi:
Parents PaXi Children Y1,…,Yk Parents of children (excluding Xi) PaY1/Xi, …,
PaYk/Xi Xi is independent of rest of network given Markov
blanket Sample xi~P(Xi|, Y1, PaY1/Xi, …, Yk, PaYk/Xi)
= 1/Z P(Xi|PaXi) P(Y1|PaY1) *…* P(Yk|PaYk) Product of Xi’s factor and the factors of its
children
HANDLING EVIDENCE Simply set each evidence variable to its
appropriate value, don’t sample Resulting walk approximates distribution
P(X/E|E=e) Uses evidence more efficiently than
likelihood weighting
GIBBS SAMPLING ISSUES Demonstrating correctness & convergence
requires examining Markov Chain random walk (more later)
Need to take many steps before the effects of poor initialization wear off (mixing time) Difficult to tell how much is needed a priori
Numerous variants Known as Markov Chain Monte Carlo techniques
NEXT TIME Continuous and hybrid distributions