Basic Steps 1.Compute the x and y image derivatives 2.Classify each derivative as being caused by either shading or a reflectance change 3.Set derivatives.

Basic Steps1. Compute the x and y image derivatives2. Classify each derivative as being caused by

either shading or a reflectance change3. Set derivatives with the wrong label to zero. 4. Recover the intrinsic images by finding the least-

squares solution of the derivatives.

Original x derivative image Classify each derivative(White is reflectance)

Learning the Classifiers• Combine multiple classifiers into a strong classifier using

AdaBoost (Freund and Schapire)• Choose weak classifiers greedily similar to (Tieu and Viola

2000)• Train on synthetic images• Assume the light direction is from the right

Shading Training Set Reflectance Change Training Set

Using Both Color and Gray-Scale Information

Results withoutconsidering gray-scale

Some Areas of the Image Are Locally Ambiguous

Input

Shading Reflectance

Is the change here better explained as

or ?

Propagating Information• Can disambiguate areas by propagating

information from reliable areas of the image into ambiguous areas of the image

• Consider relationship between neighboring derivatives

• Use Generalized Belief Propagation to infer labels

Propagating Information

Setting Compatibilities

• Set compatibilities according to image contours– All derivatives along a

contour should have the same label

• Derivatives along an image contour strongly influence each other 0.5 1.0

1

1),( jxx

i

β=

Improvements Using Propagation

Input Image Reflectance ImageWith Propagation

Reflectance ImageWithout Propagation

(More Results)

Input Image Shading Image Reflectance Image

Summary

13

Belief propagation is a feasible way to do inference in some Markov Random Fields. We showed applications of this approach to a number of low-level vision problems, including super-resolution, motion, and shading/reflectance discrimination.

next talk: presentations/bengaluruDeblur.ppt or keynote:presentations/motioninvVenice.key

14

Inference in Markov Random Fields

Gibbs sampling, simulated annealingIterated conditional modes (ICM)Belief propagation

Application examples:super-resolutionmotion analysisshading/reflectance separation

Graph cutsVariational methods

15

Gibbs Sampling and Simulated Annealing

• Gibbs sampling: – A way to generate random samples from a (potentially very

complicated) probability distribution.– Fix all dimensions except one. Draw from the resulting 1-d

conditional distribution. Repeat for all dimensions, and repeat many times

• Simulated annealing:– A schedule for modifying the probability distribution so that, at

“zero temperature”, you draw samples only from the MAP solution.

Reference: Geman and Geman, IEEE PAMI 1984.

16

Sampling from a 1-d function

1. Discretize the density function

2. Compute distribution function from density function

)(kf

)(kF

)(xf

)(kf

3. Sampling

draw ~ U(0,1);

for k = 1 to n

if

break;

;

)(kF

kxx 0

17

Gibbs Sampling

x1

x2

),,,|(~ )()(3

)(21

)1(1

tK

ttt xxxxx

),,,|(~ )()(3

)1(12

)1(2

tK

ttt xxxxπx

),,|(~ )1(1

)1(1

)1(

tK

tK

tK xxxx

Slide by Ce Liu

18

Gibbs sampling and simulated annealing

Simulated annealing as you gradually lower the “temperature” of the probability distribution ultimately giving zero probability to all but the MAP estimate.

What’s good about it: finds global MAP solution.

What’s bad about it: takes forever. Gibbs sampling is in the inner loop…

19

Gibbs sampling and simulated annealing

So you can find the mean value (MMSE estimate) of a variable by doing Gibbs sampling and averaging over the values that come out of your sampler.

You can find the MAP value of a variable by doing Gibbs sampling and gradually lowering the temperature parameter to zero.

20

Inference in Markov Random Fields

Gibbs sampling, simulated annealingIterated conditional modes (ICM)Belief propagation

Application examples:super-resolutionmotion analysisshading/reflectance separation

Graph cutsVariational methods

21

Iterated conditional modes

• For each node:– Condition on all the neighbors– Find the mode– Repeat.

• Compare with Gibbs sampling…

• Very small region over which it’s a local maximum

Described in: Winkler, 1995. Introduced by Besag in 1986.

22

Winkler, 1995

23

Region marginal probabilities

)()(),( ),(

)()( )(

\)(\)(

)(

ijNkj

kj

jiNki

kijijiij

iNki

kiiii

xMxMxxkxxb

xMxkxb

i

ji

24

Belief propagation equationsBelief propagation equations come from the

marginalization constraints.

jii

jii =

ijNk

jkjji

xi

ji xMxxxM

j \)(ij )(),( )(