(1) A probability model respecting those covariance observations: Gaussian Maximum entropy probability distribution for a given covariance observation.

(1) A probability model respecting those covariance observations:

Gaussian• Maximum entropy probability distribution for a

given covariance observation (shown zero mean for

notational convenience):

• If we rotate coordinates to the Fourier basis, the covariance matrix in that basis will be diagonal. So in that model, each Fourier transform coefficient is an independent Gaussian random variable of covariance

)exp()( 12

1 xCxxP xT Image pixels

Inverse covariance matrix

)|)((|)( 2 FED

Power spectra of typical images

Experimentally, the power spectrum as a function of Fourier frequency is observed to follow a power law.

A

FE )|)((| 2

http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf

Random draw from Gaussian spectral model

http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf

Noise removal (in frequency domain), under Gaussian assumption

)2)(exp( )2||||exp()|( 122 XXXYYXP An

Variance of white, Gaussian additive noise

Observed Fourier component

Estimated Fourier component

Power law prior probability on estimated Fourier component

Setting to zero the derivative of the the log probability of X gives an analytic form for the optimal estimate of X (or just complete the square):

Posterior probability for X

)()(ˆ2

YA

AX

n

Noise removal, under Gaussian assumption

original With Gaussian noise of std. dev. 21.4 added, giving PSNR=22.06

(1) Denoised with Gaussian model, PSNR=27.87

http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf

(try to ignore JPEG compression artifacts from the PDF file)

(2) The wavelet marginal modelHistogram of wavelet coefficients, c, for various images.

)||exp()( psccP

Parameter determining width of distribution

Parameter determining peakiness of distribution

Wavelet coefficient value

http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf

Random draw from the wavelet marginal model

http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf

And again something that is reminiscent of operations found in V1…

An application of image pyramids:noise removal

Image statistics (or, mathematically, how can you tell image from noise?)

Noisy image

Clean image

Pixel representation, image histogram

Pixel representation, noisy image histogram

bandpassed representation image histogram

Pixel domain noise image and histogram

Bandpass domain noise image and histogram

Noise-corrupted full-freq and bandpass images

But want the bandpass image histogram to look like this

P(x, y) = P(x|y) P(y)soP(x|y) P(y) = P(y|x) P(x)

P(x, y) = P(x|y) P(y)soP(x|y) P(y) = P(y|x) P(x)andP(x|y) = P(y|x) P(x) / P(y)

Bayes theorem

The parameters you want to

estimate

What you observePrior probability

Likelihood function

Constant w.r.t. parameters x.

P(x, y) = P(x|y) P(y)By definition of conditional probability

Using that twice

P(x)

Bayesian MAP estimator for clean bandpass coefficient values

Let x = bandpassed image value before adding noise.Let y = noise-corrupted observation.

By Bayes theorem

P(x|y) = k P(y|x) P(x)

P(y|x)

P(x|y)P(x|y)

P(y|x)

y

y = 25

Bayesian MAP estimatorLet x = bandpassed image value before adding noise.Let y = noise-corrupted observation.

By Bayes theorem

P(x|y) = k P(y|x) P(x) y

P(y|x)

P(x|y)

y = 50

Bayesian MAP estimatorLet x = bandpassed image value before adding noise.Let y = noise-corrupted observation.

By Bayes theorem

P(x|y) = k P(y|x) P(x) y

P(y|x)

P(x|y)

y = 115

P(x)

P(y|x)

y

y = 25

P(x|y)

y

P(y|x)

P(x|y)

y = 115

For small y: probably it is due to noise and y should be set to 0For large y: probably it is due to an image edge and it should be kept untouched

MAP estimate, , as function of observed coefficient value, y

y

x̂

x̂

http://www-bcs.mit.edu/people/adelson/pub_pdfs/simoncelli_noise.pdfSimoncelli and Adelson, Noise Removal via Bayesian Wavelet Coring

original

With Gaussian noise of std. dev. 21.4 added, giving PSNR=22.06

(1) Denoised with Gaussian model, PSNR=27.87

(2) Denoised with wavelet marginal model, PSNR=29.24

http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf

M. F. Tappen, B. C. Russell, and W. T. Freeman, Efficient graphical models for processing images IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) Washington, DC, 2004.

Motivation for wavelet joint models

Note correlations between the amplitudes of each wavelet subband.

http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf

Statistics of pairs of wavelet coefficientsContour plots of the joint histogram of various wavelet coefficient pairs

Conditional distributions of the corresponding wavelet pairs

http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf

(3) Gaussian scale mixtures

dzzPz

xzxxP zN

T

)( ||)2(

))(exp()(

212

12

1

Wavelet coefficient probability A mixture of

Gaussians of scaled covariances

observed

Gaussian scale mixture model simulation

z is a spatially varying hidden variable that can be used to(a) Create the non-gaussian histograms from a mixture of Gaussian densities, and (b) model correlations between the neighboring wavelet coefficients.

original

With Gaussian noise of std. dev. 21.4 added, giving PSNR=22.06

(1) Denoised with Gaussian model, PSNR=27.87

http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf

(3) Denoised with Gaussian scale mixture model, PSNR=30.86

(2) Denoised with wavelet marginal model, PSNR=29.24