Digital Image Processing - UGentsanja/ImageProcessingCourse/04a... · Digital Image Processing ... image enhancement is largely subjective, ... If there is no aliasing we can model

Digital Image Processing

Dr. ir. Aleksandra PizuricaProf. Dr. Ir. Wilfried Philips

7 December 2006

Aleksandra.Pizurica @telin.UGent.be Tel: 09/264.3415

Telecommunicatie en Informatieverwerking

UNIVERSITEIT GENT

Telecommunicatie en Informatieverwerking

UNIVERSITEIT GENT

Image Restoration

04.a3

version: 7/12/2006 © A. Pizurica, Universiteit Gent, 2006

Objectives of Image Restoration

• Image restoration likewise image enhancement attemts at improving the image quality

• Some overlap exists between image enhancement and restoration

• Important differences: image enhancement is largely subjective, while image restoration is mainly objective process

• Restoration attempts to recover an image that has been degraded by using a priori knowledge about degradation process

• Restoration techniques involve modelling of degradation and applying the inverse process in order to recover the image

• The restoration approach usually involves a criterion of goodness (e.g., mean squared error, smoothness, minimal desription length,…) that will yield an optimal estimate of the desired result

04.a4


Overview of restoration techniques

A categorization according to the degradation model (noise, blur or both)

Another possible categorization:• Spatial domain techniques• Frequency domain techniques• Other transform domain (e.g., wavelet) techniques

Model based approaches:• Bayesian techniques – make use of a priori knowledge about the

unknown, undegraded image statistical image modeling• Total variation – involves regularization – penalization of not-desired

local image structures

Statistical image modeling• Modeling marginal statistics (image histograms)• Context models modeling interactions among pixel intensities

–Powerful contextual models: Markov Random Field (MRF) models

Degradation model

04.a6


Reminder: camera model…

weighting function, e.g. w(x,y)=1for |x|<∆ and |y|<∆ and 0 otherwise

∫∫ −−== '')','()','(),( dydxyxwyyxxfyxff lkolkkl

A pixel sensor measures the image intensity in the neighborhood of (xk,yl)

),)(( lko yxwf ∗=

Remark: ),)('(),)((),(),( yxhfyxwhfyxyxff ilkkl f ∗=∗∗== where

⇒ Mathematical model: linear filter followed by ideal “sampling”

Camera: CCD (Charged-coupled device) pixel matrix

x, k

y, l

Optical system

fi(x,y)

h(x,y)f0(x,y)

kl

lk

fyxf

=),(

© W. Philips, Universiteit Gent, 1999-2006

04.a7


fkl=f(xk,yl)

…Reminder: camera model

The linear filter here is low-pass (attenuates high frequencies)⇒The image becomes blurred, fine details are lost

The sampling keeps only the values of f(x,y) at discrete positions (xk, yl)⇒ Aliasing appears if sampling frequency is not high enough

Remarks•Uniform sampling: xk=k∆, yl=l∆

linear filter

lensaveragingover pixels

idealsamplingfi(x,y) f(x,y)

=

lk

Vyx

l

k (sampling matrix V)•More general (sampling “lattice”)


! For compactness we shall write f(x,y) instead of f(xk,yl) !

04.a8


Degradation model…

If there is no aliasing we can model the analogue PSF by a digital filter

Scene Not-idealanti-alias filter

fx

H( fx, fy)

lklklk

lk

NFHG

,,,

,

+

=

noise

+ lkG , (DFT-coefficients)Ideal sampling

½ sampling frequency

Scene Lens with idealfrequency

characteristicIdeal

samplingEquivalent

(digital) PSF (linear filter)

+

noise

fx

Hideaal( fx, fy)

kHk,l

lkB ,'

Equivalent model

lkB ,


04.a9


Equivalent (digital) PSF +

noise

lkG ,Scene Lens with ideal

frequency characteristic

Idealsampling lkF ,

Scene Lens with ideal frequency

characteristicIdeal

sampling

…Degradation model

k

Hk,l

Digital filter: models imperfections in the lens, form of the pixels, … (if aliasing appears equivalent with analogue PSF may not hold)

lkN ,lklklklk NFHG ,,,, +=

Degradation function h(x,y) +

noisen(x,y)

),( yxg),( yxf

equivalent),(),(),(),( lklklkkk yxnyxfyxhyxg +∗=

equivalent

For compactness we write x,y instead of xk,yl

Noise reduction

04.a11


Why is denoising important

Not only visual enhancement, but also: automatic processing is facilitated!

original denoised

Example: edge detection

04.a12


Noise models…Noise models can be categorized according to

• marginal statistics (first-order statistics, marginal probability density function):• Gaussian, Rayleigh, Poisson, impulsive,…

• higher-order statistics• white noise (uncorrelated)• colored (correlated)

• type of mixing with the signal• additive• multiplicative• other (more complex)

• dependence on the signal• statistically independent of the signal• statisticaly dependent of the signal

Many techniques assume additive white Gaussian noise (AWGN) model

04.a13


Noise models: marginal statistics

* Gaussian e.g., thermal noise and a variety of noise sources* Rayleigh e.g. amplitude of random complex numbers whose real and

imaginary components are normally and independently distributed. Examples: ultrasound imaging

* Rice e.g., MRI image magnitude (Gaussian and Rayleigh are special cases of this distribution)

* Poisson models photon noise in the sensor (an average number of photons within a given observation window)

* Bipolar impulsive (e.g., salt and pepper) noise…

Rayleigh Rice Impulsive

Some common probability densitu functions (pdf’s)of noise:

04.a14


Noise in MRI

- observed noisy image (complex-valued);

- ideal, noise-free data;],...,[ 1 Nff=f

],...,[ 1 Ndd=d

)sin()cos( Im,Re, lllll nfjnfd +++= θθ

low SNR)1,0( 2 == nf σ

m

high SNR

f1

)1,( 21 == nff σ

p(m)

2Im,

2Re, )sin()cos(|| llllll nfnfdm +++== θθ

The magnitude ml

is Rician distributed

04.a15


Noise in MRI

high SNR ( f =f1 )

f1

low SNR (f=0)

m

p(m)

noise-free f

noisy m

mag

nitu

de

cont

rast

SNR

04.a16


Original image Image with white noise

Difference with the original

Noise models: correlation properties

white ⇔ uncorrelated

colored ⇔ correlatednoise

Image with colored noise

Difference with the original

© B. Goossens, Universiteit Gent, 2006

04.a17


Some reasons behind noise correlation …

interpolatedcaptured image

Bayer pattern


04.a18


RAW data – B channel Denoised RAW data - B ch.

Raw data in one color channel

noise is white

… Some reasons behind noise correlationInterpolated

noise is correlated

Resulting color imagewith correlated noise

04.a19


In many applications it is assumed that noise is additive and statistically independent of the signal

Types of mixing noise with signal

),(),(),( yxnyxfyxg =

),(),(),( yxnyxfyxg +=

In CMOS sensors there is a fixed-pattern noise and mixture ofadditive and multiplicative noise

This is a good model for example for thermal noise

Often, noise is signal-dependent. Examples: speckle, photon noise,…

Many noise sources can be modelled by a multiplicative model:

04.a20


2

4 3 2

0 9 1

1 2 3

Input image Output image

3 5

0 2

1 1

1 1 2 2 2

1 3 3 2 2

),( yxb ),( yxg

x

y

sorting 0 1 1 2 2 3 3 4 9

Order statistics filters: Median filter

Basic idea: remove “outliers”The median is a more robust statistical measure than mean


04.a21


Reduction of impulse noise

median over 3x3

Median filter removes isolated noise peaks, without blurring the image

impulse noise


04.a22


... Reduction of impulse noise …

median over 3x3Noise-free original

Median filter removes isolated noise peaks, without blurring the image


04.a23


Median filter and reduction of white noise

median over 3x3

For not-isolated noise peaks (e.g., white Gaussian noise) median filter is not very efficient.

original


04.a24


∑∈

=xySts

tsgmn

yxf),(

),(1),(ˆ

• Average within a local window Sxy• Aimed for Gaussian noise,

(but blurs edges)

Arithmetic mean

Some simple noise filters

Median

)},({median),(ˆ),(

tsgyxfxySts ∈

=

• Efficient for impulsive noise• Not efficient for Gaussian noise

MedianAlpha-trimmed mean

Discard d/2 lowest and d/2 largest values in Sxy

Denote by gr(s,t) the remaining mn-d pixels

∑∈−

=xyStsr tsg

dmnyxf

),(),(1),(ˆ

For d=0: mean filterFor d=mn-1: median filter

Digital Image Processing - UGentsanja/ImageProcessingCourse/04a... · Digital Image Processing ... image enhancement is largely subjective, ... If there is no aliasing we can model

Documents

Digital Image Processing - UGentsanja/ImageProcessingCourse/04a... · Digital Image Processing ... image enhancement is largely subjective, ... If there is no aliasing we can model