Limitation of Imaging Technology

EE465: Introduction to Digital Image Processing 1

Limitation of Imaging Technology Two plagues in image acquisition

Noise interference Blur (motion, out-of-focus, hazy

weather) Difficult to obtain high-quality

images as imaging goes Beyond visible spectrum Micro-scale (microscopic imaging) Macro-scale (astronomical imaging)

What is Noise? Wiki definition: noise means any

unwanted signal One person’s signal is another

one’s noise Noise is not always random and

randomness is an artificial term Noise is not always bad (see

stochastic resonance example in the next slide)EE465: Introduction to Digital

Image Processing 2

Stochastic Resonance


no noise heavy noiselight noise


Image Denoising Where does noise come from?

Sensor (e.g., thermal or electrical interference)

Environmental conditions (rain, snow etc.)

Why do we want to denoise? Visually unpleasant Bad for compression Bad for analysis


electrical interference

Noisy Image Examples

thermal imaging

ultrasound imagingphysical interference


(Ad-hoc) Noise Modeling Simplified assumptions

Noise is independent of signal Noise types

Independent of spatial location Impulse noise Additive white Gaussian noise

Spatially dependent Periodic noise


Noise Removal Techniques Linear filtering Nonlinear filtering

Recall

Linear system


Image Denoising Introduction Impulse noise removal

Median filtering Additive white Gaussian noise

removal 2D convolution and DFT

Periodic noise removal Band-rejection and Notch filter


Impulse Noise (salt-pepper Noise)

Definition

Each pixel in an image has the probability of p/2 (0<p<1) being contaminated by either a white dot (salt) or a black dot (pepper)

WjHi

jiXjiY

1,1

),(0

255),(

with probability of p/2with probability of p/2with probability of 1-p

noisy pixels

clean pixels

X: noise-free image, Y: noisy image

Note: in some applications, noisy pixels are not simply black or white, which makes the impulse noise removal problem more difficult


Numerical ExampleP=0.1

128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128

X

128 128 255 0 128 128 128 128 128 128 128 128 128 128 0 128 128 128 128 0 128 128 128 128 128 128 128 128 128 128 128 128 0 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 0 128 128 128 128 255 128 128 128 128 128 128 128 128 128 128 128 128 128 255 128 128 128 128 128 128 128 255 128 128

Y

Noise level p=0.1 means that approximately 10% of pixels are contaminated bysalt or pepper noise (highlighted by red color)


MATLAB Command>Y = IMNOISE(X,'salt & pepper',p)

Notes:

The intensity of input images is assumed to be normalized to [0,1].If X is double, you need to do normalization first, i.e., X=X/255;If X is uint8, MATLAB would do the normalization automatically The default value of p is 0.05 (i.e., 5 percent of pixels are contaminated) imnoise function can produce other types of noise as well (you need to change the noise type ‘salt & pepper’)


Impulse Noise Removal Problem

Noisy image Y

filteringalgorithm

Can we make the denoised image X as closeto the noise-free image X as possible?

^

X̂denoisedimage


Median Operator Given a sequence of numbers {y1,

…,yN} Mean: average of N numbers Min: minimum of N numbers Max: maximum of N numbers Median: half-way of N numbersExample ]56,55,54,255,52,0,50[y

54)( ymedian

]255,56,55,54,52,50,0[y

sorted


y(n)

W=2T+1

)](),...,(),...,([)(ˆ TnynyTnymediannx

1D Median Filtering

… …

Note: median operator is nonlinear

MATLAB command: x=median(y(n-T:n+T));


Numerical Example]56,55,54,255,52,0,50[yT=1:

],56,55,54,255,52,0,50,[ 5650y

]55,55,54,54,52,50,50[ˆ x

BoundaryPadding


x(m,n)

W: (2T+1)-by-(2T+1) window

2D Median Filtering

)],(),...,,(),...,,(),...,,(),...,,([),(ˆ

TnTmyTnTmynmyTnTmyTnTmymediannmx

MATLAB command: x=medfilt2(y,[2*T+1,2*T+1]);


Numerical Example 225 225 225 226 226 226 226 226 225 225 255 226 226 226 225 226 226 226 225 226 0 226 226 255 255 226 225 0 226 226 226 226 225 255 0 225 226 226 226 255 255 225 224 226 226 0 225 226 226 225 225 226 255 226 226 228 226 226 225 226 226 226 226 226

0 225 225 226 226 226 226 226 225 225 226 226 226 226 226 226 225 226 226 226 226 226 226 226 226 226 225 225 226 226 226 226 225 225 225 225 226 226 226 226 225 225 225 226 226 226 226 226 225 225 225 226 226 226 226 226 226 226 226 226 226 226 226 226

Y X̂

Sorted: [0, 0, 0, 225, 225, 225, 226, 226, 226]


Image ExampleP=0.1

Noisy image Y X̂denoisedimage

3-by-3 window


Image Example (Con’t)

3-by-3 window 5-by-5 window

clean noisy(p=0.2)


Reflections What is good about median

operation? Since we know impulse noise appears

as black (minimum) or white (maximum) dots, taking median effectively suppresses the noise

What is bad about median operation? It affects clean pixels as well Noticeable edge blurring after median

filtering


Idea of Improving Median Filtering Can we get rid of impulse noise

without affecting clean pixels? Yes, if we know where the clean pixels

areor equivalently where the noisy pixels

are How to detect noisy pixels?

They are black or white dots


Median Filtering with Noise Detection

Noisy image Y

x=medfilt2(y,[2*T+1,2*T+1]);Median filtering

Noise detectionC=(y==0)|(y==255);

xx=c.*x+(1-c).*y;Obtain filtering results


Image Example

clean noisy(p=0.2)

w/onoisedetection

withnoisedetection







Additive White Gaussian Noise

DefinitionEach pixel in an image is disturbed by a Gaussian random variableWith zero mean and variance 2

WjHiNjiN

jiNjiXjiY

1,1),,0(~),(

),,(),(),(2

X: noise-free image, Y: noisy image

Note: unlike impulse noise situation, every pixel in the image contaminatedby AWGN is noisy


Numerical Example2 =1

X Y

128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128

128 128 129 127 129 126 126 128 126 128 128 129 129 128 128 127 128 128 128 129 129 127 127 128 128 129 127 126 129 129 129 128 127 127 128 127 129 127 129 128 129 130 127 129 127 129 130 128 129 128 129 128 128 128 129 129 128 128 130 129 128 127 127 126


MATLAB Command

>Y = IMNOISE(X,’gaussian',m,v)

>Y = X+m+randn(size(X))*v;or

Note:rand() generates random numbers uniformly distributed over [0,1]randn() generates random numbers observing Gaussian distributionN(0,1)


Image Denoising

Noisy image Y

filteringalgorithm

Question: Why not use median filtering?Hint: the noise type has changed.

X̂denoisedimage


f(n) h(n) g(n)

)()()()()()()( nhnfnfnhknfkhngk

- Linearity

- Time-invariant property

)()()()( 22112211 nganganfanfa

)()( 00 nngnnf

Linear convolution

1D Linear Filtering

See review section


jwnenfwF )()(

dwewFnf jwn)(

21)(

forward

inverse

Note that the input signal is a discrete sequence while its FT is a continuous function

)()( nhnf time-domain convolution

)()( wHwFfrequency-domain multiplication

Fourier Series


Filter Examples

0 0.5 1 1.5 2 2.5 3 3.50

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

LP HP

w

|H(w)|Low-pass (LP)

h(n)=[1,1]

|h(w)|=2cos(w/2)

High-pass (LP)

h(n)=[1,-1]

|h(w)|=2sin(w/2)


forward transform inverse transform

• Properties- periodic

- conjugate symmetric

)()( kyNky

)()( * kykNy

1D Discrete Fourier Transform

Proof:

1

0

)()(N

n

knNWnxky

1

0

)()(N

n

knNWkynx

1

0

1

0

)( )()(N

n

knNn

N

n

nNkNn kyWxWxNky

1

0

*1

0

)( )()(N

n

knNn

N

n

nkNNn kyWxWxkNyProof:

}2exp{NjWN


Matrix Representation of 1D DFT

FA

NNN

N

NN

aa

aa

..............................

......

1

111

1,

,1

2

NN

Nj

N

klNkl

WeW

WN

a

Re

Im

NWklN

N

llk Wx

Ny

1

1

N

llklk xay

1

DFT:


Fast Fourier Transform (FFT)*

Invented by Tukey and Cooley in 1965 Basic idea: divide-and-conquer

Reduce the complexity of N-point DFT from O(N2) to O(Nlog2N)

122/12

22/2

12

)12(12

2

22

1

0

mn

kmNm

k

mn

kmNm

mn

mkNm

mn

mkNm

N

n

knNnk

WxWWx

WxWxWxy

N

N/2-point DFT N/2-point DFT

N-point DFT


Filtering in the Frequency Domain

)()()( nhnfng

convolution in the time domain is equivalent to multiplication in the frequency domain

)()()( kHkFkG

f(n) h(n) g(n) F(k) H(k) G(k)

DFT


2D Linear Filtering

f(m,n) h(m,n) g(m,n)

),(),(),(),(),(,

nmfnmhlnkmflkhnmglk

2D convolution

MATLAB function: C = CONV2(A, B)


2D Filtering=Two Sequential 1D Filtering

Just as we have observed with 2D transform, 2D (separable) filtering can be viewed as two sequential 1D filtering operations: one along row direction and the other along column direction

The order of filtering does not matter )()()()(),( 1111 mhnhnhmhnmh

h1 : 1D filter


Numerical Exampleh1(m)=[1,1], h1(n)=[1,-1]1D filter

1111

)()( 11 nhmh

1111

)()( 11 mhnh

MATLAB command: >h1=[1,1];h2=[1,-1];>conv2(h1,h2)>conv2(h2,h1)


Fourier Series (2D case)

m n

nwmwjenmfwwF )(21

21),(),(

Note that the input signal is discrete while its FT is a continuous function

),(),( nmhnmf spatial-domain convolution

),(),( 2121 wwHwwFfrequency-domain multiplication


Filter Examples Low-pass (LP)

h1(n)=[1,1]

|h1(w)|=2cos(w/2)1D

h(n)=[1,1;1,1]

|h(w1,w2)|=4cos(w1/2)cos(w2/2)2D

w1

w2

|h(w1,w2)|


Image DFT Example

Original ray image X choice 1: Y=fft2(X)


Image DFT Example (Con’t)

choice 1: Y=fft2(X) choice 2: Y=fftshift(fft2(X))Low-frequency at the centerLow-frequency at four corners

FFTSHIFT Shift zero-frequency component to center of spectrum.


Gaussian Filter

)2

exp(),( 2

22

21

21 wwwwH

)2

exp(),( 2

22

nm

nmh

FT

>h=fspecial(‘gaussian’, HSIZE,SIGMA);MATLAB code:


(=1)PSNR=24.4dB

Image Example

PSNR=20.2dB

noisy

(=25)

denoised denoised

(=1.5)PSNR=22.8dB

Matlab functions: imfilter, filter2

45

Gaussian Filter=Heat Diffusion

2

2

2

2 ),,(),,(),,(),,(y

tyxIx

tyxItyxIttyxI

Linear Heat Flow Equation:

)()0,,(),,( tGyxItyxI

scale A Gaussian filterwith zero mean and variance of t

Isotropic diffusion:

46

Basic Idea of Nonlinear Diffusion*

x

y I(x,y)

image I

image I viewed as a 3D surface (x,y,I(x,y))Diffusion should be anisotropicinstead of isotropic

Experimental Results


(Gaussian filtering)PSNR=24.4dBPSNR=20.2dB

noisy

(=25)

linear diffusion

(TV filtering)PSNR=27.5dB

nonlinear diffusion

Hammer-Nail Analogy


Gaussian filter

median filter

salt-pepper/impulse noise

Gaussian noise

periodic noise???







Periodic Noise Source: electrical or electromechanical

interference during image acquistion Characteristics

Spatially dependent Periodic – easy to observe in frequency

domain Processing method

Suppressing noise component in frequency domain


Image Example

spatial

Frequency (note the four pairs of bright dots)


Band Rejection Filter

otherwise

WDwwWDwwH1

220),(

22

21

21

w1

w2


Image Example

Before filtering After filtering

Advanced Denoising Techniques*


IN

Is

IEIW

jijiN III ,,1

jijiS III ,,1

jijiE III ,1,

jijiW III ,1,

][,1

, IcIcIcIcII WWEESSNNtji

tji

WESNdIgc dd ,,,||),(||

Basic idea: from linear diffusion (equivalent to Gaussian filtering)to nonlinear diffusion (with implicit edge-stopping criterion)

Limitation of Imaging Technology

Documents

digital image processing

denoisedimage ee465

gaussian distributionn0

random numbers

powerpoint presentation

d dft reim dft

d discrete fourier

properties periodic