Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Image Processing in

Freq. Domain

• Restoration / Enhancement

• Inverse Filtering

• Match Filtering / Pattern Detection

• Tomography

Enhancement v.s. Restoration

• Image Enhancement: – A process which aims to improve bad

images so they will “look” better.

• Image Restoration: – A process which aims to invert known

degradation operations applied to images.

Enhancement vs. Restoration

• “Better” visual representation

• Subjective

• No quantitative measures

• Remove effects of sensing environment

• Objective

• Mathematical, model dependent quantitative measures

Typical Degradation SourcesTypical Degradation Sources

Low Illumination

Atmospheric attenuation(haze, turbulence, …)

Optical distortions(geometric, blurring)

Sensor distortion(quantization, sampling,

sensor noise, spectral sensitivity, de-mosaicing)

Image Preprocessing

Enhancement Restoration

SpatialDomain

Freq.Domain

Point operations Spatial operationsFiltering

• Denoising• Inverse filtering• Wiener filtering

Examples

Hazing

Echo image Motion Blur

Blurred image Blurred image + additive white noise

Reconstruction as an Inverse ProblemReconstruction as an Inverse Problem

Distortion

Hnfg Hf

n noise

measurements

Original image

Reconstruction Algorithmg f̂

• Typically:– The distortion H is singular or ill-posed.– The noise n is unknown, only its statistical properties

can be learnt.

g f̂ ng 1H

So what is the problem?

Key point: Stat. Prior of Natural Images

fPfgPgfPfxx

maxargmaxargˆ MAP estimation:

likelihood prior

Image space

measurements

•From amongst all possible solutions, choose the one that maximizes the a-posteriori probability:

Bayesian Reconstruction (MAP)Bayesian Reconstruction (MAP)

P(g|f)

P(f)

Most probable solutionP(f | g)P(g | f) P(f)

Bayesian Denoising• Assume an additive noise model :

g=f + n

• A MAP estimate for the original f:

• Using Bayes rule and taking the log likelihood :

gfPff

|maxargˆ

fPfgPgP

fPfgPf

fflog|logminarg

|maxargˆ

Bayesian Denoising

• If noise component is white Gaussian distributed:

g=f + n where n is distributed ~N(0,)

R(f) is a penalty for non probable f

fRfgff

2minargˆ

data term prior term

Inverse Filtering

• Degradation model:

g(x,y) = h(x,y)*f(x,y)

G(u,v)=H(u,v)F(u,v)

F(u,v)=G(u,v)/H(u,v)

Inverse Filtering (Cont.)

Two problems with the above formulation:1. H(u,v) might be zero for some (u,v).

2. In the presence of noise the noise might be amplified:

F(u,v)=G(u,v)/H(u,v) + N(u,v)/H(u,v)

Solution: Use prior information

FRGHFFF

2minargˆ

data term prior term

Option 1: Prior Term• Use penalty term that restrains high F values:

where

• Solution:

FEFF

minargˆ

22 FGHFFE

022 *

FGHFH

F

FE

GHH

HF

*

*

ˆ0ˆ1),(

ˆ1),(

FvuH

HGFvuH

Degraded Image (echo)

F=G/H

GHH

HF

*

*

ˆ

Degraded Image (echo+noise)

GHH

HF

*

*

ˆ

• The inverse filter is C(H)= H*/(H*H+ )

• At some range of (u,v):

S(u,v)/N(u,v) < 1 noise amplification.

-1 -0.5 0 0.5 1 1.5 20

2

4

6

8

10

12

14

16

18

H

C(H

)

=10-3

Option 2: Prior Term1. Natural images tend to have low energy at high frequencies

2. White noise tend to have constant energy along freq.

where FEF

Fminargˆ

2222 FvuGHFFE

50 100 150 200 250

40

60

80

100

120

140

160

180

200

220

240

• Solution:

• This solution is known as the Wienner Filter• Here we assume N(u,v) is constant.• If N(u,v) is not constant:

022 22*

FvuGHFH

F

FE

GvuHH

HF

22*

*

ˆ

GvuNvuHH

HF

),(ˆ

22*

*

Degraded Image (echo+noise)

Wienner Filtering

Wienner Previous

Degraded Image (blurred+noise)

Inverse Filtering

Using Prior (Option 1)

Wienner Filtering

Matched Filter in Freq. Domain• Pattern Matching:

– Finding occurrences of a particular pattern in an image.

• Pattern:– Typically a 2D image fragment.

– Much smaller than the image.

• Image Similarity Measure:– A function that assigns a nonnegative real value to two

given images.

– Small measure high similarity– Preferable to be a metric distance (non-negative, identity,

symmetric, triangular inequality)

• Can be combined with thresholding:

Image Similarity Measures

d( - ) ≥ 0

1 ( , )( , )

0

d P Q thresholdf P Q

otherwise

• Scan the entire image pixel by pixel.

• For each pixel, evaluate the similarity between its local neighborhood and the pattern.

The Matching Approach

• Given:– k×k pattern P

– n×n image I

– kxk window of image I located at x,y - Ix,y

• For each pixel (x,y), we compute the distance:

• Complexity O(n2k2)

The Euclidean Distance as a Similarity Measure

1

0,

2

2

2

,2,2

,,1

1,

k

ji

yxyxE

jiPjyixIk

PIk

PId

• Convolution can be applied rapidly using FFT.

• Complexity: O(n2 log n)

FFT Implementation

Fixed 2( * )I P 2 * mask of 1'sI k k

2

,2,2 1

, PIk

PId yxyxE

jiPjyixIjiPjyixIk

ji

,,2,, 21

0,

2

Naïve FFT

Time Complexity

Space

Integer Arithmetic Yes No

Run time for 16×16 1.33 Sec. 3.5 Sec.

Run time for 32×32 4.86 Sec. 3.5 Sec.

Run time for 64×64 31.30 Sec. 3.5 Sec.

2( log )O n n2 2( )O n k2n2n

Performance table for a 1024×1024 image, on a 1.8 GHz PC:

Naïve vs. FFT

0

0.5

1

1.5

2

x 107

• NCC:– A similarity measure, based on a normalized cross-

correlation function.

– Maps two given images to [0,1] (absolute value).

– Measures the angle between vectors Ix,y and P

– Invariant to intensity scale and offset.

Normalized Cross Correlation

11

11,

,

,,

PPII

PPIIPId

yx

yxyxNC

• Note that

• Thus,

• The above expression can be implemented efficiently using 3 convolutions.

Efficient Implementation

YXnYXYYXX 11

11

11,

,,

,,,

PPII

PPIIPId

yxyx

yxyxyxNC

22,

22,,

,2

,

PkPPIkII

PIkPI

yxyxyx

yxyx

0

0.5

1

1.5

2

x 107

Euclidean distance similarity measure

0

0.5

1

1.5

NCC similarity measure

10 20 30 40 50 60 70

10

20

30

40

50

60

0.2

0.4

0.6

0.8

1

1.2

10 20 30 40 50 60 70

10

20

30

40

50

60

1

2

3

4

5

6

7

8

9

10

11

x 106

Euclidean distance similarity measure

NCC similarity measure

Computer Tomography using FFT

• In 1901 W.C. Roentgen won the Nobel Prize (1st in physics) for his discovery of X-rays.

CT Scanners

Wilhelm Conrad Röntgen

• In 1979 G. Hounsfield & A. Cormack, won the Nobel Prize for developing the computer tomography.

• The invention revolutionized medical imaging.

CT Scanners

Allan M. Cormack

Godfrey N. Hounsfield

f(x,y)

1

2

Tomography: Reconstruction from Projection

• Projection: All ray-sums in a direction

• Sinogram: collects all projections

Projection & Sinogram

P(t)

f(x,y)

t

y

x

X-rays Sinogramt

CT Image & Its Sinogram

K. Thomenius & B. Roysam

The Slice Theorem

spatial domain frequency domain

f(x,y)

1

x

y

1

u

vFourier

Transform

The Slice Theorem

f(x,y) = object

g(x) = projection of f(x,y) parallel to the y-axis: g(x) = f(x,y)dy

F(u,v) = f(x,y) e -2i(ux+vy) dxdyFourier Transform of f(x,y):

Fourier Transform at v=0 : F(u,0) = f(x,y) e -2iuxdxdy

= [ f(x,y)dy] e -2iuxdx

= g(x) e -2iux dx = G(u)

The 1D Fourier Transform of g(x)

• Interpolate (linear, quadratic etc) in the frequency space to obtain all values in F(u,v).

• Perform Inverse Fourier Transform to obtain the image f(x,y).

Interpolation Method

u

v

F(u,v)

THE END