Top Banner
Image Processing in Freq. Domain • Restoration / Enhancement • Inverse Filtering • Match Filtering / Pattern Detection • Tomography
55

Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Dec 26, 2015

Download

Documents

Brenda Snow
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Image Processing in

Freq. Domain

• Restoration / Enhancement

• Inverse Filtering

• Match Filtering / Pattern Detection

• Tomography

Page 2: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Enhancement v.s. Restoration

• Image Enhancement: – A process which aims to improve bad

images so they will “look” better.

• Image Restoration: – A process which aims to invert known

degradation operations applied to images.

Page 3: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Enhancement vs. Restoration

• “Better” visual representation

• Subjective

• No quantitative measures

• Remove effects of sensing environment

• Objective

• Mathematical, model dependent quantitative measures

Page 4: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Typical Degradation SourcesTypical Degradation Sources

Low Illumination

Atmospheric attenuation(haze, turbulence, …)

Optical distortions(geometric, blurring)

Sensor distortion(quantization, sampling,

sensor noise, spectral sensitivity, de-mosaicing)

Page 5: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Image Preprocessing

Enhancement Restoration

SpatialDomain

Freq.Domain

Point operations Spatial operationsFiltering

• Denoising• Inverse filtering• Wiener filtering

Page 6: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Examples

Hazing

Page 7: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Echo image Motion Blur

Page 8: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Blurred image Blurred image + additive white noise

Page 9: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Reconstruction as an Inverse ProblemReconstruction as an Inverse Problem

Distortion

Hnfg Hf

n noise

measurements

Original image

Reconstruction Algorithmg f̂

Page 10: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• Typically:– The distortion H is singular or ill-posed.– The noise n is unknown, only its statistical properties

can be learnt.

g f̂ ng 1H

So what is the problem?

Page 11: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Key point: Stat. Prior of Natural Images

fPfgPgfPfxx

maxargmaxargˆ MAP estimation:

likelihood prior

Page 12: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Image space

measurements

•From amongst all possible solutions, choose the one that maximizes the a-posteriori probability:

Bayesian Reconstruction (MAP)Bayesian Reconstruction (MAP)

P(g|f)

P(f)

Most probable solutionP(f | g)P(g | f) P(f)

Page 13: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Bayesian Denoising• Assume an additive noise model :

g=f + n

• A MAP estimate for the original f:

• Using Bayes rule and taking the log likelihood :

gfPff

|maxargˆ

fPfgPgP

fPfgPf

fflog|logminarg

|maxargˆ

Page 14: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Bayesian Denoising

• If noise component is white Gaussian distributed:

g=f + n where n is distributed ~N(0,)

R(f) is a penalty for non probable f

fRfgff

2minargˆ

data term prior term

Page 15: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Inverse Filtering

• Degradation model:

g(x,y) = h(x,y)*f(x,y)

G(u,v)=H(u,v)F(u,v)

F(u,v)=G(u,v)/H(u,v)

Page 16: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Inverse Filtering (Cont.)

Two problems with the above formulation:1. H(u,v) might be zero for some (u,v).

2. In the presence of noise the noise might be amplified:

F(u,v)=G(u,v)/H(u,v) + N(u,v)/H(u,v)

Solution: Use prior information

FRGHFFF

2minargˆ

data term prior term

Page 17: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Option 1: Prior Term• Use penalty term that restrains high F values:

where

• Solution:

FEFF

minargˆ

22 FGHFFE

022 *

FGHFH

F

FE

GHH

HF

*

*

ˆ0ˆ1),(

ˆ1),(

FvuH

HGFvuH

Page 18: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Degraded Image (echo)

Page 19: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

F=G/H

Page 20: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

GHH

HF

*

*

ˆ

Page 21: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Degraded Image (echo+noise)

Page 22: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

GHH

HF

*

*

ˆ

Page 23: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• The inverse filter is C(H)= H*/(H*H+ )

• At some range of (u,v):

S(u,v)/N(u,v) < 1 noise amplification.

-1 -0.5 0 0.5 1 1.5 20

2

4

6

8

10

12

14

16

18

H

C(H

)

=10-3

Page 24: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Option 2: Prior Term1. Natural images tend to have low energy at high frequencies

2. White noise tend to have constant energy along freq.

where FEF

Fminargˆ

2222 FvuGHFFE

50 100 150 200 250

40

60

80

100

120

140

160

180

200

220

240

Page 25: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• Solution:

• This solution is known as the Wienner Filter• Here we assume N(u,v) is constant.• If N(u,v) is not constant:

022 22*

FvuGHFH

F

FE

GvuHH

HF

22*

*

ˆ

GvuNvuHH

HF

),(ˆ

22*

*

Page 26: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Degraded Image (echo+noise)

Page 27: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Wienner Filtering

Page 28: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Wienner Previous

Page 29: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Degraded Image (blurred+noise)

Page 30: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Inverse Filtering

Page 31: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Using Prior (Option 1)

Page 32: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Wienner Filtering

Page 33: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Matched Filter in Freq. Domain• Pattern Matching:

– Finding occurrences of a particular pattern in an image.

• Pattern:– Typically a 2D image fragment.

– Much smaller than the image.

Page 34: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• Image Similarity Measure:– A function that assigns a nonnegative real value to two

given images.

– Small measure high similarity– Preferable to be a metric distance (non-negative, identity,

symmetric, triangular inequality)

• Can be combined with thresholding:

Image Similarity Measures

d( - ) ≥ 0

1 ( , )( , )

0

d P Q thresholdf P Q

otherwise

Page 35: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• Scan the entire image pixel by pixel.

• For each pixel, evaluate the similarity between its local neighborhood and the pattern.

The Matching Approach

Page 36: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• Given:– k×k pattern P

– n×n image I

– kxk window of image I located at x,y - Ix,y

• For each pixel (x,y), we compute the distance:

• Complexity O(n2k2)

The Euclidean Distance as a Similarity Measure

1

0,

2

2

2

,2,2

,,1

1,

k

ji

yxyxE

jiPjyixIk

PIk

PId

Page 37: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• Convolution can be applied rapidly using FFT.

• Complexity: O(n2 log n)

FFT Implementation

Fixed 2( * )I P 2 * mask of 1'sI k k

2

,2,2 1

, PIk

PId yxyxE

jiPjyixIjiPjyixIk

ji

,,2,, 21

0,

2

Page 38: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Naïve FFT

Time Complexity

Space

Integer Arithmetic Yes No

Run time for 16×16 1.33 Sec. 3.5 Sec.

Run time for 32×32 4.86 Sec. 3.5 Sec.

Run time for 64×64 31.30 Sec. 3.5 Sec.

2( log )O n n2 2( )O n k2n2n

Performance table for a 1024×1024 image, on a 1.8 GHz PC:

Naïve vs. FFT

Page 39: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

0

0.5

1

1.5

2

x 107

Page 40: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• NCC:– A similarity measure, based on a normalized cross-

correlation function.

– Maps two given images to [0,1] (absolute value).

– Measures the angle between vectors Ix,y and P

– Invariant to intensity scale and offset.

Normalized Cross Correlation

11

11,

,

,,

PPII

PPIIPId

yx

yxyxNC

Page 41: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• Note that

• Thus,

• The above expression can be implemented efficiently using 3 convolutions.

Efficient Implementation

YXnYXYYXX 11

11

11,

,,

,,,

PPII

PPIIPId

yxyx

yxyxyxNC

22,

22,,

,2

,

PkPPIkII

PIkPI

yxyxyx

yxyx

Page 42: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

0

0.5

1

1.5

2

x 107

Euclidean distance similarity measure

0

0.5

1

1.5

NCC similarity measure

Page 43: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

10 20 30 40 50 60 70

10

20

30

40

50

60

0.2

0.4

0.6

0.8

1

1.2

10 20 30 40 50 60 70

10

20

30

40

50

60

1

2

3

4

5

6

7

8

9

10

11

x 106

Euclidean distance similarity measure

NCC similarity measure

Page 44: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

Computer Tomography using FFT

Page 45: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• In 1901 W.C. Roentgen won the Nobel Prize (1st in physics) for his discovery of X-rays.

CT Scanners

Wilhelm Conrad Röntgen

Page 46: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• In 1979 G. Hounsfield & A. Cormack, won the Nobel Prize for developing the computer tomography.

• The invention revolutionized medical imaging.

CT Scanners

Allan M. Cormack

Godfrey N. Hounsfield

Page 47: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

f(x,y)

1

2

Tomography: Reconstruction from Projection

Page 48: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• Projection: All ray-sums in a direction

• Sinogram: collects all projections

Projection & Sinogram

P(t)

f(x,y)

t

y

x

X-rays Sinogramt

Page 49: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

CT Image & Its Sinogram

K. Thomenius & B. Roysam

Page 50: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

The Slice Theorem

spatial domain frequency domain

f(x,y)

1

x

y

1

u

vFourier

Transform

Page 51: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

The Slice Theorem

f(x,y) = object

g(x) = projection of f(x,y) parallel to the y-axis: g(x) = f(x,y)dy

F(u,v) = f(x,y) e -2i(ux+vy) dxdyFourier Transform of f(x,y):

Fourier Transform at v=0 : F(u,0) = f(x,y) e -2iuxdxdy

= [ f(x,y)dy] e -2iuxdx

= g(x) e -2iux dx = G(u)

The 1D Fourier Transform of g(x)

Page 52: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

• Interpolate (linear, quadratic etc) in the frequency space to obtain all values in F(u,v).

• Perform Inverse Fourier Transform to obtain the image f(x,y).

Interpolation Method

u

v

F(u,v)

Page 53: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.
Page 54: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.
Page 55: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.

THE END