Image Restoration. Modified from restoration.ppt by Yu Hen Hu What is Image Restoration The purpose of image restoration is to restore a degraded/distorted.

Image Restoration

Modified from restoration.ppt by Yu Hen Hu

What is Image Restoration

The purpose of image restoration is to restore a degraded/distorted image to its original content and quality.

Distinctions to Image Enhancement Image restoration assumes a degradation model that is known or can

be estimated. Original content and quality ≠ Good looking


Interactive Restoration

Example 1 (periodic noise):Manually detect peaksIn the spectrum andConstruct a band-rejectfilter.


Interactive RestorationExample 2:

Take the IDFT of thepeaks in the spectrumand construct the noiseimage (e.g. Image c here)

Subtract locally weightednoise image from thedegraded image. Theweights can be estimatedby trying to minimize thevariance of the resulting image

(a)Original (b) Spectrum (c) IDFT of the peaks (d) Result


Image Degradation Model

Spatial variant degradation model

Spatial-invariant degradation model

Frequency domain representation

( , ) ( , , , ) ( , ) ( , )g x y h x y m n f m n x y

( , ) ( , ) ( , ) ( , )g x y h x m y n f m n x y

( , ) ( , ) ( , ) ( , )G u v H u v F u v N u v


Noise Models Most types of noise are

modeled as known probability density functions

Noise model is decided based on understanding of the physics of the sources of noise. Gaussian: poor illumination Rayleigh: range image Gamma, exp: laser imaging Impulse: faulty switch during

imaging, Uniform is least used.

Parameters can be estimated based on histogram on small flat area of an image


Noise Removal Restoration Method

Mean filters Arithmetic mean filter Geometric mean filter Harmonic mean filter Contra-harmonic mean

filter Order statistics filters

Median filter Max and min filters Mid-point filter alpha-trimmed filters

Adaptive filters Adaptive local noise

reduction filter Adaptive median filter


Mean Filters

,( , )

1ˆ( , ) ( , )x ys t S

f x y g s tmn

,

1

( , )

ˆ ( , ) ( , )x y

mn

s t S

f x y g s t


Contra-Harmonic Filters

,

,

1

( , )

( , )

( , )ˆ ( , )

( , )

x y

x y

Q

s t S

Q

s t S

g s t

f x yg s t


Median Filter

,( , )

ˆ ( , ) ( , )x ys t S

f x y median g s t

Effective for removing salt-and-paper (impulsive) noise.


LSI Degradation Models(Linear Space Invariant)

Motion Blur Due to camera panning or fast

motion

Atmospheric turbulence blur Due to long exposure time

through atmosphere

Hufnagel and Stanley

Uniform out-of-focus blur:

1 0( , )

0 .

ai bjh i j

otherwise

2 2

2( , ) exp

2

i jh i j K

2 2 22

1( , )

0 .

i j Rh i j R

otherwise

2

1/ 2 , / 2

( , )0 .

L i j Lh i j L

otherwise

5/62 2( , ) exph i j k i j


Turbulence Blur Examples

5/ 62 2( , ) exph i j k i j


Motion Blur

Often due to camera panning or fast object motion.

Linear along a specific direction.

blurring filter

20 40 60

20

40

60

blurring filter mask

2 4 6 8

2

4

6

8

original image

20 40 60

10

20

30

40

50

60

blurred image

20 40 60

10

20

30

40

50

60

Blurdemo.m


Inverse Filter

Recall the degradation model:

Given H(u,v), one may directly estimate the original image by

At (u,v) where H(u,v) 0, the noise N(u,v) term will be amplified!

( , ) ( , ) ( , ) ( , )G u v H u v F u v N u v

ˆ ( , ) ( , ) / ( , )

( , )( , )

( , )

F u v G u v H u v

N u vF u v

H u v

original, f

20 40 60

20

40

60

degraded: g

20 40 60

20

40

60

inverse filter

20 40 60

20

40

60

Invfildemo.m


Wiener Filtering (Least Mean Square Filtering)

Minimum mean-square error filter Assume f and are both 2D

random sequences, uncorrelated to each other.

Goal: to minimize Solution: Frequency selective

scaling of inverse filter solution!

White noise, unknown Sf(u,v):

2ˆE f f

2

2

( , ) ( , )ˆ ( , )( , )( , ) ( , ) / ( , )f

H u v G u vF u v

H u vH u v S u v S u v

2

2

( , ) ( , )ˆ ( , )( , )( , )

H u v G u vF u v

H u vH u v K

original, f

20 40 60

20

40

60

degraded: g

20 40 60

20

40

60

Wiener filter, K=0.2

20 40 60

20

40

60

inverse filter

20 40 60

20

40

60


Derivation of Wiener Filters

Given the degraded image g, the Wiener filter is an optimal filter hwin such that E{|| f – hwin**g||2} is minimized.

Assume that f and are uncorrelated zero mean stationary 2D random sequences with known power spectrum Sf and Sn. Thus,

2 2

2 *

2 2* *

2 2

* *

( , ) ( , ) ( , )

( , ) ( , ) ( , ) ( , )

( , ) ( , ) ( , ) ( , ) ( , )

( , ) ( , ) ( , ) ( , ) ( , )

( , ) ( , ) ( , ) ( , ) (

win win

win

win win

f win f n

win f win

C E f h g E F u v H u v G u v

E F u v H u v E F u v G u v

H u v E F u v G u v H u v E G u v

S u v H u v H u v S u v S u v

H u v H u v S u v H u v H u

, ) ( , )fv S u v

2

2

*

*

( , ) ( , )

( , ) ( , )

( , ) ( , )

( , ) ( , ) 0

f

n

E F u v S u v

E N u v S u v

E F u v N u v

E F u v N u v

*

2

( , ) 0

( , ) ( , )( , )

( , ) ( , ) ( , )

Set C/ win

fwin

f n

H u v

H u v S u vH u v

H u v S u v S u v

**


Constrained Least Square (CLS) Filter

Minimize:

where is an operator that measures the “roughness” (e.g. a second derivative operator)

Subject to constraint:

where

J q m n f m n: || ( , ) ** ( , )|| 2

q m n( , )

|| ( , ) ( , ) ** ( , )||g m n h m n f m n 2 2

2 0


Solution and Iterative Algorithm

To minimize CCLS, Set

CCLS/ F = 0. This yields

The value of however, has to be determined iteratively! It should be chosen such that

Iterative algorithm (Hunt)1. Set initial value of , 2. Find , and compute

R(u,v).3. If ||R||2 - ||N||2 < - a, set =

BL, increase , else if

||R||2 - ||N||2 > a, set = Bu, decrease , else stop iteration.

4. new = (Bu+BL)/2, go to step 2.

2

2 2

( , ) ( , ) ( , )

( , ) ( , ) ( , )

CLSC G u v H u v F u v

N u v Q u v F u v

*

2 2

( , )ˆ ( , ) ( , )( , ) ( , )

H u vF u v G u v

H u v Q u v

2 2

2 2

ˆ( , ) ( , ) ( , ) ( , )

( , ) ( , )

G u v H u v F u v N u v

R u v N u v a

ˆ ( , )F u v


CLS Demonstrationiteration 1, gamma=5.0005

20 40 60

20

40

60

iteration 4, gamma=0.62594

20 40 60

20

40

60


20 40 60

20

40

60


20 40 60

20

40

60


20 40 60

20

40

60


20 40 60

20

40

60

Image Restoration. Modified from restoration.ppt by Yu Hen Hu What is Image Restoration The purpose of image restoration is to restore a degraded/distorted.

Documents

weightednoise image

thedegraded image

image restorationwhat

degradeddistorted image

image c heresubtract

degraded image g

original image byat

mean stationary