DIP Image Enhancement PART2

Course Website: http://www.comp.dit.ie/bmacnamee

Digital Image Processing

Image Enhancement(Histogram Processing)

2of32

Come To The LABS!

Day: WednesdayTime: 9:00 – 11:00Room: Aungier St. 1-005We will start by getting to grips with the basics of Scilab

– Lab details available at WebCT

Shortly, there will be a Scilab assignment which will count towards your final mark

3of32

Contents

Over the next few lectures we will look at image enhancement techniques working in the spatial domain:

– What is image enhancement?– Different kinds of image enhancement– Histogram processing– Point processing– Neighbourhood operations

4of32

A Note About Grey Levels

So far when we have spoken about image grey level values we have said they are in the range [0, 255]

– Where 0 is black and 255 is white

There is no reason why we have to use this range

– The range [0,255] stems from display technologes

For many of the image processing operations in this lecture grey levels are assumed to be given in the range [0.0, 1.0]

5of32

What Is Image Enhancement?

Image enhancement is the process of making images more usefulThe reasons for doing this include:

– Highlighting interesting detail in images– Removing noise from images– Making images more visually appealing

6of32

Image Enhancement ExamplesIm

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

7of32

Image Enhancement Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

8of32

Image Enhancement Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

9of32

Image Enhancement Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

10of32

Spatial & Frequency Domains

There are two broad categories of image enhancement techniques

– Spatial domain techniques• Direct manipulation of image pixels

– Frequency domain techniques• Manipulation of Fourier transform or wavelet

transform of an image

For the moment we will concentrate on techniques that operate in the spatial domain

11of32

Image Histograms

The histogram of an image shows us the distribution of grey levels in the imageMassively useful in image processing, especially in segmentation

Grey Levels

Freq

uenc

ies

12of32

Histogram ExamplesIm

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

13of32

Histogram Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

14of32

Histogram Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

15of32

Histogram Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

16of32

Histogram Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

17of32

Histogram Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

18of32

Histogram Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

19of32

Histogram Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

20of32

Histogram Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

21of32

Histogram Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

22of32

Histogram Examples (cont…)

A selection of images and their histogramsNotice the relationships between the images and their histogramsNote that the high contrast image has the most evenly spaced histogram

Imag

es ta

ken

from

Gon

zale

z &

Woo

ds, D

igita

l Im

age

Pro

cess

ing

(200

2)

23of32

Contrast Stretching

We can fix images that have poor contrast by applying a pretty simple contrast specificationThe interesting part is how do we decide on this transformation function?

24of32

Histogram Equalisation

Spreading out the frequencies in an image (or equalising the image) is a simple way to improve dark or washed out imagesThe formula for histogram equalisation is given where

– rk: input intensity– sk: processed intensity– k: the intensity range

(e.g 0.0 – 1.0)– nj: the frequency of intensity j– n: the sum of all frequencies

)( kk rTs

k

jjr rp

1

)(

k

j

j

nn

1

25of32

Equalisation Transformation FunctionIm

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

26of32

Equalisation ExamplesIm

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

1

27of32

Equalisation Transformation Functions

The functions used to equalise the images in the previous example

Imag

es ta

ken

from

Gon

zale

z &

Woo

ds, D

igita

l Im

age

Pro

cess

ing

(200

2)

28of32

Equalisation ExamplesIm

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002)

2

29of32

Equalisation Transformation Functions

The functions used to equalise the images in the previous example

Imag

es ta

ken

from

Gon

zale

z &

Woo

ds, D

igita

l Im

age

Pro

cess

ing

(200

2)

30of32

Equalisation Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002) 3

4

31of32

Equalisation Examples (cont…)Im

ages

take

n fro

m G

onza

lez

& W

oods

, Dig

ital I

mag

e P

roce

ssin

g (2

002) 3

4

32of32

Equalisation Transformation Functions

The functions used to equalise the images in the previous examples

Imag

es ta

ken

from

Gon

zale

z &

Woo

ds, D

igita

l Im

age

Pro

cess

ing

(200

2)

33of32

Summary

We have looked at:– Different kinds of image enhancement– Histograms– Histogram equalisation

Next time we will start to look at point processing and some neighbourhood operations

HISTOGRAM HISTOGRAM TRANSFORMATION IN IMAGE TRANSFORMATION IN IMAGE

PROCESSING AND ITS PROCESSING AND ITS APPLICATIONSAPPLICATIONS

Attila Kuba

University of Szeged

ContentsContents

HistogramHistogram transformationHistogram equalizationContrast strechingApplications

HistogramHistogram

0 1 1 2 42 1 0 0 25 2 0 0 41 1 2 4 1

The (intensity or brightness) histogram shows how many times a particular grey level (intensity) appears in an image.

For example, 0 - black, 255 – white

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6

image histogram

Histogram IIHistogram IIAn image has low contrast when the complete range of possible values is not used.  Inspection of the histogram shows this lack of contrast.

Histogram of color imagesHistogram of color imagesRGB color can be converted to a gray scale value by

    Y = 0.299R + 0.587G + 0.114B

Y: the grayscale component in the YIQ color space used in NTSC television.  The weights reflect the eye's brightness sensitivity to the color primaries.

Histogram of color images IIHistogram of color images II

Histogram:individual histograms of red, green and blue

Blue

R

R RGB

Histogram of Histogram of color images IIIcolor images III

Histogram of color images IV Histogram of color images IV

ora 3-D histogram can be produced, with

the three axes representing the red, blue and green channels, and brightness at each point representing the pixel count

Histogram transformationHistogram transformationPoint operation T(rk) =sk

rk

Tsk

Properties of T: keeps the original range of grey valuesmonoton increasing

grey values:

Histogram equalization (HE)Histogram equalization (HE)

transforms the intensity values so that the histogram of the output image approximately matches the flat (uniform) histogram

Histogram equalization II.Histogram equalization II.

As for the discrete case the following formula applies:

k = 0,1,2,...,L-1

L: number of grey levels in image (e.g., 255)

nj: number of times j-th grey level appears in image

n: total number of pixels in the image

·(L-1)

?

Histogram equalization IIIHistogram equalization III

Histogram equalization IVHistogram equalization IV

Histogram equalization VHistogram equalization V

cumulative histogram

Histogram equalization VIHistogram equalization VI

Histogram equalization VIIHistogram equalization VII

HE

histogram can be taken also on a part of the image

Histogram equalization VIIIHistogram equalization VIII

Histogram projection (HP)Histogram projection (HP)

assigns equal display space to every occupied raw signal level, regardless of how many pixels are at that same level. In effect, the raw signal histogram is "projected" into a similar-looking display histogram.

Histogram projection IIHistogram projection II

HE HP

IR image

Histogram projection IIIHistogram projection III

occupied (used) grey level: there is at least one pixel with that grey level

B(k): the fraction of occupied grey levels at or below grey level k B(k) rises from 0 to 1 in discrete uniform steps of 1/n, where n is the total number of occupied levels

HP transformation:

sk = 255 ·B(k).

Plateau equalizationPlateau equalization

By clipping the histogram count at a saturation or plateau value, one can produce display allocations intermediate in character between those of HP and HE.

Plateau equalization IIPlateau equalization II

HE PE 50

Plateau equalization IIIPlateau equalization III

The PE algorithm computes the distribution not for the full image histogram but for the histogram clipped at a plateau (or saturation) value in the count. When that plateau value is set at 1, we generate B(k) and so perform HP; When it is set above the histogram peak, we generate F(k) and so perform HE. At intermediate values, we generate an intermediate distribution which we denote by P(k).

PE transformation:

sk = 255· P(k)

Histogram specification (HS)Histogram specification (HS)

an image's histogram is transformed according to a desired function

Transforming the intensity values so that the histogram of the output image approximately matches a specified histogram.

Histogram specification IIHistogram specification II

ST

S-1*T

histogram1 histogram2

?

Contrast streching (CS)Contrast streching (CS)

By stretching the histogram we attempt to use the available full grey level range.

The appropriate CS transformation :sk = 255·(rk-min)/(max-min)

Contrast streching IIContrast streching II

Contrast streching IIIContrast streching III

CS does not help here

HE?

Contrast streching IVContrast streching IV

CS

HE

Contrast streching VContrast streching V

CS1% - 99%

Contrast streching VIContrast streching VI

HE

CS79, 136

CSCutoff fraction: 0.8

Contrast streching VIIIContrast streching VIII

a more general CS:

0, if rk < plow

sk = 255·(rk- plow)/(phigh - plow), otherwise255, if rk > phigh

Contrast streching IXContrast streching IX

Contrast streching XContrast streching X

Contrast streching XIContrast streching XI

ApplicationsApplicationsCT lung studiesThresholdingNormalizationNormalization of MRI imagesPresentation of high dynamic images (IR, CT)

CT lung studiesCT lung studies

Yinpeng Jin HE taken in a part of the image

CT lung studiesCT lung studies

R.Rienmuller

ThresholdingThresholdingconverting a greyscale image to a binary one

for example, when the histogram is bi-modal

threshold: 120

Thresholding IIThresholding IIwhen the histogram is not bi-modal

threshold: 80 threshold: 120

Normalization INormalization I

When one wishes to compare two or more images on a specific basis, such as texture, it is common to first normalize their histograms to a "standard" histogram. This can be especially useful when the images have been acquired under different circumstances. Such a normalization is, for example, HE.

Normalization IINormalization II

Histogram matching takes into account the shape of the histogram of the original image and the one being matched.

Normalization of MRI images Normalization of MRI images MRI intensities do not have a fixed meaning,

not even within the same protocol for the same body region obtained on the same scanner for the same patient.

Normalization of MRI images II Normalization of MRI images II

L. G. Nyúl, J. K. Udupa

Normalization of MRI images III Normalization of MRI images III

L. G. Nyúl, J. K. Udupa0

1000

2000

3000

4000

5000

6000

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

0

1000

2000

3000

4000

5000

6000

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

0

1000

2000

3000

4000

5000

6000

0 500 1000 1500 2000 2500

A: Histograms of 10 FSE PD brain volume images of MS patients.

B: The same histograms after scaling.

C: The histograms after final standardization.

A

B C

Normalization of MRI images IVNormalization of MRI images IV

m1 m2p1 p2m1 m2p1 p2

Method: transforming image histograms by landmark matching

Determine location of landmark i (example: mode, median, various percentiles (quartiles, deciles)).Map intensity of interest to standard scale for each volume image linearly and determine the location ’s of i on standard scale.

unimodal bimodal

Normalization of MRI images VNormalization of MRI images V

Applications IIIApplications III

A digitized high dynamic range image, such as an infrared (IR) image or a CAT scan image, spans a much larger range of levels than the typical values (0 - 255) available for monitor display. The function of a good display algorithm is to map these digitized raw signal levels into display values from 0 to 255 (black to white), preserving as much information as possible for the purposes of the human observer.

Applications IVApplications IV

The HP algorithm is widely used by infrared (IR) camera manufacturers as a real-time automated image display.

The PE algorithm is used in the B-52 IR navigation and targeting sensor.

Image Enhancement: Histogram Based Methods

The histogram of a digital image with gray values110 ,,, Lrrr

is the discrete function

nn

rp kk )(

nk: Number of pixels with gray value rk

n: total Number of pixels in the image

The function p(rk) represents the fraction of the total number of pixels with gray value rk.

What is the histogram of a digital image?

Histogram provides a global description of the appearance ofthe image.

If we consider the gray values in the image as realizations of arandom variable R, with some probability density, histogramprovides an approximation to this probability density. In otherwords,

)()Pr( kk rprR

Some Typical HistogramsThe shape of a histogram provides useful information forcontrast enhancement.

Dark image

Bright image

Low contrast image

High contrast image

Histogram Equalization

Let us assume for the moment that the input image to beenhanced has continuous gray values, with r = 0 representingblack and r = 1 representing white.

We need to design a gray value transformation s = T(r), basedon the histogram of the input image, which will enhance theimage.

What is the histogram equalization?

he histogram equalization is an approach to enhance a given image. The approach is to design a transformation T(.) such that the gray values in the output is uniformly distributed in [0, 1].

Assume that:(1) T(r) is a monotonically increasing function for 0 r 1 (preserves order from black to white). (2) T(r) maps [0,1] into [0,1] (preserves the range of allowedGray values). ie 0 ≤T( r) ≤1 for 0 ≤r≤1

inverse transformation is denoted by r T -1(s) . Weassume that the inverse transformation also satisfies the abovetwo conditions.

Consider the gray values in the input image and outputimage as random variables in the interval [0, 1].

Let pr(r) and p s(s) denote the probability density of theGray values in the input and output images.

If and T(r) are known, and r T -1(s) satisfies condition 1, we can write (result from probability theory):

One way to enhance the image is to design a transformationT(.) such that the gray values in the output is uniformlydistributed in [0, 1], i.e. ps (s) 1, 0 s1

In terms of histograms, the output image will have allgray values in “equal proportion” .

This technique is called histogram equalization.

1 ( )

( ) ( )s r

r T s

drp s p rds

This is the cumulative distribution function (CDF) of pr (r) and satisfies the previous two conditions.

From the previous equation and using the fundamentaltheorem of calculus,

( )rds p rdr

0( ) ( ) 0 1,r

rs T r p w dw r

The output probability density function is uniform, regardless of the input.

Thus, using a transformation function equal to the CDF of input gray values r, we can obtain an image with uniform gray values. This usually results in an enhanced image, with an increase in the dynamic range of pixel values.

1

1( )

( )

1( ) ( ) 1 1, 0 1( )r

rs r T sr T s

p s p r sp r

Step 1:For images with discrete gray values, compute:

nn

rp kkin )( 10 kr 10 Lk

L: Total number of gray levels

nk: Number of pixels with gray value rk

n: Total number of pixels in the image

Step 2: Based on CDF, compute the discrete version of the previous transformation :

How to implement histogram equalization?

0

( ) ( )k

rk k jj

s T r p r

0 1k L

Example:

Consider an 8-level 64 x 64 image with gray values (0, 1, …,7). The normalized gray values are (0, 1/7, 2/7, …, 1). Thenormalized histogram is given below:

NB: The gray values in output are also (0, 1/7, 2/7, …, 1).

Gray value

# pixels

Normalized gray value

Fraction of # pixels

Applying the transformation,

k

jjinkk rprTs

0

)()( we have

Notice that there are only five distinct gray levels --- (1/7, 3/7,5/7, 6/7, 1) in the output image. We will relabel them as (s0,s1, …, s4 ).

With this transformation, the output image will havehistogram

Histogram of output image

# pixels

Gray values

Note that the histogram of output image is only approximately, and not exactly, uniform. This should not be surprising, since there is no result that claims uniformity in the discrete case.

Example Original image and its histogram

Histogram equalized image and its histogram

Comments:Histogram equalization may not always produce desirableresults, particularly if the given histogram is very narrow. Itcan produce false edges and regions. It can also increaseimage “graininess” and “patchiness.”

Histogram Specification(Histogram Matching)

Histogram equalization yields an image whose pixels are (intheory) uniformly distributed among all gray levels.

Sometimes, this may not be desirable. Instead, we may want atransformation that yields an output image with a pre-specifiedhistogram. This technique is called histogram specification.

Given Information

(1) Input image from which we can compute its histogram .

(2) Desired histogram.

Goal

Derive a point operation, H(r), that maps the input image into an output image that has the user-specified histogram.

Again, we will assume, for the moment, continuous-gray values.

Input image Uniform image

Output image

s=T(r) v=G(z)

z=H(r)

Approach of derivation

= G-1(v=s=T(r))

Suppose, the input image has probability density in p(r) . Wewant to find a transformation z H (r), such that the probability density of the new image obtained by this transformation is pout(z) , which is not necessarily uniform.

This gives an image with a uniform probability density.

First apply the transformation

10)()( ,0 rdwwprTs

r

in

If the desired output image were available, then the followingtransformation would generate an image with uniform density:

10)()( ,0 zdwwpzGV

z

out (**)

(*)

From the gray values we can obtain the gray values z byusing the inverse transformation, z G-1(v)

will generate an image with the specified density out p(z) ,from an input image with density in p(r) !

If instead of using the gray values obtained from (**), weuse the gray values s obtained from (*) above (both areuniformly distributed ! ), then the point transformation

Z=H(r)= G-1[ v=s =T(r)]

For discrete gray levels, we have

k

jjinkk rprTs

0

)()( 10 Lk

k

k

jjoutkk szpzGv

0

)()( 10 Lk

If the transformation zk G(zk)is one-to-one, the inversetransformation skG-1 (sk), can be easily determined, sincewe are dealing with a small set of discrete gray values.

In practice, this is not usually the case (i.e., ) zk G(zk)is not one-to-one) and we assign gray values to match the given histogram, as closely as possible.

Algorithm for histogram specification:

(1) Equalize input image to get an image with uniform gray values, using the discrete equation:

k

jjinkk rprTs

0

)()( 10 Lk

(2) Based on desired histogram to get an image with uniform gray values, using the discrete equation:

k

k

jjoutkk szpzGv

0

)()( 10 Lk

(3) )]([ )( 11 rTGzv=sGz

Example:

Consider an 8-level 64 x 64 previous image.

Gray value

# pixels

It is desired to transform this image into a new image, using a transformation Z=H(r)= G-1[T(r)], with histogram as specified below:

Gray values

# pixels

The transformation T(r) was obtained earlier (reproducedbelow):

Now we compute the transformation G as before.

Computer z=G-1 (s)Notice that G is not invertible.

G-1(0) = ?

G-1(1/7) = 3/7

G-1(2/7) = 4/7

G-1(4/7) = ?

G-1(5/7) = 5/7

G-1(6/7) = 6/7

G-1(1) = 1

G-1(3/7) = ?

Combining the two transformation T and G-1 , compute z=H(r)= G-

1[v=s=T(r)]

Applying the transformation H to the original image yields an image with histogram as below:

Again, the actual histogram of the output image does not exactly but only approximately matches with the specified histogram. This is because we are dealing with discrete histograms.

Original image and its histogram

Histogram specified image and its histogram

Desired histogram

Part 2

CIS 350 – 3

Image ENHANCEMENTin theSPATIAL DOMAIN

Dr. Rolf Lakaemper

Most of these slides base on the textbook

Digital Image Processing by Gonzales/Woods Chapter 3

Histograms

So far (part 1) :

• Histogram definition• Histogram equalization

Now:

• Histogram statistics

Histograms

Remember:The histogram shows the number of

pixels having a certain gray-value

nu

mbe

r of p

ixel

s

grayvalue (0..1)

Histograms

The NORMALIZED histogram is the histogram divided by the total number of pixels in the source image.

The sum of all values in the normalized histogram is 1.

The value given by the normalized histogram for a certain gray value can be read as the probability of randomly picking a pixel having that gray value

Histograms

What can the (normalized) histogram tell about the image ?

Histograms

1.The MEAN VALUE (or average gray level)

M = g g h(g)1*0.3+2*0.1+3*0.2+4*0.1+5*0.2+6*0.1=2.60.3

0.20.10.0

1 2 3 4 5 6

Histograms

The MEAN value is the average gray value of the image, the ‘overall brightness appearance’.

Histograms

2. The VARIANCE

V = g (g-M)2 h(g)

(with M = mean)or similar:

The STANDARD DEVIATION

D = sqrt(V)

Histograms

VARIANCE gives a measure about the distribution of the histogram values around the mean.

0.30.20.10.0

0.30.20.10.0

V1 > V2

Histograms

The STANDARD DEVIATION is a value on the gray level axis, showing the average distance of all pixels to the mean

0.30.20.10.0

0.30.20.10.0

D1 > D2

Histograms

VARIANCE and STANDARD DEVIATION of the histogram tell us about the average contrast of the image !

The higher the VARIANCE (=the higher the STANDARD DEVIATION), the higher the image’s contrast !

Histograms

Example:

Image and blurred version

Histograms

Histograms with MEAN and STANDARD DEVIATION

M=0.73 D=0.32 M=0.71 D=0.27

Histograms

Exercise:

Design an autofocus system for a digital camera !

The system should analyse an area in the middle of the picture and automatically adjust the lens such that this area is sharp.

Spatial Filtering

End of histograms.

And now to something completely different …

Spatial Filtering

Spatial Filtering

Spatial Filtering

Spatial Filtering:

Operation on the set of ‘neighborhoods’ N(x,y) of each pixel

6 8 2 0

12 200 20 10

226

6 8

12 200

(Operator: sum)

Spatial Filtering

Neighborhood of a pixel p at position x,y is aset N(p) of pixels defined relative to p.

Example 1: N(p) = {(x,y): |x-xP|=1, |y-yP| = 1}

P

Q

Spatial Filtering

More examples of neighborhoods:

P P P P

P P

Spatial Filtering

Usually neighborhoods are used which are close to discs, since properties of the eucledian metric are often useful.

The most prominent neighborhoods are the 4-Neighborhood and the 8-Neighborhood

P P

Spatial Filtering

We will define spatial filters on the 8-Neighborhood and their bigger relevants.

PP

P

N8

N24

N48

Spatial Filtering

Index system for N8:

n3n2n1

n6n5n4

n9n8n7

Spatial Filtering

Motivation: what happens to P if we apply thefollowing formula:

P = i ni

n3n2n1

n6n5=Pn4

n9n8n7

Spatial Filtering

What happens to P if we apply this formula:

P = i ai ni

with ai given by: a3=1a2=1a1=1

a6=1a5=4a4=1

a9=1a8=1a7=1

Spatial Filtering

Lets have a look at different values of ai and their effects !

This MATLAB program creates an interesting output:

% Description: given an image 'im', % create 12 filtered versions using % randomly designed filters

for i=1:12 a=rand(7,7); % create a

% random 7x7 % filter-matrix a=a*2 - 1; % range: -1 to 1 a=a/sum(a(:)); % normalize im1=conv2(im,a);% Filter subplot(4,3,i); imshow(im1/max(im1(:)));end

Spatial Filtering

Spatial Filtering

Different effects of the previous slides included:

• Blurring / Smoothing• Sharpening• Edge Detection

All these effects can be achieved using different coefficients.

Spatial Filtering

Blurring / Smoothing(Sometimes also referred to as averaging or lowpass-filtering)

Average the values of the center pixel and its neighbors:

Purpose:

• Reduction of ‘irrelevant’ details• Noise reduction• Reduction of ‘false contours’ (e.g.

produced by zooming)

Spatial Filtering

Blurring / Smoothing

1 1 1

1 1 1

1 1 1

* 1/9

Apply this scheme to every single pixel !

Spatial Filtering

Example 2:Weighted average

1 2 1

2 4 2

1 2 1

* 1/16

Spatial Filtering

Basic idea:Weigh the center point the highest,

decrease weight by distance to center.

The general formula for weighted average:

P = i ai ni / i ai

Constant value, depending on mask, not on image !

Spatial Filtering

Blurring using different radii (=size of neighborhood)

Spatial Filtering

EDGE DETECTION

Spatial Filtering

EDGE DETECTION

Purpose:

• Preprocessing• Sharpening

Spatial Filtering

EDGE DETECTION

Purpose:

• Preprocessing• Sharpening

Spatial Filtering

Motivation: Derivatives

Spatial Filtering

First and second order derivative

Spatial Filtering

First and second order derivative

1. 1st order generally produces thicker edges2. 2nd order shows stronger response to detail3. 1st order generally response stronger to gray level step4. 2nd order produce double (pos/neg) response at step change

Spatial Filtering

Definition of 1 dimensional discrete 1st order derivative:

dF/dX = f(x+1) – f(x)

The 2nd order derivative is the derivative of the 1st order derivative…

Spatial Filtering

The 2nd order derivative is the derivative of the 1st order derivative…

F(x-1) F(x) F(x+1)

F(x)-F(x-1) F(x+1)-F(x) F(x+2)-F(x+1)

derive

F(x+1)-F(x) – (F(x)-F(x-1))

F(x+1)-F(x) – (F(x)-F(x-1))= F(x+1)+F(x-1)-2F(x)

Spatial Filtering

F(x+1)-F(x) – (F(x)-F(x-1))= F(x+1)+F(x-1)-2F(x)

1 -2 1

One dimensional 2nd derivative in x direction:

One dimensional 2nd derivative, direction y:

-2

1

1

Spatial Filtering

1 -2 1

TWO dimensional 2nd derivative:

-2

1

1

+ =0 0 0

0 0 0 0

0

0

0

0

0

-4

1

1

0

1

0

0

1

0

This mask is called the ‘LAPLACIAN’(remember calculus ?)

Spatial Filtering

Different variants of the Laplacian

Spatial Filtering

Effect of the Laplacian… (MATLAB demo)

im = im/max(im(:));

% create laplacianL=[1 1 1;1 -8 1; 1 1 1];

% Filter !im1=conv2(im,L);

% normalize and showim1=(im1-min(im1(:))) / (max(im1(:))-min(im1(:)));imshow(im1);

Spatial Filtering

Spatial Filtering

Edges detected by the Laplacian can be used to sharpenthe image !

+

Spatial Filtering

Spatial Filtering

Sharpening can be done in 1 pass:

-1 4 -1 1

0

0

+ =0 -1 0

0 -1 0 0

0

0

0

0

0

5

-1

-1

0

-1

0

0

-1

0

LAPLACIAN Original Image Sharpened Image

Spatial Filtering

Sharpening in General:

Unsharp Masking and

High Boost Filtering

Spatial Filtering

Basic Idea (unsharp masking):

Subtract a BLURRED version of an image from the image itself !

Fsharp = F – Fblurred

Spatial Filtering

Variation: emphasize original image(high-boost filtering):

Fsharp = a*F – Fblurred , a>=1

0 a 0 1

1

1

-=0 0 0

0 0 0 0

1

0

0

1

0

a-1

-1

-1

0

-1

0

0

-1

0

Spatial Filtering

Different Examples of High-Boost Filters:

0

-1

-1

0

-1

0

0

-1

0

a=1

5

-1

-1

0

-1

0

0

-1

0

a=6

1e7

-1

-1

0

-1

0

0

-1

0

a=1e7+1

Laplacian + Image !

Spatial Filtering

Spatial Filtering

Enhancement using the First Derivative:The Gradient

Definition: 2 dim. column vector,

(f) = [Gx ; Gy], G is 1st order derivative

MAGNITUDE of gradient:

Mag((f))= SQRT(Gx2 + Gy

2)

Spatial Filtering

For computational reasons the magnitude is often approximated by:

Mag ~ abs(Gx) + abs(Gy)

Spatial Filtering

Mag ~ abs(Gx) + abs(Gy)

0

-2

2

-1

0

1

-1

0

1

0

0

0

1

2

1

-1

-2

-1

“Sobel Operators”

Spatial Filtering

Sobel Operators are a pair of operators !Effect of Sobel Filtering:

Spatial Filtering

Remember the result some slides ago:

First and second order derivative

1.1st order generally produces thicker edges

2. 2nd order shows stronger response to detail3. 1st order generally response stronger to gray level step4. 2nd order produce double (pos/neg) response at step change

Spatial Filtering

In practice, multiple filters are combined to enhance images

Spatial Filtering

…continued

Spatial Filtering

Spatial Filtering

Non – Linear Filtering:

Order-Statistics Filters

MedianMinMax

Spatial Filtering

Median:The median M of a set of values is such

that half the values in the set are less than (or equal to) M, and half are greater (or equal to) M.

1 2 3 3 4 5 6 6 6 7 8 9 9

Spatial Filtering

Most important properties of the median:

• less sensible to noise than mean• an element of the original set of

values• needs sorting (O(n log(n))

Spatial Filtering

Median vs. Mean

Spatial Filtering

Min and Max

4

5

4

7

2

8

2

3

3

3

3

3

3 3 3 3

MIN

MAX

Spatial Filtering

Min and Max:

Basics of Morphological Filtering

(blackboard and MATLAB examples)

189

Image Enhancement

• Image enhancement techniques:Spatial domain methodsFrequency domain methods

• Spatial (time) domain techniques are techniques that operate directly on pixels.

• Frequency domain techniques are based on modifying the Fourier transform of an image.

Fourier Transform

• ‘Fourier Transform’ transforms one function into another domain , which is called the frequency domain representation of the original function

• The original function is often a function in the Time domain• In image Processing the original function is in the Spatial Domain• The term Fourier transform can refer to either the Frequency domain

representation of a function or to the process/formula that "transforms" one function into the other.

Our Interest in Fourier Transform

• We will be dealing only with functions (images) of finite duration so we will be interested only in Fourier Transform

Applications of Fourier Transforms

1-D Fourier transforms are used in Signal Processing 2-D Fourier transforms are used in Image Processing 3-D Fourier transforms are used in Computer Vision Applications of Fourier transforms in Image processing: –

– Image enhancement,– Image restoration,– Image encoding / decoding,– Image description

193

Discrete Fourier Transform (DFT)

• The discrete Fourier transform pair that applies to sampled functions is given by:

1

0

)/2exp()(1)(M

x

MuxjxfM

uF u=0,1,2,…,M-1

1

0

)/2exp()()(M

u

MuxjuFxf x=0,1,2,…,M-1

and

194

2-D Discrete Fourier Transform

• In 2-D case, the DFT pair is:

1

0

1

0

))//(2exp(),(1),(M

x

N

y

NvyMuxjyxfMN

vuF

1

0

1

0

))//(2exp(),(),(M

u

N

v

NvyMuxjvuFyxf

u=0,1,2,…,M-1 and v=0,1,2,…,N-1

x=0,1,2,…,M-1 and y=0,1,2,…,N-1

and:

195

Polar Coordinate Representation of FT

• The Fourier transform of a real function is generally complex and we use polar coordinates:

2 2 1/ 2

1

( , ) ( , ) ( , )

( , ) [ ( , )

( , ) ( , ) exp( ( , )

( , )

)

]

( , )( , ) tan( , )

F u v R u v j I u v

F u v R u v I u v

I u

F u v F u v j u

vu vR u v

v

Magnitude:

Phase:

Polar coordinate

04/08/23 196

Shifting the Origin to the Center

04/08/23 197

Shifting the Origin to the Center

198

Fourier Transform: shift

• It is common to multiply input image by (-1)x+y prior to computing the FT. This shift the center of the FT to (M/2,N/2).

Shift

( , ) ( , )f x y F u v

( , )( 1) ( / 2, / 2)x yf x y F u M v N

199

Symmetry of FT

• For real image f(x,y), FT is conjugate symmetric:

*( , ) ( , )F u v F u v

• The magnitude of FT is symmetric:

( , ) ( , )F u v F u v

200

FT

IFT

201

IFT

IFT

202

The central part of FT, i.e. the low frequency components are responsible for the general gray-level appearance of an image.

The high frequency components of FT are responsible for the detail information of an image.

203

u

v

Image

DetailDetail

GeneralGeneralappearanceappearance

Frequency Domain (log magnitude)

204

10 %5 % 20 % 50 %

206

Frequency Domain Filtering

207

• Edges and sharp transitions (e.g., noise) in an image contribute significantly to high-frequency content of FT.

• Low frequency contents in the FT are responsible to the general appearance of the image over smooth areas.

• Blurring (smoothing) is achieved by attenuating range of high frequency components of FT.

Frequency Domain Filtering

208

– f(x,y) is the input image– g(x,y) is the filtered– h(x,y): impulse response

Convolution in Time Domain

g(x,y)=h(x,y)f(x,y)

g x y h x y f x x y y

f x y h x yy

M

x

M( , ) ( ' , ' ) ( ' , ' )

( , )* ( , )''

0

1

0

1

209

Convolution Theorem

G(u,v)=F(u,v)H(u,v)

g(x,y)=h(x,y)f(x,y)

• Filtering in Frequency Domain with H(u,v) is equivalent to filtering in Spatial Domain with f(x,y).

Multiplication in Frequency Domain

Convolution in Time Domain

210

Spatial domain

Gaussian lowpass filter Gaussian highpass filter

Frequency domain

Examples of Filters

211

Ideal low-pass filter (ILPF)

01

),( vuH 0),( DvuD

0),( DvuD 2 2 1/ 2( , ) [( / 2) ( / 2) ]D u v u M v N

D0 is called the cutoff frequency.

(M/2,N/2): center in frequency domain

212

Shape of ILPF

Spatial domain

Frequency domain

h(x,y)

213

ringing and blurring

Ideal in frequency domain means non-ideal in spatial domain, vice versa.

FT

214

Butterworth Lowpass Filters (BLPF)• Smooth transfer function,

no sharp discontinuity, no clear cutoff frequency.

n

DvuD

vuH 2

0

),(1

1),(

21

215

No serious ringing artifacts

216

• Smooth transfer function, smooth impulse response, no ringing

02

2

2),(

),( DvuD

evuH

Gaussian Lowpass Filters (GLPF)

217

Spatial domain

Gaussian lowpass filter

Frequency domain

GLPF

218

No ringing artifacts

219

Examples of Lowpass Filtering

220

Examples of Lowpass Filtering

Original image and its FT Filtered image and its FT

Low-pass filter H(u,v)

221

High-pass Filters• Hhp(u,v)=1-Hlp(u,v)

• Ideal:

• Butterworth:

• Gaussian:

01

),( vuH0),( DvuD

0),( DvuD

20

2 2/),(1),( DvuDevuH

1( , ) 201 ( , )

H u v nDD u v

222

223

ButterworthButterworth High-pass Filtering

224

GaussianGaussian High-pass Filtering

225

IdealIdeal High-pass Filtering

ringing artifacts

226

Gaussian High-pass Filtering

Original image Gaussian filter H(u,v)

Filtered image and its FT

04/08/23 227

Example of Gaussian LPF and HPF

• A simple image model: illumination–reflection model– f(x,y) : the intensity is called gray level for monochrome image– f(x,y)=i(x,y)*r(x,y)– 0<i(x,y)<inf, the illumination– 0<r(x,y)<1, the reflection

Homomorphic filters

Image Enhancement - 3

• The illumination component– Slow spatial variations– Low frequency

• The reflectance component– Vary abruptly, particularly at the junctions of dissimilar objects– High frequency

• Homomorphic filters– Effect low and high frequency differently– Compress the low frequency dynamic range– Enhance the contrast in high frequency

Homomorphic filters (cont’)

Image Enhancement - 3Homomorphic filters (cont’)

• f(x,y)=i(x,y)*r(x,y)• z(x,y)=ln f(x,y) = ln i(x,y) + ln r(x,y)• F{z(x,y)} = F{ln i(x,y)} + F{ln r(x,y)}• S(u,v) = H(u,v) I(u,v) + H(u,v) R(u,v)• s(x,y) = i’(x,y) + r’(x,y)• g(x,y) = exp[s(x,y)] = exp[i’(x,y)]exp[r’(x,y)]

Homomorphic filters (cont’)

Image Enhancement - 3Homomorphic filters (cont’)

Image Enhancement - 3Homomorphic filters - example

LDvuDc

LH evuH )]1)[(),(20

2 /),((

234

End of Lecture