Chapter 5 Neighborhood Processing

SCCS 476 1

Chapter 5 Neighborhood Processing

• Introduction• Filters (LPF, HPF, Gaussian Filter, …)• Region of interest processing

SCCS 476 2

Basic Image Processing Operations

• Neighborhood processing– process the pixel with its neighbors

• Point operations– process according to the pixel’s value alone.

SCCS 476 3

Neighborhood Processing

3x3 Mask

Output derives from multiplying all elements in the mask by corresponding elements in the neighborhood and adding together all these products.

SCCS 476 4

Filter

• A rule or procedure for processing an image

• Combination of mask and function

• Goal: separating/attenuating a desired component of an observed image

• Type: – Linear (function), Nonlinear (function)– Low-pass filter (LPF), High-pass filter (HPF),

Band-pass filter (BPF)

SCCS 476 5

Steps of Linear Spatial Filtering

• Position the mask over the current pixel.

• Form all products of filter elements with the corresponding elements of the neighborhood.

• Add all products.

• Other names for mask:

filter, mask, filter mask, kernel, template or window, etc.

SCCS 476 6

Linear Spatial Filter: Example (1)

2 15 10 20

0 0

5 20

10 15

10 10

10

10

15

1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

1/9

Average Filter10

Find the output intensity of the blue pixel.

SCCS 476 7

Linear Spatial Filter: Example (2)

15 10 20

0

20

15

10 10

1/9 1/9

1/9 1/9

1/9 1/9 1/9

1/9

101/9

Multiply the number in the filter’s element with the corresponding pixel’s intensity.

SCCS 476 8

20/9

0/9

10/9

Linear Spatial Filter: Example (3)

15/9 20/9

10/9 15/9

10/9 10/9

Add all products for output.

15/9 + 10/9 + 20/9 + 0/9 + 10/9 + 15/9 + 20/9 + 10/9 + 10/9 = 12.22

Output intensity of blue pixel = 12.22

Spatial filtering is spatial convolution.

SCCS 476 9

Correlation

• Sum of the product of mask and intensity on each point.

][][][ nkhkfng functionk

inputoutput

SCCS 476 10

Example: Correlation (1)

10 20 10 15 5

5 10 15 10 0

20 10 20 15 5

10 5 0 0 10

10 20 15 0 10

1 3

2 3 4

3 4 5

2

Kernel20

SCCS 476 11

Example: Correlation (2)

10 15 10

10

5

15

0 0

1 3

2 4

3 4 5

2

20 3

Multiply and sum all products.

101 +152 +103 +102 +203 +154 +53 +04 +05= 225

SCCS 476 12

Exercise: Correlation

0 0 0 0 0

0 0 0 0 0

0 0 1 0 0

0 0 0 0 0

0 0 0 0 10

1 3

2 3 4

3 4 5

2

Kernel

Find the correlation of the green kernel to the above image.

SCCS 476 13

Convolution

• Sum of the response on each point

k

functioninputoutput knhkfng ][][][

SCCS 476 14

Example: Convolution (1)

10 20 10 15 5

5 10 15 10 0

20 10 20 15 5

10 5 0 0 10

10 20 15 0 10

5 3

4 3 2

3 2 1

4

Kernel

20

1 3

2 3 4

3 4 5

2

Reverse Kernel!!

SCCS 476 15

Example: Convolution (2)

10 15 10

10

5

15

0 0

5 3

4 2

3 2 1

4

20 3

Multiply and sum all products.

105 +154 +103 +104 +203 +152 +53 +02 +01= 285

SCCS 476 16

Exercise: Convolution

0 0 0 0 0

0 0 0 0 0

0 0 1 0 0

0 0 0 0 0

0 0 0 0 10

1 3

2 3 4

3 4 5

2

Kernel

Find the convolution of the green kernel to the above image.

SCCS 476 17

Notation

• A linear filter is represented as a matrix, e.g., the 3 x 3 averaging filter.

1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

1/9

SCCS 476 18

Notation (2)• Edges of the Image

– There are a number of different approaches to dealing with this problem.

• Ignore the edges: The mask is applied to all pixels except the edges and results in an output image that is smaller than the original. If the mask is very large, a significant amount of information may be lost.

• Pad with zeros: We assume that all necessary values outside the image are zero. It will return an output image of the same size as the original, but may have the effect of introducing unwanted artifacts, e.g., edges, around the image.

SCCS 476 19

Filtering in MATLAB

• Command: filter2• Syntax: filter2(filter, image, shape);

filter2(filter, image);– shape:

• ‘same’: pad edge with zeros. Size unchanged. (default)

• ‘valid’: apply mask only to inside pixel. Size smaller.

• ‘full’: pad edge with zeros and applying the filter at all places on and around the image where the mask intersects the image matrix. Size larger.

SCCS 476 20

Filter Construction in MATLAB

• Command: fspecial• Syntax: fspecial(type, parameter);

fspecial(type);– type: type of the filter

• ‘average’ : average filter

• ‘gaussian’ : Gaussian filter

• ‘laplacian’ : Laplacian filter …

– parameter: parameter of the filter (size, sigma, …). Default varies among filter. Try!!!

SCCS 476 21

Separable Filters

• Some filters can be implemented by the successive application of two simple filters.

n

mmn

n

mm a

a

aa

aa

.......

...

...

...

1

11

21

1211

SCCS 476 22

Separable Filters (2)

• Speed up the processing by filtering in one axis at a time.– working with 2D n x n matrix requires n2, O(n2),

multiplications, and n2 – 1, O(n2), addition for each pixel.

– working with 1D n x 1 matrix twice requires 2n, O(n), multiplications, and 2n – 2, O(n), additions for each pixel.

n

mmn

n

mm a

a

aa

aa

.......

...

...

...

1

11

21

1211

SCCS 476 23

Examples of Separable Filter

• Average filter

• Laplacian filter

1113

1

1

1

1

3

1

111

111

111

9

1

121

1

2

1

121

242

121

SCCS 476 24

Filter on Frequency Domain• Low-pass filter (LPF): filter that allows only

the low-frequency components and reduces or eliminates the high-frequency components.– E.g. Gaussian, average

• High-pass filter (HPF): filter that allows only the high-frequency components and reduces or eliminates the low-frequency components.– E.g. Laplacian, Prewitt, Sobel

SCCS 476 25

Frequency Components

• Spatial data (intensity) transformed by Fourier transform.

• Simplified version:– high-frequency indicates the abrupt changes in

intensity edges.– low-frequency indicates the intensity smoothness uniform region.

SCCS 476 26

Example: HPF

10 20 10 15 5

5 10 15 10 0

20 10 20 15 5

10 5 0 0 10

10 20 15 0 10

1 1

-2 4 -2

1 -2 1

-2

Kernel

Input image

SCCS 476 27

Example: HPF (2)

Filtered Image

10 20 10 15 5

5 0 20 -10 0

20 -40 25 15 5

10 5 -35 0 10

10 20 15 0 10

SCCS 476 28

Computing Consideration

• Filter may lead to the value outside [0,255]– Solution 1: Make negative values positive (use absolute

value) good when there are few negative values and the negative values are close to zero

– Solution 2: Clip values. Values larger than 255 become 255 and values less than 0 become 0. not good if there are many values outside the range.

– Solution 3: Scaling transformation. Rescale the range to [0,255] by pixel transform. Suppose gL and gH are the lowest and the highest values.

SCCS 476 29

Rescaling Intensity

gH

Filtered value (x)gL

0

255

Rescaled value (y)

1. Map gL to 0.2. Map gH to 255.

3. Interpolate for the remaining intensity. LH

L

gg

gxy

255

SCCS 476 30

Rescaling: MATLAB

• Manual:

>> gH = max(filtered_image(:));

>> gL = min(filtered_image(:));

>> scaled = (image – gmin)/(gmax – gmin);

Note: No need to rescale to 255 because intensity range for double image is [0,1].

SCCS 476 31

Rescaling: MATLAB (2)

• Command: mat2gray• Syntax: mat2gray(double_image);

What this command do?– scale the value in double_image to displayable

value. Output is double type.– minimum value is mapped to 0.– maximum value is mapped to 1.

SCCS 476 32

Gaussian Filter

• 1D Gaussian filter:

• 2D Gaussian filter:

2

2

2)( x

exf

2

22

2),( yx

eyxf

http://www-mmdb.iai.uni-bonn.de/lehre/BIT/ss03_DSP_Vorlesung/matlab_demos/

http://upload.wikimedia.org/wikipedia/su/thumb/3/38/Gaussian-pdf.png/300px-Gaussian-pdf.png

SCCS 476 33

Benefits of Gaussian Filter

• They are mathematically very well behaved. The Fourier transform of a Gaussian filter is another Gaussian.

• There are rotationally symmetric, so are very good starting points for some edge-detection algorithms.

• They are separable in x and y axes. This can lead to very fast implementations.

• The convolution of two Gaussians is another Gaussian.

SCCS 476 34

Gaussian Filter (2)

• Effect of Gaussian filter = blurring– larger leads to more blur.

average filter

• Construction of Gaussian filter:– command: fspecial(‘gaussian’, size, gamma);

size : size of the filter [row column], default [3 3]

gamma : , default 0.5.

SCCS 476 35

Gaussian Filter: MATLAB

>>gaussian1 = fspecial(‘gaussian’, [5 5], 5);Create the 55 Gaussian filter with the

value of 5.

>>gaussian2 = fspecial(‘gaussian’, 3, 0.75);

Create the 33 Gaussian filter with the value of 0.75.

SCCS 476 36

Effect of Gaussian Filter

= 0.5 = 2 = 5

Filter size = 5 5

SCCS 476 37

Effect of Gaussian Filter (2)

= 5 Filter size = 5 5Filter size = 3 3

SCCS 476 38

Edge Sharpening

• Also known as edge enhancement, edge crispening, unsharp masking.

• Process to make the edge slightly sharper and crisper.

• E.g. linear edge sharpening, unsharp masking, high boost filtering

SCCS 476 39

Linear Edge Sharpening

• High pass filter (HPF)

• Increase the edge power

• Example:

121

252

121

,

111

191

111

,

010

151

010

SCCS 476 40

Unsharp Masking

Original Image

Blurred with LPF

Scale with a < 1

Scale forDisplay

SCCS 476 41

Unsharp Masking: MALAB

>> f = fspecial(‘average’); Create LPF (E.g. average filter)

>> xf = filter2(f,x); Filter input image x by average filter.

>> xu = double(x) – xf/1.5; Substract the low passed components.

>> imshow (xu/70) Scale intensity for better viewing.

SCCS 476 42

Unsharp Filter

[unsharp_image] = [input] – (a [filter] [input])

= (I* – a [filter]) [input]

= [unsharp_filter] [input]

I* : matrix whose center member is 1 and the others are zero.

E.g. for 3 3 matrix I* =

000

010

000

SCCS 476 43

Unsharp Filter (2)

• For 33 matrix, after some rearranging term:

[unsharp_filter] =

MATLAB use this format for the unsharp filter with the default of 0.2.

1

151

1

1

1

SCCS 476 44

Unsharp Filter: MATLAB

• Command: fspecial• Syntax: fspecial(‘unsharp’, alpha);

– alpha : alpha value for unsharp filter– Size of the filter is fixed to 3 3

SCCS 476 45

Effect of Unsharp Masking

BEFORE AFTERhttp://ise.hansung.ac.kr/jun/DI/Chapter-5.htm

SCCS 476 46

High-Boost Filtering

Original Image

Blurred with LPF

Scale with A>1

Scale forDisplay

[high_boost] = A [input] – [low_passed]

Amplification factor. A=1, high boost = HPF

SCCS 476 47

High Boost Filter (2)

high_boost = A input – low_pass

= A input – (input - high_pass)

= (A – 1) input + high_pass

Rescale high_boost by w.

w high_boost = w (A – 1) input + w high_pass

Best result when w (A – 1) = 1.

w = 1/(A – 1)

SCCS 476 48

High Boost Filter (3)• General high boost filter:

Another version:

Best result when 3/5 A 5/6 • Used in dark image.• Boost the intensity of the original image.

)low_pass(1

1)original(

1

AA

A

)low_pass(12

1)original(

12

A

A

A

A

SCCS 476 49

Effect of High Boost Filter

BEFORE AFTERA = 5/6

SCCS 476 50

Non-linear Filter• A non-linear filter is obtained by a nonlinear function

of the grayscale values in the mask.• E.g. rank-order filters (minimum filter, maximum

filter, median filter); the elements under the mask are ordered.

• MATLAB command: nlfilter • A faster alternative: colfilt (rearranges the image into

column)• Good for removing impulse-like noise• Better tradeoff between smoothing and retention of

fine image details.

SCCS 476 51

MATLAB Command: nlfilter

• Syntax: nlfilter(Image,[m,n],func);– Image : input image

– [m n] : size of the neighborhood

– func : function to be applied to the neighborhood

• E.g. >> cmed = nlfilter(image, [3,3], ‘median(x(:))’);

Median function33 sliding box

SCCS 476 52

MATLAB Command: colfilt

• Columnwise neighborhood operation

• Syntax: colfilt(im, [m n], block_type, func)– im : input image – [m n] : size that image matrix to be arranged into

– block_type : how to rearrange im matrix.

– func : function to be applied to the neighborhood

• E.g. >> cmed = colfilt(image, [3,3], ‘sliding’, @median);

SCCS 476 53

Block_type in colfilt

• ‘distinct’: “Rearranges each distinct m-by-n block in the image A

into a column of B. im2col pads A with 0's, if necessary, so its size is an integer multiple of m-by-n. If A = [A11 A12; A21 A22], where each Aij is m-by-n, then B = [A11(:) A12(:) A21(:) A22(:)].”

• ‘sliding’“Converts each sliding m-by-n block of A into a column

of B, with no zero padding. B has m*n rows and contains as many columns as there are m-by-n neighborhoods of A.

SCCS 476 54

Rearrangement by ‘distinct’

• B = im2col(A,[m n],’distinct’);

Input Image

a1 a2 a3

a4 a5 a6

a7 a8 a9

A BmnBlock

a1 a2 a3

a4 a5 a6

a7 a8 a9

Pad this area with 0

Rearrange to column vector

SCCS 476 55

Input Image

Rearrangement by ‘sliding’

• B = im2col(A,[m n],’sliding’);

A BmnBlock

a0a1a2 anan+1

amn

SCCS 476 56

10 13 5

11 20 15

8 0 0

Rank-Order Filter

• Sort the intensities within the mask.

• Choose the intensity at ith position as output.

10 20 10 15 5

5 10 13 5 0

20 11 20 15 5

10 8 0 0 10

10 20 15 0 10

20

Sort intensityMedian filter

Min. filter

Max. filter

SCCS 476 57

Maximum Filter

• Output pixel is the maximum intensity of the pixels within the mask. (find brightest point)

BEFORE AFTERImage corrupted by pepper noise

http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/min_max.html

SCCS 476 58

Minimum Filter

• Output pixel is the minimum intensity of the pixels within the mask. (find darkest point)

BEFORE AFTERImage corrupted by salt noise

http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/max_min.html

SCCS 476 59

Median Filter

• Output pixel is the mid-intensity of the pixels within the mask (the median intensity).

www.ee.siue.edu/~sumbaug/ CVIPbook_PPLec/Chapter%209b.ppt

SCCS 476 60

Effect of Median Filter

BEFORE AFTER3x3 Kernel

http://www.ee.siue.edu/~cvip/CVIPtools_demos/

SCCS 476 61

Rank-Order Filtering: MATLAB

• Command: ordfilt2• Syntax: ordfilt2(image, order, domain);

– image : input image– order : which order of the sorted intensity

(minimum to maximum value) taken as output– domain : matrix indicating the neighborhood.

1 : pixels in the neighbor. 0 : pixels not in the neighbor

E.g. cmin = ordfilt2(image, 1, ones(3,3));

SCCS 476 62

Geometric Mean Filter

M

Mji

jixyxf

1

),(

),(),(ˆ

x(i, j): pixel’s intensityM : filter mask|M| : size of the mask

SCCS 476 63

Effect of Geometric Mean Filter

BEFORE AFTERImage corrupted by Gaussian noise with variance = 300, mean = 0

http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/geometric.html

SCCS 476 64

Bad Effect of Geometric Mean Filter

BEFORE AFTERImage corrupted by pepper noise with probability = 0.4

http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/geometric.html

SCCS 476 65

Alpha-Trimmed Mean Filter

• Output is the mean of the data after removing the first d/2 and the last d/2 ordered data.

10 13 5

11 20 15

8 0 0

10 20 10 15 5

5

10

13

5 0

20

11

20

15

5

10

8

0

0

10

10 20 15 0 10

20

Sort intensity

d =2

Trim the data by 2.(1 from the top.1 from the bottom.)

Output = average intensity of the remaining data. = 9.5

SCCS 476 66

Effect of Alpha-Trimmed Mean Filter

BEFORE AFTERImage corrupted by salt-and-pepper noise with variance = 200, mean = 0

http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/alpha.html

Trim size = 2, mask size =1

SCCS 476 67

Region of Interest Processing

• Process image only in the predefined area.

• The predefined area is called “Region Of Interest” (ROI).

• Function is the same as before but applied to only ROI.

SCCS 476 68

ROI Selection: MATLAB

• Command: roipoly• Syntax: roipoly(im);

roipoly(im, [x0 x1 …xm], [y0 y1 …ym];

– im : input image

– (x0, y0), (x1, y1), …, (xm, ym) : coordinate of the polygon covering region of interest.

– E.g. roi = roipoly(im, [60 100 100 60], [40 40 80 80]);

SCCS 476 69

Mask for ROI

• Binary image with the white (1) indicating ROI.

Mask coordinate: (222, 21) (272, 21) (300, 75) (270, 121) (221, 121) (191, 75)

http://www.mathworks.com/access/helpdesk/help/toolbox/images/roipoly.html

SCCS 476 70

ROI Filtering: MATLAB

• Command: roifilt2• Syntax: roifilt2(filter, image, roi);

– filter : 2D linear filter

– image : input image

– roi : region of interest (1: ROI, 0: not ROI)

SCCS 476 71

Effect of ROI Filtering

• Unsharp masking in ROI.

http://www.mathworks.com/access/helpdesk/help/toolbox/images/roifilt2.html

ROI at this coin