1 Vision industrielle 2002/2003 Christian Wolf [email protected]Département Génie Productique INSA de Lyon Session 4 - Image Processing 2 Table of contents Introduction Motivation, human vision, history, digital images, methodology Image Acquisition Image formation, sensors, cameras, scanners, illumination, color Fourier Transformation Noise, mage filtering in the frequency domain, correlation Image Processing linear and non linear filtering, histograms Image Analysis thresholding, gradients, laplacian, edge detection, region segmentation Histograms Non-linear filters Convolution Mathematical morphology, 3D binary images, erosion, dilation, neighborhood transformations, stereo vision, shape from X Questions Questions to the Lecture PDF files: http://rfv.insa-lyon.fr/~wolf/enseignement 3 Local and semi-local smoothing Instead of restoration (by inverting the degradation effects), local treatment on the image enhances its quality. Examples: • Salt and pepper noise • Holes or cuts in the object contours • ... Histograms Non-linear filters Convolution 4 Linear noise filtering: example Replace each pixel by the mean of its 4 neighbors: f'(x,y)=( f(x-1,y-1) + f(x,y-1) + f(x-1,y) + f(x,y) )/ 4 1 1 1 1 1 (, ) ( , ) 9 u v u v Fxy fx vy u = = =− =− = + + ∑∑ Replace each pixel by the mean of its 9 neighbors: Original image with noise Filtered image Histograms Non-linear filters Convolution 5 Convolution: introduction = 1 1 1 1 1 1 1 1 1 9 1 h − = 1 0 0 1 r h − = 0 1 1 0 l h Mean filter Roberts filter Histograms Non-linear filters Convolution 6 1D convolution The continuous case: The discrete case: Commutative operation! x Filtered pixel h(x) = ”filter kernel” origin f h Histograms Non-linear filters Convolution
8
Embed
Convolution Non-linear filters Histograms 2Problem: edge smoothing of linear filters 29 The median filter Example: “salt and pepper” noise Original image Filtered with the median
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Replace each pixel by the mean of its 9 neighbors:
Original imagewith noise
Filtered image
HistogramsNon-linear filtersConvolution
5
Convolution: introduction
=
111111111
91h
−
=1001
rh
−
=0110
lh
Mean filter
Roberts filter
HistogramsNon-linear filtersConvolution 6
1D convolutionThe continuous case:
The discrete case:
Commutative operation!
x
Filtered pixel
h(x) = ”filter kernel”origin
f
h
HistogramsNon-linear filtersConvolution
7
2D convolution
h0 M-1
0
N-1
f xx→
y↓
y
f⊗h x
y×
The discrete case:
HistogramsNon-linear filtersConvolution 8
Why flip the kernel?
f⊗hfh
flipped kernel
Reason: so that f⊗h = h⊗f
HistogramsNon-linear filtersConvolution
9
The impulse response (IR)“2D Dirac image”: 1D case: 2D case:
The filter kernel equals the response of the filter to a Dirac impulse:
Transfer function: the Fourier transform of the impluse response: H=FT(h)Contains for each frequency the complex factor by which a periodicstructure is multiplicated.
The impulse response of theoperator ∅:
HistogramsNon-linear filtersConvolution 10
PropertiesCommutativity:
Distributivity (over addition):
Linearity:
α,β ... scalars
Associativity (combination of filters):
HistogramsNon-linear filtersConvolution
11
PropertiesShift invariance:
Shift operator:
HistogramsNon-linear filtersConvolution 12
Linear and shift-invariant operators
Thought experiment: what kind of linear and shift-invariantoperators can there exist?
Let’s consider an operator ∅with following properties:
Linearity
Shift invariance
h is the impulse response of this operator
13
A linear andshiftinvariantoperator isequivalentto aconvolutionwith theimpulseresponse oftheoperator!!!
HistogramsNon-linear filtersConvolution 14
Iteratively convolving an image N times by a filter of sizeM corresponds to the application of a single convolutionof a filter of size M* = 2 × ( (M-1) / 2 ) × N ) +1
3
1 2 11 2 4 216
1 2 1g
=
=
1464141624164624362464162416414641
2461
5g
2 x a filter of size 3 = 1 x filter of size 53 x a filter of size 3 = 1 x filter of size 7
⇔2 ×
Iterative application
One kernel: may reduce complexitySeveral kernels: may reducing programming effort
HistogramsNon-linear filtersConvolution
15
SeparabilityThe Gaussian and the average filter are separable:
f g f g g f g gx y y x⊗ = ⊗ ⊗ = ⊗ ⊗(( ) ) ( ) )
Advantage: Computational complexity for a MxN filter:Non-separable implementation: MxN operationsSeparable implementation: M+N operations
1 1 1 1 11 1 1 1 1
1 1 1 1 1 1251 1 1 1 11 1 1 1 1
B
=
11
1 1511
Bx
=
[ ]1 1 1 1 1 15
By =
1 2 11 2 4 216
1 2 1G
=
11 241
Gy =
[ ]1 1 2 14
Gx = [ ]1 1 2 12 1 2 1 2 4 21 1 2 1
× =
since
since [ ]
1 1 1 1 1 11 1 1 1 1 1
1 1 1 1 11 1 1 1 1 11 1 1 1 1 11 1 1 1 1 1
× =
HistogramsNon-linear filtersConvolution 16
Convolution in the frequency domain
Convolution in the spatial domain corresponds to amultiplication in the frequency domain:
)v,u(H).v,u(F)y,x(h)y,x(f ⊗
Can lead to a large decrease in computational complexity(depending on the filter operations)
Example: filter bank etc.
HistogramsNon-linear filtersConvolution
17
Border treatmentWhat do we do with missing pixels on the border?
0 M-1
• Cyclic replacement (only option in thefrequency domain!!)
• Zeros
• Interpolation (linear, quadratic, ...)
HistogramsNon-linear filtersConvolution 18
g =
19
1 1 11 1 11 1 1
1 111 14
g =
The obvious filter, corresponds to a convolution with the following kernel:
F(x,y)=f⊗g ⇔1 1
1 1
1( , ) ( , )9
u v
u vF x y f x v y u
= =
=− =−
= + +∑ ∑
The average filter (“blur”)
HistogramsNon-linear filtersConvolution
19
g =
125
1 1 1 1 11 1 1 1 11 1 1 1 11 1 1 1 11 1 1 1 1
Bigger kernel = more blur
Computational complexity!
Increasing the filter size (“blur more”)
HistogramsNon-linear filtersConvolution 20
g =
116
1 2 12 4 21 2 1
Discrete approximation of a 2D Gaussianfilter:
Weighted mean, the center pixels are moreimportant than the border pixels
G x yx y
x yx ye( , ) =
− +
1
2
12
2 2
πσ σσ σ
=
1464141624164624362464162416414641
2461
5g
Gaussian low pass filter (“soften”)
HistogramsNon-linear filtersConvolution
21
Original image Average filter Gaussian filter
Comparison: Gaussian & average filter
HistogramsNon-linear filtersConvolution 22
Gaussian low pass filter: details
HistogramsNon-linear filtersConvolution
23
Sharpen (rehaussement)Sharpened image = (A-1) original image + high pass filtered image
−−−−−−−−
111181111
−−−
−
010141010 Laplacian filter (high pass filter)
−−−−−−−−
111191111
A=1: standard laplacian filterA>1: a part of the original image is added to the high pass
A=2
Noise!!!
HistogramsNon-linear filtersConvolution 24
Strong sharpenStronger sharpening lets the noise in the image appear!
Example: scanner noise appears after sharpening
HistogramsNon-linear filtersConvolution
25
Adaptive smoothingThe coefficients in the filter mask depend on the data:
Position in the maskPixel to filter
Discontinuity function:
Normalization constant:
HistogramsNon-linear filtersConvolution
Problem: edgesmoothing oflinear filters
26
The min/max filterGoal: Contrast enhancementThe original pixel value is compared to the minimum of thepixels in the neighborhood (=m) and the maximum (=M):
1218
25 32
25 36 57
4819Minimum m=12Maximum M=57The center value 19 is closer to m, i.e. the new value ism=12.
HistogramsNon-linear filtersConvolution
27
Min/max: example
Original image Filtered image
HistogramsNon-linear filtersConvolution 28
The median filterNon-linear filter developed in order to preserveedges. Chooses the median value of the theneighboring pixels, i.e. the value at rank K/2 in theordered list of gray values.
1218
25 32
25 36 57
484
12 18 25 3225 36 574841 K/2 K
replace 4 by 25
The median filter is very efficient against noise, avoidsblurring and preserves colors transitions around the objects.
HistogramsNon-linear filtersConvolution
Problem: edgesmoothing oflinear filters
29
The median filterExample: “salt and pepper” noise
Original image Filtered with the median filter
HistogramsNon-linear filtersConvolution 30
The “mode” filter
1218
25 32
25 36 57
484
The mean value is not taken from the whole ordered list L ofgray values, but from a well chosen compact part of size K/2:
12 18 254
12 18 25 25
18 25 3225
12 18 25 3225 36 57484
...
From the chosen interval, themedian value is taken
HistogramsNon-linear filtersConvolution
31
The symmetric nearest neighbor filterGoal: smoothing regions while preserving edges and thinstructures.