Prof. Sinisa Todorovic [email protected] ECE 468 / CS 519: Digital Image Processing Gradients, Harris Corners 1
Prof. Sinisa Todorovic
ECE 468 / CS 519: Digital Image Processing
Gradients, Harris Corners
1
Image Gradient along X-axis
I
x
(x, y) = I(x + 1, y)� I(x, y)
=
2
40 0 00 �1 10 0 0
3
5
| {z }D
x
(x,y)
⇤I(x, y)
2
Image Gradient along Y-axis
=
2
40 0 00 �1 00 1 0
3
5
| {z }Dy(x,y)
⇤I(x, y)
Iy(x, y) = I(x, y + 1)� I(x, y)
3
Filtering Image Gradient
convolution is associative
w(x, y;�) ⇤ I
x
(x, y) = w(x, y;�) ⇤ D
x
(x, y) ⇤ I(x, y)
= [w(x, y;�) ⇤ D
x
(x, y)] ⇤ I(x, y)
4
= [Dx
(x, y) ⇤ w(x, y;�)] ⇤ I(x, y)
Filtering Image Gradient
w(x, y;�) ⇤ I
x
(x, y) = w(x, y;�) ⇤ D
x
(x, y) ⇤ I(x, y)
= [w(x, y;�) ⇤ D
x
(x, y)] ⇤ I(x, y)
convolution is commutative
5
= [Dx
(x, y) ⇤ w(x, y;�)] ⇤ I(x, y)
Filtering Image Gradient
w(x, y;�) ⇤ I
x
(x, y) = w(x, y;�) ⇤ D
x
(x, y) ⇤ I(x, y)
= [w(x, y;�) ⇤ D
x
(x, y)] ⇤ I(x, y)
= w
x
(x, y;�) ⇤ I(x, y)
derivative of the filter6
Weighted Image Gradient
Image is discrete ⇒ Gradient is approximate
We always find the gradient of the filter !
w(x, y;�) ⇤ I
x
(x, y) = w
x
(x, y;�) ⇤ I(x, y)
w(x, y;�) ⇤ Iy(x, y) = wy(x, y;�) ⇤ I(x, y)
7
Interest Points
• Harris corners
8
Properties of Interest Points
• Locality -- robust to occlusion, noise
• Saliency -- rich visual cue
• Stable under affine transforms
• Distinctiveness -- differ across distinct objects
• Efficiency -- easy to compute
9
Example of Detecting Harris Corners
10
Harris Corner Detector
homogeneous region⇓
no change in all directions
edge⇓
no change along the edge
corner⇓
change in all directions
Source: Frolova, Simakov, Weizmann Institute
scanning window
11
Harris Corner Detector
2D convolutionSource: Frolova, Simakov, Weizmann Institute
E(x, y) = w(x, y) ⇤ [I(x + u, y + v)� I(x, y)]2
12
Harris Corner Detector
change weighting window shifted image original image
Source: Frolova, Simakov, Weizmann Institute
E(x, y) =X
m,n
w(m, n)[I(x + u + m, y + v + n)� I(x + m, y + n)]2
13
Harris Detector Example
input image
E(x, y)I(x, y)
2D map of changes
14
Harris Corner Detector
image derivatives along x and y axes
Taylor series expansion
For small shiftsu � 0v � 0
I(x + u, y + v) ⇡ I(x, y) +@I
@x
u +@I
@y
v
I(x + u, y + v) ⇡ I(x, y) + [Ix
I
y
]
u
v
�
15
Harris Corner Detector
E(x, y) = w(x, y) ⇤✓
I(x, y) + [Ix
I
y
]
u
v
�� I(x, y)
◆2
= w(x, y) ⇤✓
[Ix
I
y
]
u
v
�◆2
= w(x, y) ⇤✓
[Ix
I
y
]
u
v
�◆T ✓[I
x
I
y
]
u
v
�◆
16
Harris Corner Detector
E(x, y) = w(x, y) ⇤ [u v]
I
x
I
y
�[I
x
I
y
]
u
v
�
= [u v]✓
w(x, y) ⇤
I
2x
(x, y) I
x
(x, y)Iy
(x, y)I
x
(x, y)Iy
(x, y) I
2y
(x, y)
�◆
| {z }M(x,y)
u
v
�
17
= [Dx
(x, y) ⇤ w(x, y;�)] ⇤ I(x, y)
Filtering Image Gradient
w(x, y;�) ⇤ I
x
(x, y) = w(x, y;�) ⇤ D
x
(x, y) ⇤ I(x, y)
= [w(x, y;�) ⇤ D
x
(x, y)] ⇤ I(x, y)
= w
x
(x, y;�) ⇤ I(x, y)
derivative of the filter18
Harris Corner Detector
E(x, y) = [u v]M(x, y)
u
v
�
M(x, y) =
(wx
⇤ I)2 (wx
⇤ I)(wy
⇤ I)(w
x
⇤ I)(wy
⇤ I) (wy
⇤ I)2
�
(x,y)
19
Eigenvalues and Eigenvectors of M
• Eigenvectors of M -- directions of the largest change of E(x,y)
• Eigenvalues of M -- the amount of change along eigenvectors
��1
��2
E(x, y) = [u v]M(x, y)
u
v
�
20
Detection of Harris Corners using the Eigenvalues of M
�2
�1
21
Harris Detector
f(�) =�1(�)�2(�)
�1(�) + �2(�)=
det(M(�))trace(M(�))
objective function
22
Harris Detector Example
input image
23
Harris Detector Example
f values color codedred = high valuesblue = low values
24
Harris Detector Example
f values thresholded
25
Harris Detector Example
Harris features = Spatial maxima of f values
26
Example of Detecting Harris Corners
27
Properties of Harris Corners
• Invariance to variations of imaging parameters:
• Illumination?
• Camera distance, i.e., scale ?
• Camera viewpoint, i.e., affine transformation?
28
Harris/Hessian Detector is NOT Scale Invariant
Source: L. Fei-Fei
“edge”
“edge”“edge”
“corner”
29
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
30
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
31
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
32
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
33
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
34
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
35
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
36
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
37
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
38
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
39
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
40
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
41
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σ ""xIfmii� Source: Tuytelaars
42