EE 584 MACHINE VISION - Middle East Technical Universityeee.metu.edu.tr/~alatan/Courses/EE584LECTURES/rv04.pdf · 2016. 11. 24. · 22.03.2012 1 METU EE584 Lecture Notes by A.AydinALATAN

22.03.2012

1

METU EE584 Lecture Notes by A.Aydin ALATAN © 2012

EE 584MACHINE VISION

Edge DetectionDifferential OperatorsDiscrete Approximations

Roberts, Prewit & SobelLaplacian of Gaussian (LoG) DetectorCanny Edge Detector

Corner Detection

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Edge Finding� A curve in image where rapid changes occur� Edges usually contain important info

� Surface orientation changes� Shadows due to non-uniform illumination� Occlusions of objects� Discontinuity in the surface reflectance

� discontinuity in image brightness is expected� derivatives can be used to detect edges� Edge detection is complementary to

segmentation, since edges divide image into regions

22.03.2012

2


© Gonzalez & Woods, Digital Image Processing,2nd ed., Prentice-Hall, 2002

Types of Edges

(a)

(b)

(c)

Step edges

Roof-top edge

Line edges


Major Steps in Edge Detection

� Steps for most edge detection algorithms:� Filtering : Noise is a critical factor; filtering noise is

possible while losing edge strength� Enhancement : Using gradient information, significant

changes in intensity is located� Detection : Finding edge pixels among all the pixels with

non-zero gradient information� Localization : (Optional) non max. suppression and

subpixel resolution

(a ) ( b ) (c ) (d )True edge Noise susceptibility

Poor localization

Too many responses

22.03.2012

3


Differential Operators (1/3)

� A simple edge model : ∫∞−

=

<

=

>

=z

dttzuwhere

zfor

zfor

zfor

zu )()(

00

02

101

)( δ

� Assume edge is a line :

0cossin =+Θ−Θ ρyxΘ

y

x

ρ B1

B2

)cossin()(),( 121 ρ+Θ−Θ−+= yxuBBByxE

� Partial derivatives of the intensity field E(x,y)

)cossin()(cos),(

)cossin()(sin),(

12

12

ρδ

ρδ

+Θ−Θ−Θ−=∂

∂

+Θ−Θ−Θ=∂

∂

yxBBy

yxE

yxBBx

yxE



� Magnitude for brightness gradient

∂∂

∂∂

y

yxEx

yxE

),(

),(

� )cossin()(),(),( 22

12

22

ρδ +Θ−Θ−=

∂∂+

∂∂

yxBBy

yxE

x

yxE

)cossin()(cos),(

)cossin()(sin),(

122

2

2

122

2

2

ρδ

ρδ

+Θ−Θ′−Θ=∂

∂

+Θ−Θ′−Θ=∂

∂

yxBBy

yxE

yxBBx

yxE

� Similarly, Laplacian can be found as

)cossin()(),(),(

122

2

2

2

ρδ +Θ−Θ′−=∂

∂+∂

∂yxBB

y

yxE

x

yxE�

22.03.2012

4



� Note that the magnitude of brightness gradient and Laplacian do not depend on orientation(rotation or translation) of the edge� Isotropic operators

� Laplacian retains the sign of the brightness difference across the edge, which allows to determine the brighter side of the image

)cossin()(),(),( 22

12

22

ρδ +Θ−Θ−=

∂∂+

∂∂

yxBBy

yxE

x

yxE

)cossin()(),(),(

122

2

2

2

ρδ +Θ−Θ′−=∂

∂+∂

∂yxBB

y

yxE

x

yxE


Discrete Approximations (1/4)� Using finite-difference approximation of a derivative :

{ }

{ })()(2

1),(

)()(2

1),(

,1,,11,1

,,11,1,1

jijijiji

jijijiji

EEEEy

yxE

EEEEx

yxE

−+−≈∂

∂

−+−≈∂

∂

++++

++++

ε

ε

{ }2,11,

2,1,1

22 )()()),(

()),(

( jijijiji EEEEy

yxE

x

yxE++++ −+−≈

∂∂+

∂∂

� Discrete approximation to the magnitude of the brightness gradient can be obtained as :

� Discrete approximation to the angle of the brightness gradient is not accurate since edge pixels may have intermediate values

1, +jiE 1,1 ++ jiE

jiE ,1+jiE ,

ε

22.03.2012

5


Discrete Approximations (2/4)� Roberts operator

� Sobel operator� Averaging with emphasis to

center pixels

� Prewitt operator� Averaging to suppress noise

1 0

0 -1

0 -1

1 0

1

1−

0

1

1−

0 0

2−

21−

1−

0

1

1

2− 2

0

0

1−

1−

0

1

1

1− 1

0

0 1

1−

0

1

1−

0 0

1−

1

{ }2,11,

2,1,1

22 )()()),(

()),(

( jijijiji EEEEy

yxE

x

yxE++++ −+−≈

∂∂+

∂∂


Discrete Approximations (3/4)� Compass Gradients for different orientations

� Sobel Compass � Nevatia-Babu Compass

00

0-2-1

-121

102

-201

-101

-101

-1-10

-212

001

-112

0-10

-2

00

021

1-2-1

-10-2

20-1

10-1

10-1

110

2-1-2

00-1

1-1-2

010

2

-100-100

-100-100

-100-100

-100-100

-100-100

0 100

0 100

0 100

0 100

0 100

100

100

100

100

100

-100 -32

-100 -78

-100-100

-100-100

-100-100

100 100

92 100

0 100

-92 78

-100 -32

100

100

100

100

100

100 -100

-32 78

-100 -92

-100-100

-100-100

100 100

100 100

0 92

-100 -78

-100-100

100

100

100

32

-100

100 100

100 100

0 0

-100-100

-100-100

100 100

100 100

0 0

-100-100

-100-100

100

100

0

-100

-100

100 100

100 100

100 92

32 -78

-100-100

100 100

100 78

0 -92

-100-100

-100-100

100

-32

-100

-100

-100

100 100

100 100

100 100

-100 78

-100 -32

100 32

92 -78

0 -100

-92 -100

-100-100

-100

-100

-100

-100

-100

00 300

900

600

1200 1500

22.03.2012

6


Discrete Approximations (4/4)

� Finite-difference approximation of a Laplacian in 3x3 picture cells :

{ }

{ }1,,1,22

2

,1,,122

2

21),(

21),(

+−

−+

+−≈∂

∂

+−≈∂

∂

jijiji

jijiji

EEEy

yxE

EEEx

yxE

ε

ε

1,1 +− jiE 1, +jiE 1,1 ++ jiE

jiE ,1− jiE , jiE ,1+

1,1 −− jiE 1, −jiE 1,1 −+ jiE

−+++≈

∂∂+

∂∂

+−−+ jijijijiji EEEEEy

yxE

x

yxE,1,1,,1,122

2

2

2

)(4

14),(),(

εAverage of neighbors

� Laplacian will return 0 values around constant and linearly changing regions

� Corresponding stencil (kernel) �

1

1 4− 1

1


Detection of Edges (1/2)

� Detection of an edge differs for 1st and 2nd derivatives

� Maxima of 1st

derivative gives edge location

� Zero-crossing of 2nd

derivative shows the location of edge

x

f(x,y)

x

f’’(x,y)

x

f’(x,y)

Threshold

22.03.2012

7



Detection of Edges (2/2)



Image intensity 1st derivative 2nd derivative

No noise

Gaussian noise with σ=0.1



Effect of Noise on Edges

22.03.2012

8


Detection vs. Localization� Noise generates spurious edges

� In order to suppress noise, filtering is one option but the support of such filters are quite large

� Filtering makes the edges thicker

� Hence, edge localization becomes weaker.

This dilemma can not be avoided


Convolution

c22

+ c22 f(i,j)

f(.)

o (i,j) =

f(.)

c12

+ c12 f(i-1,j)

f(.)

c13

+ c13 f(i-1,j+1) +

f(.)

c21

c21 f(i,j-1)

f(.)

c23

+ c23 f(i,j+1) +

f(.)

c31

c31 f(i+1,j-1)

f(.)

c32

+ c32 f(i+1,j)

f(.)

c33

+ c33 f(i+1,j+1)

f(.)

c11

c11 f(i-1,j-1)

Slide from Marc Pollefeys, Lecture Notes on Computer Vision

22.03.2012

9


ConvolutionSlide from Marc Pollefeys, Lecture Notes on Computer Vision


Convolutionexp −

x2 + y2

2σ 2


22.03.2012

10


Laplacian of Gaussian (LoG)

� Smoothing filter is a Gaussian

� Enhancement step is a Laplacian

� Detection is based on zero-crossings

� Edge location can be obtained by linear int.

( )( )

2

22

24

2222

2

2

2),(

where),(),(

filterGaussian a is ),( where),(),(),(

σ

σσ yx

eyx

yxg

yxfyxg

yxgyxfyxgyxh

+−

−+=∇

∗∇=

∗∇=


Laplacian of Gaussian (LoG)2

22

24

2222 2

),( σ

σσ yx

eyx

yxg+−

−+=∇


Inverted LoG function

(Mexican hat)

22.03.2012

11


Canny Edge Detector� Smooth the image with a Gaussian� Compute the gradient magnitude and angle� Apply non-maxima suppression to magnitude

� remove all pixels except the maximum along the gradient direction

� Use double thresholding (hysteresis) the detect/link edges

� obtain two edges maps with two thresholds1. High threshold � thick edges � start tracing2. From staring points, low threshold � trace on thin edges

� link the thin edges using the other to obtain the final


Canny Edge Detector

� Non-maxima suppression� which point is the maximum?

� along the gradient direction, q should be larger than p & r (both p & r are interpolated)

� where is the next maximum?� next one should be around the line

perpendicular to the gradient vector, i.e. r or s


22.03.2012

12


Surface Fitting for Edge Detection

� Discrete approximations of derivatives limits the performance

� Let z=f(x,y) be a continuous image intensity function to be found, after fitted to the discrete pixel values at each neighborhood

� f(x,y) function can be approximated locally at every pixel of the image, so that derivatives can be found

� Let f(x,y)=k1+k2x+ k3y+ k4x2+ k5xy+ k6y2+ k7x3+k8x2y+ k9xy2+ k9y3

� First solve for ki using the discrete pixel values, then analytically find the partial derivatives to determine the location of the edges


Comparison between Edge Detectors� There are 2 main approaches in edge detection

� Extracting local maxima of the magnitude gradient in the direction of gradient

� Finding zero crossings of the Laplacian

� The idea behind these two approaches is quite similar

� Note that following two approaches are equivalent� Extracting local maxima of the magnitude gradient in the

direction of gradient� Finding points, where the 2nd directional derivative in the

direction of the gradient is zero.

� Hence, how are� Zero crossings of the Laplacian� Zero crossings of the 2nd directional derivative in the

direction of the gradient related?

22.03.2012

13


� Parameterize the directional derivative by t in the direction of gradient

S (x,y)Sx

Sy

Comparison between Edge Detectors


� Finding edges via LoG

( )22

22

2

2 2,

yx

yyyyxxyxxx

SS

SSSSSSS

n

yxS

+++

=∂

∂

( ) ( ) ( )yxSyxSyxS yyxx ,,,2 +=∇

( ) [ ] [ ]

−

−+

=∇

a

b

SS

SSab

b

a

SS

SSbayxS

yyyx

xyxx

yyyx

xyxx,2 122 =+ ba

( )2

2

2

22

2222,,let

⊥∂∂+

∂∂=∇⇒

+=

+=

n

S

n

SyxS

SS

Sb

SS

Sa

yx

y

yx

x

Gradient Magnitude Maxima LoG zero crossing

Comparison between Edge Detectors

22.03.2012

14


Simulations : Edge Detection

Original gray-scale Additive Gaussian Noise


Roberts Operator

Poor robustness to noise, low detection

22.03.2012

15


Sobel & Prewitt Operators

Better robustness to noise, better detection


Compass Gradients (Nevatia-Babu)

Good robustness to noise, noise/localization trade-off

22.03.2012

16


LoG Operator

Better robustness to noise, good detection, better localization(May fail at very nonlinear intensity gradients)


Canny Detector

Better robustness to noise, very good detection, good localization

22.03.2012

17


Performance Comparison

� Canny (5)� LoG (4)� Compass Gradients (3)� Sobel & Prewitt (2)� Roberts’ Cross (1)

� Detection vs. Localization problem still exists


Final Words on Edge Finding

� A simple model is used : unit step + noise

� Better models are emerging for more realistic situations

� Fundamental problem of detection vs. localization still exists

Although edge detection is assumed to be a simple problem, it is not possible to obtain perfect results in real applications

22.03.2012

18


Corner Detection : Intro

� Extraction of salient features from an image is necessary in many applications

� Requirements for such feature detectors� Accurate localization

� Repeatability (detectable under different views)

� Invariance under geometric and photometric transformations

� Corners are typical salient features


A corner model� An ideal corner with an edge

along the x-axis and an angle Θ

( ) >

=otherwise

xifxu

0

01

( ) ( ) ( )yuymxuyxI ⋅−=Θ ,

Θ= tanm

� Convolving with a 2-D Gaussian yields (more realistic data)

( ) ( ) ( ) ( ) dadbbubmaubygaxgyxS ⋅−⋅−⋅−= ∫ ∫∞

∞−

∞

∞−

),(

( ) 2

2

2

2

1 σ

σ

x

exg−

Π=

22.03.2012

19


Gaussian Curvature (1/4)

� Curvature of a point, p, on a curve is defined as rate of change of the tangent vector

� For a surface, there exist too many (infinitely many) curves passing through a given point, p

� Consider the intersection of this surface with the planes which pass thru the normal vector at point p� Intersection of these planes with

the surface results with curves of various curvatures

� Principal curvatures : min & max of such curvatures, Kmin & Kmax(planes always perpendicular)


Gaussian Curvature (2/4)• Parametric surface: x(u,v), xu=∂x(u,v)/∂u,

• Unit surface normal: N = (xu × xv) | xu x xv |

1

• First fundamental form: I( t, t ) t =u’ xu+ v’ xv : a vector in the tangent plane at x

I( t, t ) ≡ t . t = Eu’2 + 2Fu’v’+Gv’ 2

E=xu . xu

F=xu . xv

G=xv . xv

{• Second fundamental form: II( t, t )

II( t, t ) ≡ t . dN (t)= eu’2 + 2fu’v’+gv’ 2 e= – N . xuu = Nu . xu

f = – N . xuv = Nv . xu

g= – N . xvv = Nv . xv{

• Normal and Gaussian curvatures:

κκκκt ≡I( t, t )

II( t, t )K ≡

eg – f 2

EG – F 2


22.03.2012

20


Gaussian Curvature (3/4)Slide from Marc Pollefeys, Lecture Notes on Computer Vision

(1+Iu2+I v

2)2

IuuIvv–Iuv2

Gaussian curvature is equal to: K = .

Monge Patches I

u

v

x ( u, v )= (u, v, I( u, v ))

In this case

• N= ( –Iu , –Iv , 1)T

• E = 1+Iu2; F = IuIv,; G = 1+Iv

2

• e = ; f = ; g =

(1+Iu2+I v

2)1/2

1

(1+Iu2+I v

2)1/2

–Iuu

(1+Iu2+I v

2)1/2

–Iuv

(1+Iu2+I v

2)1/2

–Ivv


Gaussian Curvature (4/4)

1. Compute the Gaussian curvature.2. Select locations of Gaussian curvature extrema.3. Match each elliptic maxima with a hyperbolic minima by principal curvatures4. For a particular match, consider the segment joini ng the elliptic maximum

with the hyperbolic minimum. 5. The point at which the Gaussian curvature is equal to zero � corner

maxmin : curvatureGaussian KKK ⋅= ( )222

2

1 yx

xyyyxx

II

IIIK

++

−=

K > 0

K < 0

K = 0

Gaussian Curvature

Gaussian curvature is 0 for plane&cylindir, +1 for spheres, (-) for saddle points.

It determines whether a surface is locally convex or locally saddle � corner at 0.

22.03.2012

21


K-Curvature� Extract the edges in the image using

one of the edge detection methods� Represent the edges as chain codes

� Calculate k-curvature on the curve1.Let the point denoted by t.2.Subtract the difference of the

directions of the vectors defined by [ t, t+k] and [ t-k, t].

3.Define this difference to be k-curvature at t

4.Average k-curvatures with possibly different weights(emphasizing small k’s) to obtain the curvature.

� The local maximum of a curvature is taken as a corner, if its curvature is above some threshold.

t

t-1t+1

t-2 t+2

t-kt+k


Kitchen-Rosenfeld Cornerness� A cornerness measure may be proposed, as the

change of gradient direction along an edge contour multiplied by the local gradient magnitude.

� Calculating the derivative of , along

−

x

y

I

I1tan

2

1

1

1)(tan

xdx

xd

+=

−

22

1tan

yx

xxyyxxx

y

II

IIII

x

I

I

+−

=∂

∂ −

( ) 22

22

11 tantan

yx

yx

xyx

y

x

y

IIII

II

y

I

I

x

I

I

K +⋅+

−•

∂

∂

∂

∂

=

−−

[ ]2222

11

yxx

y

yyyx

xyxx

xy

yx III

I

II

IIII

IIK

+

−⋅

⋅−

+=

22.03.2012

22


Zuniga-Haralick Facet Model:� Image patch (7x7 or 5x5), can be modeled by a bi-

cubic polynomial

310

29

28

37

265

24321),( ykxykyxkxkykxykxkykxkkyxf +++++++++≈

� Cornerness is again calculated as the rate of change of the direction of the gradient along the edge contour via

( )2

322

22 2

yx

yyyyxxyxxx

II

IIIIIIIK

+

+−=

( )2

323

22

236325

2242

kk

kkkkkkkK

+

+−=

� If this measure has a local maximum and is above a threshold, an edge point, detected after the non-maxima gradient magnitude suppression � corner


Harris-Stephens Corner Detector:

� If the following function, at some point , assumes high values for any direction, (∆x,∆y) , the point can be considered to be significantly distinct.

( )( ) ( ) ( )( )

∆+∆+−= ∫∆∆

R

yx dRyqxpIqpIyxm 2, ,,,

22.03.2012

23



( )( ) ( ) ( )( )

∆+∆+−= ∫∆∆

R

yx dRyqxpIqpIyxm 2, ,,,

( ) ( ) ( ) ( )yxyIyxxIyxIyyxxI yx ,,,, ∆+∆+≈∆+∆+

( )( ) [ ] [ ]

∆∆

∆∆=

∆∆

≈∑ ∑∑

∑∑

′

∆∆ y

x

IIII

IIIIyx

y

xIIyxm

R

C

Ryy

Ryx

Ryx

Rxx

yxyx

444 3444 21

2

,,

� Using Taylor series expansion, as

� The distinctness measure can be written as



� If C’ has 2 significant eigenvalues, λ1, λ2,measure should yield high values signaling an interest point

( )( ) [ ]

∆∆

∆∆≈

′

∆∆ ∑∑

∑∑y

x

IIII

IIIIyxyxm

C

Ryy

Ryx

Ryx

Rxx

yx

444 3444 21

,,

� Instead of calculating the eigenvalues, the product of the eigenvalues is compared via

22121

2

)(

)()(

λλλλ +−⋅=−

k

CkTraceCDet

22.03.2012

24



� There is also Hessian-Laplace detector

� Note its relation to Gaussian Curvature

� Concentrating the measure around center of patch, the summations are done using a Gaussian window,

( )

=∑∑

∑∑

RyR

RyxR

RyxR

RxR

IwIIw

IIwIwyxC 2

2

,

� Search for elliptic maximas at corners or blobs


SUSAN & Fast Corner Detection

� Radically different interest point detectors� An area is defined by the pixels having brightness

similar to that of the nucleus (center point)� USAN: Univalue Segment Assimilating Nucleus

� Near a corner, USAN significantly decreases and

attains a local minimum at the corner point � brightness difference threshold is utilized for deciding

whether a pixel in the circular mask belongs to USAN

� geometrical threshold is deciding whether a local minimum

is a corner point.

22.03.2012

25


� In another method, an arbitrary line k containing thenucleus and intersecting the boundary of the circularwindow at two opposite points is assumed

� For various lines k, corner response function (CRF) iscalculated by minimizing :

� For discrete case, interpolation is used

( ) ( )[ ]22min NPNPk

C IIIIR −+−= ′

( )( ) ( )( )xrxrR

x21

1,0,min

∈=

( ) ( ) ( )221 CPCP IIIIxr −+−= ′

( ) ( ) ( )222 CQCQ IIIIxr −+−= ′

SUSAN & Fast Corner Detection


Blob Detectors : Scale Invariant Feature Transform

� The scale-normalized LoG function � σ2 is required for scale invariance. � The amplitude of the scale space representation in

general decreases with scale, and the factor σ2

compensates for this decrease.

� The maxima and minima of produce the most stable image features compared to other image functions such as Harris corner detectors.

� How to implement scale normalized LoG in an efficient manner?

Courtesy Elif Vural

22.03.2012

26


Scale Invariant Feature Transform

� Scale Invariant Feature Transform (SIFT), is a method for extracting distinctive features from images.

� The features are invariant to image scale and rotation.

� In order to assure scale invariance, the image must be searched for stable features across all possible scales.

� This requires the usage of a continuous function of scale known as scale space.

Courtesy Elif Vural


Scale Space Representation� Given a continuous signal f, the scale space

representation L of f is defined as the solution to the diffusion equation:

with initial condition

� Equivalently, this family can be defined by convolution with Gaussian kernels of variable width t.

Courtesy Elif Vural

22.03.2012

27


� Only possible scale space kernel is the Gaussianfunction. � Scale space L(x,y,σ) of an image I(x,y) as

� The method SIFT suggests that the scale-space extrema of difference of Gaussian functions with two nearby scales (σ and kσ) convolved with the image turn out to be stable features.

Courtesy Elif Vural



� The maxima and minima of the images obtained by convolution with DoG’s are detected by comparing a pixel to its 26 neighbours at the current and adjacent scales.

Courtesy Elif Vural


22.03.2012

28


180

190

200

210

220

80

90

100

110

1200

100

200

300

Difference of Gaussian

� The reason for using the DoG function is that it is a close approximation to the scale-normalized Laplacian-of-Gaussian function .

Courtesy Elif Vural



� SIFT is capable of detecting features resembling a DoG at various scales.

370

380

39056 58 60 62 64 66 68

0

50

100

150

200

250

145150155160165170175

90

100

110

120

20

40

60

80

100

120

140

160

180

140

150

160

170

180

50

60

70

80

900

100

200

Courtesy Elif Vural


22.03.2012

29


Final Words

� Different salient point detectors exist� Edge, corner, blob

� In different applications, these detectors have performances� Edges are more suitable to define segments of

objects� Corners perform better to match two scenes

that are observed from similar distance and angles

� Blobs are more salient in case of scale and view changes

EE 584 MACHINE VISION - Middle East Technical Universityeee.metu.edu.tr/~alatan/Courses/EE584LECTURES/rv04.pdf · 2016. 11. 24. · 22.03.2012 1 METU EE584 Lecture Notes by A.AydinALATAN

Documents