CS223b, Jana Kosecka Image Features Local, meaningful, detectable parts of the image. Line detection Corner detection Motivation Information content high.

CS223b, Jana Kosecka

Image Features

Local, meaningful, detectable parts of the image.• Line detection • Corner detectionMotivation• Information content high• Invariant to change of view point, illumination• Reduces computational burden• Uniqueness • Can be tuned to a task at hand


Canne Edge Detector

• Edge detection involves 3 steps:– Noise smoothing– Edge enhancement– Edge localization

• J. Canny formalized these steps to design an optimal edge detector

• How to go from derivatives to edges ?

Horizontal edges

Before:


• Compute image derivatives • if gradient magnitude > and the value is a local maximum along gradient direction – pixel is an edge candidate

Canny edge detector

gradient magnitudeoriginal image

Edge Detection


Algorithm Canny Edge detector

• The input is image I; G is a zero mean Gaussian filter (std = )

– J = I * G (smoothing)– For each pixel (i,j): (edge enhancement)

– Compute the image gradient 1. J(i,j) = (Jx(i,j),Jy(i,j))’

1. Estimate edge strength » es(i,j) = (Jx

2(i,j)+ Jy2(i,j))1/2

– Estimate edge orientation 1. eo(i,j) = arctan(Jx(i,j)/Jy(i,j))

• The output are images Es - Edge Strength - Magnitude• and Edge Orientation Eo

-


• Es has large values at edges: Find local maxima

• … but it also may have wide ridges around the local maxima (large values around the edges)

ThTh


NONMAX_SUPRESSION

• The inputs are Es & Eo (outputs of CANNY_ENHANCER)• Consider 4 directions D={ 0,45,90,135} wrt x

• For each pixel (i,j) do:• Find the direction dD s.t. d Eo(i,j) (normal to

the edge)

• If {Es(i,j) is smaller than at least one of its neigh. along d} • IN(i,j)=0

• Otherwise, IN(i,j)= Es(i,j)

1. The output is the thinned edge image IN

Edge orientationEdge orientation


Graphical Interpretation

xx xx


Thresholding

• Edges are found by thresholding the output of NONMAX_SUPRESSION

• If the threshold is too high:– Very few (none) edges

•High MISDETECTIONS, many gaps• If the threshold is too low:

– Too many (all pixels) edges•High FALSE POSITIVES, many extra edges


SOLUTION: Hysteresis Thresholding

Es(i,j)Es(i,j)> H> H

Es(i,j)<Es(i,j)<HH

Es(i,j)Es(i,j)>L>L

Es(i,j)<LEs(i,j)<LEs(i,j)Es(i,j)>L>L


Canny Edge Detection (Example)

courtesy of G. Loy

gap is gone

Originalimage

Strongedgesonly

Strong +connectedweak edges

Weakedges


Other Edge Detectors

(2nd order derivative filters)


First-order derivative filters (1D)

• Sharp changes in gray level of the input image correspond to “peaks” of the first-derivative of the input signal.

F(x)F(x)F ’(x)F ’(x)

xx


Second-order derivative filters (1D)

• Peaks of the first-derivative of the input signal, correspond to “zero-crossings” of the second-derivative of the input signal.

F(x)F(x) F ’(x)F ’(x)

xx

F’’(x)F’’(x)


NOTE:

• F’’(x)=0 is not enough!– F’(x) = c has F’’(x) = 0, but there is no edge

• The second-derivative must change sign, -- i.e. from (+) to (-) or from (-) to (+)

• The sign transition depends on the intensity change of the image – i.e. from dark to bright or vice versa.


Edge Detection (2D)

1D1D 2D2D

I(x)I(x) I(x,y)I(x,y)

dd22I(x)I(x)dxdx22

= 0= 0

xx

yy

||I(x,y)| =(II(x,y)| =(Ix x 22(x,y) + I(x,y) + Iyy

22(x,y))(x,y))1/2 1/2 > Th> Th

tan tan = I = Ixx(x,y)/ I(x,y)/ Iyy(x,y) (x,y)

F(x)F(x)

xx

dI(x)dI(x)dxdx

> Th> Th

22I(x,y) =II(x,y) =Ix x x x (x,y) + I(x,y) + Iyy yy (x,y)=0(x,y)=0

LaplaciaLaplacia

nn


Notes about the Laplacian:

• 22I(x,y) is a SCALARI(x,y) is a SCALAR Can be found using a SINGLE maskCan be found using a SINGLE mask Orientation information is lostOrientation information is lost

22I(x,y) is the sum of SECOND-order derivativesI(x,y) is the sum of SECOND-order derivatives– But taking derivatives increases noiseBut taking derivatives increases noise– Very noise sensitive!Very noise sensitive!

• It is always combined with a smoothing It is always combined with a smoothing operation:operation:


LOG Filter

• First smooth (Gaussian filter),• Then, find zero-crossings (Laplacian filter):– O(x,y) = 22(I(x,y) * G(x,y))(I(x,y) * G(x,y))

• Using linearity:– O(x,y) = 22G(x,y) * I(x,y)G(x,y) * I(x,y)– This filter is called: “Laplacian of the This filter is called: “Laplacian of the Gaussian” (LOG)Gaussian” (LOG)


1D Gaussian

2

2

2)( x

exg−

=

2

2

2

2

22

222

2

1)('

xx

ex

xexg−−

−=−=


First Derivative of a Gaussian

2

2

2

2

22

222

2

1)('

xx

ex

xexg−−

−=−=

PositivPositivee NegativeNegative

As a mask, it is also computing a difference (derivative)As a mask, it is also computing a difference (derivative)


Second Derivative of a Gaussian

2

2

23

2

)1

()(''

x

ex

xg−

−=

2D2D

““Mexican Hat”Mexican Hat”


An edge is not a line...

• How can we detect lines ?


Finding lines in an image

• Option 1:– Search for the line at every possible position/orientation

– What is the cost of this operation?

• Option 2:– Use a voting scheme: Hough transform



• Connection between image (x,y) and Hough (m,b) spaces– A line in the image corresponds to a point in Hough space

– To go from image space to Hough space:• given a set of points (x,y), find all (m,b) such that y = mx + b

x

y

m

b

m0

b0

image space Hough space



• Connection between image (x,y) and Hough (m,b) spaces– A line in the image corresponds to a point in Hough space– To go from image space to Hough space:

• given a set of points (x,y), find all (m,b) such that y = mx + b

– What does a point (x0, y0) in the image space map to?

x

y

m

b

image space Hough space

• A: the solutions of b = -x0m + y0

• this is a line in Hough space

x0

y0


Hough transform algorithm• Typically use a different

parameterization

– d is the perpendicular distance from the line to the origin

is the angle this perpendicular makes with the x axis

– Why?Idea – keep an accumulator array (Hough space)and let each edge pixel contribute to it Line candidates are the maxima in the accumulatorarray


Typical Hough Transform

Basic Hough transform algorithm

1. Initialize H[d, q]=02. For each edge point I[x,y] in the image3. For q = 0 to 180 H[d, q] += 1 where point is now a sinusoid in Hough space Find the value(s) of (d, q) where H[d, q] is maximum

The detected line in the image is given b

What’s the running time (measured in # votes)?


Radon transform


Hough Transform for Curves

• The H.T. can be generalized to detect any curve that can be expressed in parametric form:– Y = f(x, a1,a2,…ap)– a1, a2, … ap are the parameters– The parameter space is p-dimensional– The accumulating array is LARGE!


x

y

• Edge detection, non-maximum suppression (traditionally Hough Transform – issues of resolution, threshold selection and search for peaks in Hough space)• Connected components on edge pixels with similar orientation - group pixels with common orientation

Non-max suppressed gradient magnitude

Line fitting


• Line fitting lines determined from eigenvalues and eigenvectors of A• Candidate line segments - associated line quality

second moment matrixassociated with eachconnected componentv1 - eigenvector of A

Line Fitting


Corners contain more edges than lines.

• A point on a line is hard to match.

Corner detection


Finding Corners

Intuition:

• Right at corner, gradient is ill defined.

• Near corner, gradient has two different values.


Formula for Finding Corners

⎥⎥⎦

⎤

⎢⎢⎣

⎡=

∑∑∑∑

2

2

yyx

yxx

III

IIIC

We look at matrix:

Sum over a small region, the hypothetical corner

Gradient with respect to x, times gradient with respect to y

Matrix is symmetric


⎥⎦

⎤⎢⎣

⎡=

⎥⎥⎦

⎤

⎢⎢⎣

⎡=

∑∑∑∑

2

12

2

0

0

λ

λ

yyx

yxx

III

IIIC

First, consider case where:

This means all gradients in neighborhood are:

(k,0) or (0, c) or (0, 0) (or off-diagonals cancel).

What is region like if:

1. l1 = 0?

2. l2 = 0?

3. l1 = 0 and l2 = 0?

4. l1 > 0 and l2 > 0?


General Case:

From Linear Algebra, it follows that because C is symmetric:

RRC ⎥⎦

⎤⎢⎣

⎡= −

2

11

00λ

λ

With R a rotation matrix.

So every case is like one on last slide.


So, to detect corners

• Filter image.• Compute magnitude of the gradient everywhere.

• We construct C in a window.• Use Linear Algebra to find λ1and λ2.• If they are both big, we have a corner.


• Compute eigenvalues of G• If smalest eigenvalue of G is bigger than - mark pixel as candidate feature point

• Alternatively feature quality function (Harris Corner Detector)

Point Feature Extraction


% Harris Corner detector - by Kashif Shahzadsigma=2; thresh=0.1; sze=11; disp=0;

% Derivative masksdy = [-1 0 1; -1 0 1; -1 0 1];dx = dy'; %dx is the transpose matrix of dy % Ix and Iy are the horizontal and vertical edges of imageIx = conv2(bw, dx, 'same');Iy = conv2(bw, dy, 'same'); % Calculating the gradient of the image Ix and Iyg = fspecial('gaussian',max(1,fix(6*sigma)), sigma);Ix2 = conv2(Ix.^2, g, 'same'); % Smoothed squared image derivativesIy2 = conv2(Iy.^2, g, 'same');Ixy = conv2(Ix.*Iy, g, 'same'); % My preferred measure according to research papercornerness = (Ix2.*Iy2 - Ixy.^2)./(Ix2 + Iy2 + eps); % We should perform nonmaximal suppression and thresholdmx = ordfilt2(cornerness,sze^2,ones(sze)); % Grey-scale dilatecornerness = (cornerness==mx)&(cornerness>thresh); % Find maxima[rws,cols] = find(cornerness);

clf ; imshow(bw); hold on;p=[cols rws];plot(p(:,1),p(:,2),'or'); title('\bf Harris Corners')


Example (=0.1)


Example (=0.01)


Example (=0.001)


Harris Corner Detector - Example

CS223b, Jana Kosecka Image Features Local, meaningful, detectable parts of the image. Line detection Corner detection Motivation Information content high.

Documents