SIFT: Scale Invariant Feature Transformahmedeltahawy.weebly.com/uploads/4/7/2/2/47228767/sift.pdf · Outline SIFT Speeded Up Robust Feature -SURF Conclusion Introduction SIFT Steps

Outline SIFT Speeded Up Robust Feature -SURF Conclusion

SIFT: Scale Invariant Feature Transform

Ahmed Othman

Systems Design DepartmentUniversity of Waterloo, Canada

October, 23, 2012

1 / 25


1 SIFTIntroductionScale-space extrema detectionKeypoint localizationOrientation assignmentKeypoint descriptor

2 Speeded Up Robust Feature -SURF

3 Conclusion

2 / 25


Introduction

Objectives

Extract different features from object image.These features are:

Invariant to image scale and rotation.Used to recognize this object in different images.

3 / 25


Introduction

SIFT Output

Testing Data

4 / 25


Introduction

SIFT Output

Matching Process.

5 / 25


Introduction

SIFT Output

Matching Process.6 / 25


Introduction

SIFT Steps

SIFT is consist of four stages.1 Scale-space extrema detection:

Search over all scales of the image.Detect candidate key points.

2 Keypoint localization.Measure the stability of the candidate point.

3 Orientation assignment.Image gradient directions are used to assign orientations tothe keypoint locations

4 Keypoint descriptor.Each point is represented with a descriptor vector.

7 / 25


Scale-space extrema detection


Detecting key points that are invariant to different scales ofthe image.Use Gaussian filter to detect the candidate points.Define the scale space of an image L(x , y ,�) as:

L(x , y ,�) = G(x , y ,�) ⇤ I(x , y), (1)

G(x , y ,�) =1

2⇡�2 exp�(x2+y2)/2�2(2)

Difference of Gaussian function D(x , y ,�) are used todetect stable points by convolving to the image.

D(x , y ,�) = (G(x , y , k�)� G(x , y ,�)) ⇤ I(x , y) (3)

D(x , y ,�) = L(x , y , k�)� L(x , y ,�). (4)

k is a constant multiplicative factor.8 / 25



Difference of Gaussians.

9 / 25




Every point is compared to:1 Its eight neighbors in 3 ⇥ 3 map.2 Its nine neighbors in the scale above and below.

The point is selected if :1 Larger than the other 26 points (local maxima).2 Less than the other 26 points (local minima).

Key point detection.

10 / 25


Keypoint localization


The candidate points are revisited and outliers arediscarded.Two types of points are Discarded:

1 Points with low contrast.2 points that poorly localized along an edge.

Use the Taylor expansion of the scale space D(x , y ,�).The sample point is in the origin.

11 / 25



Keypoint localization - Low contrast points removal

The Taylor series of the scale space is defined as:

D(x) = D +@DT

@xx +

12

xT @2D@x2 x (5)

The location of the extremum x̂ is determined as:

x̂ = �@2D�1@x2

@D@x

(6)

The Taylor value at the extremum is:

D(x̂) = D +12@DT@xx̂ (7)

Points that its |D(x)| < 0.03 are discarded.

12 / 25



Keypoint localization - Poor edge point removal

DOG have strong response along the edge.The location along the edge may be poorly determined.The poor point in the DOG have:

1 A large principal curvature across the edge.2 A small principal curvature in the perpendicular direction.

A thresholded principle curvature (PC) is used to removepoor edge points.

13 / 25



Keypoint localization -Poor edge point removal

PC computed from 2 ⇥ 2 Hessian matrix:

H =

Dxx DxyDxy Dyy

�(8)

The trace Tr(H) and the determinant Det(H)are computed.

Tr(H) = Dxx + Dyy = ↵+ �, (9)

Det(H) = DxxDyy � (Dxy )2 = ↵�. (10)

↵ the largest magnitude eignvalue � is the lowest one.

14 / 25



Keypoint localization -Poor edge point removal

let r be:r =

↵

�. (11)

The ratio of the principle curvature is checked by checking :

Tr(H)2

Det(H)2 <(r + 1)2

r. (12)

Points with r greater than 10 are discarded.

15 / 25





16 / 25


Orientation assignment


Based on image properties, an orientation is assigned toeach keypoint.This makes the descriptor invariant to image rotation.The gradient magnitude m(x , y) and the orientation ✓(x , y)of every smoothed image L(x , y) are used.

m(x, y) =p

(L(x + 1, y) � L(x � 1, y))2 + (L(x, y + 1) � L(x, y � 1))2)) (13)

✓(x, y) = tan� 1(((L(x, y + 1) � L(x, y � 1)))/(L(x + 1, y) � L(x � 1, y))) (14)

17 / 25




The process is as follow:1 A 36 (for 360 degree) bins histogram is built from the

orientation of the points within a neighbiours of the keypoints.

2 Sample added to the histogram weighted by:Gradient magnitude,A Gaussian filter with � equal 1.5 times that of the scale ofthe keypoint.

3 The highest peak and the 80% of the highest peak aredetected.

Orientation Assignement.

18 / 25




Orientation Assignement.

19 / 25


Keypoint descriptor

Keypoint descriptor

Use the orientation histogram of the previous step.Look at 16 ⇥ 16 around the key point.Divide 16 ⇥ 16 region into four 4 ⇥ 4 blocks 8 bins each.For each block compute a histogram of gradient magnitudeand orientation.The descriptor vector of size 128 is formed for every point.

SIFT Descriptors.20 / 25


SURF- Speeded Up Robust Feature

Key points are detected using Hessian matrix.A descriptor vector is built describing the neighbourhood ofevery keypointMatching process is performed based on the descriptorvector.

21 / 25


SURF- Descriptor vector

Orientation assignment is done using Haar waveletresponse in x and y directions.A 20s rectangle is generated around each keypoint.The rectangle is splitted into 4 ⇥ 4 subregions.

22 / 25


SURF- Descriptor vector

For each subregion:the Haar wavelet response in horizontal and verticaldirection dx , and dy .dx , and dy are summed over each subregion.The sum of the absolute value of the responses |dx | and|dy | are calculated.

A four dimension vector [dxdy |dx ||dy |] is extracted.

Each sub region have 4 vectors come up to 64.Matching process is performed based on the descriptorvector.

23 / 25


SIFT - SURF

Keypoints are detected.Different orientations are assigned around the keypoints.A set of discriptors are assigned to the key points.These descriptors are used for object recognition.

24 / 25


References

Distinctive image features from scale-invariant keypoints.by David Lowe,International journal of computer visionvolume 60, pages (91–110), 2004.Vedio lecture by Dr. Mubarak Shah, Professor of ComputerScience at university of Central Florida, USA.Google images for SIFT.SURF: Speeded Up Robust Features.Herbert Bay, Tinne Tuytelaars, and Luc Van GoolComputer Vision 2006.

25 / 25

SIFT: Scale Invariant Feature Transformahmedeltahawy.weebly.com/uploads/4/7/2/2/47228767/sift.pdf · Outline SIFT Speeded Up Robust Feature -SURF Conclusion Introduction SIFT Steps

Documents