Guest Lecturer: Prof. BoqingGongbagci/teaching/computervision17/Lec4.pdf · Guest Lecturer: Prof. BoqingGong Lecture 4 –Finding Features 1. Outline Non-Deep Learning Approaches

CAP5415-ComputerVisionLecture4- FindingFeatures(introductiontofeatureengineering,

localfeatures)

GuestLecturer:Prof.Boqing Gong

Lecture4–FindingFeatures

1

Outline

Non-DeepLearningApproaches(IntroductiontoFeatureEngineering)

• Ex1.Key-pointFeatures– Harriscornerdetection

• Ex2.AffineInvariance

• ReadSzeliski,Chapter4.• ReadShah,Chapter2.• Read/ProgramCVwithPython,Chapters1and2.


2

MotivationforFeatureFinding:MatchingLecture4–FindingFeatures

3

MotivationforFeatureFinding:Hardercase


4Credit:Fei Fei Li


5

Top-down processing suggests that we form our perceptions starting with a larger object, concept or idea before working our way toward more detailed information.

In other words, top-down processing happens when we work from the general to the specific; the big picture to the tiny details.

In top-down processing, your abstract impressions can influence the sensory data that you gather.

9/3/15


6


7

FindingFeaturesinVideos


8

Subsequentstudieshaveshownthatmanycomplexactionscanberecognizedonthebasisofsuch'point-lightdisplays',including

• facialexpressions,• SignLanguage,• armmovements,• andvariousfull-bodyactions.


9

ChoosinginterestpointsLecture4–FindingFeatures

10

Wherewouldyoutellyourfriendtomeetyou?

Slide Credit: James Hays

Featuresareusedin…• Automate object tracking • Point matching for computing disparity• Motion based segmentation• Object Recognition • 3D Object Reconstruction• Robot Navigation• Image Retrieval/Indexing• ….


11

Whatisaninterestpoint?

• Expressivetexture• Thepointatwhichthedirectionoftheboundaryofobjectchangesabruptly

• Intersectionpointbetweentwoormoreedgesegments


12

Whatisaninterestpoint?Lecture4–FindingFeatures

13

PropertiesofInterestPoints

• Detect all (or most) true interest points• No false interest points • Well localized• Robust with respect to noise• Efficient detection


14

PossibleApproachesforCornerDetection(ex.interestpoint)

• Basedonbrightnessofimages• Usuallyimagederivatives

• Basedonboundaryextraction• Firststepedgedetection• Curvatureanalysisofedges


15

CorrespondenceAcrossViews

• Correspondence:matchingpoints,patches,edges,orregionsacrossimages


16

≈

GoalsforKeyPointsLecture4–FindingFeatures

17

Detectpointsthatarerepeatable anddistinctive

OverviewofKeyPoint MatchingLecture4–FindingFeatures

18K.Grauman,B.Leibe

Af Bf

A1

A2 A3

Tffd BA

MajorTrade-offsLecture4–FindingFeatures

19

More Repeatable More Interest Points

A1

A2 A3

Detection of interest points

More Distinctive More Flexible

Description of patches

Robust to occlusionWorks with less texture

Minimize wrong matches

Robust detectionPrecise localization

LocalFeatures:MainComponentsLecture4–FindingFeatures

20

1) Detection: Identify the interest points

2) Description: Extract feature vector descriptor surrounding each interest point.

3) Matching: Determine correspondence between descriptors in two views

],,[ )1()1(11 dxx !=x

],,[ )2()2(12 dxx !=x

Kristen Grauman

Goal: interest operator repeatabilityLecture4–FindingFeatures

21

• We want to detect (at least some of) the same points in both images.

• Yet we have to be able to run the detection procedure independently per image.

No chance to find true matches!


22

Goal: descriptor distinctiveness• We want to be able to reliably determine

which point goes with which.

• Must provide some invariance to geometricand photometric differences between the two views.

?

Kristen Grauman

Some patches can be localizedor matched with higher accuracy than others

why ?


23

• Edge detectors often fail in corners. Why?

• Corner point can be recognized in a window

• Shifting a window in any direction should give a large change in intensity

• LOCALIZINGandUNDERSTANDINGshapes…


24

BasicIdeainCornerDetectionLecture4–FindingFeatures

25

“flat” region:no change in all directions

“edge”:no change along the edge direction

“corner”:significant change in all directions

TemplateMatchingLecture4–FindingFeatures

26

2

4�4 5 5�4 5 5�4 �4 �4

3

5

2

45 5 5�4 5 �4�4 �4 �4

3

5


27

2

4�4 5 5�4 5 5�4 �4 �4

3

5

2

45 5 5�4 5 �4�4 �4 �4

3

5

Completesetofeighttemplatescanbegeneratedbysuccessive90degreeofrotations.


28

2

4�4 5 5�4 5 5�4 �4 �4

3

5

2

45 5 5�4 5 �4�4 �4 �4

3

5


Whythesummationoffilteris0?


29

2

4�4 5 5�4 5 5�4 �4 �4

3

5

2

45 5 5�4 5 �4�4 �4 �4

3

5


Whythesummationoffilteris0? InsensitivetoabsolutechangeInintensity!

CornerDetectionusingtheHessianMatrix

• Cornersarecharacterizedbyhigh-curvatureofintensityvalues.


30

H(p) =

Ixx

(p) Ixy

(p)Ixy

(p) Iyy

(p)

�




31

H(p) =

Ixx

(p) Ixy

(p)Ixy

(p) Iyy

(p)

�




32

H(p) =

Ixx

(p) Ixy

(p)Ixy

(p) Iyy

(p)

�

HessianMatrixatvoxelp.




33

H(p) =

Ixx

(p) Ixy

(p)Ixy

(p) Iyy

(p)

�


EigenvaluesoftheHessianindicatecornerfeaturesifbotheigenvaluesarelarge!




34

H(p) =

Ixx

(p) Ixy

(p)Ixy

(p) Iyy

(p)

�



Onelarge,onesmalleigenvalues->edgeregions

Twosmalleigenvaluesindicate->flatregions




35

H(p) =

Ixx

(p) Ixy

(p)Ixy

(p) Iyy

(p)

�



Onelarge,onesmalleigenvalues->edgeregions

Twosmalleigenvaluesindicate->flatregions

HessianmatrixsummarizesDistributionofgradients.

Reminder:Eigenvalues/EigenvectorsLecture4–FindingFeatures

36

det(H) = Ixx

Iyy

� I2xy

Tr(H) = Ixx

+ Iyy

H(p) =

Ixx

(p) Ixy

(p)Ixy

(p) Iyy

(p)

�


37

det(H) = Ixx

Iyy

� I2xy

Tr(H) = Ixx

+ Iyy

H(p) =

Ixx

(p) Ixy

(p)Ixy

(p) Iyy

(p)

�

TheeigenvaluesofthematrixHaresolutionsofitscharacteristicspolynomial

det(H� �12) = 0


38

det(H) = Ixx

Iyy

� I2xy

Tr(H) = Ixx

+ Iyy

H(p) =

Ixx

(p) Ixy

(p)Ixy

(p) Iyy

(p)

�


det(

Ixx

(p)� � Ixy

(p)Ixy

(p) Iyy

(p)� �

�) = 0

det(H� �12) = 0


39

det(H) = Ixx

Iyy

� I2xy

Tr(H) = Ixx

+ Iyy

H(p) =

Ixx

(p) Ixy

(p)Ixy

(p) Iyy

(p)

�


det(

Ixx

(p)� � Ixy

(p)Ixy

(p) Iyy

(p)� �

�) = 0

det(H� �12) = 0

(Ixx

(p)� �)(Iyy

(p)� �)� Ixy

(p)2 = 0

HarrisandStephensCornerDetector

• InsteadofusingHessian ofimageI,usefirstderivativeofsmoothedI (i.e.,L)


40

G(p,�) =

L2x

(p,�) Lx

(p,�)Ly

(p,�)Lx

(p,�)Ly

(p,�) L2y

(p,�)

�




41

G(p,�) =

L2x

(p,�) Lx

(p,�)Ly

(p,�)Lx

(p,�)Ly

(p,�) L2y

(p,�)

�

• Insteadofcalculatingeigenvalues,wewillconsidercornerness feature:




42

G(p,�) =

L2x

(p,�) Lx

(p,�)Ly

(p,�)Lx

(p,�)Ly

(p,�) L2y

(p,�)

�

• Insteadofcalculatingeigenvalues,wewillconsidercornerness feature:

H(p,�,↵) = det(G)� ↵.T r(G)forsmallalpha(=1/25)

HarrisandStephensCornerDetectorLecture4–FindingFeatures

43

H(p,�,↵) = �1�2 � ↵.(�1 + �2)


• ThesamebehaviorasHessianbaseddetector,butnowwedirectlyusesimpledeterminantandtrace functions!


44

H(p,�,↵) = �1�2 � ↵.(�1 + �2)

H(p,�,↵) = det(G)� ↵.T r(G)


• ThesamebehaviorasHessianbaseddetector,butnowwedirectlyusesimpledeterminantandtrace functions!


45

H(p,�,↵) = �1�2 � ↵.(�1 + �2)

H(p,�,↵) = det(G)� ↵.T r(G)

Othercornernessmeasures:or�1�2

�1 + �2�1 � ↵�2

Harmonicmean Triggs


46


47

Ix Iy

Ix2 Iy2 IxIy

Squareofderivatives

GaussianSmoothing

FeatureExtraction:CornersLecture4–FindingFeatures

489300 Harris Corners Pkwy, Charlotte, NC

InvariantLocalFeaturesLecture4–FindingFeatures

49

•Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging parameters

Features Descriptors

InvarianceandCovariance• We want corner locations to be invariant to photometric

transformations and covariant to geometric transformations– Invariance: image is transformed and corner locations do not

change– Covariance: if we have two transformed versions of the same

image, features should be detected in corresponding locations


50

AffineTransformationLecture4–FindingFeatures

51

translation


52

rotation

translation


53

rotation

translation

scaling


54

rotation

translation

scaling

Cornersareinvariantto

✔

✔

✖

NextLecture

• WewillcontinueAffineInvarianceconcept• Introducetheconceptofscale-spaceapproaches

• Introduce“famous”SIFTalgorithm(Lowe’sScaleinvariantfeaturetransform)


55

Questions?Lecture4–FindingFeatures

56

Guest Lecturer: Prof. BoqingGongbagci/teaching/computervision17/Lec4.pdf · Guest Lecturer: Prof. BoqingGong Lecture 4 –Finding Features 1. Outline Non-Deep Learning Approaches

Documents