Methods for 3D Shape Matching and Retrieval Marcin Novotni & Reinhard Marcin Novotni & Reinhard Klein Klein University of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Jan 19, 2016
Methods for 3D Shape Matching and Retrieval
Marcin Novotni & Reinhard KleinMarcin Novotni & Reinhard KleinUniversity of BonnUniversity of Bonn
Computer Graphics GroupComputer Graphics Group
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Our Aim #1Our Aim #1Our Aim #1Our Aim #1
Given an example:Given an example:
,, ,…
Find the most Find the most
similar object(s)similar object(s)
in a databasein a database
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
MotivationMotivationMotivationMotivation
Lots of 3D archives:Lots of 3D archives: WWW WWW Proprietary databasesProprietary databases ......
Search engines for data:Search engines for data: Text, 2D images, music (MIDI), …Text, 2D images, music (MIDI), … Emerging since 1998 for 3DEmerging since 1998 for 3D
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Our Aim #2Our Aim #2Our Aim #2Our Aim #2
Direct matching Direct matching AlignmentAlignment Establishing correspondencesEstablishing correspondences
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
MotivationMotivationMotivationMotivation
Partial matching/retrievalPartial matching/retrieval
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
MotivationMotivationMotivationMotivation
Partial matching/retrievalPartial matching/retrievalStatistical shape analysisStatistical shape analysisMorphingMorphingTexture transferTexture transferRegistrationRegistration
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
General ProblemGeneral ProblemGeneral ProblemGeneral Problem
Abstract representationAbstract representation facilitating: facilitating:
identification of salient features of 3D identification of salient features of 3D objectsobjects
description of featuresdescription of featurescomparison (matching)comparison (matching)
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
OverviewOverviewOverviewOverview
Matching for 3D Shape RetrievalMatching for 3D Shape Retrieval
Correspondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Matching for Matching for 3D Shape 3D Shape RetrievalRetrieval
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
General ProblemGeneral ProblemGeneral ProblemGeneral Problem
We need a We need a DescriptorDescriptor
D : → D( )
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
d( , )
d( , )
General ProblemGeneral ProblemGeneral ProblemGeneral Problem
We need a We need a Distance MeasureDistance Measure
:=
D( )
D( )
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
d( , ) d( , )
General ProblemGeneral ProblemGeneral ProblemGeneral Problem
We need a We need a Distance MeasureDistance Measure : : Close to (application driven) notion of resemblanceClose to (application driven) notion of resemblance Computationally cheap and robustComputationally cheap and robust
d( , )≤ ≤
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
Feature vectorsFeature vectors
XXii : 3D Zernike Descriptors : 3D Zernike Descriptors [Canterakis ’99, Novotni & Klein ’03, ’04][Canterakis ’99, Novotni & Klein ’03, ’04]Distance Measure: Euclidean DistanceDistance Measure: Euclidean Distance
D( ) ≡x1
xn
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
Retrieval performance Retrieval performance [Novotni & Klein ‘03 ’04][Novotni & Klein ‘03 ’04]
Slightly better than [Funkhouser et al. ’02]Slightly better than [Funkhouser et al. ’02]
Object class dependent performance!Object class dependent performance! Class dependent coefficient importance!Class dependent coefficient importance!
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D 3D Zernike DescriptorsZernike Descriptors3D 3D Zernike DescriptorsZernike Descriptors
Importance
Coeff No. (Frequency)
ChairsFaces Airplanes
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
Relevance feedback:Relevance feedback: User selects relevant / irrelevant itemsUser selects relevant / irrelevant items Distance measure is tunedDistance measure is tuned
Learning Machines:Learning Machines: SVM (Support vector machines) [Vapnik ‘95]SVM (Support vector machines) [Vapnik ‘95] One class SVM [SchOne class SVM [Schöölkopf et al. lkopf et al. ’’9999]] (K)BDA ((Kernel) Biased Discriminant (K)BDA ((Kernel) Biased Discriminant
Analysis) [Zhou et al. ‘01]Analysis) [Zhou et al. ‘01]
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
CorrespondenCorrespondence Matchingce Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Geometric Similarity EstimationGeometric Similarity EstimationGeometric Similarity EstimationGeometric Similarity Estimation
Idea Idea [Novotni & Klein 2001][Novotni & Klein 2001]:: Definition of „geometric“ similarity in terms of Definition of „geometric“ similarity in terms of
a geometric distancea geometric distance
Intuitive, simple, robust.Intuitive, simple, robust.
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Geometric Similarity EstimationGeometric Similarity EstimationGeometric Similarity EstimationGeometric Similarity Estimation
6.786.78 8.858.85 30.2930.29 38.0938.09 67.5367.53Normalized Normalized volumetric volumetric
errorerror0.000.00
Database objects
example
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Geometric Similarity EstimationGeometric Similarity EstimationGeometric Similarity EstimationGeometric Similarity Estimation
Classification by user set thresholdClassification by user set threshold
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Geometric Similarity EstimationGeometric Similarity EstimationGeometric Similarity EstimationGeometric Similarity Estimation
Measures deformation magnitudeMeasures deformation magnitude
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
?
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
?
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
?
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
?
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Ideally: dense mappingIdeally: dense mapping
?
22: f
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Ideally: dense mappingIdeally: dense mapping
Deformation by mapping semanticsDeformation by mapping semantics
[D’Arcy Thompson 1917: On Growth and Form ]
22: f
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Ideally: dense mappingIdeally: dense mapping
Easier: mapping Easier: mapping salient pointssalient points Curvature extremesCurvature extremes Corners (Harris points in 2D)Corners (Harris points in 2D) Etc…Etc… Scale space extremesScale space extremes
22: f
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Ideally: dense mappingIdeally: dense mapping
Easier: mapping Easier: mapping salient pointssalient points Curvature extremesCurvature extremes Corners (Harris points in 2D)Corners (Harris points in 2D) Etc…Etc… Scale space extremesScale space extremes
22: f
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Scale Space extremes Scale Space extremes [Lindeberg ‘94][Lindeberg ‘94]
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
We have:We have:Salient pointsSalient points
Spatial positionSpatial position Size of local blobsSize of local blobs
How to match???How to match???
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Criteria for correspondences:Criteria for correspondences:
SimilarSimilarLocal geometriesLocal geometriesConstellations of pointsConstellations of points
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Criteria for correspondences:Criteria for correspondences:
SimilarSimilarLocal geometriesLocal geometriesConstellations of pointsConstellations of points
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Local descriptionLocal description
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Local descriptionLocal description
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Assumption:Assumption:
Similar local descriptors Similar local descriptors
Similar local geometriesSimilar local geometries
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Criteria for correspondences:Criteria for correspondences:
SimilarSimilarLocal geometriesLocal geometriesConstellations of pointsConstellations of points
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Similar constellations of pointsSimilar constellations of points Smooth mappings leave constellations Smooth mappings leave constellations
consistentconsistent
IdeaIdea Constellations are consistent if mapping is Constellations are consistent if mapping is
smoothsmooth
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Similar constellations of pointsSimilar constellations of pointsIdea: Idea:
Constellations are consistent if mapping is Constellations are consistent if mapping is smoothsmooth
Thin Plate Spline interpolation [Brookstein ’89]Thin Plate Spline interpolation [Brookstein ’89]
minimize:minimize:2( ) ( )
d
I f f d x x
Total curvature
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
minimize:minimize:
Minimizer (Minimizer (Thin Plate SplineThin Plate Spline interpolator): interpolator):
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
2( ) ( )d
I f f d x x
1
( )( ) ( )
( )
xx lkyy
k k
wf
wf
k
xf x Ax t x x
x
Affine part Nonlinear deformation
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
minimize:minimize:
Minimizer (Minimizer (Thin Plate SplineThin Plate Spline interpolator): interpolator):
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
2( ) ( )d
I f f d x x
1
( )( ) ( )
( )
xx lkyy
k k
wf
wf
k
xf x Ax t x x
x
2( ) logr r r 2D Thin Plate Spline
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
1
( )( ) ( )
( )
xx Nkyy
k k
wf
wf
k
xf x Ax t x x
x
minimize:minimize:
Minimizer (Minimizer (Thin Plate SplineThin Plate Spline interpolator): interpolator):
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
2( ) ( )d
I f f d x x
Can be computed by a (N+4)x(N+4) matrix
inversion
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Find (sub)sets of correspondences:Find (sub)sets of correspondences: Small local descriptor distancesSmall local descriptor distances Small deformation energySmall deformation energy
Hierarchical pruning and clusteringHierarchical pruning and clusteringUsing:Using:
Local descriptorsLocal descriptors Geometrical constellation consistencyGeometrical constellation consistency
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
New avenuesNew avenues::Local Descriptions for retrievalLocal Descriptions for retrievalOnline Learning for local Online Learning for local
descriptionsdescriptionsDense matching from salient pointsDense matching from salient pointsEtc.Etc.
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Danke,Danke,DFG!DFG!
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Basis functions in the unit sphere:Basis functions in the unit sphere:
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
SH on the sphere
( , , ) ( ) ( , )m mnl nl lZ r R r Y
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Basis functions in the unit sphere:Basis functions in the unit sphere:
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
SH on the sphere
Function of the radius
( , , ) ( ) ( , )m mnl nl lZ r R r Y
Rotation invariant!
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Basis functions in the unit sphere:Basis functions in the unit sphere:
3D Zernike Moments 3D Zernike Moments [Canterakis ‘99]:[Canterakis ‘99]:
: ,m mnl nlf Z
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
( , , ) ( ) ( , )m mnl nl lZ r R r Y
Object function, e.g. voxel grid
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
3D Zernike Descriptors:3D Zernike Descriptors: Amplitudes of the Zernike decompositionAmplitudes of the Zernike decomposition
Rotation invariantRotation invariant
1
: ,
lnllnl
nl
lnl
F
0
0
. . ( ) is even
n N
l n
s t n l
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Basis functions in the unit sphere:Basis functions in the unit sphere:
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
SH on the sphere
( , , ) ( ) ( , )m mnl nl lr R r Y
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Basis functions in the unit sphere:Basis functions in the unit sphere:
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
SH on the sphere
Function of the radius
( , , ) ( ) ( , )m mnl nl lr R r Y
Rotation invariant!
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Basis functions in the unit sphere:Basis functions in the unit sphere:
3D Zernike Moments 3D Zernike Moments [Canterakis ‘99][Canterakis ‘99]::: ,m m
nl nlf Z
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
( , , ) ( ) ( , )m mnl nl lr R r Y
Object function, e.g. voxel grid
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
3D Zernike Descriptors:3D Zernike Descriptors: Amplitudes of the Zernike decompositionAmplitudes of the Zernike decomposition
Rotation invariantRotation invariant
1
: ,
lnllnl
nl
lnl
F
0
0
. . ( ) is even
n N
l n
s t n l
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
For N=22 : 155 floats as search keyFor N=22 : 155 floats as search key
Timings (1.8 GHz Pentium):Timings (1.8 GHz Pentium):Voxelization: 0.3 – 10.0 sec / objectVoxelization: 0.3 – 10.0 sec / objectComputation: 0.2 sec / objectComputation: 0.2 sec / objectRetrieval (1814 objects): 0.3 secRetrieval (1814 objects): 0.3 sec
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
Retrieval performance Retrieval performance [Novotni & Klein ’04][Novotni & Klein ’04]
Slightly better than [Funkhouser et al. ’02]Slightly better than [Funkhouser et al. ’02]
Object class dependent performance!Object class dependent performance! Class dependent coefficient importance!Class dependent coefficient importance!
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D 3D Zernike DescriptorsZernike Descriptors3D 3D Zernike DescriptorsZernike Descriptors
Importance
Coeff No.
ChairsFaces Airplanes
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
3D Zernike functions [Canterakis ‘99]3D Zernike functions [Canterakis ‘99]
are polynomials such that are are polynomials such that are orthonormal within the unit ballorthonormal within the unit ball
( , , ) : ( ) ( , )m mnl nl lZ r R r Y
mnlZnlR
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
3D Zernike functions [Canterakis ‘99]3D Zernike functions [Canterakis ‘99]
are polynomials such that are orthonormal are polynomials such that are orthonormal within the unit ballwithin the unit ball
3D Zernike Moments:3D Zernike Moments:
( , , ) : ( ) ( , )m mnl nl lZ r R r Y
mnlZnlR
: ,m mnl nlf Z
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
3D Zernike Descriptors:3D Zernike Descriptors: Amplitudes of the Zernike decompositionAmplitudes of the Zernike decomposition
Rotation invariantRotation invariant
1
: ,
lnllnl
nl
lnl
F
0
0
. . ( ) is even
n N
l n
s t n l
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
For N=20 : 121 floats as search keyFor N=20 : 121 floats as search key
Timings (1.8 GHz Pentium):Timings (1.8 GHz Pentium):Voxelization: 0.3 – 10.0 sec / objectVoxelization: 0.3 – 10.0 sec / objectComputation: 0.2 sec / objectComputation: 0.2 sec / objectRetrieval (1814 objects): 0.3 secRetrieval (1814 objects): 0.3 sec
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
Retrieval performance Retrieval performance [Novotni & Klein ’04][Novotni & Klein ’04]
Slightly better than [Funkhouser et al. ’02]Slightly better than [Funkhouser et al. ’02]
Object class dependent performance!Object class dependent performance! Class dependent coefficient importance!Class dependent coefficient importance!
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Matching should be:Matching should be:
Independent of topologyIndependent of topologyRobustRobustSuitable for partial matchingSuitable for partial matching
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Local descriptionLocal description Local shape histograms Local shape histograms
Not rotation invariantNot rotation invariant
Rotation invarianceRotation invariance
Amplitudes of the Fourier Amplitudes of the Fourier TransformTransform
( , )r
1ˆ ( , ) ( , ), ip
Sr p r e
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors3D Zernike Descriptors
155 floats as search key155 floats as search key
Retrieval (1814 objects): 0.3 secRetrieval (1814 objects): 0.3 sec
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Stuff to rememberStuff to remember::
Salient points simplify the problemSalient points simplify the problemSmooth mapping iff consistent Smooth mapping iff consistent
constellationsconstellations
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Stuff to rememberStuff to remember::
Salient points simplify the problemSalient points simplify the problem VolumetricVolumetric On the surfaceOn the surface
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Stuff to rememberStuff to remember::
Salient points simplify the problemSalient points simplify the problemSmooth mapping iff consistent Smooth mapping iff consistent
constellationsconstellations
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
New avenuesNew avenues::Local Descriptions for retrievalLocal Descriptions for retrieval
Retrieval by part selection & recognitionRetrieval by part selection & recognitionRetrieval from large scenesRetrieval from large scenes
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
New avenuesNew avenues::Local Descriptions for retrievalLocal Descriptions for retrievalOnline Learning for local Online Learning for local
descriptionsdescriptionsAdopting pattern recognition methodsAdopting pattern recognition methods
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
New avenuesNew avenues::Local Descriptions for retrievalLocal Descriptions for retrievalOnline Learning for local Online Learning for local
descriptionsdescriptionsDense matching from salient pointsDense matching from salient points
Morphing, registration, object statisticsMorphing, registration, object statistics
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Our Aim #2Our Aim #2Our Aim #2Our Aim #2
Direct matching Direct matching AlignmentAlignment
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Scale space extremes Scale space extremes [Lindeberg ‘94][Lindeberg ‘94] Blob detection by Blob detection by localizing extremes of Laplacian … localizing extremes of Laplacian … … … in scale and spacein scale and space
Size of the blob
Position of the blob
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Maxima of Laplacian over scalesMaxima of Laplacian over scales
Marcin Novotni Marcin Novotni Reinhard Klein Reinhard KleinUniversity of Bonn University of Bonn Computer Graphics Group Computer Graphics Group
Correspondence MatchingCorrespondence MatchingCorrespondence MatchingCorrespondence Matching
Spatial maximaSpatial maxima