3D Shape 3D Shape Registration using Registration using Regularized Medial Regularized Medial Scaffolds Scaffolds Ming-Ching Chang Ming-Ching Chang Frederic F. Leymarie Frederic F. Leymarie Benjamin B. Kimia Benjamin B. Kimia LEMS, Division of Engineering, Brown University 3DPVT 2004 Thessaloniki, Greece Sep. 6-9, 2004
27
Embed
3D Shape Registration using Regularized Medial Scaffolds Ming-Ching Chang Frederic F. Leymarie Benjamin B. Kimia LEMS, Division of Engineering, Brown University.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
3D Shape 3D Shape Registration using Registration using Regularized Medial Regularized Medial
ScaffoldsScaffolds
Ming-Ching ChangMing-Ching ChangFrederic F. Leymarie Frederic F. Leymarie
Benjamin B. KimiaBenjamin B. Kimia
LEMS, Division of Engineering, Brown University
3DPVT 2004 Thessaloniki, GreeceSep. 6-9, 2004
OutlineOutline Registration BackgroundRegistration Background Medial Scaffold: Representation for Medial Scaffold: Representation for
3D Shapes3D Shapes Graduated Assignment Graph Graduated Assignment Graph
Local RegistrationLocal RegistrationInitial position given. ICP and it’s improvementsInitial position given. ICP and it’s improvementsSurvey: [Campbell & Flynn CVIU’01], [3DIM’03]Survey: [Campbell & Flynn CVIU’01], [3DIM’03]
Global Registration Global Registration Skeleton-based, Surface-feature basedSkeleton-based, Surface-feature based
Fundamental for processing scanned objects, modeling, Fundamental for processing scanned objects, modeling, matching, recognition, medical applications, etc.matching, recognition, medical applications, etc.
More difficult.Main focus of this talk.
Mesh with 20K points
2K points
Local Registration: Iterative Closest Local Registration: Iterative Closest Points (ICP)Points (ICP)
[Besl & McKay PAMI’92][Besl & McKay PAMI’92] Needs a good Needs a good initial alignmentinitial alignment Local search problemsLocal search problems
Sensitive to local minimum, noiseSensitive to local minimum, noise May converge slowlyMay converge slowly Lack of surface representationLack of surface representation
Internal skeletal graphInternal skeletal graph of a of a closedclosed surface surface mesh, using an edge collapse algorithmmesh, using an edge collapse algorithm
match match largest common subgraphlargest common subgraph [Sundar [Sundar et. al. et. al. ’03]: ’03]: Skeletal Skeletal treetree from thinning from thinning
voxels via voxels via a distance transform, coarse-to-fine matchinga distance transform, coarse-to-fine matching1. Skeletons over-simplified 2. Graph topology not handled well
Proposed: Match the Medial Proposed: Match the Medial ScaffoldScaffold
Medial ScaffoldMedial Scaffold: medial structure in the form of a 3D hypergraph: medial structure in the form of a 3D hypergraph
Medial Axis is sensitive to noise & perturbations.Medial Axis is sensitive to noise & perturbations. TransitionsTransitions: sudden changes in topology : sudden changes in topology 2D examples:2D examples:
The growth of an axis with small perturbations (A1A3)
The swapping of MA branches (A1
4)
Smoothing/medial branch pruning
Pruning:
Seven Types of Transitions Seven Types of Transitions in 3Din 3D
A1A3
-I
A15
A14
A5
A1A3-II
A12A3-I
A12A3-II
[Giblin & Kimia ECCV’02]
Scaffold Scaffold RegularizationRegularization
Transition removal, i.e. remove Transition removal, i.e. remove topological instabilitytopological instability SmoothingSmoothing
Match Medial Scaffolds by Match Medial Scaffolds by Graph MatchingGraph Matching
IntractabilityIntractability Weighted graph matching: Weighted graph matching: NPNP-hard-hard One special case: Largest common subgraph: One special case: Largest common subgraph: NPNP--
completecomplete Only “good” Only “good” sub-optimalsub-optimal solutions can be found solutions can be found
by matching 2D by matching 2D shock graphsshock graphs
3D 3D hypergraphhypergraph matching: matching: Additional dimensionAdditional dimension Generally not a Generally not a treetree, might have isolated , might have isolated loopsloops No inside/outside: non-closed surfaces or surface No inside/outside: non-closed surfaces or surface
G, G: 2 undirected graphsG, G: 2 undirected graphsI: # of nodes in G, I: # of nodes in GI: # of nodes in G, I: # of nodes in G{G{Gii}, {G}, {Gii} nodes} nodes{G{Gijij}, {G}, {Gijij} edges: } edges: adjacency matricesadjacency matrices of of
graphsgraphs
The The match matrixmatch matrix MMiiii = 1 if node i in G corresponds to node i in = 1 if node i in G corresponds to node i in
G, G, = 0 otherwise= 0 otherwise
a
b cz
G: o
p
qGG:
Graduated AssignmentGraduated Assignment
Then objective function to maximize over the Then objective function to maximize over the space of space of MM is: is:
LLiijjiijj: link similarity between G: link similarity between Gijij and G and Gijij
NNiiii: node similarity between G: node similarity between Gii and G and Gii
Cost of matching Gij to Gij.If the nodes match, how similar the links are.
Cost of matching Gi to Gi
Modified Graduated Assignment Modified Graduated Assignment for 3D Medial Scaffold Matchingfor 3D Medial Scaffold Matching
Node cost: (radius)
Link cost: (length)
Sheet (hyperlink) cost: (corner angle)
α, β: weights
Results: Results: SheepSheep
Sheep 20K points, after surface reconstruction
Sheep 1-20K Full Scaffold
The scaffold matching is good enough that ICP is not required.
Result of Scaffold Graph Result of Scaffold Graph MatchingMatching
Colors to represent correct link matches;grays to represent miss matches.
Two scans of an object at the same resolution (20K points):
Results: David HeadResults: David Head20K
30K
Two sub-samples from the ground truth (42350 points)
Matching ResultsMatching ResultsScaffold matching result Scaffold matching + ICP
Validation against the ground truth: (object dimension = 69x69x76) average sq dist 3.129372 average sq dist 0.000005
Partial Shape Matching: Partial Shape Matching: Sheep with the rear portion Sheep with the rear portion
cut offcut off
Sheep 1-20K with the rear portion cutSheep 1-20K scaffold
Non-closed Surface: Non-closed Surface: Archaeological PotArchaeological Pot
Two scans of the outside surface of a pot (50K and 40K). The inner surface of the pot is missing.
Alignment by Scaffold Alignment by Scaffold MatchingMatching
The scaffold matching result
Final Registration after Final Registration after ICPICP
Two Possible Reasons for Two Possible Reasons for Incorrect MatchesIncorrect Matches
Graduated assignment matching is not optimal.Graduated assignment matching is not optimal.
Typically this does not affect the overall registration if a sufficient number of nodes are correctly assigned.
MIS-MATCH!!
Pot sherd 1 (50K) Pot sherd 2 (10K)
1. Only 8 shock vertices to match2. Transitions not completely handled Result of shock matching
Reasons for Incorrect Matches Reasons for Incorrect Matches (cont’d)(cont’d)
Medial structure transitions are not completely handled.Medial structure transitions are not completely handled.
MIS-MATCH!!
Benefits of 3D Medial Benefits of 3D Medial ScaffoldsScaffolds
A global A global hierarchicalhierarchical structure is structure is built-in.built-in.
ScaleScale is represented. is represented. Salient features are captured:Salient features are captured:
Generalized axesGeneralized axes of elongated objects of elongated objects curvature extremacurvature extrema and and ridgesridges
The medial representation is The medial representation is completecomplete. . ReconstructionReconstruction of the shape of the shape is always possible.is always possible.
RobustRobust after regularization. after regularization. Easy to handle shape Easy to handle shape
deformationsdeformations.. Data from Cyberware Inc.
ConclusionsConclusions
Take input as point clouds or partial meshes.Take input as point clouds or partial meshes. Robust to noise. Invariant under different resolutions Robust to noise. Invariant under different resolutions
Can be Can be graphsgraphs with loops (not a with loops (not a treetree).). Contains Contains sheetssheets, , linkslinks, , nodesnodes. Not over-simplified.. Not over-simplified. Carefully Regularized.Carefully Regularized.
Global Registration by Matching Medial ScaffoldsGlobal Registration by Matching Medial Scaffolds
- Skeleton:
Nearly-optimal.Nearly-optimal. Can be improved to doCan be improved to do finefine registration. registration. Can be extended to register Can be extended to register non-rigidnon-rigid objects. objects. Can be extended to do Can be extended to do recognitionrecognition..
- Match:
Thank YouThank You
This material is based upon work supported This material is based upon work supported by the National Science Foundation under by the National Science Foundation under Grants 0205477 and 0083231.Grants 0205477 and 0083231.