Geometric Registration for Deformable Shapes 2.2 Deformable Registration Variational Model· Deformable ICP
Geometric Registration for Deformable Shapes
2.2 Deformable RegistrationVariational Model· Deformable ICP
Variational ModelWhat is deformable shape matching?
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 33
Example
?
What are the Correspondences?
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 4
What are we looking for?
Problem Statement:
Given:• Two surfaces S1, S2 ⊆ ℝ3
We are looking for:• A reasonable deformation function f: S1 → ℝ3
that brings S1 close to S2
?
S1S2
f
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 55
Example
?
Correspondences? no shape match
too much deformation optimum
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 6
This is a Trade-Off
Deformable Shape Matching is a Trade-Off:• We can match any two shapes
using a weird deformation field
• We need to trade-off: Shape matching (close to data) Regularity of the deformation field (reasonable match)
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 77
Components:
Matching Distance:
Deformation / rigidity:
Variational Model
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 8
Variational Model
Variational Problem:• Formulate as an energy minimization problem:
)()()( )()( fEfEfE rregularizematch +=
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 9
Part 1: Shape Matching
Assume:• Objective Function:
• Example: least squares distance
• Other distance measures:Hausdorf distance, Lp-distances, etc.
• L2 measure is frequently used (models Gaussian noise)
S2f(S1)
( )212,1)( ),()( SSfdistfE match =
∫∈
=11
12
21)( ),()(
Sx
match dSdistfE xx
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 10
Point Cloud Matching
Implementation example: Scan matching• Given: S1, S2 as point clouds
S1 = {s1(1), …, sn(1)} S2 = {s1(2), …, sm(2)}
• Energy function:
• How to measure ?
Estimate distance to a point sampled surface
si(2)
fi(S1)( )∑
=
=m
ii
match SdistmSfE
1
2)2(1
1)( ,||)( s
( )x,1Sdist
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 11
Surface approximation
Solution #1: Closest point matching• “Point-to-point” energy
( )∑=
=m
iiSini
match sNNsdistmSfE
1
2)2()2(1)( )(,||)(1
si(2)
f(S1)
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 12
Surface approximation
Solution #2: Linear approximation• “Point-to-plane” energy• Fit plane to k-nearest neighbors• k proportional to noise level, typically k ≈ 6…20
si(2)
f(S1)
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 13
Surface approximation
Solution #3: Higher order approximation• Higher order fitting (e.g. quadratic)
Moving least squares
si(2)
f(S1)
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 14
Variational Model
Variational Problem:• Formulate as an energy minimization problem:
)()()( )()( fEfEfE rregularizematch +=
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 15
What is a “nice” deformation field?• Isometric “elastic” energies
Extrinsic (“volumetric deformation”) Intrinsic (“as-isometric-as
possible embedding”)
• Thin shell model Preserves shape (metric plus curvature)
• Thin-plate splines Allowing strong deformations, but keep shape
Part II: Deformation Model
)()( fE rregularize
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 16
Elastic Volume Model
Extrinsic Volumetric “As-Rigid-As Possible”• Embed source surface S1 in volume• f should preserve 3×3 metric tensor (least squares)
[ ]∫ −∇∇=1
2T)( )(V
rregularize dxfffE Ifirst fundamental form I (ℝ3×3)
S1
V1f
S2
∇f
f (V1)ambient space
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 17
Volume Model
Variant: Thin-Plate-Splines• Use regularizer that penalizes curved deformation
second derivative (ℝ3×3)
S1
V1f
S2
Hf =∇(∇f )
f (V1)ambient space
∫=1
2)( )()(V
frregularize dxxHfE
Eurographics 2010 Course – Geometric Registration for Deformable Shapes
How does the deformation look like?
original
as-rigid-aspossiblevolume
thinplate
splines
Eurographics 2010 Course – Geometric Registration for Deformable Shapes
Intrinsic Matching (2-Manifold)• Target shape is given and complete• Isometric embedding
[ ]∫ −∇∇=1
2T)( )(S
rregularize dxfffE Ifirst fund. form (S1, intrinsic)
19
Isometric Regularizer
19
S1
f
S2
∇f
tangent space
Eurographics 2010 Course – Geometric Registration for Deformable Shapes
“Thin Shell” Energy• Differential geometry point of view
Preserve first fundamental form I Preserve second fundamental form II …in a least least squares sense
• Complicated to implement• Usually approximated
Volumetric shells (as shown before) Other approximation (next slide)
20
Elastic “Thin Shell” Regularizer
20
S1
S2
f
III
III
Eurographics 2010 Course – Geometric Registration for Deformable Shapes
Example Implementation
Example: approximate thin shell model• Keep locally rigid
Will preserve metric & curvature implicitly
• Idea Associate local rigid transformation with surface points Keep as similar as possible Optimize simultaneously with deformed surface
• Transformation is implicitly defined by deformed surface (and vice versa)
21
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 22
Parameterization
Parameterization of S1• Surfel graph • This could be a mesh, but does not need to
edges encodetopology
surfel graph
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 23
Deformation
Ai
Orthonormal Matrix Aiper surfel (neighborhood),latent variable
Ai
predictionframe t frame t+1
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 24
Deformation
Ai
Orthonormal Matrix Aiper surfel (neighborhood),latent variable
Ai
prediction
error
frame t frame t+1
( ) ( )[ ]2)1()1()()()( ∑ ∑ ++ −−−=surfels neighbors
ti
ti
ti
ti
ti
rregularizejj
E ssssA
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 25
Unconstrained Optimization
Orthonormal matrices• Local, 1st order, non-degenerate parametrization:
• Optimize parameters α, β, γ, then recompute A0• Compute initial estimate using [Horn 87 ]
−−−=
00
0)(
γβγαβα
ti×C
)(
)exp()(
0
0
ti
ii
I ×
×
CA
CAA
+⋅=
=
Eurographics 2010 Course – Geometric Registration for Deformable Shapes 26
Variational Model
Variational Problem:• Formulate as an energy minimization problem:
)()()( )()( fEfEfE rregularizematch +=
Deformable ICP
Eurographics 2010 Course – Geometric Registration for Deformable Shapes
Deformable ICP
How to build a deformable ICP algorithm• Pick a surface distance measure• Pick an deformation model / regularizer
2828
)()()( )()( fEfEfE rregularizematch +=
Eurographics 2010 Course – Geometric Registration for Deformable Shapes
Deformable ICP
How to build a deformable ICP algorithm• Pick a surface distance measure• Pick an deformation model / regularizer• Initialize f(S1) with S1 (i.e., f = id)• Pick a non-linear optimization algorithm
Gradient decent (easy, but bad performance) Preconditioned conjugate gradients (better) Newton or Gauss Newton (recommended, but more work) Always use analytical derivatives!
• Run optimization
Eurographics 2010 Course – Geometric Registration for Deformable Shapes
Example
Example• Elastic model• Local rigid coordinate
frames• Align A→B, B→A
30
Geometric Registration for Deformable ShapesVariational Model�What is deformable shape matching?ExampleWhat are we looking for?ExampleThis is a Trade-OffVariational ModelVariational ModelPart 1: Shape MatchingPoint Cloud MatchingSurface approximationSurface approximationSurface approximationVariational ModelPart II: Deformation ModelElastic Volume ModelVolume ModelHow does the deformation look like?Isometric RegularizerElastic “Thin Shell” RegularizerExample ImplementationParameterizationDeformationDeformationUnconstrained OptimizationVariational ModelDeformable ICPDeformable ICPDeformable ICPExample