Direct Fitting of Gaussian Mixture Models Leonid Keselman, Martial Hebert Robotics Institute Carnegie Mellon University May 29, 2019 https://github.com/leonidk/direct_gmm 1
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
-
Direct Fitting of Gaussian Mixture Models
Leonid Keselman, Martial Hebert
Robotics InstituteCarnegie Mellon University
May 29, 2019
https://github.com/leonidk/direct_gmm1
https://github.com/leonidk/direct_gmm
-
Representations of 3D data
2
Point Cloud Point Cloud+ Normals
Triangular Mesh
Nearest Neighbor Plane Fit Screened Poisson Surface Reconstruction
-
Gaussian Mixture Models for 3D Shapes
3
GMM fit to object surface
Benefits• Closed-form expression• Can represent contiguous surfaces• Easy to build from noisy data• Sparse
-
Gaussian Mixture Model (GMM)
4
! " = $%&'
()* +("; .*, Σ*) $*)* = 1)* ≥ 0
Σ* is symmetric, positive-semidefinite
-
Gaussian Mixtures as a shape representation
W. Tabib, C. O’Meadhra, N. MichaelIEEE R-AL (2018)
B. Eckart, K. Kim, A. Troccoli, A. Kelly, J. Kautz. CVPR (2016)
B. Eckart, K. Kim, J. Kautz. ECCV (2018)
5
Efficient Representation Mesh Registration Frame Registration
-
Fitting a Gaussian Mixture Model
1. Obtain 3D Point Cloud2. Select Initial Parameters3. Iterate Expectation & Maximization
i. E-Step: Each point gets a likelihoodii. M-Step: Each mixture gets parameters
6
-
7
-
The E-Step (Given GMM parameters)
8
!"# =1
'" ((*#; ,", Σ")
=01'1 ((*#; ,1, Σ1)
Normalization constant for point j
Affiliation between point j & mixture i
-
The M-Step (Given point-mixture weights)
9
!" = $%&'
($)&'
*+)%log /) 0(2%; 4), Σ))
8!"8/)
= 0
mixturespoints
8!"84)
= 0 8!"8Σ)= 0
lower-bound loss
To get new parameters: takes derivatives, set equal to zero, and solve
4) =1;)$
%
-
10
-
Geometric Objects in a Probability Distribution
11
Known curve in a given 2D probability distribution
-
Geometric Objects in a Probability Distribution
12
Consider sampling N points from this curve
ℓ curve ≅()*+
,-(/))
-
Geometric Objects in a Probability Distribution
13
Take a geometric mean to account for sample number
ℓ curve ≅ ()*+
,-(/))
+,
-
Geometric Objects in a Probability Distribution
14
The curve will be the value in the limit
ℓ curve ≅ ()*+
,-(/))
+,
ℓ curve = lim,→6 ()*+
,-(/))
+,
= lim,→6exp log ()*+
,-(/))
+,
= lim,→6exp1< =)*+
,log(-(/)))
-
Geometric Objects in a Probability Distribution
15
ℓ curve ≅ ()*+
,-(/))
+,
ℓ curve = lim,→6 ()*+
,-(/))
+,
= lim,→6exp log ()*+
,-(/))
+,
= lim,→6exp1< =)*+
,log(-(/))) = exp > log(-(/)) ?/
-
Geometric Objects in a Probability Distribution
16
!= exp & log(+(,)) .,
1. If +(,) = 0 on curve, then L= 02. Invariant to reparameterization
-
17
!" Area of each triangle#" Centroid of each triangle$", &", '" Triangle vertices
-
The E-Step (Given GMM parameters)
18
!"# =1
'" ((*#; ,", Σ")
=01'1 ((*#; ,1, Σ1)
Normalization constant for point j
Affiliation between point j & mixture i
2# Area of each triangle,# Centroid of each triangle3#, 4#, Triangle vertices
-
The New E-Step (Given GMM parameters)
19
!"# =1
'"(# )(+#; +", Σ")
=01'1(1)(+#; +1, Σ1)
Normalization constant for object j
Affiliation between object j & mixture i
(# Area of each triangle+# Centroid of each triangle2#, 3#, Triangle vertices
Taylor Approximation(2 terms)
-
The M-Step (Given point-mixture weights)
20
!" =1%"&
'("')'
*" =%"+
Σ" =1%"&
'("'()'−!")()'−!")0
("' = 1"'
%" =&'("'
-
The New M-Step (Given point-mixture weights)
21
!" =1%"&
'("')'
*" =%"+
Σ" =1%"&
'("' ()'−!")()'−!")0 + Σ'
("' = 2'3"'
%" =&'("'
2' Area of each triangle!' Centroid of each triangle4', 6', 7' Triangle vertices
Σ' =112 4'4'
0 + 6'6'0 + 7'7'0 − 3 !'!'0
-
What is Σ"?
22
-
23
-
ResultsDid all that math actually help us fit better/faster GMMs?
24
-
25
Using different inputs
classic algorithm• Vertices of the mesh• Triangle centroids
our method• Approximate (E only)• Exact (E + M steps)
Measure the likelihood of a high-density point cloud (higher is better)
Evaluate across a wide range of mixtures(6 to 300)
-
Full E+M method works in all cases
26
-
Stable under even random initialization!
27
-
ApplicationsAre these models actually more useful?
28
-
Mesh Registration (P2D)
Method1. Apply a random rotation + translation to the point cloud2. Find transformation to maximize the likelihood of the points
• Perform P2D with GMMs fit toi. mesh verticesii. mesh triangles
29
Eckart, Kim, Kautz. “HGMR: Hierarchical Gaussian Mixtures for Adaptive 3D Registration.” ECCV (2018)
-
Mesh-based GMMs are more accurate
30
-
Across multiple models
31
0
50
100
150
Armadillo Bunny Dragon Happy Lucy
Rotation Error(% of ICP)
points mesh
0
100
200
Armadillo Bunny Dragon Happy Lucy
Translation Error(% of ICP)
points mesh
-
Frame Registration (D2D)
Method1. Use a sequence from an RGBD Sensor
• 2,500 frame TUM sequence from a Microsoft Kinect2. Pairwise registration between t & t-1 frames
• Optimize the D2D L2 distance• Build GMMs using square pixels as the geometric object
32
W. Tabib, C. O’Meadhra, N. Michael.“On-Manifold GMM Registration” IEEE R-AL (2018)
-
Representing points using pixel squares
33
-
D2D Registration Results
34
Compared to standard GMM• 2.4% improvement in RMSE• 22% faster D2D convergence
-
Questions?
35
! = exp & log(+(,)) .,
-
The End!
36
-
Extra Slides
37
-
How to fit a Gaussian Mixture Model?
1. Obtain any collection of objects2. Perform Expectation + Maximization
i. E-Step: Each point gets a likelihoodii. M-Step: Each mixture gets new parameters
38
-
Extension to arbitrary primitives
39Vasconcelos, Lippman. "Learning mixture hierarchies.” Advances in Neural Information Processing Systems (1999)
-
Approximation
40
area-weighted geometric mean using the primitive’s centroids
-
Product Integral Formulation
• Product integrals provide a resampling-invariant loss function• Given S samples, of M primitives, with N mixture components
• This can be evaluated in the limit of samples (with a geometric mean)
41
-
42
-
43
-
44
-
For GMMs we will use the lower bound
45
-
P2D Registration Results
46
Model Rotation Error(% of ICP)
Translation Error(% of ICP)
points mesh points meshArmadillo 127 37 161 33Bunny 50 28 41 17Dragon 68 25 40 19Happy 101 27 85 27Lucy 95 23 122 35
-
Mesh Registration with P2D
Method1. Apply a random rotation + translation to the point cloud2. Point-to-Distribution (P2D) registration of point cloud to GMM
• Perform tests with GMMs fit toi. mesh verticesii. mesh triangles
• Optimize the GMM likelihood with rigid body transformation (q & t)• BFGS Optimization using numerical gradients, starting from identity
47
Eckart, Kim, Kautz. “HGMR: Hierarchical Gaussian Mixtures for Adaptive 3D Registration.” ECCV (2018)
-
Acknowledgements
• Martial Hebert• Robotics Institute• Reviewers• Funding?
48
-
Representing points using pixel squares
49
Related Documents
