EFFICIENT VARIANTS EFFICIENT VARIANTS OF THE ICP ALGORITHM OF THE ICP ALGORITHM Szymon Rusinkiewicz Szymon Rusinkiewicz Marc Levoy Marc Levoy
Jan 14, 2016
EFFICIENT VARIANTS OF EFFICIENT VARIANTS OF THE ICP ALGORITHMTHE ICP ALGORITHM
Szymon RusinkiewiczSzymon Rusinkiewicz
Marc LevoyMarc Levoy
IntroductionIntroduction
Problem of aligning Problem of aligning 3D models, based on 3D models, based on geometry or color of geometry or color of meshesmeshes
ICP is the chief ICP is the chief algorithm usedalgorithm used
Used to register Used to register output of 3D scannersoutput of 3D scanners
[1]
ICPICP
Starting point: Two meshes and an initial Starting point: Two meshes and an initial guess for a relative rigid-body transformguess for a relative rigid-body transform
Iteratively refines the transformIteratively refines the transform
Generates pairs of corresponding points Generates pairs of corresponding points on the meshon the mesh
Minimizes an error metricMinimizes an error metric
RepeatsRepeats
Initial alignmentInitial alignment
Tracking scanner position…Tracking scanner position…
Indexing surface features…Indexing surface features…
Spin image signatures…Spin image signatures…
Exhaustive search…Exhaustive search…
User Input……User Input……
[2]
ConstraintsConstraints
Assume a rough initial alignment is Assume a rough initial alignment is availableavailable
Focus only on a single of meshesFocus only on a single of meshes
Global registration problem not addressedGlobal registration problem not addressed
Stages of the ICPStages of the ICP
SelectionSelection of the set of points of the set of points
MatchingMatching the points to the samples the points to the samples
WeightingWeighting corresponding pairs corresponding pairs
RejectingRejecting pairs to eliminate outliers pairs to eliminate outliers
AssigningAssigning an error metric an error metric
MinimizingMinimizing the error metric the error metric
FocusFocus
SpeedSpeedAccuracyAccuracyPerformance in tough scenesPerformance in tough scenesIntroducing test scenesIntroducing test scenesDiscuss combinationsDiscuss combinationsNormal-space directed samplingNormal-space directed samplingConvergence performanceConvergence performanceOptimal combinationOptimal combination
Comparison Methodology Comparison Methodology
Baseline Algorithm: [Pulli 99]Baseline Algorithm: [Pulli 99]1.1. Random sampling on both meshesRandom sampling on both meshes2.2. Matching to a point where the Matching to a point where the normal is < 45 degrees from the sourcenormal is < 45 degrees from the source3.3. Uniform weightingUniform weighting4.4. Rejection of edge vertices pairsRejection of edge vertices pairs5.5. Point-to-plane error metricPoint-to-plane error metric6.6. “ “Select-match-minimize” iterationSelect-match-minimize” iteration
AssumptionsAssumptions
2000 source points and100,000 samples2000 source points and100,000 samples
Simple perspective range images Simple perspective range images
Surface normal is based on the four Surface normal is based on the four nearest neighborsnearest neighbors
Only geometry (color, intensity excluded)Only geometry (color, intensity excluded)
Test ScenesTest Scenes
a) Wave Scene
b) Fractal Landscape
c) Incised Plane
Shamelessly stolen from [3]
Sample scanning application
Representative of different kinds of surfaces
• Low frequency
• All frequency
• High Frequency
Smooth statues
Unfinished statues
Fragments
More shameless lifts from [3]
Comparison StagesComparison Stages
SelectionSelection of the set of points of the set of points
MatchingMatching the points to the samples the points to the samples
WeightingWeighting corresponding pairs corresponding pairs
RejectingRejecting pairs to eliminate outliers pairs to eliminate outliers
AssigningAssigning an error metric an error metric
MinimizingMinimizing the error metric the error metric
Selection of point pairsSelection of point pairs
Use all available pointsUse all available points
Uniform sub-samplingUniform sub-sampling
Random samplingRandom sampling
Pick points with high intensity gradientPick points with high intensity gradient
Pick from one or both meshesPick from one or both meshes
Select points where the distribution of the Select points where the distribution of the normal between these points is as large as normal between these points is as large as possiblepossible
Normal SamplingNormal Sampling
Small features may play a critical roleSmall features may play a critical role
Distribute the spread of the points across Distribute the spread of the points across the position of the normalsthe position of the normals
SimpleSimple Low-costLow-costLow robustnessLow robustness
Comparison of performanceComparison of performance
• Uniform sub-sampling
• Random sampling
• normal-space sampling
Comparison of performanceComparison of performance
Incised Plane: Only the normal-space sampling converges
Why?Why?Samples outside Samples outside the grooves: 1 the grooves: 1 translation, 2 translation, 2 rotationsrotationsInside the grooves: Inside the grooves: 2 translations, 1 2 translations, 1 rotationrotationFewer samples + Fewer samples + noise + distortionnoise + distortion= bad results= bad results
Sampling DirectionSampling DirectionPoints from one mesh vs. points from both Points from one mesh vs. points from both meshesmeshesDifference is minimal, as algorithm is Difference is minimal, as algorithm is symmetricsymmetric
Sampling directionSampling direction
Asymmetric algorithmAsymmetric algorithm
Two meshes is betterTwo meshes is better
If overlap is small, two If overlap is small, two meshes is bettermeshes is better
Comparison StagesComparison Stages
SelectionSelection of the set of points of the set of points
MatchingMatching the points to the samples the points to the samples
WeightingWeighting corresponding pairs corresponding pairs
RejectingRejecting pairs to eliminate outliers pairs to eliminate outliers
AssigningAssigning an error metric an error metric
MinimizingMinimizing the error metric the error metric
Matching PointsMatching Points
Match a sample point with the closest in Match a sample point with the closest in the other meshthe other mesh
Normal shootingNormal shooting
Reverse calibrationReverse calibration
Project source point onto destination Project source point onto destination mesh; search in destination range imagemesh; search in destination range image
Match points compatible with source Match points compatible with source points points
Variants comparedVariants compared
Closest pointClosest point Closest compatible pointClosest compatible point Normal shootingNormal shooting Normal shooting to a Normal shooting to a compatible pointcompatible point ProjectionProjection Projection followed by a search : uses Projection followed by a search : uses steepest-descent neighbor-neighbor walksteepest-descent neighbor-neighbor walk
k-d tree
Fractal SceneFractal Scene
Best: normal shooting
Worst: closest-point
Incised PlaneIncised Plane
Closest point converges: most robust
ErrorError
Error as a function Error as a function of running timeof running time
Applications that Applications that need quick running need quick running of the ICP should of the ICP should choose algorithms choose algorithms with the fastest with the fastest performanceperformance
Best: Projection algorithm
Comparison StagesComparison Stages
SelectionSelection of the set of points of the set of points
MatchingMatching the points to the samples the points to the samples
WeightingWeighting corresponding pairs corresponding pairs
RejectingRejecting pairs to eliminate outliers pairs to eliminate outliers
AssigningAssigning an error metric an error metric
MinimizingMinimizing the error metric the error metric
AlgorithmsAlgorithms
Constant weightConstant weight
Lower weights for points with higher point-Lower weights for points with higher point-point distancespoint distances
Weight = 1 – [Dist(pWeight = 1 – [Dist(p11, p, p22)/Dist )/Dist maxmax]]
Weight based on normal compatibilityWeight based on normal compatibility
Weight = nWeight = n11* n* n22
Weight based on the effect of noise on Weight based on the effect of noise on uncertaintyuncertainty
Wave SceneWave Scene
Incised PlaneIncised Plane
Comparison StagesComparison Stages
SelectionSelection of the set of points of the set of points
MatchingMatching the points to the samples the points to the samples
WeightingWeighting corresponding pairs corresponding pairs
RejectingRejecting pairs to eliminate outliers pairs to eliminate outliers
AssigningAssigning an error metric an error metric
MinimizingMinimizing the error metric the error metric
Rejecting PairsRejecting Pairs
Pairs of points more than a given distance Pairs of points more than a given distance apartapart
Worst Worst n%n% pairs, based on a metric (n=10) pairs, based on a metric (n=10)
Pairs whose point-point distance is > Pairs whose point-point distance is > multiple multiple mm of the standard deviation of of the standard deviation of distances (m = 2.5)distances (m = 2.5)
Rejecting PairsRejecting Pairs
Pairs that are not consistent with neighboring Pairs that are not consistent with neighboring pairspairs
Two pairs are inconsistent iffTwo pairs are inconsistent iff
| Dist(p| Dist(p11,p,p22) – Dist(q) – Dist(q11,q,q22) |) |
Threshold: Threshold:
0.1 * max(Dist(p0.1 * max(Dist(p11,p,p22) – Dist(q) – Dist(q11,q,q22) )) )
Pairs containing points on mesh boundariesPairs containing points on mesh boundaries
Points on mesh boundariesPoints on mesh boundaries
• Incomplete overlap:
Low cost
Fewer disadvantages
Rejection on the wave sceneRejection on the wave scene
•Rejection of outliers does not help with initial convergence
•Does not improve convergence speed
Comparison StagesComparison Stages
SelectionSelection of the set of points of the set of points
MatchingMatching the points to the samples the points to the samples
WeightingWeighting corresponding pairs corresponding pairs
RejectingRejecting pairs to eliminate outliers pairs to eliminate outliers
AssigningAssigning an error metric an error metric
MinimizingMinimizing the error metric the error metric
Error metricsError metrics
Sum of squared distances between Sum of squared distances between corresponding pointscorresponding points
1) SVD1) SVD
2) Quaternions2) Quaternions
3) Orthonormal Matrices3) Orthonormal Matrices
4) Dual Quaternions4) Dual Quaternions
Error metricsError metrics
Point-to-point metric, taking into account Point-to-point metric, taking into account distance and color differencedistance and color difference
Point-to-plane methodPoint-to-plane method
The least-squares equations can be The least-squares equations can be solved either by using a non-linear method solved either by using a non-linear method or by linearizing the problemor by linearizing the problem
Search for the alignmentSearch for the alignment
Generate a set of points Generate a set of points Find a new transformation that minimizes the Find a new transformation that minimizes the error metricerror metricCombine with extrapolationCombine with extrapolationIterative minimization, with perturbations initially, Iterative minimization, with perturbations initially, then selecting the best resultthen selecting the best resultUse random subsets of points, select the optimal Use random subsets of points, select the optimal using a robust metricusing a robust metricUse simulated annealing and perform a Use simulated annealing and perform a stochastic search for the best transformstochastic search for the best transform
Extrapolation algorithmExtrapolation algorithm
Besl and McKay’s algorithmBesl and McKay’s algorithm
For a downward parabola, the largest x-For a downward parabola, the largest x-intercept is usedintercept is used
The extrapolation is multiplied by a The extrapolation is multiplied by a dampening factordampening factor
Increases stabilityIncreases stability
Reduces overshootReduces overshoot
Fractal SceneFractal Scene
Best: Point-to-plane error metric
Incised PlaneIncised Plane
Point-to-point cannot reach the right solution
High-Speed VariantsHigh-Speed Variants
Applications of ICP in real time:Applications of ICP in real time:
1) Involving a user in a scanning process 1) Involving a user in a scanning process for alignmentfor alignment
““Next-best-view” problemNext-best-view” problem
““Given a set of range images, to determine Given a set of range images, to determine the position/orientation of the range scanner to the position/orientation of the range scanner to scan all visible surfaces of an unknown scenescan all visible surfaces of an unknown scene” ” [4][4]
2) Model-based tracking of a rigid object2) Model-based tracking of a rigid object
Optimal AlgorithmOptimal Algorithm
Projection-based algorithm to generate Projection-based algorithm to generate point correspondencespoint correspondencesPoint-to-plane error metricPoint-to-plane error metric““Select-match-minimize” ICP iterationSelect-match-minimize” ICP iterationRandom samplingRandom samplingConstant weightingConstant weightingDistance threshold for pair rejectionDistance threshold for pair rejectionNo extrapolation of transforms (Overshoot)No extrapolation of transforms (Overshoot)
Optimal ImplementationOptimal Implementation
Former implementation using point-to-point metric
Point-to-plane is much faster
ConclusionConclusion
Compared ICP variantsCompared ICP variants
Introduced a new sampling methodIntroduced a new sampling method
Optimized ICP algorithmOptimized ICP algorithm
Future WorkFuture Work
Focus on stability and robustnessFocus on stability and robustness
Effects of noise and distortionEffects of noise and distortion
Algorithms that switch between variants Algorithms that switch between variants would increase robustnesswould increase robustness
ReferencesReferences
[1] [1] http://foto.hut.fi/opetus/ 260/luennot/9/9.htmlhttp://foto.hut.fi/opetus/ 260/luennot/9/9.html[2] [2] http://www.sztaki.hu/news/2001_07/maszk_allthrhttp://www.sztaki.hu/news/2001_07/maszk_allthree.jpgee.jpg[3][3]
http://graphics.stanford.edu/projects/mich/http://graphics.stanford.edu/projects/mich/[4] [4] http://www.cs.unc.edu/~sud/courses/comp258/fihttp://www.cs.unc.edu/~sud/courses/comp258/final_pres.ppt#257,2,Problem Statementnal_pres.ppt#257,2,Problem Statement