Range Maps Alignment - CNRvcg.isti.cnr.it/~ganovell/bertinoro/L3_RM_alignment.pdf · Range Maps Alignment. 2 The problem Many range maps for a single object Each one in its own 3D

1

Range Maps Alignment

2

The problem

Many range maps for a single object Each one in its own 3D reference frame Need to put them in a common frame

Example: Scanning the Minerva

Last scan: Minolta laser scanner (03/2002)

No. range scans: 297

Sampling resolution: ~0.3 mm

3

Rotations

2D rotation of a point p around the origin: movement of p such that its distance from the origin is constant

5

Rotations

3D rotation of a point p around a principal axis

6

Rotation group in 3D: SO(3)

Set: of all the rotations about the origin = all orthonormal matrices (det = 1)

Operation: composition Properties:

7

Matrices frames↔

every 3x3 orthonormal matrix is a orthogonal frame whose axes are the matrix's columns

every orthonormal matrix is a rotation → every frame is a rotation

8

Rotations about an arbitrary axis

Can be obtained by composing:– The rotation between the red frame and the identity– The rotation about the Z axis➔ The rotation between the identity and the red frame

9

Other representations...

Rotations can be represented in other ways:

– Euler angles– Axis-angle (without matrices)– Quaternions– ...

10

Rototranslations

A rototranslation is a rotation followed by a translation

In (handy) matrix form:

11

Manual alignment

Specifies few correspondences between two range maps and solve for the rototranslation:

12

Correspondence problem 1

Given two set of n points, find the roto-translation the maps the first on the second

– At least 3 points necessary (d in d-dimensions)– In the practical case the more the better– The “more accurate” the better

13


Given two set of n points, find the roto-translation the maps the first on the second

– At least 3 points necessary (d in d-dimensions)– In the practical case the more the better– The “more accurate” the better

14


Solution:1. solve the least square problem

2. find the rotation part of R:

15


Better solutions exists...➔ With quaternions [BeslMcKay92]

– With matrices [Horn88, Faugeras86]

..a lot of linear algebra calculations we are not interested in right now..

16

Finding Correspondences

Finding correspondences manually is ok for 2 range maps, boring for 3, unbearable for 400...

Quest for automatic solutions: finding “features” points on two (or more) surfaces and their correspondence

17

Alignment pipeline

Picking point of the surface that are notable w.r.t. to some criterion

Not just 4 points but a lot of them

Understand the correspondence between notable points

A lot of correspondence. Some correct, some uncorrect: need to tell them apart

All the range maps are aligned on the basis of the correspondences.

Finalize the alignment considering the whole surfaces

18

Alignment pipeline




A lot of correspondence. Some correct, some uncorrect: need to tell them apart



19

Feature Desc. Example [Pingi05]

Yellow: low varianceBlue: medium var.Red: high variance

A set of points are chosen in the region of medium variance

The distance between two points on two range maps is computed as:

20

Feature Descriptors

21

Feature Descriptors

22

Quality of a feature descriptor

A good descriptor finds notable pointsIt is resilient to:

– Rotation (i.e. point of view)– Noise– Scale

There is a good metric to compare two feature points

– Avoid false positives– Avoid false negatives

23

Feature desc.:Curvature Maps [Gatzke05]

Descriptor of a vertex v as a function of the curvature in the neighborhood of v

(approximated) geodesic distance from v

v Plot of a function of the distance from v

24

Feature desc.:Curvature Maps

1D descriptor: curvature values along the rings

26

Feature desc.:Curvature Maps

2D descriptor: curvature values inside the fans. Similar to 1D but:

– the spokes are sampled

– F is a function of the spoke index...

...the function depends on the starting spoke

– Try all the rotations when comparing and pick the best

– It gives a hint on the rotation between meshes

27

Curvature Maps

28

Curvature Maps

Most unique vertices are found by sorting by the sum of similarity of each vertex with all the others.

29

Feature desc.:Spin Images [Johnson97]

Estimate the normal per-vertex

Place a rectangle along the normal Accumulate projection of all the vertices on the

image while it rotates

30

Feature desc.:Spin Images

Metric: image comparison pixel-by-pixel Descriptor depends on:

– size of the image

– Resolution of the image

– Resolution of the mesh

– Angle step of the rotation

Many false positives & false negatives: it suffers from symmetries!

31

Multi-scale features [Li05]

➔ Range maps normal, curvature maps, spin images all have a crucial parameter: the scale

➔ Range maps-> size of kernel in pixels

➔ Curvature maps-> size of the ring

➔ Spin images-> radius of the ring

➔ In an uncontrolled setting it is not obvious that they work (i.e. probably they don't)

➔ Multi-scale approaches consider multiple scales at the same time....

32


➔ Smoothing a mesh: move its points so that the surface is smoother (i.e. lower curvatures)

➔ Larger-scale features “survive” to smoothing

The nose is still a feature

The hair disapperared..

34


➔ Let➔ Build a sequence of smoothed version

(called levels) j=1..n for h=F^j (i.e. increasing)

➔ Define the distance between levels as:

b(q , R)={i :∥pi−q∥<R }

d j (i):=nij ( pi

j− pij−1)

35


➔ Def: A point i on level j is a neighborhood maximum if:

➔ Def: neighborhood minimum. The same with “<” instead of “>”

d j (i)>d j−1(i ' )∀ i '∈b( pi , h j−1)d j (i)>d j (i ' )∀ i '∈b( pi , h j)d j (i)>d j+1(i ' )∀ i '∈b( pi , h j+1)

36


➔ Mapping of

Level 0

Level 8

d j(i)

37


➔ Circled points are extrema (i.e. neighborhood max or min)Level 0

Level 8

38

Alignment pipeline




A lot of correspondences. Some correct, some uncorrect: need to tell them apart



39

Random Sample Consensus [Fischler81]

Problem data– Input: a set of N correspondences– 4 enough to provide a solution – Output: the solution that satisfies as

many as possibleAlgorithm:

– For a predefined number of trials T:

● Pick 4 correspondences● Find the candidate rototranslation Rt● S* = set of satisfied correspondences

– Return the Rt corresponding to the largest set

40

RANSAC: parameters

How many trials for success?

Rule of thumb: E[k]+2*SD[k]

k=number of trials

w= probof picking a correct correspondenceb=wn probof picking acorrect set

E [k ]=average number of trial=∑i=0

∞

i P (i)

E [k ]=b+2(1−b)b+3(1−b)2b+...+i(1−b)i−1b+.......

E [k ]=1/b

41

RANSAC: parameters

How many trials for succeeding with given prob?

How do we know about b ?

ps= probability of successs(1− ps)= probability of failure

(1−b)k<(1− p s)

k>log(1− ps)

log(1−b)

42

4 Point Congruent Sets [Aiger08]

Forget notable points, find an efficient way of exploring the search space of all the transformations

– Find a way to recognize similar points quicklyAlgorithm:

– Iteration:● Pick a base: four points in the first mesh (RM)● Find four points in the second that are affine (i.e.

there is a rototranslation..)– Use Ransac to find the best match

43

4 Point Congruent Sets

How many possible corresponding quadruples?

– O(n^4), n number of verticesPrune the search space...

P Q

44


Choosing the base on mesh P:– 4 coplanar points (a,b,c,d)– Characterize the quadruple as the barycentric coordinates

of the intersection point of the segments (a,b) and (c,d)

45


Finding the corresponding quadruple on Q:– Take every pairs of points at distance ||a-b|| or ||c-d|| [ O (n) ]– For each pair (=segment) generate the intermediate points at barycentric

coordinates r1,r2,1-r1,1-r2➔ Put these 3D points in a search data structure [ O (n^2 log n) ]– Every pair of coincident points corresponds to a possibly affine quadruple

46


Robust to noiseDoes not use feature points

Amount of overlap between P and Q:– If known, the basis are chosen as wide

as the overlap– If not known, the algorithm is launched

for 1, 0.9,0.8 ...

47


If overlap is known:

α= fraction of overlapb=α

4

N=number of points

k>log (1− ps)

log (1−α4)

48

Alignment pipeline




A lot of correspondences. Some correct, some uncorrect: need to tell them apart



49

Fine Alignment

INPUTrough alignment

OutputPrecise alignment withinterror tolerance

53

ICP: Iterative Closest Point [BeslMcKay92]

Q

P

54

ICP: convergence proof

55

Improvements over ICP

Trying to avoid matching wrong correspondences. Discard matches if:

– Their normals are more than than 45° apart [Godin94]

– If the closest point is on the boundary [Turk94]

Converge faster [Pulli99]

– Keep only the p% closest points (p increasing along the iterations)

– Discard pairs more distant than a fixed threshold (decreasing along the iterations)

56

Improvements over ICP

Point-to-plane minimization– Minimize the distance from the point

and the plane tangent to the closest point

– Faster convergence

57

Global Registration

A set of range map can be registered pairwise, but pairwise alignment leads to accumulation of errors when walking across the surface of an object

An optimal solution should minimize distances between all range scans simultaneously

– i.e. to solve the global registration

58

Global Registration

Formalization: a connected graph– Nodes are RM– Arcs are pairwise alignment– Arcs are double labeled:

● Alignment value (rototranslation)● alignment error

– Goal: minimize the sum of arcs' error

59

Global Registration

Many different global registration approaches

Global optimization– Simulated annealing [Blais95]– Springs [Stoddart, Hilton 96]– Hierarchical linearized least squares [Neugebauer

97]Incremental

– Iterate moving one scan, keep others fixed [Bergevin et al 96]

– Speed up using hardware [Benjemaa, Schmitt 97]– Preserve pairwise matches [Pulli 99]

60

Global Registration [Pulli99]

Idea: incrementally add the most connected RM and distribute the error

61

Global Registration [Pulli99]

The virtual pair– Principle: trust the pairwise matching

The error is only the one due to the change from the pairwise matching

62

[BeslMcKay92] Besl, Paul J.; N.D. McKay (1992). "A Method for Registration of 3-D Shapes".

IEEE Trans. on Pattern Analysis and Machine Intelligence (Los Alamitos, CA, USA: IEEE Computer Society) 14 (2): 239-256

[Blais95]

Blais, G. and M.D. Levine, Registering Multiview Range Data to Create 3D Computer Objects. IEEE

Transactions on PAMI, 1995. 17(8): p. 820-4.

[Stoddart96] Stoddart, A.J., Hilton, A., 1996. Registration of multiple point sets. In: Proc. 13th Int. Conf. on Pattern Recognition

[Neugebauer 97] Reconstruction of real-world objects via simultaneous registration and robust

combination of multiple range images. International Journal of Shape Modeling,

3(1,2):71-90, 1997

[Bergevin96]

Bergevin, R., Soucy, M., Gagnon, H., and Laurendeau,

D. "Towards a General Multi-View Registration Technique," Trans.

PAMI, Vol. 18, No. 5, 1996

63

[Benjemaa97]

Benjemaa, R. and Schmitt, F. "Fast Global Registration of 3D Sampled Surfaces Using a Multi-Z-Buffer Technique," Proc.

3DIM, 1997.

[Pulli 99]

Pulli, K. "Multiview Registration for Large Data Sets," Proc. 3DIM, 1999.

[Horn88]

Horn, B., Hilden, H., and Negahdaripour, S. "Closed-Form Solution of Absolute Orientation Using Orthonormal Matrices," JOSA

A, Vol. 5, No. 7, 1988.

[Faugeras86] Faugeras, O. and Hebert, M. "The Representation, Recognition, and Locating of 3-D Objects,"

Int. J. Robotic Res., Vol. 5, No. 3, 1986

[Pingi05]

Exploiting the scanning sequence for automatic registration of large sets of range maps

Paolo Pingi, Andrea Fasano, Paolo Cignoni, Claudio Montani, Roberto Scopigno

Computer Graphics Forum, Volume 24, Number 3, page 517-526 - 2005

64

[Gatzke05]

Curvature Maps for Local Shape Comparison, Timothy Gatzke , Cindy Grimm , Michael Garland , Steve Zelinka

In Shape Modeling International, SMI 05

[Johnson97]

Johnson, A. and Hebert, M. "Surface Registration by Matching Oriented Points," Proc. 3DIM, 1997.

[Li05]

Xinju Li, Igor Guskov: Multiscale Features for Approximate Alignment of Point-based Surfaces. Symposium on Geometry Processing 2005: 217-226

[Fischler81]

Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography

Martin A. Fischler,Robert C. Bolles Communications of the ACM Volume 24 Issue 6, June 1981 Pages 381 - 395

[Turk94]

Turk, G. and Levoy, M. "Zippered Polygon Meshes from

Range Images," Proc. SIGGRAPH, 1994.

[GOdin94]

Godin, G., Rioux, M., and Baribeau, R. "Three-dimensional

Registration Using Range and Intensity Information," Proc. SPIE:

Videometrics III, Vol. 2350, 1994.

Range Maps Alignment - CNRvcg.isti.cnr.it/~ganovell/bertinoro/L3_RM_alignment.pdf · Range Maps Alignment. 2 The problem Many range maps for a single object Each one in its own 3D

Documents