Surface Reconstruction: Part II - Computer graphicsgraphics.stanford.edu/.../LectureSlides/04_Surface_Reconstruction.pdf · Surface Reconstruction: Part II. Surface ReconstructionSurface

SurfaceSurfaceSurface Surface Reconstruction:Reconstruction:

Part IIPart II

Surface ReconstructionSurface ReconstructionSurface ReconstructionSurface Reconstruction

Today:

Scanning

Today:

Scanning devices

RegistrationImplicit

reconstruction

physicalmodel

acquiredpoint cloud

reconstructedmodelp

2

Input DataInput Datapp

• Set of range scans– Each scan is a regular quad- or tri-

meshNormal vectorsNormal vectors are well defined–– Normal vectors Normal vectors are well defined

– Scans registered in common coordinate system

• Set of irregular sample pointsg p p– Typically without normal vectorswithout normal vectors– More general

3

Shape RepresentationsShape RepresentationsShape RepresentationsShape Representations

• Explicit representationp p– Image of parametrization

• Implicit representation– Zero set of distance function

4

Implicit RepresentationsImplicit RepresentationsImplicit RepresentationsImplicit Representations

• Level set of 2D function defines 1D curveLevel set of 2D function defines 1D curve

5

Explicit / ImplicitExplicit / ImplicitExplicit / ImplicitExplicit / Implicit• Explicit representation

– Image of parameterization– Easy to find points on shape

C d f bl– Can defer problems to paramdomain

• Implicit representation– Zero set of distance function

Easy in/out/distance test– Easy in/out/distance test–– Easy to handle different topologiesEasy to handle different topologies

6

Implicit RepresentationsImplicit RepresentationsImplicit RepresentationsImplicit Representations

• Easy to handle different topologiesEasy to handle different topologies

7

8

Implicit RepresentationsImplicit RepresentationsImplicit RepresentationsImplicit Representations

• General implicit pfunction:– Interior: F(x,y,z) < 0( ,y, )– Exterior:F(x,y,z) > 0– Surface: F(x,y,z) = 0Surface: F(x,y,z) 0

• Special case– Signed distance function

(SDF)

9

Explicit Reconstruction MethodsExplicit Reconstruction Methodspp

• Connect sample points by triangles

• Exact interpolation of sample pointsp p p

• Bad for noisy or misaligned data

• Can lead to holes or non manifold situations• Can lead to holes or non-manifold situations

10

Implicit Reconstruction MethodsImplicit Reconstruction Methodspp

• Estimate signed distance function (SDF)

• Extract zero iso-surface

• Approximation of input points

• Watertight manifold results by construction• Watertight manifold results by construction– Can have spurious components

11

Input Implicit Explicit

12

Implicit Function Approach

13

Implicit Function Approach

• Define a function 3:f R R→

with value < 0 outside the shape and > 0

:f R R→

the shape and > 0 inside

< 0 > 0014

Implicit Function Approach

• Define a function 3:f R R→

with value < 0 outside the shape and > 0 inside

:f R R→

the shape and > 0 inside

• Extract the zero set• Extract the zero-set

{ : ( ) 0}x f x =< 0 > 00

{ : ( ) 0}x f x =

15

Signed Distance FunctionSigned Distance FunctionSigned Distance FunctionSigned Distance Function

• Construct SDF from point samplesp p– Distance to points is not enough

– Need inside/outside information/

– Reconstruct normal vectors first

+ -+

16

Normal EstimationNormal EstimationNormal EstimationNormal Estimation

• To find normal ni for each sample point pip1. Examine local neighborhood for each point

• Set of k nearest neighborsSet of k nearest neighbors

2. Compute best approximating tangent plane• Covariance analysis• Covariance analysis

3. Determine normal orientation• MST propagation pi• MST propagation pi

17

Normal EstimationNormal EstimationNormal EstimationNormal Estimation

• To find normal ni for each sample point pip1. Examine local neighborhood for each point

• Set of k nearest neighborsSet of k nearest neighbors

2. Compute best approximating tangent plane• Covariance analysis• Covariance analysis

3. Determine normal orientation• MST propagation• MST propagation

18

Local NeighborhoodLocal NeighborhoodLocal NeighborhoodLocal Neighborhood

• Find k nearest neighbors (kNN) of a pointFind k nearest neighbors (kNN) of a point– Brute force: O(n) complexity

• Use BSP tree– Binary space partitioning treey p p g

– Recursively partition 3D space by planes

– Tree should be balanced, put plane at medianTree should be balanced, put plane at median

– log(n) tree levels, complexity O(log n)

19

BSP Closest PointsBSP Closest PointsBSP Closest PointsBSP Closest Points

PA A

DF B C

PA

PBED

PC

F G

E

PC

G

20

BSP Closest PointsBSP Closest PointsBSP Closest PointsBSP Closest Points

APA

DF B C

PA

ED F GPB

P

E

PC

G

21

BSP Closest PointsBSP Closest PointsBSP Closest PointsBSP Closest Points

APA

B CD

F

PA

ED F GPB

P

E

PC

G

22

BSP Closest PointsBSP Closest PointsBSP Closest PointsBSP Closest Points

APA

B CD

F

PA

ED F GPB

P

E

PC

G

23

BSP Closest PointsBSP Closest PointsBSP Closest PointsBSP Closest Points

APA

B CD

F

PA

ED F GPB

P

E

PC

G

24

BSP Closest PointsBSP Closest PointsBSP Closest PointsBSP Closest Points

Node::dist(Point x, Scalar& dmin)( ){if (leaf_node())

for each sample point p[i]dmin = min(dmin, dist(x, p[i]));

else{

d = dist_to_plane(x);if (d < 0) {{

left_child->dist(x, dmin);if (|d| < dmin) right_child->dist(x, dmin);

} else {{

right_child->dist(x, dmin);if (|d| < dmin) left_child->dist(x, dmin);

}}

}}

25

More TreesMore TreesMore TreesMore Trees

Quad-tree (oct-tree)Cells are squares (cubes)

Kd-tree Cells are axis-aligned boxes

26

Normal EstimationNormal EstimationNormal EstimationNormal Estimation

• Find normal ni for each sample point pip1. Examine local neighborhood for each point

• Set of k nearest neighborsSet of k nearest neighbors

2. Compute best approximating tangent plane• Covariance analysis• Covariance analysis

3. Determine normal orientation• MST propagation• MST propagation

27

Principal Component AnalysisPrincipal Component AnalysisPrincipal Component AnalysisPrincipal Component Analysis

• Fit a plane with center c and normal n to a set of points {p1, ..., pm}

Mi i i l t• Minimize least squares error

• Subject to non-linear constraintn

Subject to non linear constraintc pi

28

Principal Component AnalysisPrincipal Component AnalysisPrincipal Component AnalysisPrincipal Component Analysis

• Plane center is barycenter of pointsy p

• Normal is eigenvector w.r.t. smallest geigenvalue of point covariance matrix

29

Normal EstimationNormal EstimationNormal EstimationNormal Estimation

• Find normal ni for each sample point pip1. Examine local neighborhood for each point

• Set of k nearest neighborsSet of k nearest neighbors

2. Compute best approximating tangent plane• Covariance analysis• Covariance analysis

3. Determine normal orientation• MST propagation• MST propagation

30

Naïve Orientation PropagationNaïve Orientation PropagationNaïve Orientation PropagationNaïve Orientation Propagation

31

Normal OrientationNormal OrientationNormal OrientationNormal Orientation

• Build graph connecting neighboring pointsu d g ap co ec g e g bo g po s– Edge (ij) exists if pi ∈ kNN(pj) or pj∈ kNN(pi)

• Propagate normal orientation through graph• Propagate normal orientation through graph– For neighbors pi,pj: Flip nj if niTnj < 0– Fails at sharp edges/corners

• Propagate along “safe” paths (parallel normals)Minimum spanning tree with angle based edge weights– Minimum spanning tree with angle-based edge weights wij = 1- | niTnj |

32

SDF from Point & NormalsSDF from Point & NormalsSDF from Point & NormalsSDF from Point & Normals

• Signed distance from tangent planes– Points + normals determine local tangent planes

– Use distance from closest center’s tangent planeg p

– Linear approximation in Voronoi cell

– Simple and efficient, but SDF is only C-1Simple and efficient, but SDF is only Cx( )F x

33

Smooth SDF ApproximationSmooth SDF ApproximationSmooth SDF ApproximationSmooth SDF Approximation

• Scattered data interpolation problemScattered data interpolation problem– On-surface constraints dist(pi) = 0

Avoid trivial solution dist ≡ 0– Avoid trivial solution dist ≡ 0– Off-surface constraints dist(pi + εni) = ε

11

1

0

0

0 01

10

34

Smooth SDF ApproximationSmooth SDF ApproximationSmooth SDF ApproximationSmooth SDF Approximation

• Scattered data interpolation problemScattered data interpolation problem– On-surface constraints dist(pi) = 0

Avoid trivial solution dist ≡ 0– Avoid trivial solution dist ≡ 0– Off-surface constraints dist(pi + εni) = ε

• Radial basis functions– Well suited for smooth interpolationWell suited for smooth interpolation

– Sum of shifted, weighted kernel functions

35

Radial Basis FunctionsRadial Basis Functions InterpolationInterpolationRadial Basis FunctionsRadial Basis Functions InterpolationInterpolation

How do we find the weights?

36

Radial Basis FunctionsRadial Basis FunctionsRadial Basis FunctionsRadial Basis Functions

2n equations2n equations

2n variables

The on- and off-surface points are the centers ci

37

Radial Basis FunctionsRadial Basis FunctionsRadial Basis FunctionsRadial Basis Functions

2n equations2n equations

2n variables

The on- and off-surface points are the centers ci

0

00εK w dε

38

Radial Basis FunctionsRadial Basis FunctionsRadial Basis FunctionsRadial Basis Functions

• Solve the system Kw = d for w• Implicit function is defined as:

39

RBF Basis FunctionsRBF Basis FunctionsRBF Basis FunctionsRBF Basis Functions

• Tri-harmonic basis functions

– Globally supported

– Leads to dense symmetric linear systemy y

– C2 smoothness

– Provably smoothProvably smooth

– Works well for highly irregular sampling

40

Other Radial Basis FunctionsOther Radial Basis FunctionsOther Radial Basis Functions Other Radial Basis Functions • Polyharmonic spline

• Multiquadratic

• Gaussian

• B-Spline (compact support)

41

How Big isHow Big is εε??How Big is How Big is εε??

Without normal length lid ti

With normal length lid tivalidation validation

42

ExamplesExamplesExamplesExamples

43

ComparisonComparisonComparisonComparison

Distance Compact RBF Global RBFDistanceto plane

Compact RBF Global RBFTriharmonic

44

Complexity IssuesComplexity IssuesComplexity IssuesComplexity Issues

• Solve the linear system for RBF weightsSolve the linear system for RBF weights– Hard to solve for large number of samples

• Compactly supported RBFs– Sparse linear system– Efficient solvers

• Greedy RBF fittingGreedy RBF fitting– Start with a few RBFs only– Add more RBFs in region of large errorAdd more RBFs in region of large error

45

Extracting the SurfaceExtracting the SurfaceExtracting the SurfaceExtracting the Surface

F(x) = 0 surface

olog

y.co

m

F(x) < 0 inside

e fr

om: w

ww

.farf

ield

tech

no

F(x) > 0 outside

Imag

e46

Sample the SDFSample the SDFSample the SDFSample the SDF

47

Sample the SDFSample the SDFSample the SDFSample the SDF

48

2D: Marching Squares2D: Marching Squares

?

49

3D: Marching Cubes3D: Marching Cubes3D: Marching Cubes3D: Marching Cubes

• Classify grid nodes as inside/outsideClassify grid nodes as inside/outside

• Classify cell: 28 configurations

i i l i l d• Linear interpolation along edges

• Look-up table for patch configuration– Disambiguation more complicated

50

Marching CubesMarching CubesMarching CubesMarching Cubes

• Cell classification:Cell classification:– Inside

Outside– Outside

– Intersecting

51

Marching CubesMarching Cubes256 cases 15 cases

InversionRotationMarching CubesMarching Cubes

52

Marching CubesMarching CubesMarching CubesMarching Cubes

53

Marching Cubes ProblemsMarching Cubes ProblemsMarching Cubes ProblemsMarching Cubes Problems

• AmbiguityH l– Holes

• Generates HUGE meshes– Millions of polygons

54

AmbiguityAmbiguityAmbiguityAmbiguity

Separate pink

Separate blue

55

The Inversion ProblemThe Inversion ProblemThe Inversion ProblemThe Inversion Problem

• Reduction from 256 to 15 cases includesReduction from 256 to 15 cases includes inversion

56

The Inversion ProblemThe Inversion ProblemThe Inversion ProblemThe Inversion Problem

Inversion

InversionMismatch

57

Ambiguity SolutionAmbiguity SolutionAmbiguity SolutionAmbiguity Solution• 256 cases 23 cases

– Rotation only– Rotation only– Always separate same “color”– Ambiguous faces triangulated consistently

• 8 new cases

58

Without InversionWithout InversionWithout InversionWithout Inversion

Match

59

Ambiguity

NoAmbiguity

60

Ambiguity No Ambiguity

61

Marching Cubes IssuesMarching Cubes IssuesMarching Cubes IssuesMarching Cubes Issues

• Grid not adaptiveGrid not adaptive

• Many polygons required to represent small featuresfeatures

Images from: “Dual Marching Cubes: Primal Contouring of Dual Grids” by Schaeffer et al.

62

63

Beyond RBFBeyond RBFBeyond RBFBeyond RBF

• Poisson reconstructionPoisson reconstruction[Kazhdan 06]

64

Beyond RBFBeyond RBFBeyond RBFBeyond RBF

• Interactive approachInteractive approach[Sharf, 2007]

65