3dpho

8/8/2019 3dpho

1/38

5/1/2000 Deepak Bandyopadhyay / UNCChapel Hill

1

3D Photography

(Image-based Model Acquisition)

Funky Image Goes Here

8/8/2019 3dpho

2/38

11/6/2000 Deepak Bandyopadhyay / 258 / 3D Photography 2

Analog 3D photography !

3D stereoscopic imaging

been around as long as cameras have

Use camera with 2 or more lenses (or stereo attachment) Use stereo viewer to create impression of 3D

8/8/2019 3dpho

3/38


Motivation Digitizing real world objects

Getting realistic models

humans

objects

places

8/8/2019 3dpho

4/38


3D Photography : Definition

Sometimes called 3D Scanning

Use cameras and light to capture the shape &

appearance of real objects

Shape == geometry (point sampling + surface

reconstruction + fairing)

Appearance == surface attributes (color/texture,material properties, reflectance)

Final result = richly detailed model

8/8/2019 3dpho

5/38


Applications in Industry

Human body / head / face scans

Avatar creation for virtual worlds

3d conferencing

medical applications

product design

Platforms:

Cyberware RD3030

Others (Geomagic, Metacreations, Cyrax, Geometrix)

8/8/2019 3dpho

6/38


More applications

Historical preservation, dissemination of museum

artifacts (Digital Michelangelo, Monticello, )

CAD/CAM (eg. Legacy motorcycle parts scannedby Geomagic for Harley-Davidson).

Marketing (models of products on the web)

3D games & simulation Reverse engineering

8/8/2019 3dpho

7/38


Technology Overview The Imaging Pipeline

Real World

Optics

Recorder

Digitizer

Vision & Graphics

8/8/2019 3dpho

8/38


Quick Notes on Optics

Model lenses with all their properties -

aberration, distortion, flare, vignetting etc.

We correct for some of these effects (eg.

distortion) in the calibration, ignore others.

CCD (charged coupled devices) are the

most popular recording media.

8/8/2019 3dpho

9/38


Theory : Passive Methods

Stereo pair matching

Structure from motion

Shape from shading

Photometric stereo

8/8/2019 3dpho

10/38


Stereo Matching

Stereo Matching Basics

Needs two images, like stereoscopy

Given correspondence betweenpoints in 2 views, we can find

depth by triangulation

But correspondence is hard prob!

A lot of literature on solving it

Stereo Matching output 3D point cloud Remove outliers and pass through surface reconstructor

8/8/2019 3dpho

11/38


Structure from Motion

Camera moving, objects static

Compute camera motion and object geometry from

motion of image points

Assumption - orthographic projn (use telephoto)

If: world origin = 3D centroidcamera origin = 2D centroid

Then: camera translation drops out

8/8/2019 3dpho

12/38



Camera moving, objects static

Compute camera motion and object geometry from

motion of image points

8/8/2019 3dpho

13/38



Factorization [Tomasi & Kanade, 92]

Find M, S using Singular Value Decomposition of

W.

SVD gives:

S}S modulo linear transform A.

Solve for A using constraints on M.

8/8/2019 3dpho

14/38


More methods

Shape from shading, [Horn]

Invert Lamberts Law (L=I k cos E)

knowing the intensity at image pointto solve for normal

Photometric stereo [Woodham]

An extension of the above

Two or more images under different illumination conditions. Each image provides one normal

Three images provide unique solution for a pixel.

8/8/2019 3dpho

15/38


Active Sensing Passive methods (eg. stereo matching) suffer from

ambiguities - many similar regions in an image

correspond to a point in the other.

Project known / regular pattern (structured light)

into scene to disambiguate

get precise reconstruction by combining views Laser rangefinder

Projectors and imperceptible structured light

8/8/2019 3dpho

16/38


Desktop 3D PhotographyJean-Yves Bouguet, Pietro Perona

An active sensing technique using weak

structured lighting

Need: camera, lamp, chessboard, pencil, stick

Idea:

Light object with lamp & aim camera at it

Move stick around & capture shadow sequence Use image of deformed shadow to calc 3D shape

8/8/2019 3dpho

17/38


Desktop 3D PhotographyJean-Yves Bouguet, Pietro Perona

Computation of 3d position from the plane of

light source, stick and shadow

8/8/2019 3dpho

18/38


Volumetric MethodsChevette Project, Debevec, 1991

8/8/2019 3dpho

19/38


Voxel Models from Images When there are 2 colors in the image - use volume

intersection [Szeliski 1993]

Back-project silhouettes from camera views &

intersect

8/8/2019 3dpho

20/38


Voxel Models from Images

With more colors but constrained viewpoints, we

use voxel coloring [Seitz & Dyer, 1997]

Choose a voxel & project to it from all views Color if enough matches

Prob - determining visibility

of a point from a view

Solution - depth orderedtraversal using a view indep.

d.o. (dist from separating plane)

8/8/2019 3dpho

21/38


Voxel Models from Images

A view-independent depth order may not exist

(for some configuration of viewpoints / scene geometry).

Use Space Carving [Kutulakos & Seitz, 1998]

Computes 3D (voxel) shape from multiple color photos

Computes maximally photo-consistent shape

maximal superset of all 3D shapes that produce the given photos

8/8/2019 3dpho

22/38


Space Carving

Algorithm:

a) Initialize V to volume

containing true sceneb) For each voxel,

check if photo-consistent

if not, remove (carve) it.

Can be shown to converge to maximal photo-consistentscene (union of all photo-consistent scenes).

8/8/2019 3dpho

23/38


Space Carving : Results

House walkthru - 24 rendered input views

Results best as seen from one of the original views

8/8/2019 3dpho

24/38


Modeling from a single view(Criminisi et al, 1999)

Compute 3D affine measurements of the scene

from single perspective image

Use minimal geom info

vanishing line for a pencil of

planes || to reference plane

vanishing point of parallellines along a direction

outside reference plane

8/8/2019 3dpho

25/38


Modeling from a single view(Criminisi et al, 1999)

Compute ratio of parallel distances

Creating a 3D model from a photograph

horizontal lines used to compute vanishing line

parallel vertical lines used to compute vanishing point

Can generate geometrically correct model from a

Renaissance painting (with correct perspective)

8/8/2019 3dpho

26/38


Extracting color, reflectance Photographs have lighting/shading effects that we

estimate (reflectance function) and compensate for

(specular highlight removal) or change (relighting) Work of Paul Debevec & others at Berkeley

(acquiring reflectance field)

Wood et al at U. Washington (surface light lield

for 3D photography)

8/8/2019 3dpho

27/38


Surface Light Field

[Wood et al, 2000] A 4D function on the surface - at surface

parameter (u,v), for every direction (U,J), stores

the color. Fixed illumination conditions.

Photographs taken from a lot of different

directions sample the surface light field.

Continuous function (piecewise linear overU,J)

estimated bypointwise fairing.

8/8/2019 3dpho

28/38


Reflectance from Photographs

(Yu, Debevec et al, 1999) Estimating reflectance for entire scenes

Too general a problem, parameterize thus:

Assume surface can be divided into patches Diffuse reflectance function (albedo), varies across a patch

Specular reflectance function taken as const across a region

Assume known lighting, calib, geometry known

Approach - Inverse Global Illumination

Estimate BRDF for direct illumination - f(u,v,U,J)

8/8/2019 3dpho

29/38


Reflectance from Photographs

(Yu, Debevec et al, 1999) Inverse Global Illumination

Known Li (measure), Ii (calc fm known light sources) at

every pixel Estimate BRDF for direct illumination - f(u,v,Ui,Ji,Ur,Jr)

Write BRDF as a constant diffuse term and a specular term

which is a function of incoming & outgoing U and roughness.

Solve for the constants

(Vd,Vs,E) For indirect illumination - estimate the parameters (and indirect

illumination coeffs with other patches) iteratively

8/8/2019 3dpho

30/38


Case study - Faade

Debevec, Taylor & Malik, 1996

Modeling architectural scenes from photographs

Not fully automatic (user inputs blocky 3D model)

Using blocks leads to fewer params in architectural models

User marks corresponding features on photo

Computer solves for block size, scale, camera rotation

by minimizing error of corresponding features

Reprojects textures from the photographs onto the

reconstructed model

8/8/2019 3dpho

31/38


Modeling and Rendering Architecture from Photographs

(Debevec, Taylor, and Malik 1996)

Block ModelBlock Model User User--Marked EdgesMarked Edges Recovered ModelRecovered Model

8/8/2019 3dpho

32/38


Arches and

Surfaces of Revolution

Taj Mahal

modeled from

one photograph

8/8/2019 3dpho

33/38


Case study - Digital

Michelangelo Project

3D scanning of large statues (SIGGRAPH 00)

Separate geometry and color scans

custom rig : laser scanner & camera mounted concurrently

Range scan post-processing

Combine range scans from different positions

Use volumetric modeling methods (Curless, Levoy 1996)

Fill holes using space carving

8/8/2019 3dpho

34/38


Case study - Digital

Michelangelo Project Color scan processing

Compensate for ambient lighting

subtract image with & without spotlight Subtract out shadows & specularities

find surface orientation (inverse lighting computation)

convert color to RGB reflectance (acquire light field)

Using estimated BRDF of marble

modeling subsurface scattering

8/8/2019 3dpho

35/38


Digital Michelangelo

Scanning a large object

calibrated motions pitch (yellow) pan (blue)

horizontal translation (orange)

uncalibrated motions vertical translation remounting the scan head

moving the entire gantry

8/8/2019 3dpho

36/38


References

[Bouguet98] Bouguet, J.-Y., P. Perona. 3D Photography on your Desk. In

Proc. ICCV 1998

[Bouguet00] Bouguet, J.-Y. Presentation on Desktop 3D Photography, in

SIGGRAPH course notes on 3D Photography, 2000

[Criminisi99] Criminisi, A., I. Reid and A. Zisserman. Single View Metrology.In Proc. ICCV, pp 434-442, September 1999

[Curless96] Curless, B. and M. Levoy. A Volumetric Method for Building

Complex Models from Range Images. In Proc. SIGGRAPH 1996

[Debevec96] Debevec, P., C. Taylor and J. Malik. Faade - Modeling and

Rendering Architectural Scenes from Photographs. In Proc. SIGGRAPH 1996

[Debevec00a] Debevec, P. Presentation on the Faade, from SIGGRAPH

course notes on 3D Photography, 1999, 2000.

[Debevec00b] Debevec, P., T. Hawkins, C. Tchou, H.P.Duiker, W. Sarokin and

M. Sagar. Acquiring the Reflectance Field of a Human Face. In Proc.

SIGGRAPH 2000.

8/8/2019 3dpho

37/38


More References

[Horn70] Horn, B.K.P. Shape from Shading : A Method for Obtaining the

Shape of a Smooth Opaque Object from One View. Ph.D. Thesis, Dept of EE,

MIT, 1970.

[Kutulakos98] Kutulakos, K. N. and S. Seitz. A Theory of Shape by Space

Carving. URCS TR#692, May 1998, appeared in Proc. ICCV 1999. [Levoy96] Levoy, M. and P. Hanrahan. Light Field Rendering. In Proc.

SIGGRAPH 1996.

[Levoy00a] Levoy, M., Pulli, K., Curless, B. et al. The Digital Michelangelo

Project - 3D Scanning of Large Statues. In Proc. SIGGRAPH 2000.

[Levoy00b] Levoy, M. Presentation on the Digital Michelangelo Project, in

SIGGRAPH course notes on 3D Photography, 2000.

[Seitz97] Seitz & Dyer. Photorealistic Scene Reconstruction by Voxel

Coloring. In Proc. CVPR 1997, pp. 1067-1073.

8/8/2019 3dpho

38/38


Still More References

[Seitz00] Seitz, S. SIGGRAPH course notes on 3D photography, 1999, 2000.

[Szeliski93] Szeliski, R. Rapid Octree Construction from Image Sequences.

CGVIP : Image Understanding, vol. 58, no. 1, pp 23-32, 1993.

[Wood00] Wood, D., D. I. Azuma, K. Aldinger, B. Curless, T. Duchamp, D.H.Salesin and W. Stuetzle. Surface Light Fields for 3D Photography. In Proc.

SIGGRAPH 2000.

[Woodham80] Woodham, R. Photometric Stereo for Determining Surface

Orientation from Multiple Images. Journal of Optical Engineering, vol. 19,

no. 1, pp 138-144, 1980.

[Yu99] Yu, Y., P. Debevec, J. Malik and T. Hawkins. Inverse GlobalIllumination - Recovering Reflectance Models of Real Scenes from

Photographs.

3dpho

Documents