Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy.

Interactive Deformation Interactive Deformation of Light Fieldsof Light Fields

Billy Chen

Eyal Ofek

Heung-Yeung Shum

Marc Levoy

Image-based renderingImage-based rendering

Levoy et al., 1996 Shum et al., 1999

Zitnick et al., 2004 Uyttendaele et al., 2004

Previous workPrevious work

Seitz, S., Kutulakos, K. N. “Plenoptic image editing,” In Proc. ICCV 1998.

Zhang, Z., Wang, L., Guo, B., Shum, H.-Y, “Feature-based light field morphing,” In Proc. SIGGRAPH 2002.

Weyrich, T., Pfister, H., Gross, M., “Rendering deformable surface reflectance fields,” In IEEE Transactions on Computer Graphics and Visualization.

Light field deformationLight field deformation

OutlineOutline

Coaxial light fields Framework for deformation

Splitting Deforming Rendering

Results

OutlineOutline

Coaxial light fields Framework for deformation

Splitting Deforming Rendering

Results

Coaxial light fieldsCoaxial light fields

… …

OutlineOutline

Coaxial light fields Framework for deformation

Splitting Deforming Rendering

Results

Splitting a light fieldSplitting a light field

A sub-light field consists of a bundle of rays a deformation box

u

s

Sub-light field exampleSub-light field example

Projector-based light field Projector-based light field segmentationsegmentation

…

OutlineOutline

Coaxial light fields Framework for deformation

Splitting Deforming Rendering

Results

DeformationDeformation

Ray warpingRay warping

• preserves mapping specified by the animator• preserves C0

continuity of points• maps lines to lines

Inverse view ray warpingInverse view ray warping

Inverse view ray warpingInverse view ray warping

Hardware accelerated warpingHardware accelerated warping

front id map back id map deformation

• 256 x 256 resolution texture map• anisoptric filter for mip-mapped texture• 64 x 64 triangles• real-time with a NVIDIA GeForce 3

OutlineOutline

Coaxial light fields Framework for deformation

Splitting Deforming Rendering

Results

For each camera pixel1. Segment ray2. Warp ray3. Render ray4. Mask ray

Rendering and visibilityRendering and visibility

ResultsResults

ConclusionConclusion

A pipeline for deforming light fields Light field segmentation Ray warping Rendering multiple light fields

Coaxial light fields Enables interactive deformation of photo-

realistic objects Applications: photo-realistic rapid prototyping,

games

Thank youThank you

Reviewers NSF (IIS-0219856-001) DARPA (NBCH-1030009) Elizabeth Chen, Vaibhav Vaish, Gaurav Garg

Failure casesFailure cases

Video of acquisition processVideo of acquisition process

DeformationDeformationand compositingand compositing