Structured Light in Scattering Media Srinivasa Narasimhan Sanjeev Koppal Robotics Institute Carnegie Mellon University Sponsor: ONR Shree Nayar Bo Sun.

Structured Light in Scattering Media

Srinivasa Narasimhan

Sanjeev Koppal

Robotics Institute

Carnegie Mellon University

Sponsor: ONR

Shree Nayar

Bo Sun

Computer Science

Columbia University

ICCV ConferenceOctober 2005, Beijing, China

Natural illumination in Scattering Media

[ Narasimhan and Nayar, 99 - 03, Schechner et al, 01, 04 ]

Active illumination in Scattering Media

[Levoy et al., Narasimhan-Nayar, Kocak-Caimi, Jaffe et al., Schechner et al., Negahdaripour et al. ]

Floodlighting is Bad in Scattering Media

Structured Light Critical for Good Visibility

Light Stripe Range Finding in Clear Air

Camera

Source

Surface

Light plane

Light plane

Camera

Source

Surface

Light plane

Light Stripe Range Finding in Scattering Media

Light Striping Model in Scattering Media

Extinction coefficient

Dv

• Irradiance due to Medium:

Camera

Source

SurfaceLight plane

)()( smediumssurface DxEDxEE

)(0 PeeLE yxmedium

)(0

vs DDsurface eRLE

)cos1(4

)(0

geLE yx

medium

Radiance

αx

y

Ds

• Irradiance due to Surface:

• Final Image Irradiance:

0L

Phase Function

Light Striping Algorithm in Scattering Media

Surface Intersection from Brightness Profile:

3D by Triangulation or Temporal Analysis : Same as in clear air.

Medium from Fall-off :

“Clear-Air” Scene Appearance:

NoScattering

ModerateScattering

SignificantScattering

E

x

mediumE surfaceE

2)(

0 ||)cos1(4

||min geLE yx

medium ),( g

)(0

vs DDsurface eERL

Experimental Setup

Calibration technique similar in spirit to [Grossberg-Nayar 01 ]

Floodlit Image Computed Appearance

Smoke and Mirrors

Milk and Mirrors

[Discussions with Marc Levoy]

Planar Mirror seen through Dilute Milk

Light Striping of Mirrors

(Dark Intersections)

Reconstruct surfaces with any BRDF if light plane visible

Three images

required.

Photometric Stereo in Clear Air

[ Woodham 80, Horn 86 ]

Distant Source

Orthographic

Camera ns

PSurface

Pure Air

sn .0 LEsurface

Image Irradiance:Surface normal

Source directionAlbedo

Photometric Stereo in Scattering MediaScattering Medium

Parallel Rays from Distant Source

Orthographic

Camera α n s

P

Ds

DvSurface

sn .)cos1(0

sDsurface eLE

Image Irradiance:

)1()cos1( cos0

ss DDmedium eegLE

+ E

E

Optical Thickness

Phase Function

Photometric Stereo in Scattering MediaScattering Medium

Parallel Rays from Distant Source

Orthographic

Camera α n s

P

Ds

DvSurface

5 Parameter Non-linear Optimization (4 per pixel, 1 global) :

E

Five Non-degenerate Sources are Necessary and Sufficient

||)(||min mediumsurface EEE

Simulations: Error Histograms

0 0.05 0.10

50

100

150

200

250

300 ( x 10 )

Fractional Error for Albedo

Fractional Error for Phase Function, g

Fractional Error for Optical Thickness

Angular Error for Normals

0 0.05 0.10

50

100

150

200

250

300 ( x 10 )

0 0.05 0.10

50

100

150

200

250

300 ( x 10 )

0 0.05 0.10

50

100

150

200

250

300 ( x 10 )

Zero error with zero noise.

Robust estimation with 5% uniform noise.

Trials Trials Trials Trials

Experiments: Teapot in Pure Water

Experiments: Teapot in Dilute Milk

Low Contrast, Flat Appearance

Results: Traditional Photometric Stereo

3D Shape from Normals

Too Flat

Albedos

Scattering effects present

Results: Our Five-Source Algorithm

3D Shape from Normals

Albedos

Results: Depth from Photometric Stereo

3D Shape from Normals

Depth map

Impossible using traditional method

% RMS Error

3 ml 4 ml 5 ml 6 ml 12 ml15 ml

2.0 2.5 3.0 3.3 5.8 6.3

Milk Concentration

• Surprising results possible

because of scattering

• Structured light improves visibility

Summary

• Physics of scattering crucial