Seeing through Optical Barriers Using Visible Light Guy Satat May 25, 2017
Seeing through Optical Barriers Using Visible Light
Guy Satat
May 25, 2017
RadarCameraLidar
X rays UV IR Microwave Radio Waves
Visible
Wavelength
โข Resolution
โข Optical Contrast
Wildlife Rehabilitation Center of Minnesota
Optical Contrast
Visible light X-Ray
Light Scatters
Light and Matter in a Nutshell
Object LensAbsorption Scattering
How to Overcome Scattering
Hardware
Image processing
Computational imaging
Lessons learned from seeing into the body
Information Carried by Light
โข The plenoptic function:
๐ผ(๐ผ(๐,๐ผ(๐, ๐๐ผ(๐, ๐, ๐ก,๐ผ(๐, ๐, ๐ก, ๐,๐ผ(๐, ๐, ๐ก, ๐, ๐๐ผ(๐, ๐, ๐ก, ๐, ๐, ๐,๐ผ(๐, ๐, ๐ก, ๐, ๐, ๐, ฮฆ)
Irradiance Position AngleTimeWavelength Polarization Bounce Phase
Information Carried by Light
โข The plenoptic function:
๐ผ(๐ผ(๐,๐ผ(๐, ๐๐ผ(๐, ๐, ๐ก,๐ผ(๐, ๐, ๐ก, ๐,๐ผ(๐, ๐, ๐ก, ๐, ๐๐ผ(๐, ๐, ๐ก, ๐, ๐, ๐,๐ผ(๐, ๐, ๐ก, ๐, ๐, ๐, ฮฆ)
Irradiance Position AngleTimeWavelength Polarization Bounce Phase
Optics Based Solutions
โข Angleโข Timeโข Polarization
Not enough photons
Photon gating:
Use All Photons!
Computationally Invert Scattering
Satat, Heshmat, Raviv, Raskar Nature Scientific Reports 2016
โข Estimate targetโข Estimate scattering
Time
10,000,000,000 Slower
Time
AScene Scatterer Measurement
Sharp Blurred
๐ก
๐ฅ
๐ฆ
2D 3D
๐ ๐ฅ, ๐ฆ โ ๐พ ๐ฅ, ๐ฆ, ๐ก = ๐ ๐ฅ, ๐ฆ, ๐ก
Estimating the Scattering - ๐พ ๐ฅ, ๐ฆ, ๐ก
โข Point Spread Function
โข Probabilistic interpretation:โข Probability to measure photon at specific location and time
โข Bayes rule
)๐พ ๐ฅ, ๐ฆ, ๐ก = ๐๐ ๐ก ๐(๐ฅ, ๐ฆ|๐ก
Probability to measure a photon at time ๐ก
Given the time, probability to measure a photon at location ๐ฅ, ๐ฆ
Estimating the Scattering - ๐พ ๐ฅ, ๐ฆ, ๐ก
โข ๐๐ ๐ก , ๐ ๐ฅ, ๐ฆ ๐ก โ Easier to estimate
โข Assumptions:โข Enough samples to satisfy law of large numbers
๐พ ๐ฅ, ๐ฆ, ๐ก = ๐๐ ๐ก ๐ ๐ฅ, ๐ฆ ๐ก
Recovery of Slits
Results
5๐๐
Invariant to Layered Material
Satat, Heshmat, Raskar COSI 2017
Properties of All Photons Imaging
โข Recovers scatterer and targetโข Calibration free
โข Minimal assumptions
โข Works with layered materials
โข Doesn't require raster scan
Challenges
Will It Scale?
โข Emerging sensors
โข Can build on LIDAR
โข Framework for low pixel count
Lensless Imaging with a Femto-Pixel
Satat, Tancik, Raskar IEEE Trans. Computational Imaging 2017
Lensless Imaging with a Femto-Pixel
Traditional Our approach
Regular pixel Femto-pixel
Framework for Imaging with a Femto-Pixel
Summary
โข What if we could simply see through obstructions?
โข Each photon has a story
โข Super driver requires super vision
media.mit.edu/[email protected]