Light Field Technology Chen Zhao, Chen Zhi, John Allen Ray, Ung Guan Wah, Zhou Xintao For information on other technologies, please see Jeff Funk’s slide share account ( http :// www.slideshare.net/Funk98/presentations ) or his book with Chris Magee: Exponential Change: What drives it ? What does it tell us about the future? http:// www.amazon.com/Exponential - Change - drives - about - future - ebook/dp/B00HPSAYEM/ref=sr_1_1?ie=UTF8&qid=1398325920&sr=8 - 1&keywords=exponential+change
These slides use concepts from my (Jeff Funk) course entitled analyzing hi-tech opportunities to analyze how Light Field Technology is becoming economic feasible for an increasing number of applications. Light Field Cameras record all of the light fields in a picture instead of just one light field. This capability enables users to change the focus of pictures after they have been taken and to more easily record 3D data. These features are becoming economically feasible improvements because of rapid improvements in camera chips and micro-lens arrays (an example of micro-electronic mechanical systems, MEMS). These features offer alternative ways to do 3D sensing for automated vehicles and augmented reality and can enable faster data collection with telescopes.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Light Field TechnologyChen Zhao, Chen Zhi, John Allen Ray, Ung Guan Wah, Zhou Xintao
For information on other technologies, please see Jeff Funk’s slide share account(http://www.slideshare.net/Funk98/presentations) or his book with Chris Magee:
Exponential Change: What drives it? What does it tell us about the future? http://www.amazon.com/Exponential-Change-drives-about-future-
What is light field technology?• Light goes through the main lens and then hit a special lens• The special lens incorporates a compound lens known as a
micro-lens array (MLA)• MLA re-group sensors (e.g. CCD, CMOS) into small groups • Capture information on the direction, color, and luminosity of
millions of individual rays of light• Computationally reconstruct the light field (3D)• http://www.youtube.com/watch?v=7babcK2GH3I
Lytro cameraGeneralized conceptual diagram of light field camera
A light-field picture contains more information about depth than simply correspondence which allow us to capture the real world in unparalleled detail (4D).
Rendering complexity is independent of scene complexity.
Processing speed is fast
No need to worry about the focus of the scanner lenses.
Why light field for 3D sensing is better
1. Stereoscopic vision:
Currently most common 3D sensor approach
Passive range determination via stereoscopic vision utilizes the disparity in viewpoints between a pair of near-identical cameras to measure the distance to a subject of interest.
Replaces the stereoscopic vision sensor's second imaging sensor with a projection component.
Similar to stereoscopic vision techniques, this approach takes advantage of the known camera-to-projector separation to locate a specific point between them and compute the depth with triangulation algorithms.
3D sensing Technology
3. Time of flight
An indirect system to obtain travel time information by measuring the delay or phase shift of a modulated optical signal for all pixels in the scene.
The ToF sensor in the system consists of an array of pixels, where each pixel is capable of determining the distance to the scene.
3D sensing Technology
Stereoscopic vision
Structured light Time of flight(TOF)
Light Field
Software complexity
High High Low Low
Material Cost Low High/Middle Middle Low
Response time Middle Slow Fast Fast
Low light Weak Light source dep(IR or visible)
Good (IR, laser) Good
Outdoor Good Weak Fair Good
Depth (“z”) accuracy
cm μm ~ cm mm ~ cm μm ~ cm
Range Mid range Very short range(cm) to mid range (4-
6m)
Short range(<1m) to
long range(~40m)
Very short range(cm) to long
rang (~100m)
Application
Device control
3D movie
3D scanning
3D-vision gesture control system, which is a highly precise and reliable user interface for interacting with any display screen from any distance. Whether on a personal computer, set top box, television set, mobile device, game console, digital sign or interactive kiosk, The depth tracking software enables users to control onscreen interaction with simple hand motions instead of a remote control, keyboard or touch screen.
3D sensing control
3D Dental Scanner
3D scanning for arch
Interactive Billboard
Interactive mirror
Future Retail Industry
Mechanical Parts ScanningFor Lockheed Martin, 3D scanner assures the right
a) Depicts the representation of a hologram. b) and c) show two different representations of a
light field.
Reference: Eurographics 2007/D.Cohen-Or and P.Slavik, A Bidirectional Light Field-Hologram Transform Volume 26 (2007), Number 3
Light Field Mapping
• The possibility to transform a light field into a holographic representation and vice versa.
• The holographic data representation is similar to a light field
• “M” transforms the light field into a holographic representation.
• Method to extract depth from the input light field.
• If accurate depth information is available for the light field it can
optionally be added to the input
Reference: Eurographics 2007/D.Cohen-Or and P.Slavik, A Bidirectional Light Field-Hologram Transform Volume 26 (2007), Number 3
Light Field Mapping
• Depth Reconstruction from Light Fields • Effects of Loss of Data from Hologram
Reference: Eurographics 2007/D.Cohen-Or and P.Slavik, A Bidirectional Light Field-Hologram Transform Volume 26 (2007), Number 3
Hologram Vs Light Field
Reference: Eurographics 2007/D.Cohen-Or and P.Slavik, A Bidirectional Light Field-Hologram Transform Volume 26 (2007), Number 3
Hologram Vs Light Field
• Hologram - an illustration of direct output of holographic content on future generation holographic displays.
• Light field - far more efficient for conventional 2D frame buffer displays.
• Light Field - the versatility and the power of transformation on synthetic light fields, real light fields and digitally recorded holograms.
• The rendered images can be evaluated directly from the holographic representation or through light field rendering.
• Light field capable of simulating different aperture sizes as well as focal length – Versatile displays
Future• Holograms can be captured using a light field camera • Take advantage of the realism and detail preserving benefits of
a real light field while giving the possibility of a 3D output on a holographic screen
AR Light Field Possible Improvements
• See-through displays• New tracking sensors• Interfaces and Interactions
Applications
Cameras
3D Sensing
Augment Reality
Telescope/Microscope
Autonomous vehicles
Scientific Applications
Light-Field Telescope
Light-Field Microscope
Light-Field Telescope
Source: JonathanWedd, Jan van der Laan, Eric Lavelle, and David Stoker. “A High-Magnification Light-Field Telescope forExtended Depth-of-Field Biometric Imaging”
Light-Field Telescope
Take the image faster and refocus the image after taking
Increase magnification
Achieve large depth of field and high lateral image resolution simultaneously
Capture different wavelengths
Light Field Microscope
A compact Light Field Microscope Designed by Stanford
Consists of an ordinary research microscope and cooled scientific camera
A microlens array is inserted
Source: Marc Levoy, Ren Ng, Andrew Adams, Matthew Footer, Mark Horowitz. “Light Field Microscopy”. Stanford University
Light Field Microscope
Source: Marc Levoy, Ren Ng, Andrew Adams, Matthew Footer, Mark Horowitz. “Light Field Microscopy”. Stanford University
Light Field Microscope
Captures light fields of biological specimens in a single snapshot
Offers 3D functional imaging of neuronal activity in entire organisms at single cell level
Separates image acquisition from the selection of viewpoint and focus
Captures video of high speed moving specimens
Applications
Cameras
3D Sensing
Augment RealityTelescope/Microscope
Autonomous vehicles
Autonomous Vehicle Applications
Consumer
Passenger Vehicles
Agricultural Vehicles
Military
Combat Vehicles
Logistics/Supplies
Search & Rescue Vehicles
M.Bellone, et al, “Unevenness Point Descriptor for Terrain Analysis in Mobile Robot Applications”, 2012http://cdn.intechopen.com/pdfs-wm/45459.pdf
LFC Vs. Laser Based Systems
Advantages:
Easier data interpretation
Significantly Lower Cost
Lower Power Requirement
Comparable performance
Disadvantages:
Smaller field of view (multiple cameras required)
D. Stavens, “LEARNING TO DRIVE: PERCEPTION FOR AUTONOMOUS CARS”, 2011http://purl.stanford.edu/pb661px9942
Light Field Depth Map
M.Tao, et al, “Depth from Combining Defocus and Correspondence Using Light-Field Cameras”, 2013http://www.cs.berkeley.edu/~ravir/lightfield_ICCV.pdf
Light Field Terrain Analysis
M.Bellone, et al, “Unevenness Point Descriptor for Terrain Analysis in Mobile Robot Applications”, 2012http://cdn.intechopen.com/pdfs-wm/45459.pdf
Light Field Terrain Analysis
M.Bellone, et al, “Unevenness Point Descriptor for Terrain Analysis in Mobile Robot Applications”, 2012http://cdn.intechopen.com/pdfs-wm/45459.pdf
Future of Light Field
We have analyzed advantages of LF
We showed applications of LF in cameras, mobilephones, 3D scanning, AR, scientific research, andautonomous vehicles
Future applications of LF will enable us to do moresimple and fast 3D applications