A Personal Surround Environment: Projective Display with Correction for Display Surface Geometry and Extreme Lens Distortion Tyler Johnson, Florian Gyarfas, Rick Skarbez, Herman Towles and Henry Fuchs Department of Computer Science University of North Carolina at Chapel Hill IEEE VR– March 14, 2007
29
Embed
A Personal Surround Environment: Projective Display with Correction for Display Surface Geometry and Extreme Lens Distortion Tyler Johnson, Florian Gyarfas,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
A Personal Surround Environment:
Projective Display with Correction forDisplay Surface Geometry and Extreme Lens Distortion
Tyler Johnson, Florian Gyarfas, Rick Skarbez, Herman Towles and Henry Fuchs
Department of Computer Science
University of North Carolina at Chapel Hill
IEEE VR– March 14, 2007
A Personal Surround Environment2
Multi-Projector Display
A Personal Surround Environment3
Wide-FOV Projectors
Conventional Projector Fisheye-Lens Projector
Conventional vs. fisheye lens projector
A Personal Surround Environment4
Outline
Distortion due to Non-Planar SurfacesLens Distortion Correction for ProjectorsResultsContributions and Future Work
Hemispherical display screen with fisheye-lens projector for immersive display.Static projector/display surface relationship.Distortion correction based on optical/mechanical design specifications.
Allows exploration of volume data using a deformable hand-held screen.Dynamic display surface.
A Personal Surround Environment12
Our Goals
Our goals were to providecamera-based calibration for wide-FOV projectorsgeometric correction for arbitrary display surfacessupport for tracked viewers
A Personal Surround Environment13
3-Pass Rendering
Pass-1 Pass-2
Display Surface
GeometryPinhole Projection
Pinhole Projection
Lens Distortion Correction
A Personal Surround Environment14
3-Pass Rendering Issuesfor Wide-FOV Lenses
Fisheye lenses are highly non-linearPinhole approximation in Pass-2 leads to sampling issues during lens distortion correction in Pass-3.
A Personal Surround Environment15
Sampling Artifacts
Display Surface
P
3-Pass Rendering
All Passes at Projector Resolution (1024 x 768)
A Personal Surround Environment16
Pass-3 Sampling Issues
Fisheye and pinhole lenses have very different spatial sampling characteristics
Leads to sampling issues during lens distortion correction.
Fisheye lens
(output)
Pinhole lens
(input)
Under-sampled! Over-
sampled!
Over-sampled!
A Personal Surround Environment17
Sampling Artifact Reduction
Pass 1-2: 4x Projector Resolution (4096x3072)
Pass 3: (1024x768)
P
Display Surface
A Personal Surround Environment18
A New Approach
Remove the pinhole approximation of the wide-FOV lens.
Use the actual lens model of the projector.
Allows simultaneous compensation for lens distortion and non-planar surface geometry.
A Personal Surround Environment19
Changes to Correction Pipeline
Pass-1 Pass-2
Display Surface
GeometryPinhole Projection
Pass-3
Pinhole Projection
Lens Distortion Correction
Projector Lens Model
Use actual projector lens model to avoid
resampling artifacts!
A Personal Surround Environment20
A New Approach
Display Surface
Viewer
Projector
Pre-compute where projector rays intersect
the display surface!
Projector Image
A Personal Surround Environment21
Implementation
Pre-compute the projector to display surface (ray) mapping
Store this projector-resolution, 2D to 3D map on GPU as a floating-point texture.Static map - independent of viewer location.
New Pixel ShaderAt each projector pixel:• Look up <x,y,z> vertex on the display surface.• Project the vertex into ideal image to obtain
output color.
A Personal Surround Environment22
Results
All Passes at Projector Resolution (1024x768)
Display Surface
P
A Personal Surround Environment23
Calibration
Capture structured light patterns with camera-pair
Calibrate projector• Linear calibration followed by
non-linear optimization incorporating projector lens model
Could apply simple 2D-2D mappingApplies only to stationary viewer!
Our approach is independent of the viewer location
Viewer location can be updated each frame
Pass-1 Combined Pass-2 & -3 Remapping
2D - Mapping
Only applicable for Stationary
Viewer
A Personal Surround Environment25
Heterogeneous 2-Projector Display
Also applicable to conventional projectors
Hybrid Display with Conventional and Fisheye-lens
Projectors
Fisheye-lens
Projector
Display Surface
Conventional
Projector
A Personal Surround Environment26
Results
Video
A Personal Surround Environment27
Contributions
Single-pass geometric remapping algorithmCorrecting for both extreme lens distortion and display surface geometryMinimizes sampling and FOV issues with fisheye lenses
Simple GPU shader program implementationStatic 2D:3D mapping stored as 3D floating point texture
Demonstrated applicabilityPersonal surround environment with a single fisheye lens projectorConventional or Wide-FOV fisheye AND Heterogeneous displays
Applicable to real-time VR applications with user-head tracking
A Personal Surround Environment28
Future Work
Methods to further reduce sampling artifacts
Mip-mapped Pass-1 textures for geometry optimized sampling.
Application of more general lens models[Kannala, IEEE TPAMI, Aug. 06]
Real-time, continuous calibrationAccommodate projector or display surface changes while the application running.Dynamic vs. static Pass-2 re-map function.
A Personal Surround Environment29
Thank You
Funding support: ONR N00014-03-1-0589Funding support: ONR N00014-03-1-0589DARWARS Training Superiority program DARWARS Training Superiority program
VIRTE – Virtual Technologies and Environments programVIRTE – Virtual Technologies and Environments program
Elumenati LLC for loan of fisheye-lens projectorElumenati LLC for loan of fisheye-lens projector