Saarbrücker IT-Dialog Automobilindustrie A Cave System for Interactive Modeling of Global Illumination in Car Interior Kirill Dmitriev, Thomas Annen, Grzegorz Krawczyk, Rafal Mantiuk, Karol Myszkowski, and Hans-Peter Seidel Max-Planck-Institut für Informatik, Saarbrücken, Germany Karol Myszkowski
33
Embed
Saarbr ü cker IT-Dialog Automobilindustrie A Cave System for Interactive Modeling of Global Illumination in Car Interior Kirill Dmitriev, Thomas Annen,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Saarbrücker IT-Dialog Automobilindustrie
A Cave System for Interactive Modeling of Global Illumination in
Car Interior
Kirill Dmitriev, Thomas Annen, Grzegorz Krawczyk, Rafal Mantiuk,
Karol Myszkowski, and Hans-Peter Seidel
Max-Planck-Institut für Informatik,Saarbrücken, Germany
Karol Myszkowski
Saarbrücker IT-Dialog Automobilindustrie
[log cd/m^2]Luminance
-6 -4 -2 0 2 4 6 8
Bright projector
LCD Monitor
CRT Monitor
Human eye
High Dynamic RangeHigh Dynamic Range
Human eye adjusts comfortably up to 12 orders of magnitude and can see simultaneously up to 4 orders of magnitude
Saarbrücker IT-Dialog Automobilindustrie
HDR Pipeline: Acquisition• Global illumination• Products: HDR cameras
– Lars III (Silicon Vision), Autobrite (SMal Camera Technologies), HDRC (IMS Chips), LM9628 (National), Digital Pixel System (Pixim)
– Solution: tone mapping – Important factors in tone mapping selection
• Dynamic range of display device• Video display conditions
– Background lighting• Application
– Just nice looking video– Visually plausible results– Optimizing visibility of details– Improving contrast …
• HDR displays start to appear – Sunnybrook Technologies and University of British
Columbia
Saarbrücker IT-Dialog Automobilindustrie
HDR Pipeline: Applications
IMS ChipsIMS Chips
Saarbrücker IT-Dialog Automobilindustrie
Light Reflection
emissiongeometryBRDF
radiometricvalues
displayedimage
Visual DisplayLight TransportSimulation
27
15
13
22
Display Observer
Realistic Image Synthesis
Greenberg et al. Siggraph’97. Cornell UniversityGreenberg et al. Siggraph’97. Cornell University
HDRHDR LDR
Saarbrücker IT-Dialog Automobilindustrie
Acquisition: Materials• Shift variant Bi-
directional Reflectance Distribution Function (BRDF)– Lensch et al.
• Transluscency – Goesele et al.
Saarbrücker IT-Dialog Automobilindustrie
Acquisition: Luminaires• Luminaire spatial power distribution (Goesele et al.)
– Near field photometry– Emitted energy represented as 4D light field
• Relevant for light sources installed in the car interior
measurement
VR: rendering
Saarbrücker IT-Dialog Automobilindustrie
Acquisition: Luminaires• Natural lighting
– Light probes and multi-exposure techniques– SpheroCam from Spheron VR– Emitted energy represented as an environment map
• Relevant for external car illumination
VR: rendering
Light probe
Paul Debevec
Saarbrücker IT-Dialog Automobilindustrie
Lighting Simulation + Rendering
1) Photographs of mirror sphere at varying exposure times
2) High-dynamicrange environment map
3) Use as light source in Monte Carlo radiosity algorithm
Philippe Bekaert
Saarbrücker IT-Dialog Automobilindustrie
Our Goal• CAVE system for car
interior rendering– Real-time lighting simulation– Dynamic real world lighting
• HDR video environment maps– Free observer position
• Head tracking
• Special focus: – Predict impact of quickly changing lighting conditions on the visibility
of information displayed at the LCD panel• Precise modeling of light reflections from the LCD panel
• Predicting effective contrast and information readability for various viewing angles
• Taking into account light adaptation conditions for the human eye
Saarbrücker IT-Dialog Automobilindustrie
Requirements for Rendering Algorithm• 5x2 full screen resolution frames at interactive rates• LCD panel illumination must be computed precisely• Higher error tolerance for the car interior illumination
– Only low spatial frequencies in reconstructed lighting acceptable
Such a dot product must be computed for each mesh vertex: n = 25
Saarbrücker IT-Dialog Automobilindustrie
PRT for Arbitrary Meshes
BeforeBefore AfterAfter
Saarbrücker IT-Dialog Automobilindustrie
Handling Two-Sided Geometry• Car geometry is two sided
– Need to store SH coefficients for both sides and render geometry twice with back face culling
– This leads to lower performance and twice larger memory consumption
• Better store SH coefficients only for one side of geometry:– Only the rays going through the windows contribute to SH
coefficients– User has to point out the mesh parts representing car windows
Saarbrücker IT-Dialog Automobilindustrie
LCD Panel Modeling• Emission characteristic of the LCD-sandwich:
– Spectral emission-function of the backlight– Transmission characteristic of the polarizer and other
layers (e.g. BEF, d-BEF, LCF, rgb-filters, …)– Transmission characteristic of the LC-cell– Transmission of other optical elements – e.g. glass
• Reflective characteristic in case of incoming light– Surface coating (e.g. AR/AG)– Reflection characteristic of the LCD sandwich
observer
LCD-sandwich
Incoming lighting
backlight
glassplate
Saarbrücker IT-Dialog Automobilindustrie
Computing LCD Panel Lighting• DIMOS or SPECTER systems can be used for lighting simulation
within the LCD panel– We use just external spectral emissivity and reflectance data
provided with a very high angular resolution
• Algorithm– Cover geometry of LCD panel with texture– For each texel
• Compute energy emitted in the observer direction
– Use tabulated emissivity data
– Modulate emissivity as a function of the displayed information
• Compute energy reflected in the observer direction
– Use the final gathering method
– Improve performance using BRDF-weighted importance sampling
» If traced ray hits the car window, query the environment map
» If traced ray hits the car interior, query PRT data structures
Saarbrücker IT-Dialog Automobilindustrie
System Architecture
Saarbrücker IT-Dialog Automobilindustrie
Distributing the Computation
Camera parametersCamera parameters
• Use “Lightning” system that takes care of OpenGL synchronization between different PCs
• LCD display computation is distributed in an asynchronous way
Saarbrücker IT-Dialog Automobilindustrie
Distributing the Computation• Use “Lightning” system that
takes care of OpenGL synchronization between different PCs
• LCD display computation is distributed in an asynchronous way
Broadcast the display image as it is readyBroadcast the display image as it is ready
Saarbrücker IT-Dialog Automobilindustrie
Tone Mapping in the CAVE• Visual adaptation is influenced by light projected on a
small area around the center of retina • Head tracking enables precise estimation of the gaze
direction– We assume that the adaptation is affected by scene luminances
within the region of 10° surrounding the gaze direction– This region can be mapped to 1-3 walls in the CAVE– Luminance data within this region is collected and send to the
master
• Master computes common tone mapping parameters and broadcasts them to all computers in the cluster– There is a small delay in illumination update, but it is hidden by
temporal adaptation model anyway
Saarbrücker IT-Dialog Automobilindustrie
Rendering with Tone Mapping• 3 Passes:
– Compute tone mapping parameters
• Preview rendering of 128x128 HDR images for the CPU processing
– Select visible geometry for tone mapping
• Lighting OFF• Render geometry to z-buffer
– Final rendering with tone mapping
• Lighting ON• Use z-buffer test to tone map
only visible fragments
Saarbrücker IT-Dialog Automobilindustrie
Results (Car Interior Part)• Car model contains approx. 500K triangles• Preprocessing of SH coefficients takes about 100
minutes• Full model size with SH coefficients is about 60 MB• Rendering frame rate is about 10 FPS
– For each frame current environment map is projected into SH basis and lighting in every vertex is computed
– Full image tone mapping is applied
Saarbrücker IT-Dialog Automobilindustrie
Results (LCD Panel Part)• One processor on each PC is
busy with sending data to OpenGL
• Another processor computes draft images (40 samples per pixel) of LCD panel and sends them to other PCs
• Draft image display is available almost immediately after each head movement or environment maps rotation, converged image is computed in approx. 2 seconds
Saarbrücker IT-Dialog Automobilindustrie
Conclusions• We proposed efficient global illumination and tone
mapping solutions for CAVE VR applications involving static geometry and dynamic environment lighting
• We successfully applied those solutions to the car interior modeling, efficiently utilizing all CPUs and GPUs resources on the CAVE cluster
• We proposed an accurate algorithm for the LCD panel simulation and rendering based on measured BRDF and emission data.
Saarbrücker IT-Dialog Automobilindustrie
Future work• Use HDR video stream as dynamic environment map
lighting– Trivial to do, we are just waiting for a fish-eye lens suitable for our
HDR camera
• Use HDR display (0.05-3,000 cd/m2) to render the LCD panel with luminance levels similar to real world driving conditions– Display-in-display rendering problem
• Consider recently proposed techniques for all frequency PRT lighting
Saarbrücker IT-Dialog Automobilindustrie
Acknowledgements
• We would like to thank the following people– Michael Arnold (Virtual Reality Center, DaimlerChrysler AG)– Thomas Ganz (Research & Technology Displays and