INFORMATIK INFORMATIK An An E E fficient Spatio-Temporal fficient Spatio-Temporal Architecture for Animation Architecture for Animation Rendering Rendering Vlastimil Havran Vlastimil Havran , Cyrille Damez, Karol , Cyrille Damez, Karol Myszkowski, and Hans-Peter Seidel Myszkowski, and Hans-Peter Seidel Max-Planck-Institut f Max-Planck-Institut f ür Informatik, ür Informatik, Saarbrücken, Germany Saarbrücken, Germany
27
Embed
An E fficient Spatio-Temporal Architecture for Animation Rendering
An E fficient Spatio-Temporal Architecture for Animation Rendering. Vlastimil Havran , Cyrille Damez, Karol Myszkowski, and Hans-Peter Seidel Max-Planck-Institut f ür Informatik, Saarbrücken, Germany. Motivation. - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
INFORMATIKINFORMATIK
An An EEfficient Spatio-Temporal fficient Spatio-Temporal Architecture for Animation RenderingArchitecture for Animation Rendering
Vlastimil HavranVlastimil Havran, Cyrille Damez, , Cyrille Damez, Karol Myszkowski, and Hans-Peter SeidelKarol Myszkowski, and Hans-Peter Seidel
Max-Planck-Institut fMax-Planck-Institut für Informatik, ür Informatik, Saarbrücken, GermanySaarbrücken, Germany
Vlastimil HavranVlastimil Havran, Cyrille Damez, , Cyrille Damez, Karol Myszkowski, and Hans-Peter SeidelKarol Myszkowski, and Hans-Peter Seidel
Max-Planck-Institut fMax-Planck-Institut für Informatik, ür Informatik, Saarbrücken, GermanySaarbrücken, Germany
INFORMATIKINFORMATIKMotivationMotivation
In the traditional approach to rendering of high In the traditional approach to rendering of high quality animation sequences every frame is quality animation sequences every frame is considered separately.considered separately.
• The temporal coherence is poorly exploited:The temporal coherence is poorly exploited:
redundant computations.redundant computations.
• The visual sensitivity to temporal detail cannot be properly The visual sensitivity to temporal detail cannot be properly accounted for:accounted for:
too conservative stopping conditions,too conservative stopping conditions,
temporal aliasing.temporal aliasing.
In the traditional approach to rendering of high In the traditional approach to rendering of high quality animation sequences every frame is quality animation sequences every frame is considered separately.considered separately.
• The temporal coherence is poorly exploited:The temporal coherence is poorly exploited:
redundant computations.redundant computations.
• The visual sensitivity to temporal detail cannot be properly The visual sensitivity to temporal detail cannot be properly accounted for:accounted for:
too conservative stopping conditions,too conservative stopping conditions,
temporal aliasing.temporal aliasing.
INFORMATIKINFORMATIKGoalGoal
Developing an architecture for efficient rendering Developing an architecture for efficient rendering of high-quality animations which better exploits the of high-quality animations which better exploits the spatio-temporal coherence between frames:spatio-temporal coherence between frames:• Visibility: Multi-frame ray tracingVisibility: Multi-frame ray tracing
• Global illumination: Bi-directional Path Tracing extended to Global illumination: Bi-directional Path Tracing extended to re-using samples between framesre-using samples between frames
• Texturing and shading: sharing information between Texturing and shading: sharing information between framesframes
• Motion blur: conservative computation in 2D image spaceMotion blur: conservative computation in 2D image space
• Cache use: coherent patterns of access to data structures Cache use: coherent patterns of access to data structures in memory (along motion compensation trajectories)in memory (along motion compensation trajectories)
Developing an architecture for efficient rendering Developing an architecture for efficient rendering of high-quality animations which better exploits the of high-quality animations which better exploits the spatio-temporal coherence between frames:spatio-temporal coherence between frames:• Visibility: Multi-frame ray tracingVisibility: Multi-frame ray tracing
• Global illumination: Bi-directional Path Tracing extended to Global illumination: Bi-directional Path Tracing extended to re-using samples between framesre-using samples between frames
• Texturing and shading: sharing information between Texturing and shading: sharing information between framesframes
• Motion blur: conservative computation in 2D image spaceMotion blur: conservative computation in 2D image space
• Cache use: coherent patterns of access to data structures Cache use: coherent patterns of access to data structures in memory (along motion compensation trajectories)in memory (along motion compensation trajectories)
• Industrial solutions such as PhotorealisticRenderman, Industrial solutions such as PhotorealisticRenderman, Maya Rendering System, or SoftimageMaya Rendering System, or Softimage
– postprocessing of animation to decrease temporal postprocessing of animation to decrease temporal aliasing aliasing
• Industrial solutions such as PhotorealisticRenderman, Industrial solutions such as PhotorealisticRenderman, Maya Rendering System, or SoftimageMaya Rendering System, or Softimage
– Static vs. dynamic objects [Besuievsky and Pueyo 01, Static vs. dynamic objects [Besuievsky and Pueyo 01, Lext and Moller 01, and Wald et al. 02]Lext and Moller 01, and Wald et al. 02]
Temporal Coherence in Ray TracingTemporal Coherence in Ray Tracing• Coherence of shaded pixelsCoherence of shaded pixels
– Interpolation between frames [Maisel and Hegron 92]Interpolation between frames [Maisel and Hegron 92]
– Reprojection [Adelson and Hughes 95]Reprojection [Adelson and Hughes 95]
– 4D radiance interpolants [Bala et al. 99]4D radiance interpolants [Bala et al. 99]
• Coherence in acceleration data structuresCoherence in acceleration data structures
– Static vs. dynamic objects [Besuievsky and Pueyo 01, Static vs. dynamic objects [Besuievsky and Pueyo 01, Lext and Moller 01, and Wald et al. 02]Lext and Moller 01, and Wald et al. 02]
INFORMATIKINFORMATIKPrevious WorkPrevious Work
Temporal Coherence in Global IlluminationTemporal Coherence in Global Illumination• Coherence of illumination samplesCoherence of illumination samples
– Render cache [Walter et al. 99]Render cache [Walter et al. 99]
– Shading cache [Tole et al. 02]Shading cache [Tole et al. 02]
– Reusing photon hit points [Myszkowski et al. 01]Reusing photon hit points [Myszkowski et al. 01]
• Coherence in visibility computationCoherence in visibility computation
– Global lines [Besuievsky and Pueyo 01]Global lines [Besuievsky and Pueyo 01]
• Coherence in generation of random numbersCoherence in generation of random numbers
– Random number sequence associated with each light Random number sequence associated with each light transport path [Lafortune 96, Jensen 01, Wald et al. 02]transport path [Lafortune 96, Jensen 01, Wald et al. 02]
Temporal Coherence in Global IlluminationTemporal Coherence in Global Illumination• Coherence of illumination samplesCoherence of illumination samples
– Render cache [Walter et al. 99]Render cache [Walter et al. 99]
– Shading cache [Tole et al. 02]Shading cache [Tole et al. 02]
– Reusing photon hit points [Myszkowski et al. 01]Reusing photon hit points [Myszkowski et al. 01]
• Coherence in visibility computationCoherence in visibility computation
– Global lines [Besuievsky and Pueyo 01]Global lines [Besuievsky and Pueyo 01]
• Coherence in generation of random numbersCoherence in generation of random numbers
– Random number sequence associated with each light Random number sequence associated with each light transport path [Lafortune 96, Jensen 01, Wald et al. 02]transport path [Lafortune 96, Jensen 01, Wald et al. 02]
INFORMATIKINFORMATIKMulti-Frame Ray TracingMulti-Frame Ray Tracing• Aggregate queries Aggregate queries
– Single ray geometrySingle ray geometry
– Results possible for all frames in Results possible for all frames in [f[f(i-R-S)’(i-R-S)’ff(i+R)(i+R)]]
• Two types of visibility queriesTwo types of visibility queries
– Ray shootingRay shooting
– Visibility between two pointsVisibility between two points
• Three settings of visibility queriesThree settings of visibility queries
– Exact result for single frame Exact result for single frame ff(i)(i)
– Exact for single frame Exact for single frame ff(i) (i) + validity for all other frames+ validity for all other frames
– Exact results for all frames in Exact results for all frames in [f[f(i-R)’(i-R)’ff(i+R)(i+R)]]
• Aggregate queries Aggregate queries
– Single ray geometrySingle ray geometry
– Results possible for all frames in Results possible for all frames in [f[f(i-R-S)’(i-R-S)’ff(i+R)(i+R)]]
• Two types of visibility queriesTwo types of visibility queries
– Ray shootingRay shooting
– Visibility between two pointsVisibility between two points
• Three settings of visibility queriesThree settings of visibility queries
– Exact result for single frame Exact result for single frame ff(i)(i)
– Exact for single frame Exact for single frame ff(i) (i) + validity for all other frames+ validity for all other frames
– Exact results for all frames in Exact results for all frames in [f[f(i-R)’(i-R)’ff(i+R)(i+R)]]
INFORMATIKINFORMATIKMulti-Frame Ray TracingMulti-Frame Ray Tracing
Additional techniques usedAdditional techniques used• Mail boxes (cache) for ray transforms, objects, and kd-treesMail boxes (cache) for ray transforms, objects, and kd-trees
• Frame masks for inserted kd-trees and shadow cacheFrame masks for inserted kd-trees and shadow cache
Additional techniques usedAdditional techniques used• Mail boxes (cache) for ray transforms, objects, and kd-treesMail boxes (cache) for ray transforms, objects, and kd-trees
• Frame masks for inserted kd-trees and shadow cacheFrame masks for inserted kd-trees and shadow cache
Construction of Spatial Data StructureConstruction of Spatial Data Structure• Instantiation of dynamic objects for range of framesInstantiation of dynamic objects for range of frames
• Static Static versus versus dynamicdynamic objects separation objects separation
• Construction of global kd-tree over static objectsConstruction of global kd-tree over static objects
• Hierarchical clustering over dynamic objectsHierarchical clustering over dynamic objects
• Construction of kd-trees for clusters of objectsConstruction of kd-trees for clusters of objects
• Insertion of kd-trees for clusters into global kd-tree leavesInsertion of kd-trees for clusters into global kd-tree leaves
• Refinement of global kd-tree leaves + empty space cutting Refinement of global kd-tree leaves + empty space cutting off (efficiency improvement methods)off (efficiency improvement methods)
Construction of Spatial Data StructureConstruction of Spatial Data Structure• Instantiation of dynamic objects for range of framesInstantiation of dynamic objects for range of frames
• Static Static versus versus dynamicdynamic objects separation objects separation
• Construction of global kd-tree over static objectsConstruction of global kd-tree over static objects
• Hierarchical clustering over dynamic objectsHierarchical clustering over dynamic objects
• Construction of kd-trees for clusters of objectsConstruction of kd-trees for clusters of objects
• Insertion of kd-trees for clusters into global kd-tree leavesInsertion of kd-trees for clusters into global kd-tree leaves
• Refinement of global kd-tree leaves + empty space cutting Refinement of global kd-tree leaves + empty space cutting off (efficiency improvement methods)off (efficiency improvement methods)
– The same sample point is considered for multiple framesThe same sample point is considered for multiple frames
In other frame-by-frame architectures the motion of In other frame-by-frame architectures the motion of objects must be computed explicitely by additional objects must be computed explicitely by additional samples.samples.
– Temporal changes in shading are properly accounted forTemporal changes in shading are properly accounted for
Difficult in other architecturesDifficult in other architectures
• EfficiencyEfficiency
– 2D computation2D computation
• Accuracy & Quality Accuracy & Quality
– The same sample point is considered for multiple framesThe same sample point is considered for multiple frames
In other frame-by-frame architectures the motion of In other frame-by-frame architectures the motion of objects must be computed explicitely by additional objects must be computed explicitely by additional samples.samples.
– Temporal changes in shading are properly accounted forTemporal changes in shading are properly accounted for
Difficult in other architecturesDifficult in other architectures
• EfficiencyEfficiency
– 2D computation2D computation
INFORMATIKINFORMATIKResultsResults
Speedup for bi-directional path Speedup for bi-directional path tracingtracing• Moving camera, moving objects: 7.7Moving camera, moving objects: 7.7
Proportion of native samples 2.4 - 4.7 %Proportion of native samples 2.4 - 4.7 %Proportion of native samples 2.4 - 4.7 %Proportion of native samples 2.4 - 4.7 %
Cost of native samples (profiler) 44 - 64 % Cost of native samples (profiler) 44 - 64 % of the whole computation time. of the whole computation time.Cost of native samples (profiler) 44 - 64 % Cost of native samples (profiler) 44 - 64 % of the whole computation time. of the whole computation time.
Disc caching overhead 10 % slowdown Disc caching overhead 10 % slowdown (for 1% of main memory used)(for 1% of main memory used)Disc caching overhead 10 % slowdown Disc caching overhead 10 % slowdown (for 1% of main memory used)(for 1% of main memory used)
Time per frame 240 – 415 sec.Time per frame 240 – 415 sec.Time per frame 240 – 415 sec.Time per frame 240 – 415 sec.
Time per frame 120 sec. Time per frame 150 sec.Time per frame 120 sec. Time per frame 150 sec.Time per frame 120 sec. Time per frame 150 sec.Time per frame 120 sec. Time per frame 150 sec.
INFORMATIKINFORMATIKVideosVideos
INFORMATIKINFORMATIKConclusionsConclusions• We presented an efficient architecture for rendering of high-We presented an efficient architecture for rendering of high-
quality animationsquality animations
• In our architecture path tracing, texturing and shading, motion In our architecture path tracing, texturing and shading, motion blur can be efficiently computed.blur can be efficiently computed.
• Our architecture is efficient in particular for scenes in which a Our architecture is efficient in particular for scenes in which a limited number of objects moves locally in the scene and limited number of objects moves locally in the scene and camera motion is slow. camera motion is slow.
– For more complex motion scenario the animation segment For more complex motion scenario the animation segment length can be reduced which in the limit may boil down to length can be reduced which in the limit may boil down to the traditional frame-by-frame computation.the traditional frame-by-frame computation.
• Data structures handling dynamic objects require additional Data structures handling dynamic objects require additional memory, which is acceptable on modern computers. memory, which is acceptable on modern computers.
• The memory overhead involved in storing multiple images is The memory overhead involved in storing multiple images is negligible due to efficient buffering.negligible due to efficient buffering.
• We presented an efficient architecture for rendering of high-We presented an efficient architecture for rendering of high-quality animationsquality animations
• In our architecture path tracing, texturing and shading, motion In our architecture path tracing, texturing and shading, motion blur can be efficiently computed.blur can be efficiently computed.
• Our architecture is efficient in particular for scenes in which a Our architecture is efficient in particular for scenes in which a limited number of objects moves locally in the scene and limited number of objects moves locally in the scene and camera motion is slow. camera motion is slow.
– For more complex motion scenario the animation segment For more complex motion scenario the animation segment length can be reduced which in the limit may boil down to length can be reduced which in the limit may boil down to the traditional frame-by-frame computation.the traditional frame-by-frame computation.
• Data structures handling dynamic objects require additional Data structures handling dynamic objects require additional memory, which is acceptable on modern computers. memory, which is acceptable on modern computers.
• The memory overhead involved in storing multiple images is The memory overhead involved in storing multiple images is negligible due to efficient buffering.negligible due to efficient buffering.
INFORMATIKINFORMATIKFuture WorkFuture Work
• Reusing samples for many pixels in the same frame Reusing samples for many pixels in the same frame as suggested in [Bekaert et al. 2002, Szirmay-Kalos as suggested in [Bekaert et al. 2002, Szirmay-Kalos 2002]2002]
• Improving the efficiency of visibility tests for Improving the efficiency of visibility tests for reprojected samples using a shaft culling approach.reprojected samples using a shaft culling approach.
• Skipping some visibility tests based on the spatio-Skipping some visibility tests based on the spatio-temporal coherence of neighboring samples as temporal coherence of neighboring samples as implemented in the Maya Rendering System [Sung et implemented in the Maya Rendering System [Sung et al. 2002]al. 2002]
• Reusing samples for many pixels in the same frame Reusing samples for many pixels in the same frame as suggested in [Bekaert et al. 2002, Szirmay-Kalos as suggested in [Bekaert et al. 2002, Szirmay-Kalos 2002]2002]
• Improving the efficiency of visibility tests for Improving the efficiency of visibility tests for reprojected samples using a shaft culling approach.reprojected samples using a shaft culling approach.
• Skipping some visibility tests based on the spatio-Skipping some visibility tests based on the spatio-temporal coherence of neighboring samples as temporal coherence of neighboring samples as implemented in the Maya Rendering System [Sung et implemented in the Maya Rendering System [Sung et al. 2002]al. 2002]
INFORMATIKINFORMATIKThanks!Thanks!
• Polina Kondratieva for rewriting RenderMan ShadersPolina Kondratieva for rewriting RenderMan Shaders
• Markus Weber for help with preparing the scenesMarkus Weber for help with preparing the scenes
• Partial support of IST-2001-34744 Partial support of IST-2001-34744 project RealReflectproject RealReflect
• Polina Kondratieva for rewriting RenderMan ShadersPolina Kondratieva for rewriting RenderMan Shaders
• Markus Weber for help with preparing the scenesMarkus Weber for help with preparing the scenes
• Partial support of IST-2001-34744 Partial support of IST-2001-34744 project RealReflectproject RealReflect
• All anonymous reviewers for their commentsAll anonymous reviewers for their comments• All anonymous reviewers for their commentsAll anonymous reviewers for their comments
Rendering animations – Rendering animations – ray tracing with shadersray tracing with shaders
Speedup ray tracingSpeedup ray tracing• Moving camera, moving objects: 2.6Moving camera, moving objects: 2.6
Speedup ray tracingSpeedup ray tracing• Moving camera, moving objects: 2.6Moving camera, moving objects: 2.6
( deterministic reflections must be recomputed! )( deterministic reflections must be recomputed! ) ( deterministic reflections must be recomputed! )( deterministic reflections must be recomputed! )
Time per frame 770 sec.Time per frame 770 sec.Time per frame 770 sec.Time per frame 770 sec.