Advanced Computer Graphics (Spring 2005) COMS 4162, Lecture 15: Ray Tracing/Global Illumination Ravi Ramamoorthi cs4162 Slides.

Advanced Computer Graphics Advanced Computer Graphics (Spring 2005) (Spring 2005)

COMS 4162, Lecture 15: Ray Tracing/Global Illumination

Ravi Ramamoorthi

http://www.cs.columbia.edu/~cs4162

Slides adapted from unit 5 (lectures 21-22) of COMS 4160

To DoTo Do

Enjoy spring break!! If you want, you can work on raytracer/project during

break, but this is not required

Course OutlineCourse Outline

3D Graphics Pipeline

Rendering(Creating, shading images from geometry, lighting, materials)

Modeling(Creating 3D Geometry)

Course OutlineCourse Outline

3D Graphics Pipeline

Rendering(Creating, shading images from geometry, lighting, materials)

Modeling(Creating 3D Geometry)

Unit 1: Foundations of Signal and Image ProcessingUnderstanding the way 2D images are formed and displayed, the important concepts and algorithms, and to build an image processing utility like PhotoshopWeeks 1 – 3. Assignment 1 due Feb 15

Unit 2: Meshes, ModelingWeeks 4 – 6. Ass 2 due Mar 10

Unit 3: Ray TracingWeeks 7 – 9. Ass 3 due Apr 12

This lectureThis lecture

High-level overview and summary of ray tracing / global illumination (from 4160)

After spring break: technical details of writing a ray tracer for assignment 3 and more advanced topics

Chapters 15.10 and 16 are useful background, although they don’t follow treatment here

Effects needed for RealismEffects needed for Realism

(Soft) Shadows

Reflections (Mirrors and Glossy)

Transparency (Water, Glass)

Interreflections (Color Bleeding)

Complex Illumination (Natural, Area Light)

Realistic Materials (Velvet, Paints, Glass)

And many more

Image courtesy Paul Heckbert 1983

OutlineOutline

Ray Tracing

Global Illumination and Rendering Equation

Ray Tracing: HistoryRay Tracing: History

Appel 68

Whitted 80 [recursive ray tracing] Landmark in computer graphics

Lots of work on various geometric primitives

Lots of work on accelerations

Current Research Real-Time raytracing (historically, slow technique) Ray tracing architecture

Ray TracingRay Tracing

Different Approach to Image Synthesis as compared to Hardware pipeline (OpenGL)

Pixel by Pixel instead of Object by Object

Easy to compute shadows/transparency/etc

Demo applet: http://www.cs.berkeley.edu/~efros/java/tracer/tracer.html

Ray Casting

Virtual Viewpoint

Virtual Screen Objects

Ray misses all objects: Pixel colored blackRay intersects object: shade using color, lights, materialsMultiple intersections: Use closest one (as does OpenGL)

Shadows

Virtual Viewpoint

Virtual Screen Objects

Light Source

Shadow ray to light is unblocked: object visibleShadow ray to light is blocked: object in shadow

Shadows: Numerical Issues• Numerical inaccuracy may cause intersection to be below surface (effect exaggerated in figure)

• Causing surface to incorrectly shadow itself• Move a little towards light before shooting shadow ray

Mirror Reflections/Refractions

Virtual Viewpoint

Virtual Screen ObjectsGenerate reflected ray in mirror direction, Get reflections and refractions of objects

Recursive Ray TracingRecursive Ray Tracing

For each pixel Trace Primary Eye Ray, find intersection

Trace Secondary Shadow Ray(s) to all light(s) Color = Visible ? Illumination Model : 0 ;

Trace Reflected Ray Color += reflectivity * Color of reflected ray

Problems with RecursionProblems with Recursion

Reflection rays may be traced forever

Generally, set maximum recursion depth

Same for transmitted rays (take refraction into account)

Turner Whitted 1980

Effects needed for Realism

• (Soft) Shadows

• Reflections (Mirrors and Glossy)

• Transparency (Water, Glass)

• Interreflections (Color Bleeding)

• Complex Illumination (Natural, Area Light)

• Realistic Materials (Velvet, Paints, Glass)

Discussed in this lecture so farNot discussed but possible with distribution ray tracingHard (but not impossible) with ray tracing; radiosity methods

Ray/Object IntersectionsRay/Object Intersections

Heart of Ray Tracer One of the main initial research areas Optimized routines for wide variety of primitives

Various types of info Shadow rays: Intersection/No Intersection Primary rays: Point of intersection, material, normals Texture coordinates

Work out examples in next lecture Triangle, sphere, polygon, general implicit surface

Ray-Tracing Transformed ObjectsRay-Tracing Transformed Objects

We have an optimized ray-sphere test But we want to ray trace an ellipsoid…

Solution: Ellipsoid transforms sphere Apply inverse transform to ray, use ray-sphere Allows for instancing (traffic jam of cars)

Mathematical details worked out in class

AccelerationAcceleration

Testing each object for each ray is slow Fewer Rays

Adaptive sampling, depth control Generalized Rays

Beam tracing, cone tracing, pencil tracing etc. Faster Intersections

Optimized Ray-Object Intersections Fewer Intersections

Acceleration StructuresAcceleration Structures

Bounding boxes (possibly hierarchical) If no intersection bounding box, needn’t check objects

Bounding Box

Ray

Spatial Hierarchies (Oct-trees, kd trees, BSP trees)

Acceleration Structures: GridsAcceleration Structures: Grids

Interactive RaytracingInteractive Raytracing

Ray tracing historically slow

Now viable alternative for complex scenes Key is sublinear complexity with acceleration; need not

process all triangles in scene

Allows many effects hard in hardware

OpenRT project real-time ray tracing (http://www.openrt.de)

Raytracing on Graphics HardwareRaytracing on Graphics Hardware

Modern Programmable Hardware general streaming architecture

Can map various elements of ray tracing

Kernels like eye rays, intersect etc.

In vertex or fragment programs

Convergence between hardware, ray tracing

[Purcell et al. 2002, 2003]

http://graphics.stanford.edu/papers/photongfx

OutlineOutline

Ray Tracing

Global Illumination and Rendering Equation

Global IlluminationGlobal Illumination

Diffuse interreflection, color bleeding [Cornell Box]

Global IlluminationGlobal IlluminationCaustics: Focusing through specular surface

Major research effort in 80s, 90s till today

Reflectance Equation (review)

ir

x

( , ) ( , ) ( , ) ( , , )( )r r e r i i i r iL x L x L x f x n Reflected Light(Output Image)

Emission Incident Light (fromlight source)

BRDF Cosine of Incident angle

Reflectance Equation (review)

ir

x

( , ) ( , ) ( , ) ( , , )( )r r e r i i i r iL x L x L x f x n Reflected Light(Output Image)

Emission Incident Light (fromlight source)

BRDF Cosine of Incident angle

Sum over all light sources

Reflectance Equation (review)

ir

x

( , ) ( , ) ( , ) ( , , ) cosr r e r i i i r iiL x L x L x df x

Reflected Light(Output Image)

Emission Incident Light (fromlight source)

BRDF Cosine of Incident angle

Replace sum with integral

id

Global Illumination

ir

x

( , ) ( , ) ( , , ) cos( , )r r e r i r ir iiL x dL x L x f x

Reflected Light(Output Image)

Emission ReflectedLight (fromsurface)

BRDF Cosine of Incident angle

id

Surfaces (interreflection)

dAx

i x x

Rendering Equation

ir

x

( , ) ( , , ) c( , ) ( , ) ose r i rr r i ir iL x L xL x f x d

Reflected Light(Output Image)

Emission ReflectedLight

BRDF Cosine of Incident angle

id

Surfaces (interreflection)

dAx

UNKNOWN UNKNOWNKNOWN KNOWN KNOWN

Rendering Equation (Kajiya 86)Rendering Equation (Kajiya 86)

Rendering Equation as Integral Equation

Reflected Light(Output Image)

Emission ReflectedLight

BRDF Cosine of Incident angle

UNKNOWN UNKNOWNKNOWN KNOWN KNOWN

( ) ( )( ) ( , )l u e u K u dvl v v

Is a Fredholm Integral Equation of second kind [extensively studied numerically] with canonical form

( , ) ( , , ) c( , ) ( , ) ose r i rr r i ir iL x L xL x f x d

Kernel of equation

Linear Operator Equation

( ) ( )( ) ( , )l u e u K u dvl v v Kernel of equationLight Transport Operator

L E KL Can be discretized to a simple matrix equation[or system of simultaneous linear equations] (L, E are vectors, K is the light transport matrix)

Ray Tracing and extensions– General class numerical Monte Carlo methods– Approximate set of all paths of light in scene

L E KL IL K EL

( )I K EL 1( )I KL E

Binomial Theorem2 3( ...)I KL K K E

2 3 ...E KE K E K EL

Ray Tracing2 3 ...E KE K E K EL

Emission directlyFrom light sources

Direct Illuminationon surfaces

Global Illumination(One bounce indirect)[Mirrors, Refraction]

(Two bounce indirect) [Caustics etc]

Ray Tracing2 3 ...K EE K K EEL

Emission directlyFrom light sources

Direct Illuminationon surfaces

Global Illumination(One bounce indirect)[Mirrors, Refraction]

(Two bounce indirect) [Caustics etc]

OpenGL Shading

Rendering Equation

ir

x

( , ) ( , , ) c( , ) ( , ) ose r i rr r i ir iL x L xL x f x d

Reflected Light(Output Image)

Emission ReflectedLight

BRDF Cosine of Incident angle

id

Surfaces (interreflection)

dAx

UNKNOWN UNKNOWNKNOWN KNOWN KNOWN

i x x

Change of Variables

Integral over angles sometimes insufficient. Write integral in terms of surface radiance only (change of variables)

( , ) ( , ) ( , ) ( , , ) cosr r e r r i i r i iL x L x L x df x

x

x

dA

i

i

i

o

id2

cos

| |o

i

dAd

x x

Change of Variables

Integral over angles sometimes insufficient. Write integral in terms of surface radiance only (change of variables)

( , ) ( , ) ( , ) ( , , ) cosr r e r r i i r i iL x L x L x df x

2

cos

| |o

i

dAd

x x

all visible2

to

cos cos( , ) ( , ) ( , ) ( , , )

| |i o

r r e r r i i r

x x

L x L x L x f xx

dx

A

2

cos cos( , ) ( , )

| |i oG x x G x x

x x

Rendering Equation: Standard Form

Integral over angles sometimes insufficient. Write integral in terms of surface radiance only (change of variables)

Domain integral awkward. Introduce binary visibility fn V

( , ) ( , ) ( , ) ( , , ) cosr r e r r i i r i iL x L x L x df x

2

cos

| |o

i

dAd

x x

all visible2

to

cos cos( , ) ( , ) ( , ) ( , , )

| |i o

r r e r r i i r

x x

L x L x L x f xx

dx

A

2

cos cos( , ) ( , )

| |i oG x x G x x

x x

all surfaces

( , ) ( , ) ( , ) ( , , ) ( , ) ( , )r r e r r

x

i i rL x L x L x f x G x dAx x V x

Same as equation 2.52 Cohen Wallace. It swaps primedAnd unprimed, omits angular args of BRDF, - sign.Same as equation above 16.3 in Shirley, except he has no emission, slightly diff. notation