Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Lighting.

Computer Graphics Inf4/MSc

Computer Graphics

Lecture Notes #16

Image-Based Lighting

Computer Graphics Inf4/MSc

13/11/2007 2

Global Illumination and Image-Based Lighting

• Traditional Computer Graphics involves:

– Modelling with matter: geometry with reflectance properties.

• Image-Based Lighting allows:

– Combination of real and synthetic graphics with consistent illumination, using images as light sources.

– Extension of Environment mapping

Computer Graphics Inf4/MSc

Environment Mapping

• Computing the color according to the reflection vector

• No self-shadowing, no inter-reflection

Computer Graphics Inf4/MSc

13/11/2007 4

Image Based Lighting

• Add models or objects to scenes and allow them to be manipulated in the scenes.

• Modelling with light allows added objects to be illuminated consistent with image existent lighting - photorealistic inclusions.

• We start with the answer by finding the scene illumination.

Computer Graphics Inf4/MSc

13/11/2007 5

Image Based Lighting

• Real Scene

• Goal: place synthetic objects on table

(all pictures P. Debevec 98-99)

Computer Graphics Inf4/MSc

13/11/2007 6

Extracting scene lighting

• Capture illumination using illumination sphere

Computer Graphics Inf4/MSc

13/11/2007 7

Image Based Lighting

Real scene

Computer Graphics Inf4/MSc

13/11/2007 8

Image Based Lightingcaptured illumination field

Computer Graphics Inf4/MSc

13/11/2007 9

Image Based Lighting

Real scene

light based model

local scene

synthetic objects

Computer Graphics Inf4/MSc

13/11/2007 10

Image Based Lighting• Use global illumination - compute effects of

synthetic objects on local scene

local scene (brdf estimated)synthetic objects (brdf known)

light based model

Computer Graphics Inf4/MSc

13/11/2007 11

Image Based Lighting

• Render into the scene

background

Computer Graphics Inf4/MSc

13/11/2007 12

Image Based Lighting

• Render synthetic objects

Computer Graphics Inf4/MSc

13/11/2007 13

Image Based Lighting• Effect of local scene on real scene

Computer Graphics Inf4/MSc

13/11/2007 14

Image Based Lighting• Add differences to image

Computer Graphics Inf4/MSc

Some more results

Computer Graphics Inf4/MSc

Light Probe Images

• Spherical Environment Maps• In most digital images, pixel values

aren’t proportional to the light levels in the scene. – light levels are encoded nonlinearly so they

appear either more correctly or more pleasingly on CRTs

Computer Graphics Inf4/MSc High-Dynamic Range

Photography• Standard digital images typically represent

only a small fraction of the dynamic range—the ratio between the dimmest and brightest regions accurately represented—present in most realworld lighting environments.

• The bright light saturates the pixel colour• Need a high dynamic range image which can

be produced by combining images of different shutter speed

Computer Graphics Inf4/MSc High-Dynamic Range

Photography

Computer Graphics Inf4/MSc

HDR images

Computer Graphics Inf4/MSc

Computer Graphics Inf4/MSc Comparison: Radiance map

versus single image

Computer Graphics Inf4/MSc

Eyes For Relighting

• Extracting environment maps from eyes for use in integrating and relighting scenes

• Apply IBL to the 3D objects added into the image

• http://www.youtube.com/watch?v=MllTSofXt8E

Computer Graphics Inf4/MSc

Shape from Shading

• Obtaining the 3D shape from an image• Assuming Lambertian reflection• Can recover the normal vector if we cast the

light from 3 different directions

zzyyxx nLnLnL nL

LnL, )cos(

Object

N

L

Example: sphere(lit from left)

Computer Graphics Inf4/MSc

• Integrate the normals over the surface to compute the 3D surface

• We can also estimate the colour information and use this for texture

Computer Graphics Inf4/MSc

Surface inserted into new scenes• lit according to the global illumination of that scene.• This illumination can of course come from eyes!

Computer Graphics Inf4/MSc

Acquiring the Reflectance Field of a Human Face

• Simulating reflectance on human faces requires a lot of effort and computation– Need to acquire the reflectance model – Subsurface scattering

• Instead of simulating the global illumination, we can simply illuminate the face by real-lights from different directions

• Creating images of arbitrary light conditions by combining the captured data

Computer Graphics Inf4/MSc

The Light Stage:60-second exposure

Computer Graphics Inf4/MSc Light Stage Data

Original Resolution:

6432

Original Resolution:

6432

Lighting through image recombination: Haeberli ‘92, Nimeroff ‘94, Wong ‘97Lighting through image recombination: Haeberli ‘92, Nimeroff ‘94, Wong ‘97

Computer Graphics Inf4/MSc Reflectance Functions

Computer Graphics Inf4/MSc

Lighting Reflectance Functions

normalized light map

normalized light map

reflectance function

reflectance function

lighting productlighting product

rendered pixel

rendered pixel

1

Computer Graphics Inf4/MSc

13/11/2007 31

References

• Rendering Synthetic Objects into Real Scenes, SIGGRAPH 98

• Acquiring the Reflectance Field of a Human Face, SIGGRAPH 2000 http://www.debevec.org

• Eyes For Relighting, Nishino SIGGRAPH 2004