Shape from Shading Course web page: vision.cis.udel.edu/cv February 26, 2003 Lecture 6.

Shape from Shading

Course web page:vision.cis.udel.edu/cv

February 26, 2003 Lecture 6

Announcements

• Changed HW1 page to a PDF file to eliminate font problems (let me know if hyperlinks don’t work)

• Read about color in Forsyth & Ponce, Chapter 6-6.1, 6.3, 6.5 for Friday

Outline

• Light sources– Exitance– Point sources

• Photometric stereo– Shape from point sources at infinity

• Complications—e.g., interreflections• More kinds of light sources

Light Source Exitance

• Just like radiosity, but generated internally – Foreshortening because off-normal

viewing angles make patch look smaller

Point Light Source

• Model as tiny (radius ²) light-emitting sphere with constant exitance

• Good approximation when light is small relative to distance to viewer (e.g., light bulb, the sun)

• Solid angle: Recall that d! = (dA cos®)/r2

– For sphere, patch area is of circle and normal is always aligned with viewing direction, so d! = ¼²2/r2

• Radiosity scales as 1/r2

Illumination by a Point Source at Infinity

• Consider a distant Lambertian object (so orthographic projection is reasonable) in camera coordinates

• Image brightness at a pixel (based on reflectance equation):

I(x, y) = k ½(x, y) s ¢ n(x, y) where k includes the BRDF (with albedo factored out), the

light’s exitance, and a photometric factor, and s is the direction of the light

Let n(x, y) be

normal at I(x, y) (these are all column vectors)

Normal Information from a Point Source at Infinity

• Suppose k and s are known, and let g(x, y) = ½(x, y) n(x, y) and v = k s, so we can write:

I(x, y) = v ¢ g(x, y)• Not enough information!

– Image brightness constrains the polar angle of the normal at each point on the object surface, but not the azimuth—i.e., we only know that the solution is on a circle

– We do know that where I(x, y) is maximal over the entire object, n(x, y) = s. This is where the highlight is on a specular object

• Additional sources of information – More lights (one per image—in the same image their effects

sum like a single “virtual” light) – Assume normal vector varies smoothly over object

Solving for the Normals with Multiple Point Sources

• Need 3 circles for unique intersection point ! need 3 light sources to solve for g(x, y)

• Formally, we must solve a linear system at each point, which we can write as:

• In Matlab, solve using B = V\g• Albedo is just the length of g(x, y), and

n(x, y) is the result of normalizing it

VB

Photometric Stereo: Example

From Forsyth & Ponce

Inputimages

Recoveredalbedo

Recoverednormals

Normals from Multiple Point Sources at Infinity:

Considerations• Shadows

– When surface patch at (x, y) is occluded,

I(x, y) = 0 and the image brightness equation will be invalid there ! Zero out that

row of V• Intuition about error

– More lights help... (least-squares solution)• So that no point is illuminated by < 3 lights• To reduce effects of noise

– Geometry of lights matters (close together is bad)

Shape from Normals: Getting the Gradient

• Shape typically means depth—the z values. With these we can make a height map like Matlab’s mesh

• Suppose z = f(x, y) and n = (n1, n2, n3)T. By definition, the gradient is:

In these terms, n = (p, q, 1)T, so we can

compute p = n1/n3 and q = n2/n3.

Shape from Gradient

• We can integrate partial derivatives along a path to get the function value at the end of the path

• Simple path: Starting at image origin (0, 0), follow row to x coordinate, then

column to y coordinate for each point

Photometric Stereo Example: Recovered Height Map

From Forsyth & Ponce

Shape Computation:Complications

• Usefulness of photometric stereo for specular objects inversely proportional to magnitude of diffuse component of BRDF– For mirror objects, mitigating cue is

distorted reflection of environment (analogous to camera calibration)

• Interreflections– Light from other objects = light

sources we don’t know about– Ambient illumination approximation can help

—add constant to radiosity everywhere (results in extra term in photometric stereo)

courtesy ofP. Debevec

More Light Source Types

• Spotlight– Point source constrained to a small solid

angle

• Line

– For long lines, radiosity scales as 1/r• Area (e.g., overcast sky)

– For big areas, radiosity is uniform for nearby viewers.

– Shape from shading more difficult—in the worst case, the cosine term disappears and there is no shape information at all