1 CSE152, Spring 2020 Shape from Shading and Photometric Stereo Introduction to Computer Vision CSE 152 Lecture 11 1 CSE152, Spring 2020 Announcements • HW2 posted, due Friday 2/14 – HW as modified (1.b extra credit) • Presidents day (No Class), Mon 2/17 • Midterm, Wed 2/19 – Sample questions are posted to the class web page. 2
26
Embed
Shape from Shading and Photometric StereoTwo Shape-from-X methods that use shading 1.Shape from shading: Single Image, known light direction, known BRDF. 2.Photometric stereo: Single
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
CSE152, Spring 2020
Shape from Shadingand
Photometric Stereo
Introduction to Computer VisionCSE 152
Lecture 11
1
CSE152, Spring 2020
Announcements
• HW2 posted, due Friday 2/14– HW as modified (1.b extra credit)
• Presidents day (No Class), Mon 2/17• Midterm, Wed 2/19
– Sample questions are posted to the class web page.
2
2
CSE152, Spring 2020
MidtermWed, Feb. 19
• In class• Full period• Coverage – everything up to today’s lecture• The primary source of material to understand is
what was presented in class and on HW. If it only in the readings, then it’s very unlikely to be in the midterm.
• “Cheat sheet” – you can prepare a one-sided sheet of notes. It must be handwritten. (After the midterm, save your sheet since you can use the other side for the final).
• No calculators.
3
CSE152, Spring 2020
Incomplete list of topics covered…• Camera models• Factors in producing
– Thoroughly Pitted Surfaces [Koenderink et al 1999]
– Hair, Skin, Cloth
Common Models Arbitrary Reflectance
• Phenomenological[Koenderink, Van Doorn 1996]
• Non-parametric models• Anisotropic• Non-uniform over surface• BRDF Measurement [Dana et
al, 1999], [Marschner ]
11
CSE152, Spring 2020
Lambertian Surface
The intensity (irradiance) I(u,v) of a pixel at (u,v) is:
• a(u,v) is the albedo of the surface projecting to (u,v)• n(u,v) is the direction of the surface normal• s0 is the light source intensity• s is the direction to the light source
ns
a
I(u,v)
^
I (u,v) = a(u,v)n(u,v)• s0s
12
7
CSE152, Spring 2020
Full Phong Reflectance Model
where
ia, im,d, im,s are brightness of ambient, diffuseand specular light sources.
ka, kd, ks,𝛼 parameterize Phong BRDF at a point
13
CSE152, Spring 2020
Shadows cast by a point source• A point that can’t see the source is in shadow• For point sources, two types of shadows: cast
shadows & attached shadows
Cast Shadow
Attached Shadow
14
8
CSE152, Spring 2020
Penumbra and Umbra(soft vs. hard shadow)
15
CSE152, Spring 2020
Hard vs soft shadows for different light source
Hard shadow(point light)
Soft shadow(small square light source)
Z. Dimitry, https://slideplayer.com/slide/5184420/
16
9
CSE152, Spring 2020
Hard vs Soft Shadow
17
CSE152, Spring 2020Figure from “Mutual Illumination,” by D.A. Forsyth and A.P. Zisserman, Proc. CVPR, 1989, copyright 1989 IEEE
At the top, geometry of a gutter with triangular cross-section; below, predicted radiosity solutions, scaled to lie on top of each other, for different albedos of the geometry. When albedo is close to zero, shading follows a local model; when it is close to one, there are substantial reflexes.
– a(u,v) is the albedo of the surface projecting to (u,v).– /𝒏(u,v) is the direction of the surface normal.– s0 is the light source intensity.– ,𝒔 is the direction to the light source.
And– 𝒔 = 𝒔𝟎,𝒔– 𝒃 𝑢, 𝑣 = 𝑎(𝑢, 𝑣)/𝒏(𝑢, 𝑣)
ns
^ ^
a
e(u,v)
32
17
CSE152, Spring 2020
Normal Field – Normals at every location
33
CSE152, Spring 2020
Lambertian Photometric stereo• If the light sources s1, s2, and s3 are known, then we can
recover b from as few as three images. [Silver 80, Woodham81]
• For a single pixel location in 3 gray scale images, we have
[e1 e2 e3 ] = bT[s1 s2 s3 ] where e1, e2, and e3 are measured pixel intensities and we know s1, s2, and s3. We can then compute b at the pixel location by solving a linear system.
• Normal is: = b/|b|, albedo is: a = |b|
[ ][ ] 1321-= 321
T sssb eeen
34
18
CSE152, Spring 2020
Computing b for every pixel• Note: S-1 = [s1 s2 s3 ]-1 only needs to be computed. Not per
pixel.• Slow way: Loop over every pixel and apply
• Better: For an image with n pixels,• Let
where ei,j is the measured image intensity at pixel i in image j, and row i of B is the surface normal scaled by the albedo (b) for pixel i
• So image formation equation is E = BS• Solving for B, we have: B = ES-1
E =
e1,1 e1,2 e1,3e2,1 e2,2 e2,3e3,1 e3,2 e3,3
!
⎡
⎣
⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥
, B =
b1,1 b1,2 b1,3b2,1 b2,2 b2,3b3,1 b3,2 b3,3
!
⎡
⎣
⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥
, S = s1 s2 s3⎡⎣⎢
⎤⎦⎥
[ ][ ] 1321-= 321
T sssb eee
35
CSE152, Spring 2020
What if we have more than 3 Images?Linear Least Squares
[e1 e2 e3…en] =bT[s1 s2 s3…sn]
Rewrite as e = Sb
wheree is n by 1b is 3 by 1S is n by 3
Let the residual ber = e - Sb
Squaring this: r2 = rTr = (e-Sb)T (e-Sb)
= eTe - 2bTSTe + bTSTSb
∇𝒃(r2) = 0 --zero gradient with respect to b is a necessary condition for a minimum, or-2STe+2STSb=0;
Solving for b yields
b= (STS)-1STe
36
19
CSE152, Spring 2020
Plastic Baby Doll: Normal Field
37
CSE152, Spring 2020
Next step:Go from normal field to surface
38
20
CSE152, Spring 2020
Depth recover as solution to partial differential equation
• From estimated n =(nx, ny, nz) at each pixel (x,y), compute the slant p(x,y) and tilt q(x,y)