Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams mages/Demo from Steve Seitz, Wikipedia
Mar 31, 2015
Photometric Image Formation
CSE 559: Computer VisionGuest Lecturer: Austin Abrams
Images/Demo from Steve Seitz, Wikipedia
How are images made?
• One half: geometric vision– “how the pixel projected onto the image”
• Today: photometric vision (aka radiometric)– “how the pixel got its color”
Vision and Graphics
Properties of a sceneImage
Computer Graphics
Vision
Image Formation Approach
• Come up with a model for how the scene was created
• Given images, find the most likely properties that fit that model
Diffuse Surfaces
Brightness of a pixel depends on:• object color• lighting direction• surface normal
But NOT view direction!
Lambertian Cosine Law
• The intensity of an observed diffuse object is proportional to the cosine of the angle between the normal and lighting direction
= ρ L N
I = ρ cos θ
= ρ |L||N| cos θL Nθ
=
L N = L N
= x
I = ρ L N
Recovering Albedo and Normals
• Can you decompose a single image into its albedo and normal images?
=
x
x
x
Photometric Stereo
• Given multiple images taken with varying illumination, recover albedo and normals.– take pictures in dark room with varying
illumination.– estimate lighting directions L.– recover albedo and normals.
Side note 1: How to get the lighting direction?
• Put a shiny sphere in the scene• Sphere’s geometry (normals) are known• Find specular highlight
Side-note 2: Why “Stereo”?
Surface normals provide constraints on depth differences
Photometric Stereo
• If L is known, and albedo is grayscale this is a linear problem.
I = ρ(L N) = ρ (Lx Nx + Ly Ny + Lz Nz ) = Lx Nxρ + Ly Nyρ + Lz Nzρ = Lx a + Ly b + Lz c
Lx1 Ly1 Lz1
Lx2 Ly2 Lz2
Lx3 Ly3 Lz3
…
Lxn Lyn Lzn
I1I2I3…In
abc
=
I = ρ(L N) = Lx a + Ly b + Lz c
Then:ρ = sqrt(a2 + b2 + c2)N = (a,b,c) / ρ
For each pixel:
Demo
When does this model fail?
I ≠ ρ (L N)
Attached shadows
I = ρ max(L N, 0)
L N > 0
L N = 0
L N < 0
Cast Shadows, Ambient Light
I = ρ (S L N + a) S = 0 or 1
Radiometric Camera Calibration
• Pixel intensities are usually not proportional to the energy that hit the CCD
RAW image Published image
Radiometric Camera Calibration
f
RAW
Published
Radiometric Camera Calibration
Observed = f(RAW)
(Grossberg and Nayar)
f -1 (Observed) = RAW
Radiometric Camera Calibration
• How do you model f -1?
f -1(x) = xγ
f -1(x) = c0 + c1x + c2x2 + c3x3 + …
f -1(x) = f0(x) + f1(x) c1 + f2(x)c2 + …
mean camera curve basis camera curves
Radiometric Camera Calibration
I = f (ρ (S L N + a))
Adding exposure:
I = f (e ρ (S L N + a))
Heliometric StereoGiven lots of images from a stable webcam,
use lighting from the sun to recover:
I = f (e ρ (S L N + a))
Heliometric Stereo
Heliometric Stereo
Heliometric Stereo