CS 691 Computational Photography Instructor: Gianfranco Doretto 3D to 2D Projections.

CS 691 Computational Photography

Instructor: Gianfranco Doretto3D to 2D Projections

Pinhole camera model

• Pinhole model:– Captures pencil of rays – all rays through a single point– The point is called Center of Projection (COP)– The image is formed on the Image Plane– Effective focal length f is distance from COP to Image

Plane

Point of observation

Dimensionality Reduction Machine (3D to 2D)

3D world 2D image

What have we lost?• Angles• Distances (lengths)

Projection can be tricky…

Slide source: Seitz

Projection can be tricky…

Slide source: Seitz

Projective Geometry

What is lost?• Length

Which is closer?

Who is taller?

Lengths can’t be trusted...

B’

C’

A’

…but humans adopt!

http://www.michaelbach.de/ot/sze_muelue/index.html

Müller-Lyer Illusion

We don’t make measurements in the image plane

Projective Geometry

What is lost?• Length• Angles

Perpendicular?

Parallel?

Parallel lines aren’t…

Projective Geometry

What is preserved?• Straight lines are still straight

Vanishing points and lines

Parallel lines in the world intersect in the image at a “vanishing point”

Vanishing points and lines

oVanishing Point oVanishing Point

Vanishing Line

Vanishing points and lines

Vanishing point

Vanishing line

Vanishing point

Vertical vanishing point

(at infinity)

Credit: Criminisi

Vanishing points and lines

Photo from online Tate collection

Note on estimating vanishing points

Use multiple lines for better accuracy… but lines will not intersect at exactly the same point in practice

One solution: take mean of intersecting pairs… bad idea!

Instead, minimize angular differences

Vanishing objects

Modeling projection

• The coordinate system– We will use the pin-hole model as an approximation– Put the optical center (Center Of Projection) at the origin– Put the image plane (Projection Plane) in front of the COP

• Why?

– The camera looks down the negative z axis• we need this if we want right-handed-coordinates

–

Modeling projection

• Projection equations– Compute intersection with PP of ray from (x,y,z) to COP– Derived using similar triangles (on board)

• We get the projection by throwing out the last coordinate:

Homogeneous coordinates• Is this a linear transformation?

Trick: add one more coordinate:

homogeneous image coordinates

homogeneous scene coordinates

Converting from homogeneous coordinates

• no—division by z is nonlinear

Homogeneous coordinates

Invariant to scaling

Point in Cartesian is ray in Homogeneous

wywx

kwkykwkx

kw

ky

kx

w

y

x

k

Homogeneous Coordinates

Cartesian Coordinates

Perspective Projection• Projection is a matrix multiply using homogeneous coordinates:

divide by third coordinate

This is known as perspective projection• The matrix is the projection matrix• Can also formulate as a 4x4

divide by fourth coordinate

Orthographic Projection

• Special case of perspective projection– Distance from the COP to the PP is infinite

– Also called “parallel projection”– What’s the projection matrix?

Image World

Scaled Orthographic Projection

• Special case of perspective projection– Object dimensions are small compared to

distance to camera

– Also called “weak perspective”– What’s the projection matrix?

1000

000

000

z

y

x

s

f

f

w

v

u

Spherical Projection

• What if PP is spherical with center at COP?• In spherical coordinates, projection is trivial:• (θ,Φ) = (θ,Φ,d)• Note: doesn’t depend on focal length!

Projection matrix

XtRKx x: Image Coordinates: w(u,v,1)K: Intrinsic Matrix (3x3)R: Rotation (3x3) t: Translation (3x1)X: World Coordinates: (X,Y,Z,1)

Ow

iw

kw

jw

R,T

X0IKx

10100

000

000

1z

y

x

f

f

v

u

w

K

Projection matrix

Intrinsic Assumptions• Unit aspect ratio• Principal point at (0,0)• No skew

Extrinsic Assumptions• No rotation• Camera at (0,0,0)

Remove assumption: known optical center

X0IKx

10100

00

00

10

0

z

y

x

vf

uf

v

u

w

Intrinsic Assumptions• Unit aspect ratio• No skew

Extrinsic Assumptions• No rotation• Camera at (0,0,0)

Remove assumption: square pixels

X0IKx

10100

00

00

10

0

z

y

x

v

u

v

u

w

Intrinsic Assumptions• No skew

Extrinsic Assumptions• No rotation• Camera at (0,0,0)

Remove assumption: non-skewed pixels

X0IKx

10100

00

0

10

0

z

y

x

v

us

v

u

w

Intrinsic Assumptions Extrinsic Assumptions• No rotation• Camera at (0,0,0)

Note: different books use different notation for parameters

Oriented and Translated Camera

Ow

iw

kw

jw

t

R

Allow camera translation

XtIKx

1100

010

001

100

0

0

10

0

z

y

x

t

t

t

v

u

v

u

w

z

y

x

Intrinsic Assumptions Extrinsic Assumptions• No rotation

3D Rotation of Points

Rotation around the coordinate axes, counter-clockwise:

100

0cossin

0sincos

)(

cos0sin

010

sin0cos

)(

cossin0

sincos0

001

)(

z

y

x

R

R

R

p

p’

y

z

Slide Credit: Saverese

Allow camera rotation

XtRKx

1100

0

1 333231

232221

131211

0

0

z

y

x

trrr

trrr

trrr

v

us

v

u

w

z

y

x

Degrees of freedom

XtRKx

1100

0

1 333231

232221

131211

0

0

z

y

x

trrr

trrr

trrr

v

us

v

u

w

z

y

x

5 6

Vanishing Point = Projection from Infinity

R

R

R

z

y

x

z

y

x

z

y

x

KpKRptRKp

0

R

R

R

z

y

x

vf

uf

v

u

w

100

0

0

10

0 0uz

fxu

R

R

0vz

fyv

R

R

Lens Flaws

Lens Flaws: Chromatic Aberration

• Dispersion: wavelength-dependent refractive index– (enables prism to spread white light beam into rainbow)

• Modifies ray-bending and lens focal length: f(λ)

• color fringes near edges of image• Corrections: add ‘doublet’

lens of flint glass, etc.

Chromatic Aberration

Near Lens Center Near Lens Outer Edge

Radial Distortion (e.g. ‘Barrel’ and ‘pin-cushion’)

straight lines curve around the image center

Radial Distortion

• Radial distortion of the image– Caused by imperfect lenses– Deviations are most noticeable for rays

that pass through the edge of the lens

No distortion Pin cushion Barrel

Radial Distortion

Slide Credits

• This set of sides also contains contributionskindly made available by the following authors– Alexei Efros– Stephen E. Palmer– Steve Seitz– Derek Hoiem– David Forsyth– George Bebis– Silvio Savarese