Image Formation I Chapter 1 (Forsyth&Ponce) Camerasgerig/CS6320-S2013/Materials/CS... · Chapter 1 (Forsyth&Ponce) Cameras ... Affine projection models: Orthographic projection ...

Image Formation I Chapter 1 (Forsyth&Ponce)

Cameras

Guido Gerig

CS 6320 Spring 2013

Acknowledgements:

• Slides used from Prof. Trevor Darrell,

(http://www.eecs.berkeley.edu/~trevor/CS280.html)

• Some slides modified from Marc Pollefeys, UNC Chapel Hill. Other

slides and illustrations from J. Ponce, addendum to course book.

GEOMETRIC CAMERA MODELS

• The Intrinsic Parameters of a Camera

• The Extrinsic Parameters of a Camera

• The General Form of the Perspective Projection

Equation

• Line Geometry

Reading: Chapter 1.

Images are two-dimensional patterns of brightness values.

They are formed by the projection of 3D objects.

Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of

Naval Personnel. Reprinted by Dover Publications, Inc., 1969.

Animal eye:

a looonnng time ago.

Pinhole perspective projection: Brunelleschi, XVth Century.

Camera obscura: XVIth Century.

Photographic camera:

Niepce, 1816.

Camera model

Relation between pixels and rays in space

?

Camera obscura + lens

The camera obscura (Latin for 'dark room') is an

optical device that projects an image of its

surroundings on a screen (source Wikipedia).

http://en.wikipedia.org/wiki/Image

Limits for pinhole cameras

Physical parameters of image formation

• Geometric – Type of projection – Camera pose

• Photometric – Type, direction, intensity of

light reaching sensor – Surfaces’ reflectance

properties • Optical

– Sensor’s lens type – focal length, field of view,

aperture • Sensor

– sampling, etc.

Physical parameters of image formation

• Geometric – Type of projection – Camera pose

• Optical – Sensor’s lens type – focal length, field of view, aperture

• Photometric – Type, direction, intensity of light reaching sensor – Surfaces’ reflectance properties

• Sensor – sampling, etc.

Perspective and art

• Use of correct perspective projection indicated in 1st century B.C. frescoes

• Skill resurfaces in Renaissance: artists develop systematic methods to determine perspective projection (around 1480-1515)

Durer, 1525 Raphael K. Grauman

Perspective projection equations

• 3d world mapped to 2d projection in image plane

Forsyth and Ponce

Camera frame

Image plane

Optical axis

Focal length

Scene / world points

Scene point Image coordinates

‘’ ‘ ’

Affine projection models: Weak perspective projection

0

'where

'

'

z

fm

myy

mxx

is the magnification.

When the scene relief is small compared to its distance from the

Camera, m can be taken constant: weak perspective projection.

Affine projection models: Orthographic projection

yy

xx

'

' When the camera is at a

(roughly constant) distance

from the scene, take m=1.

Homogeneous coordinates

Is this a linear transformation?

Trick: add one more coordinate:

homogeneous image coordinates

homogeneous scene coordinates

Converting from homogeneous coordinates

• no—division by z is nonlinear

Slide by Steve Seitz

Perspective Projection Matrix

divide by the third coordinate to convert back to non-homogeneous coordinates

• Projection is a matrix multiplication using homogeneous coordinates:

'/1

0'/100

0010

0001

fz

y

x

z

y

x

f

),()','( '' yxz

yf

z

xf

Slide by Steve Seitz

Complete mapping from world points to image pixel positions?

Points at infinity, vanishing points

Points from infinity represent

rays into camera which are

close to the optical axis.

Image source: wikipedia

Perspective projection & calibration

• Perspective equations so far in terms of camera’s reference frame….

• Camera’s intrinsic and extrinsic parameters needed to calibrate geometry.

Camera frame

K. Grauman

The CCD camera


Camera frame

Intrinsic: Image coordinates relative to camera Pixel coordinates

Extrinsic: Camera frame World frame

World frame

World to camera coord. trans. matrix

(4x4)

Perspective projection matrix

(3x4)

Camera to pixel coord.

trans. matrix (3x3)

= 2D

point (3x1)

3D point (4x1)

K. Grauman

Intrinsic parameters: from idealized world coordinates to pixel values

Forsyth&Ponce

z

yfy

z

xfx

'

'Perspective projection: Worls point and pixels in camera coordinates

W. Freeman

Intrinsic parameters

ion magnificat represents

* with ,

* with ,

kfz

yv

kfz

xu

But “pixels” are in some arbitrary spatial units, which can be described by #pixels per mm.

W. Freeman

Pixel dimensions: 1/k*1/k

k: cells/mm, units [mm-1]


ionsmagnificatrepresent ,

*with ,

*with ,

lfz

yv

kfz

xu

Maybe pixels are not square and have different horizontal and vertical dimensions. (u,v): pixel numbers, x,y,z): World point in camera coordinates.

W. Freeman

Pixel dimensions: 1/k*1/l

k,l: cells/mm, units [mm-1]


0

0

vz

yv

uz

xu

We don’t know the origin of our camera pixel coordinates: (u0,v0) represent intersection of optical axis with image plane: (u0, v0): image center in pixel coordinates.

W. Freeman


0

0

)sin(

)cot(

vz

yv

uz

y

z

xu

May be skew between camera pixel axes due to manufacturing errors and eventually line-by-line readouts.

v

u

v

u

vuvuu

vvvv

)cot()cos(

)sin( )sin(

W. Freeman

pz

p C (K)

1

Intrinsic parameters, homogeneous coordinates

0

0

)sin(

)cot(

vz

yv

uz

y

z

xu

10

0

0

100

)sin(0

)cot(

1

1

0

0

z

y

x

v

u

zv

u

Using homogenous coordinates, we can write this as:

or:

World point in camera-based coordinates

In pixels

W. Freeman


Camera frame

Intrinsic: Image coordinates relative to camera Pixel coordinates

Extrinsic: Camera frame World frame

World frame

World to camera coord. trans. matrix

(4x4)

Perspective projection matrix

(3x4)

Camera to pixel coord.

trans. matrix (3x3)

= 2D

point (3x1)

3D point (4x1)

K. Grauman

Coordinate Changes: Pure Translations

OBP = OBOA + OAP , BP = AP + BOA

Coordinate Changes: Pure Rotations

),,(

...

...

...

A

B

A

B

A

B

BABABA

BABABA

BABABA

B

A R kji

kkkjki

jkjjji

ikijii

A

B

A

B

A

Bkji

T

B

A

T

B

A

T

B

A

k

j

i

Coordinate Changes: Rotations about the k Axis

100

0cossin

0sincos

RB

A

A rotation matrix is characterized by the following properties:

• Its inverse is equal to its transpose, and

• its determinant is equal to 1.

Or equivalently:

• Its rows (or columns) form a right-handed

orthonormal coordinate system.

Coordinate Changes:

Pure Rotations

PRP

z

y

x

z

y

x

OP

AB

A

B

B

B

B

BBB

A

A

A

AAA

kjikji

Coordinate Changes: Rigid Transformations

A

BAB

A

B OPRP

Block Matrix Multiplication

2221

1211

2221

1211

BB

BBB

AA

AAA

What is AB ?

2222122121221121

2212121121121111

BABABABA

BABABABAAB

Homogeneous Representation of Rigid Transformations

11111

PT

OPRPORP A

B

A

A

BAB

A

A

T

A

BB

A

B

0

Extrinsic parameters: translation and rotation of camera frame

tpRp C

W

WC

W

C

Non-homogeneous coordinates

Homogeneous coordinates

ptRp WC

W

C

W

C

1000

|

|

W. Freeman

Combining extrinsic and intrinsic calibration parameters, in homogeneous coordinates

Forsyth&Ponce

ptR

Kp W

C

W

C

W

10,0,0z

1

pp CK

z

1

pMp W

z

1

Intrinsic

Extrinsic

ptRp WC

W

C

W

C

1000

|

|

World coordinates

Camera coordinates

pixels

W. Freeman

pixels (u,v,1)

World coordinates (x,y,z,1)

Other ways to write the same equation

1...

...

...1

1 3

2

1

z

W

y

W

x

W

T

T

T

p

p

p

m

m

m

zv

u

pMz

p W

1

pixel coordinates

world coordinates

Conversion back from homogeneous coordinates leads to (note that z = mT

3*P) :

Pm

Pmv

Pm

Pmu

3

2

3

1

W. Freeman

Extrinsic Parameters

Explicit Form of the Projection Matrix

Note:

M is only defined up to scale in this setting!!

Calibration target

http://www.kinetic.bc.ca/CompVision/opti-CAL.html

Find the position, ui and vi, in pixels, of each calibration object feature point.

Image Formation I Chapter 1 (Forsyth&Ponce) Camerasgerig/CS6320-S2013/Materials/CS... · Chapter 1 (Forsyth&Ponce) Cameras ... Affine projection models: Orthographic projection ...

Documents