Page 1
EECS 442 – Computer vision
Cameras without cameras we wouldn’t have C.V.
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Other camera models
Reading: [FP] Chapters 1 – 3
[HZ] Chapter 6
Some slides in this lecture are courtesy to Profs. J. Ponce, S. Seitz, F-F Li
Page 2
EECS 442 – Computer vision
Cameras without cameras we wouldn’t have C.V.
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Other camera models
Reading: [FP] Chapters 1 – 3
[HZ] Chapter 6
Some slides in this lecture are courtesy to Profs. J. Ponce, S. Seitz, F-F Li
Page 3
How do we see the world?
• Let’s design a camera – Idea 1: put a piece of film in front of an object
– Do we get a reasonable image?
Page 4
Pinhole camera
• Add a barrier to block off most of the rays
– This reduces blurring
– The opening known as the aperture
Page 5
Pinhole perspective projection Pinhole camera f
f = focal length
c = center of the camera
c
Page 6
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
Some history…
Page 7
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
Some history…
Page 8
Photography (Niepce, ―La
Table Servie,‖ 1822)
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
• Joseph Nicephore Niepce (1822):
first photo - birth of photography
Some history…
Page 9
Photography (Niepce, ―La
Table Servie,‖ 1822)
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
• Joseph Nicephore Niepce (1822):
first photo - birth of photography
• Daguerréotypes (1839)
• Photographic Film (Eastman, 1889)
• Cinema (Lumière Brothers, 1895)
• Color Photography (Lumière Brothers,
1908)
Some history…
Page 10
Let’s also not forget…
Motzu
(468-376 BC)
Aristotle
(384-322 BC)
Also: Plato, Euclid
Al-Kindi (c. 801–873)
Ibn al-Haitham
(965-1040) Oldest existent
book on geometry
in China
Page 11
z
yf'y
z
xf'x
y
xP
z
y
x
P
Pinhole camera
Derived using similar triangles
Page 12
O
P = [x, z]
P’=[x’, f]
f
z
x
f
x
x
z
Pinhole camera
Page 13
Common to draw image plane in front of
the focal point. Moving the image plane
merely scales the image.
f f
Pinhole camera
z
yf'y
z
xf'x
Page 14
Kate lazuka ©
Pinhole camera
Is the size of the aperture important?
Page 15
Shrinking
aperture
size
- Rays are mixed up
Adding lenses!
-Why the aperture cannot be too small?
-Less light passes through
-Diffraction effect
Page 16
Cameras & Lenses
• A lens focuses light onto the film
Page 17
• A lens focuses light onto the film
– Rays passing through the center are not deviated
– All parallel rays converge to one point on a plane
located at the focal length f
focal point
f
Cameras & Lenses
Page 18
• A lens focuses light onto the film
– There is a specific distance at which objects are ―in
focus‖
[other points project to a ―circle of confusion‖ in the image]
―circle of
confusion‖
Cameras & Lenses
Page 19
Snell’s law
n1 sin 1 = n2 sin 2
Cameras & Lenses
1 = incident angle
2 = refraction angle
ni = index of refraction
Page 20
Thin Lenses
Snell’s law:
n1 sin 1 = n2 sin 2
Small angles:
n1 1 n2 2
n1 = n (lens)
n1 = 1 (air)
zo
z
y'z'y
z
x'z'x
ozf'z
)1n(2
Rf
Page 21
A compound lens Lenses are combined in various ways…
Source wikipedia
Page 22
f
f
f
Sourc
e w
ikip
edia
Dolly zooms
Compressed perception of depth
Exaggerated perception of depth
Page 23
Issues with lenses: Chromatic Aberration
• Lens has different refractive indices for different
wavelengths: causes color fringing
)1n(2
Rf
Page 24
Issues with lenses: Spherical aberration
• Rays farther from the
optical axis focus closer
Page 25
No distortion
Pin cushion
Barrel (fisheye lens)
Issues with lenses: Radial Distortion
– Deviations are most noticeable for rays that pass through
the edge of the lens
Image magnification
decreases with distance
from the optical axis
Page 26
Cameras
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Other camera models
Page 27
Pinhole perspective projection Pinhole camera f
f = focal length
c = center of the camera
c
)z
yf,
z
xf()z,y,x(
2E
3
Page 28
Is this a linear transformation?
No — division by z is nonlinear
How to make it linear?
)z
yf,
z
xf()z,y,x(
Page 29
Homogeneous coordinates
homogeneous image
coordinates
homogeneous scene
coordinates
• Converting from homogeneous
coordinates
Page 30
Perspective Projection
Transformation
1
z
y
x
0100
00f0
000f
z
yf
xf
X
z
yf
z
xf
iX
XMX
M
3H
4
Page 31
From retina plane to images
Pixels, bottom-left coordinate systems
Page 32
Coordinate systems
xc
yc
Page 33
Converting to pixels
x
y
xc
yc
C=[cx, cy]
)cz
yf,c
z
xf()z,y,x( yx
1. Off set
Page 34
Converting to pixels
)cz
ylf,c
z
xkf()z,y,x( yx
1. Off set
2. From metric to pixels
x
y
xc
yc
C=[cx, cy] Units: k,l : pixel/m
f : m : pixel
, Non-square pixels
Page 35
Matrix form?
x
y
xc
yc
C=[cx, cy]
)cz
y,c
z
x()z,y,x( yx
Converting to pixels
Page 36
Camera Matrix
)cz
y,c
z
x()z,y,x( yx
1
z
y
x
0100
0c0
0c0
z
zcy
zcx
X y
x
y
x
x
y
xc
yc
C=[cx, cy]
Page 37
XMX
X0IK
Camera Matrix
1
z
y
x
0100
0c0
0c0
X y
x
Camera
matrix K
Page 38
Finite projective cameras
1
z
y
x
0100
0c0
0cs
X y
x
Skew parameter
x
y
xc
yc
C=[cx, cy] K has 5 degrees of freedom!
Page 39
World reference system
Ow
iw
kw
jw
R,T
•The mapping so far is defined within the camera
reference system
• What if an object is represented in the world
reference system
Page 40
3D Rotation of Points Rotation around the coordinate axes, counter-clockwise:
100
0cossin
0sincos
)(
cos0sin
010
sin0cos
)(
cossin0
sincos0
001
)(
z
y
x
R
R
R
p
x
Y’
p’
x’
y
z
Page 41
wXMX wXTRK
World reference system
Ow
iw
kw
jw
R,T
wXTR
X44
10
In 4D homogeneous coordinates:
Internal parameters External parameters
Page 42
wx XMX 4313
14w4333 XTRK
Projective cameras
Ow
iw
kw
jw
R,T
How many degrees of freedom?
5 + 3 + 3 =11!
100
c0
cs
K y
x
Page 43
Projective cameras
Ow
iw
kw
jw
R,T
)X
X,
X
X()z,y,x(
w3
w2
w3
w1w
m
m
m
m M is defined up to scale!
Multiplying M by a scalar
won’t change the image
w13 XMX
14w4333 XTRK
3
2
1
m
m
m
M
Page 44
Theorem (Faugeras, 1993)
][ bATKRKTRKM
3
2
1
a
a
a
A
100
0 y
x
c
cs
K
lf
;kf
Page 45
Properties of Projection •Points project to points
•Lines project to lines
Page 46
Properties of Projection
•Distant objects look smaller
z
y'f'y
z
x'f'x
Image plane
scene
Page 47
Properties of Projection
•Angles are not preserved
•Parallel lines meet! Vanishing point
Page 48
Vanishing points
• Each set of parallel lines meets at a different point
[The vanishing point for this direction]
•Sets of parallel lines on the same plane lead to collinear
vanishing points [The line is called the horizon for that plane]
Page 49
One-point perspective • Masaccio, Trinity,
Santa Maria
Novella, Florence,
1425-28
Credit slide S. Lazebnik
Page 50
Cameras
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Other camera models
Page 51
Weak perspective projection
yz
fy
xz
fx
yz
fy
xz
fx
0
0
''
''
''
''
Magnification m
P ~
When the relative scene depth is small compared to its distance from the camera
Page 52
Orthographic (affine) projection
Distance from center of projection to image plane is infinite
yy
xx
yz
fy
xz
fx
'
'
''
''
Page 53
Pros and Cons of These Models
• Weak perspective much simpler math. – Accurate when object is small and distant.
– Most useful for recognition.
• Pinhole perspective much more accurate for scenes. – Used in structure from motion.
Page 54
Qingming Festival by the Riverside Zhang Zeduan ~900 AD
Weak perspective projection
Page 55
The Kangxi Emperor's Southern Inspection Tour (1691-1698) By Wang Hui
Page 56
58
http://www.eecs.umich.edu/~silvio/teaching/EECS442_2009/UCSD_videos/3DVisi
on.avi
HW 0.2: watch:
Marilia Maschion Producer & Director UCSD ICAM
undergraduate
Vincent Rabaud Technical Advisor UCSD CSE student
Serge Belongie Project Advisor UCSD CSE Department
Page 57
Next lecture
•How to calibrate a camera?