amera Geometry and Calibrati Thanks to Martial Heb
Camera Geometry and Calibration Thanks to Martial Hebert.
Dec 28, 2015
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Camera Geometry and Calibration
Thanks to Martial Hebert
Basic transformations (reminder)….
Pz
x
y
C
p
u
v
Retinal plane
f
Normalized imageplane
uC
v p
1
tRv
u
M o
o
100sin
0
cot
Scale in x direction between world coordinates and image coordinates
Skew of camera axes. = 90o if the axes are perpendicular
Principal point =Image coordinates of the projection of camera origin on the retina
Rotation between world coordinate system and camera
Translation between world coordinate system and camera
Scale in y direction between world coordinates and image coordinates
Standard Perspective Camera Model
Alternate Notations
T
T
T
m
m
m
M
3
2
1
tRKM bAM
T
T
T
a
a
a
A
3
2
1
4
3
3x3
3x3 3x1
By rows: By blocks:By components:
3x3 3x1
Intrinsic parametermatrix
Extrinsic parametermatrix
Q: Is a given 3x4 matrix M the projection matrix of some camera?A: Yes, if and only if det(A) is not zero
Q: Is the decomposition unique?A: There are multiple equivalent solutions
Applying the Projection Matrix
1
v
u
p
1
Z
Y
X
P
MPp
Pm
Pm
Pm
Pmv
Pm
Pm
Pm
Pmu
T
T
T
T
.
.
.
.
3
2
3
2
3
1
3
1
Homogeneous coordinates of point in image
Homogeneous coordinates of point in world
Homogeneous vector transformation: p proportional to MP
Computation of individual coordinates:
Observations:
•
is the equation of a plane of normal direction a1
• From the projection equation, it is also the plane defined by: u = 0
• Similarly: • (a2,b2) describes the plane defined by: v = 0• (a3,b3) describes the plane defined by:
That is the plane passing through the pinhole (z = 0)
Geometric Interpretation
33
11
3
1
b
Z
Y
X
a
b
Z
Y
X
a
Pm
Pmu
T
T
T
T
011
b
Z
Y
X
aT
vu
Projection equation:
u
v
a3
C
Geometric Interpretation: The rows of the projection matrix describe the three planes defined by the image coordinate system
a1
a2
p P
Other useful geometric properties
Q: Given an image point p, what is the direction of the corresponding ray in space?
A:
Q: Can we compute the position of the camera center ?
A:
pAw 1
bA 1
Affine Cameras
• Example: Weak-perspective projection model
• Projection defined by 8 parameters
• Parallel lines are projected to parallel lines
• The transformation can be written as a direct linear transformation
PtRKPMy
xaffine 222
2x4 projection matrix
2x2 intrinsic
parameter matrix
2x3 matrix = first 2 rows of the rotation matrix between world and camera frames
First 2 components of the translation between world and camera frames
Note: If the last row is the coordinates equations degenerate to:
The mapping between world and image coordinates becomes linear. This is an affine camera.
PmPmv
PmPmuT
T
.
.
22
11
10003 Tm
Calibration: Recover M from scene points P1,..,PN and the corresponding projections in the image plane p1,..,pN
In other words: Find M that minimizes the distance between the actual points in the image, pi, and their predicted projections MPi
Problems:• The projection is (in general) non-linear• M is defined up to an arbitrary scale factor
ii MPp
iT
iT
i
iT
iT
iPm
Pmv
Pm
Pmu
3
2
3
1
0)(
0)(
32
31
iiT
iT
iiT
iT
vPmPm
uPmPm
Write relation between image point, projection matrix, and point in space:
Write non-linear relations between coordinates:
Make them linear:
The math for the calibration procedure follows a recipe that is used in many (most?) problems involving camera geometry, so it’s worth remembering:
TNN
TN
TNN
TN
TT
TT
TT
PvP
PuP
PvP
PuP
LLmLmLm
0
0
0
0
111
111
2
1m
0
0
0
0
0
111
111
m
PvP
PuP
PvP
PuP
TNN
TN
TNN
TN
TT
TT
Put all the relations for all the points into a single matrix:
Solve by minimizing:
Subject to:
3
2
1
00
0m
m
m
mmPv
Pu
P
PT
ii
Tii
Ti
TiWrite them in
matrix form:
Slight digression: Homogeneous Least Squares
VXXXVVXXV Ti
Ti
i
T
ii 2
1X
VXXMin T
Suppose that we want to estimate the best vector of parameters X from matrices Vi computed from input data such that VXi = 0
We can do this by minimizing:
Since V=0 is a trivial solution, we need to constrain the magnitude of V to be non-zero, for example: |V| = 1. The problem becomes:
The key result (which we will use 50 times in this class) is:For any symmetric matrix V, the minimum of XTVX is reached at X = eigenvector of V corresponding to the smallest eigenvalue.
P1
z
x
y
Pi
(ui,vi)
(u1,v1)
MPi
2
3
2
2
3
1
i
ii
i
ii Pm
Pmv
Pm
Pmu
Calibration
We can follow exactly the same recipe with non-linear distortion:
iT
iT
i
iT
iT
iPm
Pmv
Pm
Pmu
3
2
3
1 11
021 iT
iiT
i PmuPmv
Write non-linear relations between coordinates:
Make them linear:
TNN
TNN
TT
TT
PuPv
PuPv
LLmLmLm
1111
2
1m
0
1111
m
PuPv
PuPv
TNN
TNN
TT
Put all the relations for all the points into a single matrix:
Solve by minimizing:
Subject to:
2
10m
mmmPuPv T
iiT
iiWrite them in matrix form:
From Faugeras
p
• http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/links.html
• http://www.ai.sri.com/~konolige/svs/svs.htm
• MATLAB calibration toolbox
• Intel OpenCV calibration package
Radiometric Calibration
• Important preprocessing step for many vision and graphics algorithms
• Use a color chart with precisely known reflectances.
Irradiance = const * Reflectance
Pix
el V
alu
es
3.1%9.0%19.8%36.2%59.1%90%
• Use more camera exposures to fill up the curve.• Method assumes constant lighting on all patches and works best when source is far away (example sunlight).
• Unique inverse exists because g is monotonic and smooth for all cameras.
0
255
0 1
g
?
?1g
Measured Response Curves of Cameras
[Grossberg, Nayar]
Dark Current Noise Subtraction
• Dark current noise is high for long exposure shots
• To remove (some) of it:
• Calibrate the camera (make response linear)
• Capture the image of the scene as usual
• Cover the lens with the lens cap and take another picture
• Subtract the second image from the first image
Dark Current Noise Subtraction
Original image + Dark Current Noise Image with lens cap on
Result of subtractionCopyright Timo Autiokari, 1998-2006
Related Documents
