Camera Calibration The purpose of camera calibration is to determine the intrinsic camera parameters (c 0 ,r 0 ), f, s x , s y , skew parameter (s = cot α), and the lens distortion (radial distortion coefficient k 1 ). Skew parameter defines the angle between the c and r axes of a pixel. For most CCD camera, we have rectangular pixel. The skew angle is 90. The skew parameter is therefore zero. k 1 is often very small and can be assumed to be zero. The intrinsic camera matrix is 1
82
Embed
The purpose of camera calibration is to determine theqji/CV/camera_calibrationX.pdf · Camera Calibration The purpose of camera calibration is to determine the intrinsic camera parameters
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Camera Calibration
The purpose of camera calibration is to determine the
intrinsic camera parameters (c0, r0), f, sx, sy, skew
parameter (s = cotα), and the lens distortion (radial
distortion coefficient k1). Skew parameter defines the
angle between the c and r axes of a pixel. For most CCD
camera, we have rectangular pixel. The skew angle is 90.
The skew parameter is therefore zero. k1 is often very
small and can be assumed to be zero. The intrinsic
camera matrix is
1
W =
fsx cotα c0
0 fsysinα
r0
0 0 1
If we assume α = 90, then we have
W =
fsx 0 c0
0 fsy r0
0 0 1
fsx and fsy are collectively treated as two separate
parameters instead of three parameters.
2
Camera Calibration Methods
• Conventional method: use a calibration pattern that
consists of a 2D and 3D data to compute camera
parameters. It needs a single image of the calibration
pattern.
• camera self-calibration: only need 2D image data of
the calibration pattern but need the 2D image data
from multiple views. Problem is often less
constrained than the conventional method.
3
Conventional Camera Calibration
Given a 3D calibration pattern, extract image features
from the calibration pattern. Use 2D image features and
the corresponding 3D features on the calibration pattern.
The features can be points, lines, or curves or their
combination.
• Determine the projection matrix P
• Derive camera parameters from P
4
Pfull =
sxfrt1 + c0r
t3 sxftx + c0tz
syfrt2 + r0r
t3 syfty + r0tz
rt3 tz
5
Compute P: Linear Method using 2D/3D Points
Given image points m2×1i = (ci, ri)
t and the
corresponding 3D points M 3×1i = (xi yi zi)
t, where i=1,
. . . , N, we want to compute P . Let P be represented as
P =
pt1 p14
pt2 p24
pt3 p34
(1)
where pi, i=1,2,3 are 3× 1 vectors and pi4, i=1,2,3, are
scalers.
6
Compute P: Linear Method (cont’d)
Then for each pair of 2D-3D points, we have
M ti p1 + p14 − ciM
ti p3 − cip34 = 0
M ti p2 + p24 − riM
ti p3 − rip34 = 0
For N points, we can setup a system of linear equations
AV = 0
where A is a 2N × 12 matrix depending only on the 3-D
and 2-D coordinates of the calibration points, and V is a
12× 1 vector (pt1 p14 pt2 p24 pt3 p34)t.
7
A =
M t1 1 ~0 0 −c1M
t1 −c1
~0 0 M t1 1 −r1M
t1 −r1
...
M tN 1 ~0 0 −cNM
tN −cN
~0 0 M tN 1 −rNM
tN −rN
where ~01×3 = [0 0 0]
8
Compute P: Linear Method (cont’d)
For the linear method to work, we need N >= 6 and the
N points cannot be coplanar points.
9
Rank of A
In general, the rank of A is 11 (for 12 unknowns), which
means the solution is up to a scale factor. But due to
effect of noise and locational errors, A may be full rank,
which may make the solution (corresponding to the
eigenvector of the smallest eigen value) unique.
The rank of A may also change for certain special
configurations of input 3D points, for example collinear
points, coplanar points, etc. The issue is of practical
relevance.
10
Rank of A
Rank of A is a function of the input points configurations
(see section 3.4.1.3 of Faugeras). If 3D points are
coplanar, then rank(A) < 11 (in fact, it equals 8), which
means there is an infinite number of solutions. Faugeras
proves that 1) in general Rank(A)=11; 2) for coplanar
points (N ≥4), rank(A)=8 since three points are needed
to determine a plane (11-3=8).
How about the rank of A if points are located a sphere or
on the planes that are orthogonal or parallel to each
other ? Hint: how many points are needed to determine
a sphere ?
11
Linear Solution 1
Minimize
||AV ||2
to solve for V . Can we solve for V ? Solution to AV = 0
is not unique (up to a scale factor). It lies in the null
space of A. If rank(A)=11, then V is the null vector of A,
multiplied by a scaler. If on the other hand, rank(A) ≤11, the the solution to X is the linear combinations of all
null vectors of A.
If rank(A)=11, then V is the null vector of A, multiplied
by a scaler. The null vector A can be obtained by
12
performing SVD on A, yielding
Am×n = Um×mDm×n(St)n×n
The null vector A is the last column of S matrix. Note V
is solved only up to a scale factor. The scale factor can
be recovered using the fact that ||p3||2 = 1. Let the null
vector of A be V’. The scale factor
α =√
1V ′2(9)+V ′2(10)+V ′2(11)
. Hence, V = αV ′.
Alternatively, we can also solve V by minimizing ||AV ||2,which yields (AtA)V = 0 or (AtA)V = λV , where λ = 0.
As a result, solution to V is the eign vector of matrix
(AtA) corresponding to zero eigen value. This implies
that the eignvectors of AtA correspond to the columns of
13
the S matrix.
14
Linear Solution 2
Let A = [B b], where
B =
M t1 1 ~0 0 −c1M
t1
~0 0 M t1 1 −r1M
t1
...
M tN 1 ~0 0 −cNM
tN
~0 0 M tN 1 −rNM
tN
b = (−c1 − r1 . . . − cN − rN)t
15
V = p34
Y
1
Y = (pt1 p14 pt2 p24 pt3)t/p34
Then, AV = p34(BY + b). Since p34 is a constant,
minimizing ||AV ||2 is the same as minimizing ||BY + b||2,whose solution is Y = −(BtB)−1Btb. The rank of matrix
B must be eleven.
16
Linear Solution 2 (cont’d)
The solution to Y is up to a scale factor p34. To recover
the scale factor, we can use the fact that |p3| = 1. The
scale factor p34 can be recovered as
p34 =1√
Y 2(9)+Y 2(10)+Y 2(11), where Y (9), Y (10), and Y (11)
are the last 3 elements of Y . The final projection matrix
(vector) is therefore equal to
V =
p34Y
p34
17
Linear Solution 3
Imposing the orthonormal constraint, Rt = R−1, i.e.,
minimize
||AV ||2
subject to Rt = R−1. Solution to this problem is
non-linear !.
18
Linear Solution 3 (cont’d)
To yield a linear solution, we can impose one of the
normal constraints, i.e., ||p3||2 = 1, then the problem is
converted to a constrained linear least-squares problem.
That is, minimize ||AV ||2 subject to ||p3||2 = 1.
ǫ2 = ||AV ||2 + λ(||p3||2 − 1) (2)
19
Decomposing A into two matrices B and C, and V into
Y and Z
A = (B C)
V =
Y
Z
B2N×9
=
Mt
11 ~0 0 −c1
~0 0 Mt
11 −r1
.
.
.
Mt
N1 ~0 0 −cN
~0 0 Mt
N1 −rN
C2N×3
=
−c1Mt
1
−r1Mt
1
.
.
.
−cNMt
N
−rNMt
N
20
Y =
p1
p14
p2
p24
p34
Z = p3
21
Then equation 2 can be rewritten as
ǫ2 = ||BY + CZ||2 + λ(||Z||2 − 1)
22
Taking partial derivatives of ǫ2 with respect to Y and Z
and setting them to zeros yield
Y = −(BtB)−1BtCZ
Ct(I − B(BtB)−1Bt)CZ = λZ
Apparently, the solution to Z is the eigenvector of matrix
Ct(I − B(BtB)−1Bt)C. Given Z, we can then obtain
solution to Y .
23
Substituting Y into ||BY + CZ||2 leads to
||BY + CZ||2
= || −B(BtB)−1BtCZ + CZ||2
= ||(I −B(BtB)−1Bt)CZ||2
= ZtCt(I −B(BtB)−1Bt)t(I − B(BtB)−1Bt)CZ
= ZtCt(I −B(BtB)−1Bt)CZ
= ZtλZ
= λ
This proves that solution to Z corresponds to the eigen
vector of the smallest positive eigenvalue of matrix
Ct(I − B(BtB)−1Bt)C.
24
Note
(I−B(BtB)−1Bt)(I−B(BtB)−1Bt) = (I−B(BtB)−1Bt)
25
Linear Calibration with Planar Object
Since planar points reduces the rank of A matrix to 8, we
cannot follow the conventional way of camera calibration
using planar object. But this becomes possible if we
acquire two images of the planar object, producing two A
matrices. With each A providing 8 independent
equations, we can have a total of 16 independent
equations, theoretically sufficient to solve for the intrinsic
matrix W. But since the two A matrices share the same
W but different extrinsic parameters, the extrinsic
parameters must be eliminated from the system of linear
equations to only determine W. This can be done using