7/31/2019 GeometryCalibration_Ioana2
1/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Computer VisionProjective Geometry
and Calibration
Professor Hagerhttp://www.cs.jhu.edu/~hager
Ioana Fleminghttp://lcsr.jhu.edu/IoanaFleming
.
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Camera parameters
Summary: points expressed in external frame points are converted to canonical camera coordinates points are projected points are converted to pixel units
=
T
Z
Y
X
W
V
U
parametersextrinsic
ngrepresenti
tionTransforma
modelprojection
ngrepresenti
tionTransforma
parametersintrinsic
ngrepresenti
tionTransforma
point in cam.coords.
point in metricimage coords.
point in pixelcoords.
point inworld coords.
7/31/2019 GeometryCalibration_Ioana2
2/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Intrinsic Parameters + Projection Model
U V W
=
f 0 0 00 f 0 00 0 1 0
X Y Z T
U V W
pix
=
fs u 0 o u 0 fs v o v 0 0 1
U V W
mm
=
u 00 v 00 0 1
U V W
Ap
cameraimage (mm)
U V W
pix
=
s u 0 o u 0 s v o v 0 0 1
U V W
mm
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Putting it All Together
Now, using the idea of homogeneous transforms , we can write:
p ' = R T
0 0 0 1
p
R and T both require 3 parameters. These correspond
to the 6 extrinsic parameters needed for camera calibration
Finally, we can write:
U V W
pix
=
u 00 v 00 0 1
X Y Z T
world
7/31/2019 GeometryCalibration_Ioana2
3/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Compute the camera intrinsic (4 or 5) and extrinsicparameters (6) using only observed camera data.
Camera Calibration: Problem Statement
10/16/2008
A Quick Aside: Least Squares
= the pseudoinverse of A Ainv = pinv(A) in Matlab
Least Squares courtesy of Rene Vidal
7/31/2019 GeometryCalibration_Ioana2
4/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
A Quick Aside: Least Squares
Least Squares courtesy of Rene Vidal
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Camera CalibrationMethods: Photogrammetric Calibration Self Calibration Multi-Plane Calibration
Think of it like of a least squares problem (error minimization):
U V W
pix
=
u 00 v 00 0 1
X Y Z T
world
General strategy: view calibration object identify image points obtain camera matrix by
minimizing error obtain intrinsic parameters from
camera matrix
Most modern systems employthe multi-plane method avoids knowing absolute
coordinates of calibration points
7/31/2019 GeometryCalibration_Ioana2
5/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Photogrammetric Calibration
Calibration is performed through imaging a pattern whosegeometry in 3d is known with high precision.
PRO:Calibration can be performed very efficiently
CON:Expensive set-up apparatus is required;Multiple orthogonal planes.
Approach 1: Direct Parameter Calibration Approach 2: Projection Matrix Estimation
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Direct Calibration - Basic Equations
From world to cameracoordinates
(extrinsic parameters)
From world to imagemetric coordinates
(projection model +extrinsic parameters)
7/31/2019 GeometryCalibration_Ioana2
6/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Basic Equations
From image metric toimage pixel coordinates
(intrinsic parameters)
, where s x and s y = horizontal, vertical effectivepixel size
f x = focal length inhorizontal pixels;
= aspect ratio
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Basic Equations
one of these for each point
7/31/2019 GeometryCalibration_Ioana2
7/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Basic Equations
N x 8
, where m is defined up to scale.
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Properties of SVD
Recall the singular values of a matrix are related to its rank.
Recall that Am = 0 can have a nonzero m as solution only if A issingular.
Finally, note that the matrix V of the SVD is an orthogonal basisfor the domain of A; in particular the zero singular values are thebasis vectors for the null space.
Putting all this together, we see that A must have rank 7 (in thisparticular case) and thus m must be a vector in this subspace.
Clearly, m is defined only up to scale.m = * m
7/31/2019 GeometryCalibration_Ioana2
8/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Basic Equations
We now know R x and R y up to a sign and , and R z = R x x R yHave to use another SVD to orthogonalize this system
(R = U D V; set D to I and multiply).
m defined up to a scale factor ,||t|| -> retrieve ||r|| -> retrieve
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Last Details We still need to compute the correct sign.
note that the denominator of the original equations must be positive(points must be in front of the cameras)
Thus, the numerator and the projection must disagree in sign. We know everything in numerator and we know the projection, hence
we can determine the sign.
We still need to compute T z and f x we can formulate this as a least squares problem on those two
values using the first equation.
7/31/2019 GeometryCalibration_Ioana2
9/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Direct Calibration: The Algorithm
Compute the A matrix Compute solution with SVD: get m Compute gamma and alpha from m Compute R (and normalize) Compute f x and and T z If necessary, estimate distortion parameters
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Indirect Calibration: The Basic Idea
We know that we can also just write u h = M p h x = (u/w) and y = (v/w), u h = (u,v,1) As before, we can multiply through (after plugging in for u,v, and w)
Once again, we can write A m = 0
Once again, we use an SVD to compute m up to a scale factor.
7/31/2019 GeometryCalibration_Ioana2
10/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Self-Calibration
Calculate the intrinsic parameters solely from pointcorrespondences from multiple images.
Static scene and intrinsics are assumed. No expensive apparatus. Highly flexible but not well-established.
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Multi-Plane Calibration
Hybrid method: Photogrammetric and Self-Calibration. Uses a planar pattern imaged multiple times (inexpensive). Used widely in practice and there are many implementations. Based on a group of projective transformations called
homographies.
m be a 2d point [u v 1] and M be a 3d point [x y z 1].
Projection is
7/31/2019 GeometryCalibration_Ioana2
11/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Planar HomographiesFirst Fundamental Theorem of Projective Geometry: There exists a uniquehomography H that performs a change of basis between two projectivespaces of the same dimension.
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Estimating A Homography
7/31/2019 GeometryCalibration_Ioana2
12/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Estimating A Homography
10/16/2008
Multi-Plane Calibration
7/31/2019 GeometryCalibration_Ioana2
13/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Planar Homographies
First Fundamental Theorem of Projective Geometry: There exists a unique homography that performs a change of basis
between two projective spaces of the same dimension.
Projection Becomes
Notice that the homography H is defined up to scale (s).
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
We know that
From one homography, how many constraints on the intrinsicparameters can we obtain?
Extrinsics have 6 degrees of freedom. The homography supplies 8 values. Thus, we should be able to obtain 2 constraints per homography.
Use the constraints on the rotation matrix columns
Multi-Plane Calibration
7/31/2019 GeometryCalibration_Ioana2
14/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Computing Intrinsics
Rotation Matrix is orthogonal.
Write the homography in terms of its columns
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Computing Intrinsics
Derive the two constraints:
7/31/2019 GeometryCalibration_Ioana2
15/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Closed-Form Solution
Notice B is symmetric, 6 parameters can be written as a vector b. From the two constraints, we have h 1T B h 2 = v 12 T b
Stack up n of these for n images and build a 2n*6 system. Solve with SVD. Extrinsics fall-out of the result easily.
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Computing Extrinsics
Orthogonalize using SVD to get rotation
7/31/2019 GeometryCalibration_Ioana2
16/18
10/16/2008
In general, lens introduce minor irregularities into images, typically radial distortions:
x = x d(1 + k 1r2 + k2r4)y = y d(1 + k 1r2 + k2r4)r2 = xd2 + yd2
Distortion Correction: make lines straight.
The values k 1 and k 2 are additional parameters that must be estimated in order tohave a model for the camera system.
Lens Distortion
f (r) xd
yd
r
xy
10/16/2008
Lens Distortion
7/31/2019 GeometryCalibration_Ioana2
17/18
7/31/2019 GeometryCalibration_Ioana2
18/18
10/16/200810/16/2008 CS 441, Copyright G.D. Hager
Calibration Summary
Two groups of parameters: internal (intrinsic) and external (extrinsic)
Many methods direct and indirect, flexible/robust
The form of the equations that arise here and the way they aresolved is common in vision: bilinear forms Ax = 0 Orthogonality constraints in rotations
Most modern systems use the method of multiple planes (matlabdemo) more difficult optimization over a large # of parameters more convenient for the user