Camera Calibration to ARToolkitkowon.dongseo.ac.kr/~lbg/web_lecture/imageprocessing/... · 2015-02-06 · Projective transformations 11 lbg@dongseo.ac.kr 11/26/2012 A projectivity
Post on 12-Aug-2020
1 Views
Preview:
Transcript
Camera Calibration
to ARToolkit
2012.12.
lbg@dongseo.ac.kr
Agenda
11/26/2012 lbg@dongseo.ac.kr 2
Homogeneous Coordinates
Projective Transformation
Pinhole Camera Model
Camera Calibrations
Zhengyou Zhang
http://www.cs.unc.edu/~marc/
Homogeneous Coordinates
lbg@dongseo.ac.kr
Homogeneous coordinates
11/26/2012 lbg@dongseo.ac.kr 4
0 cbyax Ta,b,c
0,0)()( kkcykbxka TTa,b,cka,b,c ~
Homogeneous representation of lines
equivalence class of vectors, any vector is representative
Set of all equivalence classes in R3(0,0,0)T forms P2
Homogeneous representation of points
0 cbyax Ta,b,cl x , ,1x yTon if and only if
0l 11 x,y,a,b,cx,y,T 0,1,,~1,, kyxkyx
TT
The point x lies on the line l if and only if xTl=lTx=0
Homogeneous coordinates
Inhomogeneous coordinates Tyx,
, ,x y zT
but only 2DOF
Points from lines and vice-versa
11/26/2012 lbg@dongseo.ac.kr 5
l'lx
Intersections of lines
The intersection of two lines and is l l'
Line joining two points
The line through two points and is x'xl x x'
Example
1x
1y
Ideal points and the line at infinity
11/26/2012 lbg@dongseo.ac.kr 6
T0,,l'l ab
Intersections of parallel lines
TTand ',,l' ,,l cbacba
Example
1x 2x
Ideal points T0,, 21 xx
Line at infinity T1,0,0l
l22RP
tangent vector
normal direction
ab ,
ba,
Note that in P2 there is no distinction
between ideal points and others
A model for the projective plane
11/26/2012 lbg@dongseo.ac.kr 7
exactly one line through two points
exactly one point at intersection of two lines
Duality
11/26/2012 lbg@dongseo.ac.kr 8
x l
0xl T0lx T
l'lx x'xl
Duality principle:
To any theorem of 2-dimensional projective geometry
there corresponds a dual theorem, which may be
derived by interchanging the role of points and lines in
the original theorem
Projective Transformations
lbg@dongseo.ac.kr
Projective 2D Geometry
11/26/2012 lbg@dongseo.ac.kr 10
Projective transformations
11/26/2012 lbg@dongseo.ac.kr 11
A projectivity is an invertible mapping h from P2 to itself
such that three points x1,x2,x3 lie on the same line if and
only if h(x1),h(x2),h(x3) do.
Definition:
A mapping h:P2P2 is a projectivity if and only if there
exist a non-singular 3x3 matrix H such that for any point
in P2 reprented by a vector x it is true that h(x)=Hx
Theorem:
Definition: Projective transformation
3
2
1
333231
232221
131211
3
2
1
'
'
'
x
x
x
hhh
hhh
hhh
x
x
x
xx' Hor
8DOF
projectivity=collineation=projective transformation=homography
Mapping between planes
11/26/2012 lbg@dongseo.ac.kr 12
central projection may be expressed by x’=Hx
(application of theorem)
Removing projective distortion
11/26/2012 lbg@dongseo.ac.kr 13
333231
131211
3
1
'
''
hyhxh
hyhxh
x
xx
333231
232221
3
2
'
''
hyhxh
hyhxh
x
xy
131211333231' hyhxhhyhxhx
232221333231' hyhxhhyhxhy
select four points in a plane with know coordinates
(linear in hij)
(2 constraints/point, 8DOF 4 points needed)
11/26/2012 lbg@dongseo.ac.kr 14
11/26/2012 lbg@dongseo.ac.kr 15
http://www.youtube.com/watch?v=NOU1lXR4JKo
http://www.youtube.com/watch?v=-tuS3GbHtV8
11/26/2012 lbg@dongseo.ac.kr 16
http://www.youtube.com/watch?v=i7woG0pqFjs
http://www.youtube.com/watch?v=zFMOtzpkgQo
More Examples
11/26/2012 lbg@dongseo.ac.kr 17
Transformation for lines
11/26/2012 lbg@dongseo.ac.kr 18
Transformation for lines
ll' -TH
xx' HFor a point transformation
Isometries
11/26/2012 lbg@dongseo.ac.kr 19
1100
cossin
sincos
1
'
'
y
x
t
t
y
x
y
x
1
11
orientation preserving:
orientation reversing:
x0
xx'
1
tT
RHE IRR T
special cases: pure rotation, pure translation
3DOF (1 rotation, 2 translation)
Invariants: length, angle, area
Similarities
11/26/2012 lbg@dongseo.ac.kr 20
1100
cossin
sincos
1
'
'
y
x
tss
tss
y
x
y
x
x0
xx'
1
tT
RH
sS IRR T
also know as equi-form (shape preserving)
metric structure = structure up to similarity (in literature)
4DOF (1 scale, 1 rotation, 2 translation)
Invariants: ratios of length, angle, ratios of areas, parallel lines
Affine transformations
11/26/2012 lbg@dongseo.ac.kr 21
11001
'
'
2221
1211
y
x
taa
taa
y
x
y
x
x0
xx'
1
tT
AH A
non-isotropic scaling! (2DOF: scale ratio and orientation)
6DOF (2 scale, 2 rotation, 2 translation)
Invariants: parallel lines, ratios of parallel lengths, ratios of areas
DRRRA
2
1
0
0
D
Projective transformations
11/26/2012 lbg@dongseo.ac.kr 22
xv
xx'
vP T
tAH
Action non-homogeneous over the plane
8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity)
Invariants: cross-ratio of four points on a line (ratio of ratio)
T21,v vv
vv
sPAS TTTT v
t
v
0
10
0
10
t AIKRHHHH
Ttv RKA s
K 1det Kupper-triangular, decomposition unique (if chosen s>0)
Overview Transformations
11/26/2012 lbg@dongseo.ac.kr 23
100
2221
1211
y
x
taa
taa
100
2221
1211
y
x
tsrsr
tsrsr
333231
232221
131211
hhh
hhh
hhh
100
2221
1211
y
x
trr
trr
Projective
8dof
Affine
6dof
Similarity
4dof
Euclidean
3dof
Concurrency, collinearity,
order of contact (intersection,
tangency, inflection, etc.),
cross ratio
Parallellism, ratio of areas,
ratio of lengths on parallel
lines (e.g midpoints), linear
combinations of vectors
(centroids).
The line at infinity l∞
Ratios of lengths, angles.
The circular points I,J
lengths, areas.
Line at infinity
11/26/2012 lbg@dongseo.ac.kr 24
2211
2
1
2
1
0v
xvxv
x
x
x
x
v
AAT
t
000 2
1
2
1
x
x
x
x
v
AAT
t
Projective Transformation : Line at infinity becomes finite,
allows to observe vanishing points, horizon,
Affine Transformation : Line at infinity stays at infinity
The line at infinity
11/26/2012 lbg@dongseo.ac.kr 25
v1 v2
l1
l2 l4
l3
l∞
21 vvl
211 llv
432 llv
The line at infinity l is a fixed line under a projective transformation H if and only if H is an affinity
l
1
0
0
1t
0ll
A
AH
TT
A
Recovery of affine properties from images
11/26/2012 lbg@dongseo.ac.kr 26
Projective transformation Affine rectification
Affine transformation
Affine Rectification
11/26/2012 lbg@dongseo.ac.kr 27
v1 v2
l1
l2 l4
l3
l∞
21 vvl
211 llv
432 llv
Projective 3D geometry
11/26/2012 lbg@dongseo.ac.kr 28
vTv
tAProjective
15dof
Affine
12dof
Similarity
7dof
Euclidean
6dof
Intersection and tangency
Parallellism of planes,
Volume ratios, centroids,
The plane at infinity π∞
The absolute conic Ω∞
Volume
10
tAT
10
tRT
s
10
tRT
Pinhole Camera Model
lbg@dongseo.ac.kr
Pinhole Camera Model
11/26/2012 lbg@dongseo.ac.kr 30
TT ZfYZfXZYX )/,/(),,(
101
0
0
1
Z
Y
X
f
f
Z
fY
fX
Z
Y
X
101
01
01
1Z
Y
X
f
f
Z
fY
fX
Pinhole Camera Model
11/26/2012 lbg@dongseo.ac.kr 31
camX0|IKx
1
y
x
pf
pf
K
Principal point offset Calibration Matrix
Internal Camera Parameters
11/26/2012 lbg@dongseo.ac.kr 32
Camera rotation and translation
11/26/2012 lbg@dongseo.ac.kr 33
C~
-X~
RX~
cam
X10
RCR
1
10
C~
RRXcam
Z
Y
X camX0|IKx
XC~
|IKRx
t|RKP C~
Rt
PXx
Camera Parameter Matrix P
11/26/2012 lbg@dongseo.ac.kr 34
CCD Cameras
11/26/2012 lbg@dongseo.ac.kr 35
Correcting Radial Distortion of Cameras
11/26/2012 lbg@dongseo.ac.kr 36
11/26/2012 lbg@dongseo.ac.kr 37
Calibration Process
11/26/2012 lbg@dongseo.ac.kr 38
Correcting Radial Distortion of Camera with
Rotation Angle
11/26/2012 lbg@dongseo.ac.kr 39
2 4 6
1 2 3
2 4 6
1 2 3
( )(1 )cos
( )(1 )sin
u x d x d d d
d y d d d
x c x c k r k r k r
y c k r k r k r
2 4 6
1 2 3
2 4 6
1 2 3
( )(1 )sin
( )(1 )cos
u y d x d d d
d y d d d
y c x c k r k r k r
y c k r k r k r
0 1 2
6 7 1
u up
u u
m x m y mx
m x m y
3 4 5
6 7 1
u up
u u
m x m y my
m x m y
11/26/2012 lbg@dongseo.ac.kr 40
Camera Calibration
lbg@dongseo.ac.kr
Camera Models
11/26/2012 lbg@dongseo.ac.kr 42
“An empirical evaluation of factors influencing camera calibration accuracy using three publicly
available techniques”,Wei Sun and Jeremy R. Cooperstock
Machine Vision and Applications Volume 17, Number 1 (2006), 51-67,
Roger Y. Tsai
11/26/2012 lbg@dongseo.ac.kr 43
"An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision", Roger Y. Tsai,
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL,
1986, pages 364-374,
Zhengyou Zhang
11/26/2012 lbg@dongseo.ac.kr 44
Z. Zhang. Flexible Camera Calibration By Viewing a Plane From Unknown Orientations. International Conference on Computer Vision, Corfu, Greece, pages 666-673, September 1999.
Basic Equations
11/26/2012 lbg@dongseo.ac.kr 45
Basic Equations
11/26/2012 lbg@dongseo.ac.kr 46
Basic Equations
11/26/2012 lbg@dongseo.ac.kr 47
Solving Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 48
Solving Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 49
Solving Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 50
Solving Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 51
Solving Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 52
Solving Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 53
Solving Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 54
Solving Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 55
Zhengyou Zhang
11/26/2012 lbg@dongseo.ac.kr 56
Zhengyou Zhang
11/26/2012 lbg@dongseo.ac.kr 57
http://research.microsoft.com/en-us/um/people/zhang/
Camera Calibration Toolbox
11/26/2012 lbg@dongseo.ac.kr 58
http://www.vision.caltech.edu/bouguetj/calib_doc/
Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 59
Calibrating a Stereo System
11/26/2012 lbg@dongseo.ac.kr 60
Robust Multi-camera Calibration
11/26/2012 lbg@dongseo.ac.kr 61
http://graphics.stanford.edu/~vaibhav/projects/calib-
cs205/cs205.html
Multi-Camera Self Calibration
11/26/2012 lbg@dongseo.ac.kr 62
http://cmp.felk.cvut.cz/~Esvoboda/
Multi-Camera Self Calibration
11/26/2012 lbg@dongseo.ac.kr 63
ARToolKit Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 64
http://www.hitl.washington.edu/artoolkit/
OpenCV
11/26/2012 lbg@dongseo.ac.kr 65
http://opencv.willowgarage.com/documentation/
camera_calibration_and_3d_reconstruction.html
11/26/2012 lbg@dongseo.ac.kr 66
http://www.youtube.com/watch?v=DrXIQfQHFv0
11/26/2012 lbg@dongseo.ac.kr 67
http://www.youtube.com/watch?v=O6PEFJhsCEo
OpenCV 2.1 Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 68
http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html
OpenCV 2.1 Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 69
http://mmlab.disi.unitn.it/wiki/index.php/Camera_Calibration_Tool_in_OpenCV
OpenCV 2.1 Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 70
xcorrected = x + [2P1xy + P2(r2 + 2x
2)]
ycorrected = y + [P1(r2 + 2y
2) + 2P2xy]
Using the Polynomial Distortion Model to Correct Tangential Distortion The polynomial distortion model uses two parameters, P1 and P2, to characterize tangential distortion.
The distortion model for tangential distortion can be represented as:
http://zone.ni.com/reference/en-XX/help/372916L-01/nivisionconcepts/spatial_calibration_indepth/
xcorrected = x(1+ K1r2+ K2r
4 + K3r
6 + Knr
(n × 2))
ycorrected = y(1+ K1r2+ K2r
4 + K3r
6 + Knr
(n × 2))
Brown, D. C. Decentric distortion of lenses.
Journal of Photogrammetric Engineering and Remote Sensing, 32(3), 444-462, 1966.
OpenCV 2.1 Camera Calibration
11/26/2012 lbg@dongseo.ac.kr 71
http://www.nowpublishers.com/product.aspx?product=CGV&doi=0600000001§ion=x1-10r1
calibrateCamera
11/26/2012 lbg@dongseo.ac.kr 72
calibrateCamera
11/26/2012 lbg@dongseo.ac.kr 73
calibrateCamera
11/26/2012 lbg@dongseo.ac.kr 74
solvePnP
11/26/2012 lbg@dongseo.ac.kr 75
solvePnP
11/26/2012 lbg@dongseo.ac.kr 76
findChessboardCorners
11/26/2012 lbg@dongseo.ac.kr 77
drawChessboardCorners
11/26/2012 lbg@dongseo.ac.kr 78
rodrigues2
11/26/2012 lbg@dongseo.ac.kr 79
11/26/2012 lbg@dongseo.ac.kr 80
http://dprg.geomatics.ucalgary.ca/system/files/AKAM_ENGO667_CH_1Handouts.pdf
11/26/2012 lbg@dongseo.ac.kr 81
11/26/2012 lbg@dongseo.ac.kr 82
11/26/2012 lbg@dongseo.ac.kr 83
Camera Calibration
lbg@dongseo.ac.kr
11/26/2012 lbg@dongseo.ac.kr 85
Capture chessboard image
11/26/2012 lbg@dongseo.ac.kr 86
Chessboard no of row and col
10 : no of calibration images
Auto : automatically add images after 3*150 milliseconds
Start : start calibration
Calibration parameters
11/26/2012 lbg@dongseo.ac.kr 87
Focal length f_x, f_y
Center c_x, c_y
Radial : k1, k2, k3(check box)
Tangential : p1, p2
Und : Undistorted live image
Debug images
11/26/2012 lbg@dongseo.ac.kr 88
Filenames
11/26/2012 lbg@dongseo.ac.kr 89
Capture072309511500
month day hour min sec index
Corner072309511500
Image0723095115.xml
Object0723095115.xml
Distortion0723095115.xml
Intrinsics0723095115.xml
Image & Object xml
11/26/2012 lbg@dongseo.ac.kr 90
<?xml version="1.0"?>
<opencv_storage>
<Image0723095127 type_id="opencv-matrix">
<rows>480</rows>
<cols>2</cols>
<dt>f</dt>
<data>
2.25150803e+002 1.92720139e+002 2.42047302e+002 1.88396301e+002
2.59520660e+002 1.83857040e+002 2.77741913e+002 1.79775314e+002
2.96598541e+002 1.75898483e+002 3.15514801e+002 1.72283661e+002
<?xml version="1.0"?>
<opencv_storage>
<Object0723095127 type_id="opencv-matrix">
<rows>480</rows>
<cols>3</cols>
<dt>f</dt>
<data>
0. 1.37500000e+001 0. 2.75000000e+000 1.37500000e+001 0.
5.50000000e+000 1.37500000e+001 0. 8.25000000e+000 1.37500000e+001
0. 11. 1.37500000e+001 0. 1.37500000e+001 1.37500000e+001 0.
Distortion & Intrinsics xml
11/26/2012 lbg@dongseo.ac.kr 91
<?xml version="1.0"?>
<opencv_storage>
<Distortion0723095127 type_id="opencv-matrix">
<rows>4</rows>
<cols>1</cols>
<dt>f</dt>
<data>
-4.46110874e-001 2.08227739e-001 9.08007473e-003 -3.17067979e-003</data></Distortion0723095127>
</opencv_storage>
<?xml version="1.0"?>
<opencv_storage>
<Intrinsics0723095127 type_id="opencv-matrix">
<rows>3</rows>
<cols>3</cols>
<dt>f</dt>
<data>
2.66292694e+002 0. 3.24350769e+002 0. 2.58133820e+002
2.33235382e+002 0. 0. 1.</data></Intrinsics0723095127>
</opencv_storage>
top related