Robot Vision SS 2008 Matthias Rüther 1 ROBOT VISION Lesson 2: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer Science University of North Carolina, Chapel Hill
Robot Vision SS 2008 Matthias Rüther 1
ROBOT VISION Lesson 2: Projective Geometry
Matthias Rüther
Slides courtesy of Marc Pollefeys Department of Computer Science
University of North Carolina, Chapel Hill
Robot Vision SS 2008 Matthias Rüther 2
Content
Linear Algebra: basic vector and matrix operations
Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation.
Single View: Camera model, Calibration, Single View Geometry.
Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies.
Three Views: Trifocal Tensor, Computing T.
More Views: N-Linearities, Multiple view reconstruction, Bundle adjustment, auto-calibration, Dynamic SfM, Cheirality, Duality
Robot Vision SS 2008 Matthias Rüther 3
Vector Basics
nT
n
xxxv
x
x
x
v ...,... 21
2
1
vx2
x1
nnnn xx
xx
xx
vv
xx
xx
xx
vv
'
...
'
'
',
'
...
'
'
' 22
11
22
11
vv‘
v+v‘
n
ii
T xvvv1
2
wvyxyxyxwv Tnn ...2211
coswvwv v
w
Φ
Robot Vision SS 2008 Matthias Rüther 4
Vector Spaces
Linear combination– Given vectors v1, ..., vk and scalars c1, ..., ck, the vector
v c1v1 c2v2 . . . ckvk
is obtained as a linear combination of the vectors v1, ..., vk
– Example: all vectors in R3 are linear combinations of i (1, 0, 0),
j (0, 1, 0), and k (0, 0, 1)
v (a, b, c) a(1, 0, 0) b(0, 1, 0) c(0, 0, 1)
Spanning– S=(v1, v2, . . . , vk ) spans a space W if every vector in W can be written as a
linear combination of the vectors in S
Linear Indepedence– v1, ..., vk is linearly indepedent if
c1v1 c2v2 . . . ck vk 0 implies c1 c2 . . . ck 0
Robot Vision SS 2008 Matthias Rüther 5
Vector Spaces
Basis– S=(v1, ..., vk) is a basis for a vector space W if
• vj are linearly indepedent
• S spans W
! S need not be orthogonal !
Dimension– All bases for W have the same number of vectors.
– The dimension of W is the size of its sets of basis vectors.
Coordinates Relative to a Basis– If S=(v1, ..., vn) is a basis for a vector space W, then any v W has a
unique vector expansion:
nn
n
iii vxvxvxvxv
...22111
Robot Vision SS 2008 Matthias Rüther 6
Example:– In R3 the vectors
relative to and
relative to are identical.
Vector Spaces
1
1
1
v
3
2
1
w
1
0
0
,
0
1
0
,
0
0
1
S
0
1
1
,
0
1
2
,
1
2
2
'S
Robot Vision SS 2008 Matthias Rüther 7
Matrix Basics
Addition/SubtractionC A B implies cij aij bij
C A B implies cij aij bij
MultiplicationCmq = Amn*Bnq implies
Determinants– det(A)=0 A is singular A is not invertible
– det(A-I) is characteristic polynomial, roots are the eigenvalues
m
kkjikij bac
1
122122112221
1211 )det(, aaaaAaa
aaA
Robot Vision SS 2008 Matthias Rüther 8
Matrix Basics
Inverse– AA1=A1A=I– Exists, iff det(A) != 0, A is not singular– Properties:
Pseudo-InverseA(AT A)1AT
A+A=I
RankMaximum number of linearly independent columns or rows.– If A is mn, rank(A) <= min(m,n)– If A is nn, rank(A) = n iff A nonsingular (invertible, det(A)!=0)– If A is nn, rank(A) < n iff A singular
TT AA
ABAB
AA
11
111
1
)(
)(
)det(
1)det(
Robot Vision SS 2008 Matthias Rüther 9
Solving Linear Equation Systems
Common problem: solve the equation system Amnx=b
– If m>n, the system is over-determined
– If m<n, the system is under-determined
– If b=0, the system is homogeneous
– The system has either no solutions, exactly one or infinitely many
– Methods for solving Amnx=b include Gauss elimination, LU factorization, computing pseudo-inverse using SVD etc.
Conditions for solutions of Ax=b (A is square)– If A is invertible, Ax=b has exactly one solution.
– A is invertible Ax=0 has only trivial solution
– If rank(A|b)>rank(A), the system has no solution
– If rank(A)<n, and rank(A|b)=rank(A), the system has infinite number of solutions
– Ax=0 has non-trivial solutions iff rank(A)<n
Robot Vision SS 2008 Matthias Rüther 10
Points, lines & conics
Transformations & invariants
1D projective geometry and
the Cross-ratio
Projective 2D Geometry
Robot Vision SS 2008 Matthias Rüther 11
Euclidean 1D vector space:
– Geometric Primitives: points
– Representation: 1D vector: p = (x)
– Transformations:• Translation: p‘ = (x+t)
• Scaling: p‘ = (s*x)
• Translation and Scaling: p‘ = (s*x+t)
1D Projective Geometry
ℝ1
p1 p2 p3 p4x
p1 p2 p3 p4 x
p‘1 p‘2 p‘3 p‘4 x
p‘1 p‘2 p‘3 p‘4 x
dp1p2
Robot Vision SS 2008 Matthias Rüther 12
1D Projective space:
1D Projective Geometry
ℙ1
p1 p2 p3 p4
x
w
W=1
Representation: 1D vector: p‘ = (x‘ w‘)T, where p = (x) = (x‘/w‘)
e.g. p‘ = (2 1)T = (4 2)T = (1 0,5)T are all equivalent to p = (2)
Robot Vision SS 2008 Matthias Rüther 13
1D Projective Geometry
1D Projective space:
– Geometric Primitives: points
– Representation: 1D vector: p = (x w)T
– Transformations:• Translation: p‘ = T * p1DOF
• Scaling: p‘ = S * p 1DOF
• Translation and Scaling: p‘ = M * p, where M=S*T
2DOF
• Projective Mapping: p‘ = P * p, where P is a 2x2 Matrix 3DOF
10
1 tT
sS
/10
01
s
tM
/10
1
Robot Vision SS 2008 Matthias Rüther 14
1D Projective Geometry
Translation:
x
w
W=1
p‘1 p‘2 p‘3 p‘4
pt
p
10
1'
Robot Vision SS 2008 Matthias Rüther 15
1D Projective Geometry
Scaling:
x
w
W=1
p‘1 p‘2 p‘3 p‘4
ps
p
/10
01'
Robot Vision SS 2008 Matthias Rüther 16
1D Projective Geometry
Projective Mapping:
x
w
W=1
p‘1p‘2
p‘3p‘4
pa
aap
1'
3
21
Analogy to central projective camera!
Robot Vision SS 2008 Matthias Rüther 17
1D Projective Geometry
Invariants: – transformations form a hierarchy, some geometric features remain unchanged
under the transformations
– Translation (isometry)• Length, overall scale
– Translation and Scaling (similarity)• Ratio of lengths (d12 : d23)
– Projective Mapping (projectivity)• Cross ratio (ratio of ratios):
d12*d34 = d13*d24
Robot Vision SS 2008 Matthias Rüther 18
Homogeneous coordinates
0 cbyax Ta,b,c
0,0)()( kkcykbxka TT a,b,cka,b,c ~
Homogeneous representation of lines
equivalence class of vectors, any vector is representative
Set of all equivalence classes in R3(0,0,0)T forms P2
Homogeneous representation of points
0 cbyax Ta,b,cl Tyx,x on if and only if
0l 11 x,y,a,b,cx,y, T 0,1,,~1,, kyxkyx TT
The point x lies on the line l if and only if xTl=lTx=0
Homogeneous coordinates
Inhomogeneous coordinates Tyx,
T321 ,, xxx but only 2DOF
Robot Vision SS 2008 Matthias Rüther 19
Points from lines and vice-versa
l'lx
Intersections of lines
The intersection of two lines and is l l'
Line joining two points
The line through two points and is x'xl x x'
Example
1x
1y
Robot Vision SS 2008 Matthias Rüther 20
Ideal points and the line at infinity
T0,,l'l ab
Intersections of parallel lines
TTand ',,l' ,,l cbacba
Example
1x 2x
Ideal points T0,, 21 xx
Line at infinity T1,0,0l
l22 RP
tangent vector
normal direction
ab , ba,
Note that in P2 there is no distinction
between ideal points and others
Robot Vision SS 2008 Matthias Rüther 21
A model for the projective plane
exactly one line through two points
exaclty one point at intersection of two lines
Robot Vision SS 2008 Matthias Rüther 22
Duality
x l
0xl T0lx T
l'lx x'xl
Duality principle:
To any theorem of 2-dimensional projective geometry there corresponds a dual theorem, which may be derived by interchanging the role of points and lines in the original theorem
Robot Vision SS 2008 Matthias Rüther 23
Conics
Curve described by 2nd-degree equation in the plane
022 feydxcybxyax
0233231
2221
21 fxxexxdxcxxbxax
3
2
3
1 , xxyx
xx or homogenized
0xx CT
or in matrix form
fed
ecb
dba
2/2/
2/2/
2/2/
Cwith
fedcba :::::5DOF:
Robot Vision SS 2008 Matthias Rüther 24
Five points define a conic
For each point the conic passes through
022 feydxcyybxax iiiiii
or
2 2, , , , ,1 0i i i i i ix x y y x y c Tfedcba ,,,,,c
0
1
1
1
1
1
552555
25
442444
24
332333
23
222222
22
112111
21
c
yxyyxx
yxyyxx
yxyyxx
yxyyxx
yxyyxx
stacking constraints yields
Robot Vision SS 2008 Matthias Rüther 25
Tangent lines to conics
The line l tangent to C at point x on C is given by l=Cx
lx
C
Robot Vision SS 2008 Matthias Rüther 26
Dual conics
0ll * CTA line tangent to the conic C satisfies
Dual conics = line conics = conic envelopes
1* CCIn general (C full rank):
Robot Vision SS 2008 Matthias Rüther 27
Degenerate conics
A conic is degenerate if matrix C is not of full rank
TT mllm C
e.g. two lines (rank 2)
e.g. repeated line (rank 1)
TllC
l
l
m
Degenerate line conics: 2 points (rank 2), double point (rank1)
CC **Note that for degenerate conics
Robot Vision SS 2008 Matthias Rüther 28
Projective transformations
A projectivity is an invertible mapping h from P2 to itself such that three points x1,x2,x3 lie on the same line if and
only if h(x1),h(x2),h(x3) do.
Definition:
A mapping h:P2P2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P2 reprented by a vector x it is true that h(x)=Hx
Theorem:
Definition: Projective transformation
3
2
1
333231
232221
131211
3
2
1
'
'
'
x
x
x
hhh
hhh
hhh
x
x
x
xx' Hor
8DOF
projectivity=collineation=projective transformation=homography
Robot Vision SS 2008 Matthias Rüther 29
Mapping between planes
central projection may be expressed by x’=Hx(application of theorem)
Robot Vision SS 2008 Matthias Rüther 30
Removing projective distortion
333231
131211
3
1
'
''
hyhxh
hyhxh
x
xx
333231
232221
3
2
'
''
hyhxh
hyhxh
x
xy
131211333231' hyhxhhyhxhx 232221333231' hyhxhhyhxhy
select four points in a plane with known coordinates
(linear in hij)
(2 constraints/point, 8DOF 4 points needed)
Remark: no calibration at all necessary, better ways to compute (see later)
Robot Vision SS 2008 Matthias Rüther 32
Transformation of lines and conics
Transformation for lines
ll' -TH
Transformation for conics-1-TCHHC '
Transformation for dual conicsTHHCC **'
xx' HFor a point transformation
Robot Vision SS 2008 Matthias Rüther 33
A hierarchy of transformations
Projective linear group
Affine group (last row (0,0,1))
Euclidean group (upper left 2x2 orthogonal)
Oriented Euclidean group (upper left 2x2 det 1)
Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant
e.g. Euclidean transformations leave distances unchanged
Robot Vision SS 2008 Matthias Rüther 34
Class I: Isometries
(iso=same, metric=measure)
1100
cossin
sincos
1
'
'
y
x
t
t
y
x
y
x
1
11
orientation preserving:orientation reversing:
x0
xx'
1
tT
RHE IRR T
special cases: pure rotation, pure translation
3DOF (1 rotation, 2 translation)
Invariants: length, angle, area
Robot Vision SS 2008 Matthias Rüther 35
Class II: Similarities
(isometry + scale)
1100
cossin
sincos
1
'
'
y
x
tss
tss
y
x
y
x
x0
xx'
1
tT
RH
sS IRR T
also know as equi-form (shape preserving)
metric structure = structure up to similarity (in literature)
4DOF (1 scale, 1 rotation, 2 translation)
Invariants: ratios of length, angle, ratios of areas, parallel lines
Robot Vision SS 2008 Matthias Rüther 36
Class III: Affine transformations
11001
'
'
2221
1211
y
x
taa
taa
y
x
y
x
x0
xx'
1
tT
AH A
non-isotropic scaling! (2DOF: scale ratio and orientation)
6DOF (2 scale, 2 rotation, 2 translation)
Invariants: parallel lines, ratios of parallel lengths,
ratios of areas
DRRRA
2
1
0
0
D
Robot Vision SS 2008 Matthias Rüther 37
Class VI: Projective transformations
xv
xx'
vP T
tAH
Action non-homogeneous over the plane
8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity)
Invariants: cross-ratio of four points on a line
(ratio of ratio)
T21,v vv
Robot Vision SS 2008 Matthias Rüther 38
Action of affinities and projectivities on line at infinity
2211
2
1
2
1
0v
xvxvx
xx
x
vAA
T
t
000 2
1
2
1
x
xx
x
vAA
T
t
Line at infinity becomes finite,
allows to observe vanishing points, horizon,
Line at infinity stays at infinity,
but points move along line
Robot Vision SS 2008 Matthias Rüther 39
Overview of transformations
1002221
1211
y
x
taa
taa
1002221
1211
y
x
tsrsr
tsrsr
333231
232221
131211
hhh
hhh
hhh
1002221
1211
y
x
trr
trr
Projective8dof
Affine6dof
Similarity4dof
Euclidean3dof
Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio
Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l∞
Ratios of lengths, angles.The circular points I,J
lengths, areas.