Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry, the essential matrix Camera calibration, the fundamental matrix 3D reconstruction (Stereo algorithms) next week. Many of the slides are courtesy of Prof. Ronen Basri
66
Embed
Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Two-views geometryOutline
Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry:
Homography Epipolar geometry, the essential matrix Camera calibration, the fundamental matrix
3D reconstruction (Stereo algorithms) next week.
Many of the slides are courtesy of Prof. Ronen Basri
3-D Scene
u
u’
What can 2 images tell us about ….Faugeras et. al. ECCV 92
Objective
3-D Scene
u
u’
Study the mathematical relations between corresponding image points.
“Corresponding” means originated from the same 3D point.
Objective
World Cup 66: England-Germany
World Cup 66: Second View
World Cup 66: England-Germany
Conclusion: no goal (missing 3 inches)
(Reid and Zisserman, “Goal-directed video metrology”)
Camera Obscura
"Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January 24, 1544, and later he used this illustration of the event in his book De Radio Astronomica et Geometrica, 1545. It is thought to be the first published illustration of a camera obscura..." Hammond, John H., The Camera Obscura, A Chronicle
A few words about Cameras
Camera obscura dates from 15th century First photograph on record shown in the book - 1822 Basic abstraction is the pinhole camera Current cameras contain a lens and a recording device
(film, CCD, CMOS) The human eye functions very much like a camera
Ideal LensesLens acts as a pinhole (for 3D points at the focal depth).
Regular LensesE.g., the cameras in our lab.
To learn more on lens-distortion see Hartley & Zisserman Sec. 7.4 p.189.Not part of this class.
Pinhole Camera
Single View Geometry
f
X
P Y
Z
x
p y
f
∏x
p y
f
Notation
O – Focal center π – Image plane Z – Optical axis f – Focal length
Projection
x y f
X Y Z
f
x
y
Z
X
Y
Perspective Projection
f Xx
Zf Y
yZ
Origin (0,0,0) is the Focal center X,Y (x,y) axis are along the image axis (height / width). Z is depth = distance along the Optical axis f – Focal length
Orthographic Projection
•Projection rays are parallel•Image plane is fronto-parallel(orthogonal to rays)
•Focal center at infinity
x X
y Y
Scaled Orthographic ProjectionAlso called “weak perspective”
x sX
y sY
0
fs
Z
Pros and Cons of Projection Models Weak perspective has simpler math.
Accurate when object is small and distant. Useful for object recognition.
Pinhole perspective much more accurate. Used in structure from motion.
When accuracy really matters (SFM), we must model the real camera (exact imaging processes): Perspective projection, calibration parameters (later), and
all other issues (radial distortion).
Two-views geometryOutline
Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry:
Homography Epipolar geometry, the essential matrix Camera calibration, the fundamental matrix
3D reconstruction from two views (Stereo algorithms)
Hartley & Zisserman: Sec. 2 Proj. Geom. of 2D.Sec. 3 Proj. Geom. of 3D.
Reading
Hartley & Zisserman:
Sec. 2 Proj. Geo. of 2D:• 2.1- 2.2.3 point lines in 2D• 2.3 -2.4 transformations • 2.7 line at infinity
Sec. 3 Proj. Geo. of 3D. • 3.1 – 3.2 point planes & lines. • 3.4 transformations
Euclidean Geometry is good for
questions like:
what objects have the same shape (= congruent)
Same shapes are related by rotation and translation
Why projective Geometry (Motivation)
Why Projective Geometry (Motivation) Answers the question what appearances
(projections) represent the same shape
Same shapes are related by a projective transformation
Where do parallel lines meet?
Parallel lines meet at the horizon (“vanishing line”)
Why Projective Geometry (Motivation)
Coordinates in Euclidean Space
0 1 2 3 ∞
Not in space
Coordinates in Projective Line P1
-1 0 1 2 ∞
k(0,1)
k(1,0)
k(2,1)k(1,1)k(-1,1)
Points on a line P1 are represented as rays from origin in 2D,Origin is excluded from space
“Ideal point”
Coordinates in Projective Plane P2
k(0,0,1)
k(x,y,0)
k(1,1,1)
k(1,0,1)
k(0,1,1)
“Ideal point”
Take R3 –{0,0,0} and look at scale equivalence class (rays/lines trough the origin).
epipolar lineepipolar lineepipolar lineepipolar line
BaselineBaseline
PP
OO O’O’
To define an epipolar plane, we define the plane through the two camera centers O and O’ and some point P. This can be written algebraically (in some worldcoordinates as follows:
' ' 0T
OP OO O P
Essential Matrix (algebraic constraint between corresponding image points) Set world coordinates around the first camera
What to do with O’P? Every rotation changes the observed coordinate in the second image
We need to de-rotate to make the second image plane parallel to the first
Replacing by image points
' ' 0T
OP OO O P
' 0TP t RP
, 'P OP t OO
' 0Tp t Rp Other derivations Hartley & Zisserman p. 241
Essential Matrix (cont.)
Denote this by:
Then
Define
E is called the “essential matrix”
t p t p
' ' 0T Tp t Rp p t Rp
E t R
' 0Tp Ep
' 0Tp t Rp
Properties of the Essential Matrix E is homogeneous Its (right and left) null spaces are the two epipoles 9 parameters Is linear E, E can be recovered up to scale using 8 points. Has rank 2.
The constraint detE=0 7 points suffices In fact, there are only 5 degrees of freedom in E,
3 for rotation 2 for translation (up to scale), determined by epipole
0 ': l plpE t
' 0Tp Ep
e) trough lines ( : : 12 all PPEThus
BackgroundThe lens optical axis does not coincide with
the sensor
We model this using a 3x3 matrix the Calibration matrix
Camera Internal Parameters or Calibration matrix
Camera Calibration matrix
The difference between ideal sensor ant the real one is modeled by a 3x3 matrix K:
(cx,cy) camera center, (ax,ay) pixel dimensions, b skew
We end with
0
0 0 1
x x
y y
a b c
K a c
q Kp
Fundamental Matrix
F, is the fundamental matrix.
1 1
1
1
' 0 ( ) ( ') 0
( ) ' 0
T T
T T
T
p Ep K q E K q
q K EK q
F K EK
Properties of the Fundamental Matrix F is homogeneous Its (right and left) null spaces are the two epipoles 9 parameters Is linear F, F can be recovered up to scale using 8 points. Has rank 2.