1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained.

1

Preview

• At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure

• Multiple views can be obtained by several cameras or by moving a camera

Chapter 10: The Geometry of Multiple Views

2

Since and

10.1 Two Views

1 1

1

,X x

Z

11 1( ).

xX Z

2

2 2( )x

X Z

Similarly,

2 1X X B 2

1 ( ),x

X B Z

1 2 ,Z Z Z

1 2( ) ( )x x

Z B Z

1 2

Z Bx x

3

10.1.1 Epipolar Geometry

4

◎ The Calibrated Case -- The intrinsic parameters of cameras are known

5

which relates frames O and O’.

6

7

8

9

10

11

12

13

• The matrix associated with the rotation whose

axis is the unit vector a and whose angle is

can be shown to be

[ ]

0

1( [ ])

!i

i

ei

a a (Exercise 10.2)

Therefore, [ ]R Id t ω

14

15

16

◎ The Uncalibrated Case -- Intrinsic parameters of cameras are unknown

17

18

◎ Estimates based on corresponding points between images

19

20

21

○ 8-point algorithm (Longuet-Higgins, 1981)

。 Given 8 point correspondences

This method does not take advantage of rank = 2

( , )i i s p p

x bA

22

23

24

○ Normalized 8-point algorithm (Hartley, 1995)

(1) Translate and scale the image versions of data points so that they are centered at the origin and the average distance to the origin is pixels , 2 : ip piT : ip piT

2

1

( )p pn

Ti i

i

(2) Compute by minimizing (3) Enforce the rank-2 constraint using

the Luong et al. method

25

26

White points: data points White lines: epipolar lines

27

10.2 Three Views ○ Calibrated case:

28

(i) The three constraints are not independent since the three associated planes are intersected at a point P. Any two of them are independent.

(ii) The position of a point (say ) can be predicted from the corresponding two points ( ).

Each pair of cameras define an epipolar constraint

1 12 2 2 23 3 3 31 10, 0, 0p p p p p pT T T

1p

2 3,p p

29

30

31

Image line segment l is the intersection of the plane segment L and the image plane , or is the projection of l onto

Plane segment L is formed by the spatial line segment l and the viewpoint O.

Consider

10.2.1 Trifocal Geometry

32

Let P(x,y,z) be a point on l, and p and l be their i

mage projections. Then, and 0,T M l P

where M: 3 by 4 projection matrix,

( , , ,1)P Tx y z( , , ) ,Ta b cl

0l PT M : the equation of plane L.

where

Rewrite as

0,T L P .TML l

0T l p

33

10.2.2 The Calibrated Case ○ Let the world coordinate system be attached to the first camera. Then, the projection matrices

1 2 2 2 3 3 3( ), ( ), ( )M Id M R M R 0 t t

34

35

Their determinants:

36

37

Combine the above determinants into a vector3 2 1 1

2 3 2 3 2 3 2 31 3 2 2

2 3 2 3 2 3 1 2 32 1 3 3

2 3 2 3 2 3 2 3

T T T T

T T T T

T T T T

b G c G a G G

c G a G b G G

a G b G c G G

l l l l l l l l

l l l l l l l l l

l l l l l l l l

The fourth minor:

The fourth determinant:

38

○ Trilinear Constraints

-- All the above determinants are zero because of degeneration of A

12 3

21 2 3

32 3

0

T

T

T

G

G

G

l l

l l l

l l1 2 3// T iG l l l

1 1 1 1 1 2 30,T T iG p l p l p l l

12 3

21 2 3

32 3

0

T

T T

T

G

G

G

l l

p l l

l l

39

10.2.3 The Uncalibrated Case

40

。 The projection matrices

1 1 2 2 1 2( ), ( ),M K M A K 0 b

。 Trilinear constraints 1

2 32

1 2 33

2 3

0,

T

T T

T

G

G

G

l l

p l l

l l

2 3 2 3i iT i TG b A A b

10.2.4 Estimation of the Trifocal Tensor

○ The three matrix define thetrifocal tensor with 27 coefficients.

3 3 iG 3 3 3

3 3 1 3( ) bM A K

1 2 2 2 3 3 3( ), ( ), ( )M Id M R M R 0 t t

(calibrated)

41

(1) Estimate the trifocal tensor G from point and line correspondences (a) Each triple of matching points provides 4 independent linear equations (b) Each triple of matching lines provides 2 independent linear equations e.g., p points and l lines,

4 2 27p l

(2) Improve the numerical stability of tensor estimation by normalizing image coordinates of points and lines

and the trifocal tensor have 5, 7 and 18 independent coefficients, respectively.

,

42

10.3 More Views

From1

p PMz

43

3 1

3 2

( ) 0

( ) 0

m m P

m m P

T T

T T

u

v

3 1

3 2

0m m

Pm m

T T

T T

u

v

i TiM m

44

45

46

47

48

Each quadrilinearity expresses the four associated planes intersecting at a point