Algorithm Evaluation and Error Analysis class 7 Multiple View Geometry Comp 290-089 Marc Pollefeys
Dec 22, 2015
Algorithm Evaluation and Error Analysis
class 7
Multiple View GeometryComp 290-089Marc Pollefeys
Multiple View Geometry course schedule(subject to change)
Jan. 7, 9 Intro & motivation Projective 2D Geometry
Jan. 14, 16
(no class) Projective 2D Geometry
Jan. 21, 23
Projective 3D Geometry (no class)
Jan. 28, 30
Parameter Estimation Parameter Estimation
Feb. 4, 6 Algorithm Evaluation Camera Models
Feb. 11, 13
Camera Calibration Single View Geometry
Feb. 18, 20
Epipolar Geometry 3D reconstruction
Feb. 25, 27
Fund. Matrix Comp. Structure Comp.
Mar. 4, 6 Planes & Homographies Trifocal Tensor
Mar. 18, 20
Three View Reconstruction
Multiple View Geometry
Mar. 25, 27
MultipleView Reconstruction
Bundle adjustment
Apr. 1, 3 Auto-Calibration Papers
Apr. 8, 10
Dynamic SfM Papers
Apr. 15, 17
Cheirality Papers
Apr. 22, 24
Duality Project Demos
• Maximum Likelihood Estimation
• DLT not invariant normalization• Geometric minimization invariant
• Iterative minimization• Cost function• Parameterization• Initialization• Minimization algorithm
22XXXX
iiii
iii
Automatic computation of HObjective
Compute homography between two imagesAlgorithm
(i) Interest points: Compute interest points in each image
(ii) Putative correspondences: Compute a set of interest point matches based on some similarity measure
(iii) RANSAC robust estimation: Repeat for N samples
(a) Select 4 correspondences and compute H
(b) Calculate the distance d for each putative match
(c) Compute the number of inliers consistent with H (d<t)
Choose H with most inliers
(iv) Optimal estimation: re-estimate H from all inliers by minimizing ML cost function with Levenberg-Marquardt
(v) Guided matching: Determine more matches using prediction by computed H
Optionally iterate last two steps until convergence
Algorithm Evaluation and Error Analysis
• Bounds on performance• Covariance estimation
?
? residual error
uncertainty
Algorithm evaluation
measured coordinates
estimated quantities
true coordinates
xH ,x
H ,x
Test on real data ortest on synthetic data
ii xx • Generate synthetic correspondences
• Add Gaussian noise, yielding ii xx
• Estimate from using algorithmii xx Hmaybe also ii x,x
• Verify how well or ii xHx ii xHx • Repeat many times (different noise, same )
• Error in one image
• Error in two images
ii xx )σ,0(xx Gii
2/1
1
2x,x2
1
n
iires dn
e
Estimate , thenH
Note: residual error ≠ absolute measure of quality of He.g. estimation from 4 points yields eres=0
more points better results, but eres will increase
2/1
1
2
1
2 x,xx,x4
1
n
ii
n
iires ddn
e
)σ,0(xx 2Gii ii xx )σ,0(xx 2Gii Estimate so that , thenii x,x,H ii xHx
Optimal estimators (MLE)
f
XP
Estimate expected residual error of MLE,Other algorithms can then be judged to this standard
f : M → N (parameter space to measurement space)
X N P M XP f
M NSM
dimension of submanifold SM = #essential parameters
nX
X
X
SM
2σ,XX NG
)X(X MLE
Assume SM locally planar around X
projection of isotropic Gaussian distribution on N with total variance N2 onto a subspace of dimension s is an isotropic Gaussian distribution with total variance s2
N measurements (independent Gaussian noise ) model with d essential parameters(use s=d and s=(N-d))
(i) RMS residual error for ML estimator
(ii) RMS estimation error for ML estimator
2/12/12
/1/XX NdNEeres
2/12/12
//XX NdNEeest
nX
X
X
SM
Error in one image Error in two images
2/1/1 Nderes
2/1/ Ndeest
2/1/41 neres
2/1/4 neest
2/1
2
4
n
neres
2/1
2
4
n
neest
nNd 2 and 8 nNnd 4 and 28
Covariance of estimated model
• Previous question: how close is the error to smallest possible error?
• Independent of point configuration
• Real question: how close is estimated model to real model?
• Dependent on point configuration (e.g. 4 points close to a line)
Forward propagation of covariance
Let v be a random vector in M with mean v and covariance matrix , and suppose that f: M →N is an affine mapping defined by f(v)=f(v)+A(v-v). Then f(v) is a random variable with mean f(v) and covariance matrix AAT.
Note: does not assume A is a square matrix
Example:
22 2,0,1,0 GyGx
723),( yxyxfx
7)0,0( fx
40
01 23A
252
3
40
0123AA T
5)( xstd
Example:
22 2,0,1,0 GyGx
yxy
yxx
23
23
40
01
23
23A
257
725
22
33
40
01
23
23AA T
;7;25;25 yxEyyExxE
Non-linear propagation of covariance
Let v be a random vector in M with mean v and covariance matrix , and suppose that f: M →N
differentiable in the neighborhood of v. Then, up to a first order approximation, f(v) is a random variable with mean f(v) and covariance matrix JJT, where J is the Jacobian matrix evaluated at v
)vv(J)v()v( ffj
iij
fJ
v
Note: good approximation if f close to linear within variability of v
Example:
40
01,
0
0),(x T Gyx
523),( 2 yxxyxfx
dxdyyxfyxPx ),(),(
dxdyxyxfyxP 22 ),(),(
222 2/)4/(
π8
1),( yxeyxP
2σ5x422 σ2σ25σ x
5x 222 σ25
2
3
4
123σσ
x
23J
Example:
feydxcybxyaxyxf 22),(
fca 2y
2x σσmean
2y
2x
2y
22y
2x
4x
2 eσσσ2σσσ2variance dcba
fmean est.2y
2x eσσ varianceest. d
Backward propagation of covariance
f : M → N
X N P M
PX1 f
X
f -1
P
X
XX η
PX1 f
Backward propagation of covariance
PX1 f
PPJPP -ff XP f
PXXX f
PPXX J
XXJJJPP 1T-11T
XP 1 f XP 1 f
PX1 f XXXJJJ 11T-11T f
assume f is affineX
f -1PX
what about f -1o ?
solution:
minimize:
XXXJJJ 11T-11T f
Backward propagation of covariance
PX1 fX
f -1PX
1T-11T JJJA
T1T-11T1T-11T JJJJJJ1 f
-T1TT1T-11T JJJJJJ
-11T JJ
Backward propagation of covariance
PX1 fX
f -1PX
-11x
TP JJ If f is affine, then
non-linear case, obtain first order approximations by using Jacobian
Over-parameterization
In this case f is not one-to-one and rank J<M
-11x
TP JJ so can not hold
e.g. scale ambiguity infinite variance!
However, if constraints are imposed, then ok.
T11x
TTA1x
TP AJAJAAJJ
A
Invert d xd in stead of MxM
Over-parameterization
When constraint surface is locally orthogonal to the null space of J
JJ 1x
TP
e.g. usual constraint ||P||=1
nullspace||P||=1
(pseudo-inverse)
Example: error in one image
(i) Estimate the transformation from the data(ii) Compute Jacobian , evaluated at(iii) The covariance matrix of the estimated is
given by JJ 1
xT
h
h/XJ f
Hh
h
TT
TT
x~x~0
x~0x~1h/xJ
iii
iii
iii
y
x
w
iiii JJJJ 1T1
xT
h
Example: error in both images
BBAB
BAAAJJ
1x
T1x
T
1x
T1x
T1
xT
B|AJ
separate in homography and point parameters
Using covariance matrix in point transfer
Thhhx JJ
Txxx
Thhhx JJJJ
Error in one image
Error in two images
(if h and x independent, i.e. new points)
=1 pixel =0.5cm (Crimisi’97)
Example:
=1 pixel =0.5cm (Crimisi’97)
Example:
(Crimisi’97)
Example:
Monte Carlo estimation of covariance
• To be used when previous assumptions do not hold (e.g. non-flat within variance) or to complicate to compute.
• Simple and general, but expensive
• Generate samples according to assumed noise distribution, carry out computations, observe distribution of result
Next class: Camera models