Uncalibrated Epipolar - Calibration Jana Kosecka CS223b.

Uncalibrated Epipolar - Calibration

Jana KoseckaCS223b

CS223b2

Uncalibrated Camera

pixelcoordinates

calibratedcoordinates

Linear transformation

CS223b3

Overview

Calibration with a rig

Uncalibrated epipolar geometry

CS223b4

Uncalibrated Camera

• Pixel coordinates

• Projection matrix

Uncalibrated camera

• Image plane coordinates • Camera extrinsic parameters

• Perspective projection

Calibrated camera

CS223b5

Taxonomy on Uncalibrated Reconstruction

is known, back to calibrated case

is unknown Calibration with complete scene knowledge (a rig) – estimate

Uncalibrated reconstruction despite the lack of knowledge of

Autocalibration (recover from uncalibrated images)

Use partial knowledge Parallel lines, vanishing points, planar motion, constant intrinsic

Ambiguities, stratification (multiple views)

CS223b6

Calibration with a Rig

Use the fact that both 3-D and 2-D coordinates of feature points on a pre-fabricated object (e.g., a cube) are known.

CS223b7

Calibration with a Rig

• Eliminate unknown scales

• Factor the into and using QR decomposition

• Solve for translation

• Recover projection matrix

• Given 3-D coordinates on known object

CS223b8

More details• Direct calibration by recovering and decomposing the projection matrix

2 constraints per point

CS223b9

More details

• Factor the into and using QR decomposition

• Solve for translation

• Recover projection matrix

• Collect the constraints from all N points into matrix M (2N x 12)

• Solution eigenvector associated with the smallest eigenvalue

• Unstack the solution and decompose into rotation and translation

CS223b10

Calibration with a planar pattern

To eliminate unknown depth, multiply both sides by

CS223b11

Calibration with a planar pattern

Because are orthogonal and unit norm vectors of rotation matrixWe get the following two constraints

• Unknowns in K (S)

Skew is often close 0 -> 4 unknowns

• We want to recover S

• S is symmetric matrix (6 unknowns) in general we need at least 3 views• To recover S (2 constraints per view) - S can be recovered linearly • Get K by Cholesky decomposition of directly from entries of S

CS223b12

Alternative camera models/projections

Orthographic projection

Scaled orthographic projection

Affine camera model

CS223b13

Barrel and Pincushion Distortion

telewideangle

CS223b14

Models of Radial Distortion

)1( 42

21 rkrk

y

x

y

x

d

d ++⎟⎟⎠

⎞⎜⎜⎝

⎛=⎟⎟

⎠

⎞⎜⎜⎝

⎛

distance from center

CS223b15

Tangential Distortion

cheapglue

cheap CMOS chipcheap lens image

cheap camera

CS223b16

Barrel distortion

CS223b17

Distorted Camera Calibration

Set k1k2, solve for undistorted case

Find optimal k1k2 via nonlinear least squares

Iterate

Tends to generate good calibrations

CS223b18

Calibration Software: Matlab

CS223b19

Calibration Software: OpenCV

CS223b20

Calibration by nonlinear Least Squares

Least Mean Square

Gradient descent:

J

},,,,]},[{]},[{]},[{]},[{{ yxyx oossfkTkkkX ψϕφ=

0X

0X

J

∂∂

)(1.0 11 −− ∂∂

⋅−← kkk XXJ

XX

CS223b21

The Calibration Problem Quiz

Given Calibration pattern with N corners

K views of this calibration pattern

How large would N and K have to be?

Can we recover all intrinsic parameters?

N 1 3 1 3 4 4 6

K 1 1 3 3 3 4 6

CS223b22

Constraints

N points K images 2NK constraints

4 intrinsics (distortion: +2) 6K extrinsics need 2NK ≥ 6K+4

(N-3)K ≥ 2

Hint: may not be co-linear

CS223b23

The Calibration Problem Quiz

N 1 3 1 3 4 4 6

K 1 1 3 3 3 4 6

No No No No Yes Yes Yes

need (N-3)K ≥ 2

Hint: may not be co-linear

CS223b24

Problem with Least Squares

Many parameters (=slow) Many local minima! (=slower)