Optical Flow Methods 2007/8/9
Optical Flow Methods 2007/8/9. Outline Introduction to 2-D Motion The Optical Flow Equation The Solution of Optical Flow Equation Comparison of different.
Dec 20, 2015
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Optical Flow Methods
2007/8/9
Outline
Introduction to 2-D Motion The Optical Flow Equation The Solution of Optical Flow
Equation Comparison of different methods Reference
The 2-D Motion
The projection of 3-D motion into the image plane.
X
Z
Y
The 2-D Motion(2)
A 2-D displacement field is a collection of 2-D displacement vectors.
Definition of Optical Flow
Optical flow is a vector field of pixel velocities based on the observable variations form the time-varying image intensity patter.
Difference between Optical flow and 2-D displacement(1)
There must be sufficient gray-level variation for the actual motion to be observable.
Difference between Optical flow and 2-D displacement(2)
An observable optical flow may not always correspond to actual motion. For example: changes in external illumination.
Outline
Introduction to 2-D Motion The Optical Flow Equation The Solution of Optical Flow
Equation Comparison of different methods Reference
The Optical Flow Equation(1)
Let the image brightness at the point (x, y) in the image plane at time t be denoted by
The brightness of a particular point in the pattern is constant, so that
Using the chain rule for differentiation we see that, 0
t
E
dt
dy
y
E
dt
dx
x
E
0dt
dE
),,( tyxE
The Optical Flow Equation(2)
If we let and , for the partial derivatives, we have a single linear equation in two unknowns: u and v.
Writing the equation in the two unknowns u and v,
dt
dxu
dt
dyv ),( yx EE
0 tyx EvEuE
The Optical Flow Equation(3) Writing the equation in another form,
The component of the movement in the direction of the brightness gradient equals
tyx EvuEE ,,
22),(
yx
t
EE
Evu
The Optical Flow Equation(4)
The velocity has to lie along a line perpendicular to the brightness gradient vector.
x
y
),( yx EE
Constraint Line
tyx EvuEE ,,
22),(
yx
t
EE
Evu
(u,v)
Outline
Introduction to 2-D Motion The Optical Flow Equation The Solution of Optical Flow
Equation Comparison of different methods Reference
Second-Order Differential Methods(1)
Based on the conservation of the spatial image gradient.
The flow field is given by
0)),,((
dt
tyxEd
yt
E
xt
E
y
E
yx
E
xy
E
x
E
v
v
y
x
2
21
2
22
2
2
2
,
,
Second-Order Differential Methods(2)
The deficiencies: The constraint does not allow for
some motion such as rotation and zooming.
Second-order partials cannot always be estimated with sufficient accuracy.
Block Motion Model (1) (Lucas and Kanade Method) Based on the assumption that the mo
tion vector remains unchanged over a particular block of pixels.
for x,y inside block B
)(),,( tvtyxv
2
,)()(
t
Etv
y
Etv
x
EE yx
Byx
Block Motion Model (2)
Computing the partials of error with respect to and , then setting them equal to zero, we have
xv yv
0)()(,
x
E
t
Etv
y
Etv
x
Eyx
Byx
0)()(,
y
E
t
Etv
y
Etv
x
Eyx
Byx
Block Motion Model (3)
Solving the equations, we have
t
E
y
Et
E
x
E
y
E
y
E
y
E
x
E
y
E
x
E
x
E
x
E
tv
tv
Byx
Byx
ByxByx
ByxByx
y
x
,
,
1
,,
,,
,
,
)(
)(
Block Motion Model (4)
It is possible to increase the influence of the constraints towards the center of the block by weighted summations.
The accuracy of estimation depends on the accuracy of the estimated spatial and temporal partial derivatives.
Horn and Schunck Method(1) The additional constraint is to mini
mize the sum of the squares of the Laplacians of the optical flow velocity:
and2
2
2
22
y
u
x
uu
2
2
2
22
y
v
x
vv
Horn and Schunck Method(2) The minimization of the sum of the
errors in the equation for the rate of changes of image brightness.
and the measure of smoothness in the velocity flow.
2
2
2
2
2
2
2
2
y
v
x
v
y
u
x
uc
tyxb EvEuE
Horn and Schunck Method(3) Let the total error to be minimized be
The minimization is to be accomplished by finding suitable values for optical flow velocity (u ,v).
The solution can be found iteratively.
dxdybc 2222
Horn and Schunck Method: Directional-Smoothness constraint
The directional smoothness constraint:
W is a weight matrix depending on the spatial changes in gray level content of the video.
The directional-smoothness method minimizes the criterion function:
yTyx
Txds vWvvWv 2
dxdybds 2222
Gradient Estimation Using Finite Differences(1)
To obtain the estimates of the partials, we can compute the average of the forward and backward finite differences.
Gradient Estimation Using Finite Differences(2) The three partial derivatives of images
brightness at the center of the cube are estimated form the average of differences along four parallel edges of the cube.
Gradient Estimation by Local Polynomial Fitting(1) An approach to approximate E(x,y,t) local
ly by a linear combination of some low-order polynomials in x, y, and t; that is,
Set N equal to 9 and choose the following basis functions
1
0
),,(),,(N
iii tyxatyxE
ytxtxyyxtyxtyxi ,,,,,,,,1),,( 22
Gradient Estimation by Local Polynomial Fitting(2) The coefficients are estimated by using
the least squares method.
The components of the gradient can be found by differentiation,
x y t
N
iii tyxatyxEe 2
1
0
2 )),,(),,((
yaxaat
tyxE
taxayaay
tyxE
tayaxaax
tyxE
873
8652
7641
),,(
2),,(
2),,(
Estimating the Laplacian of the Flow Velocities(1) The approximation takes the following
form
and
The local averages u and v are defined as:
)( ,,,,2
kjikji vvv )( ,,,,2
kjikji uuu
Estimating the Laplacian of the Flow Velocities(2) The Laplacian is estimated by subtrac
ting the value at a point form a weighted average of the values at neighboring points.
Outline
Introduction to 2-D Motion The Optical Flow Equation The Solution of Optical Flow
Equation Comparison of different methods Reference
Comparison of different methods(1)
Three different method to be compared: Lucas-Kanade method based on block moti
on model. (11x11 blocks with no weighting) Horn-Schunck method imposing a global s
moothness constraint.( , allowed for 20 to 150 iterations)
The directional-smoothness method of Nagel( with 20 iterations)
6252
5,252
Comparison of different methods(2) These methods have been applied to the
7th and 8th frames of a video sequence, known as the “Mobile and Calendar.”
The gradients have been approximated by average finite differences and polynomial fitting.
The images are spatially pre-smoothed by a 5x5 Gaussian kernel with the variance 2.5 pixels.
Comparison of different methods(3)
Comparison of the differential methods.
278
10)),(),,((),(
255255log10
yxdyyxdxEyxEPSNR
yx
Method PSNR(dB) Entropy(bits)Polynomial Difference Polynomia
lDifference
Frame-DifferenceLucas-KanadeHorn-SchunckNagel
23.4530.8928.1429.08
-32.0930.7131.84
-6.444.225.83
-6.825.045.95
Outline
Introduction to 2-D Motion The Optical Flow Equation The Solution of Optical Flow
Equation Comparison of different methods Reference
Reference A. M. Tekalp, Digital Video Processing. Engle
wood Cliffs, NJ: Prentice-Hall, 1995. Horn, B.K.P. and Schunck, B.G. Determining
optical flow:A retrospective, Artificial Intelligence, vol. 17, 1981, pp.185-203.
J.L. Barron, D.J. Fleet, and S.S. Beauchemin, “Performance of Optical Flow Techniques,” in International Journal of Computer Vision, February 1994, vol. 12(1), pp. 43-77.
Related Documents
