Dept. of ECE 1 Feature-based Object Tracking Dr. Dapeng Oliver Wu Joint Work with Bing Han, William Roberts, and Jian Li Department of Electrical and Computer.

Dept. of ECE 1

Feature-based Object Tracking

Dr. Dapeng Oliver WuJoint Work with Bing Han, William

Roberts, and Jian LiDepartment of Electrical and Computer

EngineeringUniversity of Florida

Dept. of ECE 2

Outline

What is object tracking?

How to track an object?

KLT feature-based tracking algorithm

Our feature-based tracking algorithm

Experimental results

Conclusions

Dept. of ECE 3

What is Object Tracking?

Object tracking:

Track an object over a sequence of images

Dept. of ECE 4

Why Object/Vehicle Tracking is Challenging?

Occlusion or partial occlusion Re-appearance A vehicle makes a left/right turn A vehicle passes another vehicle in the same direction Tracking of a small imaged object (consisting of 4 pixels) Shadow effect: use local histogram-based equalization Cloud effect Tracking of many vehicles (e.g., thousands of vehicles in a city) Clutters Parallax problems (caused by high-rise buildings) Multi-camera based tracking (e.g., urban surveillance networks)

Dept. of ECE 5

How to Track an Object?

Kanade-Lucas-Tomasi (KLT) feature-based tracking algorithm

Han, Roberts, Wu, Li (HRWL) feature-based tracking algorithm

……

Dept. of ECE 6

KLT Feature-based Tracker

1. Identification of feature points

2. Determination of an optimization criterion for feature correspondence

3. Computational method to solve the optimization problem

Web: http://www.ces.clemson.edu/~stb/klt/References [1] Bruce D. Lucas and Takeo Kanade. An Iterative Image Registration Technique with an Application to Stereo Vision. International Joint Conference on Artificial Intelligence, pages 674-679, 1981. [2] Carlo Tomasi and Takeo Kanade. Detection and Tracking of Point Features. Carnegie Mellon University Technical Report CMU-CS-91-132, April 1991. [3] Jianbo Shi and Carlo Tomasi. Good Features to Track. IEEE Conference on Computer Vision and Pattern Recognition, pages 593-600, 1994. [4] Stan Birchfield. Derivation of Kanade-Lucas-Tomasi Tracking Equation. Unpublished.

Dept. of ECE 7

Identification of Feature Points

Harris criterion:

GGx 2tracedet kC

2

2where x x y

x y y

I I I

I I I

G

If threhold, declare as a feature point;

otherwise, declare as a non-feature-point.

C x x

x

For low threshold For high threshold

Dept. of ECE 8

Feature Correspondence

Determination of an optimization criterion for feature correspondence (find a similarity measure)

Correlation Information distance: e.g., mutual information

1

1 1 2 1 2

norm:

E.g., norm of [ , , , ] is (| | | | | |).TK K

l

l X x x x x x x

2

2 2 2 1/ 22 1 2

norm:

E.g., norm of is ( ) .K

l

l X x x x

Dept. of ECE 9

How to Solve the Optimization problem?

Key difficulties: Discrete optimization problem Gradient methods not directly applicable

Exhaustive search: incurs exponential complexity Lucas-Kanade computational method:

Newton-Raphson type search

Dept. of ECE 10

More about KLT Tracker

Hierarchical search Work for any dimensional vector Can address affine motion (translation, rotation, scaling) Can address intensity adjustment

Dept. of ECE 11

Hierarchical Search

Dept. of ECE 12

Limitation of KLT Tracker

Limitation: KLT tracker does not guarantee the corresponding point in the next frame is a feature point.

Why? Because KLT tracker only uses the Harris criterion for the first frame

but not for other frames.

What problem it may cause? KLT may not handle occlusion well.

How to improve it? Evaluate the quality of the corresponding point using the Harris

criterion Refer to:

B. Han, W. Roberts, D. Wu, J. Li, ``Robust Feature-based Object Tracking,'' SPIE Defense & Security Symposium 2007, Orlando, FL, USA, April 9–13, 2007.

Dept. of ECE 13

Our Tracking approach

Feature tracking: SSD criteria applied to find the feature point whose window minimizes the following energy function:

Feature quality: If the quality of a tracked feature point decreases below a chosen threshold, that point is removed from consideration. To compensate, new features are identified in the same window. The feature point with the minimum SSD criteria is retained if its SSD is below a certain threshold.

2 tyxIdttdyydxxIdydxEt ,,,,,

Dept. of ECE 14

Tracking Diagram

Feature point in current frame, new frame

Find point that min. SSD criteria

Meets threshold?

Retain feature

Identify new feature points

Select feature that min. SSD

Meets threshold?

Feature lost

Retain new feature

N

N

Y

Y

Dept. of ECE 15

Experimental Results

Biker image sequence

50 frame sequence of bikers.

Video contains global and local motions.

Objects undergo scaling, translation, and rotation.

Dept. of ECE 16

Experimental Results (2)

Biker image sequence (Cont’d)

1st frame. 258 features

selected.

50th frame. 241 features

tracked.

Dept. of ECE 17

Experimental Results (3)

Biker image sequence (Cont’d)

Blue: our approach. Red: KLT algorithm.

Dept. of ECE 18

Conclusions

Number of successfully tracked features increased by over 10% versus Tomasi-Kanade’s approach.

Computationally inexpensive and robust against various types of object motion.

For pure translational object motion, the combined algorithm does not offer improved performance to Tomasi-Kanade’s method.

Dept. of ECE 19

Future Research Direction

Design tracking schemes, which are proven to converge.KLT algorithm does not guarantee convergence; no

proof of convergenceIt is important to design convergence-guaranteed

tracking algorithms.We will systematically study this problem.

Dept. of ECE 20

THANK YOU!

THANK YOU!