A KLT-Based Approach for Occlusion Handling in Human Tracking Chenyuan Zhang, Jiu Xu, Axel Beaugendre and Satoshi Goto 2012 Picture Coding Symposium.

A KLT-Based Approach for Occlusion Handling in Human Tracking

Chenyuan Zhang, Jiu Xu, Axel Beaugendre and Satoshi Goto

2012 Picture Coding Symposium

2

Outline

• Introduction• Related Work• Proposed Approach• Experimental Result• Conclusion

3

Introduction

• Among the area of the tracking algorithms, the particle filtering approach is playing a significant role.

• Particle filter based tracking algorithms have suffered from occlusion, large deformation of the target objects and illumination changes.

4

Cont.

• Several new methods [1,2] added to the particle filter to overcome these kind of problems.

• Our proposal is mainly focused on the occlusion problem during human tracking.

[1] H. Wang, X. Hou, and C. Liu, “Object Tracking by Bidirectional Learning with Feature selection”, IEEE International Conference on Image Processing, 2009, pp.893-896.[2] J. Pan and B. Hu, “Robust Occlusion Handling in Object Tracking”, IEEE Conference of Computer Vision and Pattern Recognition, 2007, pp.1-8.

5

Related Work• [1]:– A novel bidirectional learning framework for

discrimination between the object and background is presented.

– The bidirectional learning rule is reasonable but the nonparametric kernel density based estimation for discrimination is not adaptive and not stable during the tracking.

6

Cont.• [2]:– The occlusion situation is analyzed by exploiting

the spatiotemporal context information.– And they rectify the occlusion-interfered

erroneous target location by employing a variant-mask template matching operation.

– For rigid object, the correct location can be obtained by their method, but for human tracking, spatiotemporal context information may not work well.

7

Outline Of Proposed Approach

• The main idea is that occlusion detection and handling strategy are added before model update stage.

• We use the particle filter algorithm to first estimate the positions of the targets

• We introduce the classical tracking algorithm KLT (Kanade-Lucas-Tomasi) [3] to do the occlusion detection.

[3] C. Tomasi and T. Kanade. “Detection and Tracking of Point Features ”, Carnegie Mellon University Technical Report CMU-CS-91-132, 1991.

8

Particle Filter

• Monte Carlo.• Bayesian recursion is a probability model of

estimation.

x : statez : observation valuek : time

9

Cont.

• Particle filter use Monte Carlo to deal with Bayesian recursion.

10

Cont.

11

The Proposed Approach For Occlusion Detection

• Most works utilize the similarity of color histogram, size of targets or distance between targets when dealing with the occlusion detection.

• Occlusion detection is divided into two parts, occlusion detection and drift detection.

12

Occlusion Detection

• Distance between color histogram of target model and candidate.

• Number of tracked feature points.• Motion directions of those tracked points.

13

Color Factor

• The distance between the model histogram rΜ and the candidate histogram r is defined as

• ρ is the Bhattacharyya coefficient. And the occlusion detection rule of color part is defined as

ρmax is max value among all the candidates in particle filter.

14

Feature Factor• KLT feature is presented here.• It performs by selecting the key feature points

in the first frame and finding the corresponding points in next frame.

• If the occlusion happens, most points in tracking window will be lost.

Nthr is set to 3, and 25 feature points are selected in the model blob. If tracked N is higher than 20, this frame’s tracking results seems to be good.

15

Motion Factor

• Estimated tracking window is compared with last frame by KLT to get the motion vectors of feature points

• Use K-means algorithm to cluster these vectors into two groups according to direction.

Ng1 and Ng2 are the number of the vectors in two groups.

16

Drift Detection• If the velocity of feature points is a little large,

it means that significant drift may be happened.

• The average vertical velocity and average horizontal velocity can be calculated according to the KLT feature tracker.

Δvx and Δvy are the average horizontal velocity and the average vertical velocity among tracked points.

17

18

The Proposed Approach For Occlusion Handling

• Occlusion handling strategy– If an occlusion happens, the updating of the target

model must be stopped.– Average of values in a bigger group is used to

represent the speed of target in current frame.– Use the average value of nearly 10 frames’ un-

occluded target speed to represent the estimated speed of occluded target.

– For the moving object detection, we do the subtraction between two adjacent frames.

19

Cont.

• Drift handling strategy– In order to remedy the drift problem, we should

utilize the vertical and horizontal velocity of the feature points to change the position of estimated tracking window.

20

Experimental Result

• Standard datasets：– The resolution of videos in CAVIAR dataset is

384×288 and in PETS2009 is 768×576.• The proposed algorithm was implemented in C++,

and we used a 2.93GHz CPU and 4GB RAM.

21

22

23

[6] W. Qu, D. Schonfeld, and M. Mohamed, “Real-Time Interactively Distributed Multi-Object Tracking Using a Magnetic-Inertia Potential Model’’, IEEE International Conference on Computer Vision, 2005, vol.1, pp.535-540.[7] M. Yang, T. Yu, and Y. Wu, ‘‘Game-Theoretic Multiple Target Tracking’’, IEEE International Conference on Computer Vision, 2007, pp.1-8.

24

Conclusion

• In this paper, we proposed a KLT based occlusion handling strategy in a particle filter track frame.

• Instead of using KLT as a tracker, we used it in occlusion detection to help tracking.

• Our result showed that our method is really effective.