Top Banner
Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Patt ern Recognition (ICPR’06)
27

Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Dec 20, 2015

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Multiple Human Objects Tracking in Crowded Scenes

Yao-Te Tsai, Huang-Chia Shih,and Chung-Lin HuangDept. of EE, NTHU

International Conference on Pattern Recognition (ICPR’06)

Page 2: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Page 3: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Page 4: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Introduction human objects tracking systems

Pfinder Utilize stochastic, region-based feature

McKenna et al. Adaptive Gaussian mixture to model color distributi

on M2Tracker

Combine presence probability with color model to classify each pixel

Tsutsui et al. Exchange the optical flow information within multip

le cameras

Page 5: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Multiple human objects tracking system

System consist of Model-based object segmentation Remove noise of segmented region Optical flow-based position estimation Occlusion detection Object separation from occlusion

Contribution Track occlusion, separate object and

track it individually afterwards

Page 6: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Page 7: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Gaussian Mixture Color Model Condition probability for pixel i belong to

object O is

Parameters:mean , and covariance matrix

Expectation-maximization (EM) algorithm To determine the maximum likelihood

parameters of a mixture of m Gaussian

1)(,0)( jj j

m

j

ii

j

jeOipjj

Tj

1

)()(2

1

)(2

1)(

1

Page 8: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Color Model Use HIS space to reduce ambient illumination change

Each pixel i has 2-D feature vectorwhere hi is the hue, si is the saturation

Likelihood pixel i belonging to torso (n=0) or the bottom (n=1) of a person O is

),( iii shv

m

j

vv

j

n jeOipjij

Tji

1

)()(2

1

)(2

1)(

1

Page 9: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Color similarity

The color of the torso of object 1 is similar to the color of the bottom of object 2

(b) is the result of applying the torso color model of object 2 for all pixels

Page 10: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Initialize Presence Map Presence map

The set of presence probabilities of the pixels inside the object

Head line Scan the torso projection profile H0(yi) yHL=arg minyiH0(yi)

Torso line Central vertical axis

Probabilities of the pixels will be larger

Page 11: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Bayesian Classification Only consider pixels in the neighborhood of an object

Pposteriori(Ok|i) : posterior prob. of pixel i belong to object Ok P(i|Ok) : probability defined by torso or bottom model Ppriori(Ok) : presence probability of Ok

Relative coordinate Defined by the head line and central axis

Color model selection for torso or bottom If Pposteriori(Ok|i) >=0.05, then i classified to Ok

)()()( ),(,nkkyxprior

nkposterior OiPOPiOP

rr

Page 12: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Page 13: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Single Object Tracking Flow chart of single object tracking

Newcomer detection By using background subtraction

Page 14: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Tracking process Calculate angles and magnitudes of the flow vectors in the

neighborhood of window Quantize the direction into 12 bins (30 degree/bin) and det

ermine which bin object belong to Find the most significant bin and calculate average flow Shift object window by average flow

Page 15: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Update presence map Size and shape of a moving object change over ti

me Need to update the presence map

If pixel at (xr, yr) classified correctly, increase the corresponding priori prob. for every 10 frames

Page 16: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Page 17: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Tracking Occluded Objects

Optical flow and presence probability are unreliable

Only use color models to estimate object’s central vertical axis

Use distance between central axes to determine object becomes separable

Page 18: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Occlusion detection Each individual object has five attributes

based on its activity

Two object windows touch and form an occlusion window

Two object windows overlap and form an occlusion window

A single object joins an occlusion and form a new occlusion window

Page 19: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Occlusion group separation Compute distance between every two objects i

n an occlusion group as

If Two extreme objects Oi and Oi+1

If |di| > threshold, then determine Oi or Oi+1 can be separate from the original occlusion

kc

kck xxd 1

ki dd max

Page 20: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Object separation example

separate separate

Page 21: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Resume tracking One an object separate from occlusion, we

need to update: Object window location, head line, and torso line Central vertical axis

Left and right boundary Scan the vertical projection profile of Ok From the central vertical axis leftward and then

rightward Head line and torso line

Analyze the horizontal projection profile

Page 22: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Page 23: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Tracking example 1 Format:

Image frame is 160x120x24 bits, 15 frames/sec

Occlusion 1

Occlusion 2

Object 2 separate and join occlusion 1 single object 4

occlusion 1

Page 24: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Tracking example 2 Two occlusion groups merge as one and then separate

to another two occlusion groups

Page 25: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

System evaluation and error analysis Three separation events:

2-object, 3-object, and 4-object separation event Define separation occurs’ accuracy based on:

A single object leaves an occlusion and track him correctly afterward

If an occlusion splits into two, system identify the correct objects in the two pairs.

More 2-object separation events

Page 26: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Page 27: Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Conclusion Object tracking consists of:

Gaussian mixture model Presence probability Optical flow

Objects under occlusion Use color model to distinguish each object and

locate central vertical axes Object separation

Determine by distances between the central vertical axes of objects