Video Surveillance E6998 -007 Senior/Feris/Tian 1 Emerging Topics in Video Surveillance Rogerio Feris IBM TJ Watson Research Center [email protected].

1 Video Surveillance E6998 -007 Senior/Feris/Tian

Emerging Topics in Video Surveillance

Rogerio Feris

IBM TJ Watson Research Center

[email protected]

http://rogerioferis.com


Outline

Video Surveillance in Crowded Scenarios

Online Learning – Self-adaptation in Surveillance

Other Recent Topics


Simple Scenarios

Few Objects – Background Subtraction + Tracking + High-level Event/Alert Detection

Current systems work well


Crowded Scenarios

Many objects, occlusions, shadows, etc.

Object Segmentation, Tracking and Event Analysis in crowded scenarios: Open Problem!


Parts-based Detectors

[Pedro et al, A discriminatively trained, multiscale, deformable part model, CVPR’08]

Root filter (low-res) + Parts filters (high-res)

Occlusion Handling


Parts-based Detectors

Score of a window: score of root + score of parts

Score of Parts: Appearance + Geometry

Efficient localization of parts through Dynamic Programming

SVM Classification (Structured prediction)


Detecting Pedestrians in Crowds

[Leibe et al, Pedestrian Detection in Crowded Scenes, CVPR’05]

Combination of different models: bag of features, segmentation, and chamfer matching


Tracking in Crowds

[Andriluka et al, People-tracking-by-detection and people-detection-by-tracking, CVPR’08]

Extends [Leibe et al, CVPR’05] to temporal-domain and person articulation (parts) estimation

Click for Video Demo


Crowd Segmentation

[Dong et al, Fast Crowd Segmentation Using Shape Indexing, ICCV’07]


Crowd Analysis

[Ali & Shah, A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis, CVPR’07]


Online Learning


Initialize sample weights For each cycle:

Find a classifier/rectangle feature that performs well on the weighted samples

Increase weights of misclassified examples

Return a weighted combination of classifiers

Adaboost ensembles many weak classifiers into one single strong classifier

Offline Adaboost Learning


Offline Adaboost Learning

Major Problems:

Large number of examples required to train a robust classifier

time consuming to label data

slow training (may take several days)

No Adaptation to particular surveillance scenarios


Learning from Small Sets

Choice of Features (Levi & Weiss, CVPR’04)

Co-Training (Levin & Viola, ICCV’2003)

Online Adaptation:

Online Boosting (Oza’01, Javed’05, Bischof’06, Pham’07)


Online Boosting [Oza,2001]

Train a generic strong classifier (set of weak classifiers, # of weak classifiers fixed) on a small training set.

Online Process:

Given one single example with known label:

“Slide” the example over each weak classifier

When the weak classifier receives the example

update the weak classifier online

update the weight of the example and pass to the next weak classifier






Online Boosting Car and People Detection [Omar Javed, CVPR’05]

Train a generic strong classifier (set of weak classifiers, # of weak classifiers fixed) on a small training set.

While running the classifier on unlabeled data, if an example is confidently predicted by a subset of weak classifiers use it for online learning

“Co-training framework”

BGS used for efficiency, for using more expensive features, and for balancing the number of positive and negative examples


Online Boosting Car and People Detection [Omar Javed, CVPR’05]


More Recent Work

[Bo Wu & Nevatia, Improving Part-based Object Detection by Unsupervised, Online Boosting, CVPR’07]

[Helmut & Hurst, Online Boosting and Vision, CVPR’06]

[Pham & Cham, Online Learning Asymmetric Boosted Classifiers for Object Detection, CVPR’07]

[Huang et al.,Incremental Learning of Boosted Face Detector, ICCV’07] – Boosting Adaptation

IEEE Online Learning for Classification Workshop (CVPR’08)


Other Recent Topics


High-Resolution Imagery[Kopf et al, Capturing and Viewing Gigapixel Images, SIGGRAPH’07]

How can we make use of high-resolution in video analytics?

Much more info – e.g., in face reco: skin texture, iris, etc.


Next Generation Neural Networks[Hinton, Reducing the dimensionality of data with neural networks, Science 2006]

New algorithm for learning deep belief nets

State-of-the art results in MNIST digit dataset (better than SVMs)

Youtube talk at Google: http://www.youtube.com/watch?v=AyzOUbkUf3M

Matlab Code: http://www.cs.toronto.edu/~hinton/


Learning with lots of data

How can we recognize thousands of products in a retail store for loss prevention?

80 Million Tiny Images (http://www.cs.nyu.edu/~fergus/)

Surveillance with Moving Cameras

Cameras in vehicles, or even wearable cameras. New challenges: object detection, etc.

[Leibe et al, Dynamic 3D Scene Analysis from a Moving Vehicle, CVPR 2007]


Many more recent topics:

Check for papers in recent computer vision conferences (like CVPR, ICCV, and ECCV) and also specialized workshops/conferences such as AVSS and PETS

Video Surveillance E6998 -007 Senior/Feris/Tian 1 Emerging Topics in Video Surveillance Rogerio Feris IBM TJ Watson Research Center [email protected].

Documents

video surveillance e6998

outline video surveillance

video demo slide

online learning slide

weak classifier slide

video analytics

cvpr05 slide

cvpr07 slide