1 Video Surveillance E6998 -007 Senior/Feris/Tian
Emerging Topics in Video Surveillance
Rogerio Feris
IBM TJ Watson Research Center
http://rogerioferis.com
2 Video Surveillance E6998 -007 Senior/Feris/Tian
Outline
Video Surveillance in Crowded Scenarios
Online Learning – Self-adaptation in Surveillance
Other Recent Topics
3 Video Surveillance E6998 -007 Senior/Feris/Tian
Simple Scenarios
Few Objects – Background Subtraction + Tracking + High-level Event/Alert Detection
Current systems work well
4 Video Surveillance E6998 -007 Senior/Feris/Tian
Crowded Scenarios
Many objects, occlusions, shadows, etc.
Object Segmentation, Tracking and Event Analysis in crowded scenarios: Open Problem!
5 Video Surveillance E6998 -007 Senior/Feris/Tian
Parts-based Detectors
[Pedro et al, A discriminatively trained, multiscale, deformable part model, CVPR’08]
Root filter (low-res) + Parts filters (high-res)
Occlusion Handling
6 Video Surveillance E6998 -007 Senior/Feris/Tian
Parts-based Detectors
Score of a window: score of root + score of parts
Score of Parts: Appearance + Geometry
Efficient localization of parts through Dynamic Programming
SVM Classification (Structured prediction)
7 Video Surveillance E6998 -007 Senior/Feris/Tian
Detecting Pedestrians in Crowds
[Leibe et al, Pedestrian Detection in Crowded Scenes, CVPR’05]
Combination of different models: bag of features, segmentation, and chamfer matching
8 Video Surveillance E6998 -007 Senior/Feris/Tian
Tracking in Crowds
[Andriluka et al, People-tracking-by-detection and people-detection-by-tracking, CVPR’08]
Extends [Leibe et al, CVPR’05] to temporal-domain and person articulation (parts) estimation
Click for Video Demo
9 Video Surveillance E6998 -007 Senior/Feris/Tian
Crowd Segmentation
[Dong et al, Fast Crowd Segmentation Using Shape Indexing, ICCV’07]
10 Video Surveillance E6998 -007 Senior/Feris/Tian
Crowd Analysis
[Ali & Shah, A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis, CVPR’07]
11 Video Surveillance E6998 -007 Senior/Feris/Tian
Online Learning
12 Video Surveillance E6998 -007 Senior/Feris/Tian
Initialize sample weights For each cycle:
Find a classifier/rectangle feature that performs well on the weighted samples
Increase weights of misclassified examples
Return a weighted combination of classifiers
Adaboost ensembles many weak classifiers into one single strong classifier
Offline Adaboost Learning
13 Video Surveillance E6998 -007 Senior/Feris/Tian
Offline Adaboost Learning
Major Problems:
Large number of examples required to train a robust classifier
time consuming to label data
slow training (may take several days)
No Adaptation to particular surveillance scenarios
14 Video Surveillance E6998 -007 Senior/Feris/Tian
Learning from Small Sets
Choice of Features (Levi & Weiss, CVPR’04)
Co-Training (Levin & Viola, ICCV’2003)
Online Adaptation:
Online Boosting (Oza’01, Javed’05, Bischof’06, Pham’07)
15 Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting [Oza,2001]
Train a generic strong classifier (set of weak classifiers, # of weak classifiers fixed) on a small training set.
Online Process:
Given one single example with known label:
“Slide” the example over each weak classifier
When the weak classifier receives the example
update the weak classifier online
update the weight of the example and pass to the next weak classifier
16 Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting [Oza,2001]
17 Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting [Oza,2001]
18 Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting Car and People Detection [Omar Javed, CVPR’05]
Train a generic strong classifier (set of weak classifiers, # of weak classifiers fixed) on a small training set.
While running the classifier on unlabeled data, if an example is confidently predicted by a subset of weak classifiers use it for online learning
“Co-training framework”
BGS used for efficiency, for using more expensive features, and for balancing the number of positive and negative examples
19 Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting Car and People Detection [Omar Javed, CVPR’05]
20 Video Surveillance E6998 -007 Senior/Feris/Tian
More Recent Work
[Bo Wu & Nevatia, Improving Part-based Object Detection by Unsupervised, Online Boosting, CVPR’07]
[Helmut & Hurst, Online Boosting and Vision, CVPR’06]
[Pham & Cham, Online Learning Asymmetric Boosted Classifiers for Object Detection, CVPR’07]
[Huang et al.,Incremental Learning of Boosted Face Detector, ICCV’07] – Boosting Adaptation
IEEE Online Learning for Classification Workshop (CVPR’08)
21 Video Surveillance E6998 -007 Senior/Feris/Tian
Other Recent Topics
22 Video Surveillance E6998 -007 Senior/Feris/Tian
High-Resolution Imagery[Kopf et al, Capturing and Viewing Gigapixel Images, SIGGRAPH’07]
How can we make use of high-resolution in video analytics?
Much more info – e.g., in face reco: skin texture, iris, etc.
23 Video Surveillance E6998 -007 Senior/Feris/Tian
Next Generation Neural Networks[Hinton, Reducing the dimensionality of data with neural networks, Science 2006]
New algorithm for learning deep belief nets
State-of-the art results in MNIST digit dataset (better than SVMs)
Youtube talk at Google: http://www.youtube.com/watch?v=AyzOUbkUf3M
Matlab Code: http://www.cs.toronto.edu/~hinton/
24 Video Surveillance E6998 -007 Senior/Feris/Tian
Learning with lots of data
How can we recognize thousands of products in a retail store for loss prevention?
80 Million Tiny Images (http://www.cs.nyu.edu/~fergus/)
Surveillance with Moving Cameras
Cameras in vehicles, or even wearable cameras. New challenges: object detection, etc.
[Leibe et al, Dynamic 3D Scene Analysis from a Moving Vehicle, CVPR 2007]
25 Video Surveillance E6998 -007 Senior/Feris/Tian
Many more recent topics:
Check for papers in recent computer vision conferences (like CVPR, ICCV, and ECCV) and also specialized workshops/conferences such as AVSS and PETS