Who is There? Video Surveillance & Face Recognition
Anil K. Jain
Michigan State University
March 1, 2016
~30M surveillance cameras in the United States 1
Face Recognition
Probe Gallery
MATCH
How to compute the similarity between two faces?
2
Boston Marathon Bombing Suspects
3
Tamerlan Tsarnaev
Dzhokhar Tsarnaev April, 2013
Search For Dzhokhar Tsarnaev
Law Enforcement
Who is this?
One of them?
Search System
Gallery (80 million faces)
Forensic expert
Wang, Otto, Jain, “Face Search at Scale: 80 Million Gallery“, arXiv:1507.07242v2 [cs.CV] 4
Probe
1 2 3 4 5
6 7 8 9 10
11
Search For Tammy Among 120 Million Faces
5
How Does Face Recognition Work?
6
Face Detection
7
Landmark Detection
8
Image Normalization
9
Feature Extraction
10
Comparison
Distance = 7
11
Same Person Comparison
Distance = 1
12
Faces in a Crowd
13
Pose Variations
Images of one subject in NIST IJB-A data, overlaid with V-J detector & dlib landmarks 14
Driver License Information
2009 driver license photo Gallery: 34 million (30M DMV photos, 4M mugshots)
Face Search in Action
15
1 2 3 4 5
6 7 8 9 10
Top-10 retrievals
Gallery: 34 million (30M DMV photos, 4M mugshots)
Face Search in Action
16
Summary
• Cameras are everywhere: mobile phones to surveillance systems
• Need to determine who is in the image and what they are doing
• Automatic face recognition is difficult: pose, illumination, expression, aging • ~99% accuracy for cooperative subjects with
face captured in controlled conditions • 1.5 million face comparisons/sec/core
17
Security v. Privacy
18
Privacy Concerns Will surveillance cameras be used
• to track people?
• only for the intended purpose?
• to link multiple databases? (function creep)
19
