Who is There? Video Surveillance & Face Recognitionbiometrics.cse.msu.edu/.../AnilJain...March2016.pdf · Who is There? Video Surveillance & Face Recognition Anil K. Jain Michigan

Who is There? Video Surveillance & Face Recognition

Anil K. Jain

Michigan State University

March 1, 2016

~30M surveillance cameras in the United States 1

Face Recognition

Probe Gallery

MATCH

How to compute the similarity between two faces?

2

Boston Marathon Bombing Suspects

3

Tamerlan Tsarnaev

Dzhokhar Tsarnaev April, 2013

Search For Dzhokhar Tsarnaev

Law Enforcement

Who is this?

One of them?

Search System

Gallery (80 million faces)

Forensic expert

Wang, Otto, Jain, “Face Search at Scale: 80 Million Gallery“, arXiv:1507.07242v2 [cs.CV] 4

Probe

1 2 3 4 5

6 7 8 9 10

11

Search For Tammy Among 120 Million Faces

5

How Does Face Recognition Work?

6

Face Detection

7

Landmark Detection

8

Image Normalization

9

Feature Extraction

10

Comparison

Distance = 7

11

Same Person Comparison

Distance = 1

12

Faces in a Crowd

13

Pose Variations

Images of one subject in NIST IJB-A data, overlaid with V-J detector & dlib landmarks 14

Driver License Information

2009 driver license photo Gallery: 34 million (30M DMV photos, 4M mugshots)

Face Search in Action

15

1 2 3 4 5

6 7 8 9 10

Top-10 retrievals

Gallery: 34 million (30M DMV photos, 4M mugshots)

Face Search in Action

16

Summary

• Cameras are everywhere: mobile phones to surveillance systems

• Need to determine who is in the image and what they are doing

• Automatic face recognition is difficult: pose, illumination, expression, aging • ~99% accuracy for cooperative subjects with

face captured in controlled conditions • 1.5 million face comparisons/sec/core

17

Security v. Privacy

18

Privacy Concerns Will surveillance cameras be used

• to track people?

• only for the intended purpose?

• to link multiple databases? (function creep)

19