Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et.

Computer Science Department

Detection, Alignment and Recognitionof Real World FacesErik Learned-Miller

with Vidit Jain, Gary Huang, Andras Ferencz, et al.

Faces in the Wild

2Computer Science

Is Face Recognition Solved?

3Computer Science

Is Face Recognition Solved?

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

“100% Accuracy in Automatic Face Recognition” [!!!]

Science 25 January 2008

4Computer Science

Is Face Recognition Solved?

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

“100% Accuracy in Automatic Face Recognition” [!!!]

Science 25 January 2008

A history of overstated results.

5Computer Science

The Truth

Many different face recognition problems• Out of context, accuracy is meaningless!

Many problems are REALLY HARD!• For some problems

state of the art is 70% or worse!

We have a long way to go!

6Computer Science

Face Recognition at UMass

Labeled Faces in the Wild The Detection-Alignment-Recognition pipeline Congealing and automatic face alignment Hyper-features for face recognition New directions in recognition

7Computer Science

Labeled Faces in the Wild

http://vis-www.cs.umass.edu/lfw/

8Computer Science

The Many Faces of Face Recognition

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.QuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.

Labeled Faces in the Wild

9Computer Science

The Many Faces of Face Recognition

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.QuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.

Labeled Faces in the Wild

10Computer Science

The Many Faces of Face Recognition

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.QuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.

Labeled Faces in the Wild

11Computer Science

The Many Faces of Face Recognition

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.QuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.

Labeled Faces in the Wild

12Computer Science

The Many Faces of Face Recognition

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Labeled Faces in the Wild

13Computer Science

Labeled Faces in the Wild

13,233 images, with name of each person 5749 people 1680 people with 2 or more images

Designed for the “unseen pair matching problem”.• Train on matched or mismatched pairs.• Test on never-before-seen pairs.

Distinct from problems with “galleries” or training data for each target image.

Best accuracy: currently about 73%!

14Computer Science

Detection-Alignment-Recognition Pipeline

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

DetectionRecognitionAlignment

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

“Same”

15Computer Science

Detection-Alignment-Recognition Pipeline

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

DetectionRecognitionAlignment

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

“Same”

Parts should work together.

16Computer Science

Labeled Faces in the Wild

All images are output of a standardface detector.

Also provides aligned images. Consequence: any face recognition algorithm

that works well on LFW can easily be turned into a complete system.

17Computer Science

Congealing (CVPR 2000)

QuickTime™ and a decompressor

are needed to see this picture.

18Computer Science

Criterion of Joint Alignment

Minimize sum of pixel stack entropies by transforming each image.

A pixel stack

19Computer Science

Congealing Complex Images

Window around pixel SIFT vector and clusters

SIFT clusters

vector representingprobability of each cluster,or “mixture” of clusters

QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.

21Computer Science

Crash Course on Martian Identification

?

Test: Find Bob after one meeting

Martian training set

=

=

=

Bob

22Computer Science

Training Data

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

“same”

“different”

23Computer Science

General Approach to Hyper-feature method

Carefully align objects Develop a patch-based model of

image differences. Score match/mismatch based on patch

differences.

24Computer Science

Three Models

1. Universal patch model:P(patchDistance|same)P(patchDistance|different)

2. Spatially dependent patch model:P(patchDistance |same,x,y)P(patchDistance |different,x,y)

3. Hyper-feature dependent model:1. P(patchDistance |same,x,y,appearance)2. P(patchDistance |different,x,y,appearance)

25Computer Science

Universal Patch Model

A single P(dist | same) for all patches

Different blue patches are evidence against a match!

26Computer Science

Spatial Patch Model

P(dist|same,x1,y1) estimated separately from P(dist|same,x2,y2)

Greatly increases discriminativeness of model.

27Computer Science

Hyper-Feature Patch Model

Is the patch from a matching face going tomatch this patch?

28Computer Science

Hyper-Feature Patch Model

Is the patch from a matching face going tomatch this patch? Probably yes

29Computer Science

Hyper-Feature Patch Model

What about this patch?

30Computer Science

Hyper-Feature Patch Model

What about this patch?Probably not.

31Computer Science

Ridiculous Errors from the World’s Best Unconstrained Face Recognition System

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.QuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.

32Computer Science

Ridiculous Errors from the World’s Best Unconstrained Face Recognition System

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

33Computer Science

The New Mission: Estimate Higher Level Features

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

34Computer Science

The New Mission: Estimate Higher Level Features

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Can we guesspose?

35Computer Science

The New Mission: Estimate Higher Level Features

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Can we guessgender?

36Computer Science

The New Mission: Estimate Higher Level Features

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Can we guessdegree of balding,

beardedness,moustache?

37Computer Science

The New Mission: Estimate Higher Level Features

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Can we say thatnone of these individuals are

the same person?

38Computer Science

What can we do with a good segmentation?

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

39Computer Science

CRF Segmentations

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

40Computer Science

CRF Segmentations

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

41Computer Science

Who’s This?

42Computer Science

Who’s This?

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

43Computer Science

Who’s This?

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

from www.coolopticalillusions.com

Computer Science Department

Thanks