Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et.

Computer Science Department

Detection, Alignment and Recognitionof Real World FacesErik Learned-Miller

with Vidit Jain, Gary Huang, Andras Ferencz, et al.

Faces in the Wild

2Computer Science

Is Face Recognition Solved?

3Computer Science


QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

“100% Accuracy in Automatic Face Recognition” [!!!]

Science 25 January 2008

4Computer Science



“100% Accuracy in Automatic Face Recognition” [!!!]

Science 25 January 2008

A history of overstated results.

5Computer Science

The Truth

Many different face recognition problems• Out of context, accuracy is meaningless!

Many problems are REALLY HARD!• For some problems

state of the art is 70% or worse!

We have a long way to go!

6Computer Science

Face Recognition at UMass

Labeled Faces in the Wild The Detection-Alignment-Recognition pipeline Congealing and automatic face alignment Hyper-features for face recognition New directions in recognition

7Computer Science

Labeled Faces in the Wild

http://vis-www.cs.umass.edu/lfw/

8Computer Science

The Many Faces of Face Recognition

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.QuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.


9Computer Science






10Computer Science






11Computer Science






12Computer Science



are needed to see this picture.




13Computer Science


13,233 images, with name of each person 5749 people 1680 people with 2 or more images

Designed for the “unseen pair matching problem”.• Train on matched or mismatched pairs.• Test on never-before-seen pairs.

Distinct from problems with “galleries” or training data for each target image.

Best accuracy: currently about 73%!

14Computer Science

Detection-Alignment-Recognition Pipeline



QuickTime™ and aTIFF (LZW) decompressor


DetectionRecognitionAlignment




“Same”

15Computer Science

Detection-Alignment-Recognition Pipeline





DetectionRecognitionAlignment




“Same”

Parts should work together.

16Computer Science


All images are output of a standardface detector.

Also provides aligned images. Consequence: any face recognition algorithm

that works well on LFW can easily be turned into a complete system.

17Computer Science

Congealing (CVPR 2000)

QuickTime™ and a decompressor


18Computer Science

Criterion of Joint Alignment

Minimize sum of pixel stack entropies by transforming each image.

A pixel stack

19Computer Science

Congealing Complex Images

Window around pixel SIFT vector and clusters

SIFT clusters

vector representingprobability of each cluster,or “mixture” of clusters

QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.

21Computer Science

Crash Course on Martian Identification

?

Test: Find Bob after one meeting

Martian training set

=

=

=

Bob

22Computer Science

Training Data



“same”

“different”

23Computer Science

General Approach to Hyper-feature method

Carefully align objects Develop a patch-based model of

image differences. Score match/mismatch based on patch

differences.

24Computer Science

Three Models

1. Universal patch model:P(patchDistance|same)P(patchDistance|different)

2. Spatially dependent patch model:P(patchDistance |same,x,y)P(patchDistance |different,x,y)

3. Hyper-feature dependent model:1. P(patchDistance |same,x,y,appearance)2. P(patchDistance |different,x,y,appearance)

25Computer Science

Universal Patch Model

A single P(dist | same) for all patches

Different blue patches are evidence against a match!

26Computer Science

Spatial Patch Model

P(dist|same,x1,y1) estimated separately from P(dist|same,x2,y2)

Greatly increases discriminativeness of model.

27Computer Science

Hyper-Feature Patch Model

Is the patch from a matching face going tomatch this patch?

28Computer Science


Is the patch from a matching face going tomatch this patch? Probably yes

29Computer Science


What about this patch?

30Computer Science


What about this patch?Probably not.

31Computer Science

Ridiculous Errors from the World’s Best Unconstrained Face Recognition System




32Computer Science

Ridiculous Errors from the World’s Best Unconstrained Face Recognition System





33Computer Science

The New Mission: Estimate Higher Level Features



34Computer Science




Can we guesspose?

35Computer Science




Can we guessgender?

36Computer Science




Can we guessdegree of balding,

beardedness,moustache?

37Computer Science




Can we say thatnone of these individuals are

the same person?

38Computer Science

What can we do with a good segmentation?



39Computer Science

CRF Segmentations



40Computer Science

CRF Segmentations



41Computer Science

Who’s This?

42Computer Science

Who’s This?


43Computer Science

Who’s This?

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

from www.coolopticalillusions.com

Computer Science Department

Thanks

Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et.

Documents