Face Detection&Rec

8/2/2019 Face Detection&Rec

1/46

FACE DETECTION

1


2/46

OUTLINE

What is face detection.

The History.

Challenges.

2D-Image Scan.

Biometrics : Skin Texture Analysis. Evaluation of face detection

The Viola and Jones method

Applications.

2


3/46

INTRODUCTION

Steps:

Face Detection:differentiate a human face

from the background of the image or a realtime video.

Feature Detection: record its features.

Face Recognition: Compare it to a database.

3


4/46

INTRODUCTION

Face interface

Face detection

Face recognition

4

Face detection Face recognitionMr.Chan

Prof..Cheng

Face database

Output:


5/46

WHAT IS FACE DETECTION

Technique

employed to

distinguish aHuman face fromthe rest of the

background ofthe image.

5


6/46

FACE DETECTION [1] To detect faces in an image (Not recognize it yet)

Challenges

A picture has 0,1 or many faces

Faces are not the same: with spectacles, mustache etc Sizes of faces vary

Available in most digital cameras nowadays

The simple method

Slide a window across the window and detect faces

Too slow, pictures have too many pixels(1280x1024=1.3M pixels)

6


7/46

THE HISTORY

During 1964 and 1965, Bledsoe, alongwith Helen Chan and Charles Bisson ,worked on using the computer to

recognize human faces.

He was proud of this work, but because

the funding was provided by anunnamed intelligence agency that didnot allow much publicity, little of thework was published.

7


8/46

IMPORTANCE OF FACE DETECTION The first step for any automatic face recognition

system system.

First step in many Human Computer Interactionsystems.

Expression Recognition

Cognitive State/Emotional State Recognition

First step in many surveillance and securitysystems.

Video coding

Automatic Target Recognition(ATR)

8


9/46

CHALLENGES

In Plane Rotation

Out Plane

Rotation Lighting

Aging Effects

Facial Expressions Face Covered by

long Hairs or Hand.9


10/46

CHALLENGES

10


11/46

2D IMAGE SCAN

Different

Approaches:

Knowledge Based

Approach

Feature Invariant

MethodTemplate

Matching Method

11


12/46

KNOWLEDGE-BASED APPROACH

It uses human-

coded rules to model

facial features, such

as two symmetric

eyes, a nose in the

middle and a mouth

underneath thenose.

12


13/46

KNOWLEDGE-BASED APPROACH-

SUMMARY

pros: Easy to come up with simple rules

Based on the coded rules, facial features in an input image areextracted first, and face candidates are identified

Work well for face localization in uncluttered background

con:

Difficult to translate human knowledge into rules precisely: detailedrules fail to detect faces and general rules may find many falsepositives

Difficult to extend this approach to detect faces in different poses:implausible to enumerate all the possible cases

13


14/46

FEATURE INVARIANT METHOD

Feature invariant methods

try to find facial features

which are invariant to

pose, lighting condition orrotation.

Skin colors, edges and

shapes fall into thiscategory.

14


15/46

FEATURE INVARIANT METHOD-

NODAL POINT ANALYSIS

Every face has numerous,distinguishable landmarks,the different peaks and

valleys that make up theface

o Distance between theeyes

o Width of the nose

o Depth of the eye sockets

o The shape of the

cheekbones 15


16/46

FEATURE INVARIANT METHOD-

SUMMARY

Pros:

Features are invariant to pose and change inorientation.

Cons:

Difficult to locate facial features due toseveral corruption (illumination, noise,occlusion)

Difficult to detect features in complex 16


17/46

TEMPLATE MATCHING METHOD

Template matchingmethods calculatethe correlation

between a testimage and a pre-selected facial

templates.

17


18/46

TEMPLATE MATCHING METHOD-

SUMMARY

Pros:

Simple

Cons:

Templates needs to be initialized near the faceimages

Difficult to enumerate templates for different

poses (similar to knowledge-based methods)

18


19/46

BIOMETRICS

SKIN TEXTURE ANALYSIS

Using skin color to find facesegments is a vulnerabletechnique.

Non-animate objects with

the same color as skin canbe picked up since thetechnique uses colorsegmentation.

Then the face can be pickedup using any of theapproaches.

19


20/46

SKIN TEXTURE ANALYSIS:

ADVANTAGES

Lack of restriction to orientation or size of faces.

A good algorithm can handle complexbackgrounds.

It is relatively insensitive to changes inexpression, including blinking, frowning orsmiling

Has the ability to compensate for mustache orbeard growth and the appearance ofeyeglasses.

20


21/46

EVALUATION OF FACE DETECTION Detection rate

Give positive results in locations where faces exist

Should be high > 95%

False positive rate

The detector output is positive but it is false (there is actually noface).Definition of False positive: A result that is erroneously positive when a situation is normal. Anexample of a false positive: a particular test designed to detect cancer of the toenail is positive but theperson does not have toenail cancer. (http://www.medterms.com/script/main/art.asp?articlekey=3377)

Should be low


22/46

EXERCISEWhat are the detection rate and false

detection rate here?

Answer

detection rate=(8/9)*100%

false detection rate=(1/9)*100%

22

False positive result

9 faces in the picture, 8

are correctly detected.

1 window reportedto have face is infact not a face


23/46

THE VIOLA AND JONES METHOD [1] The most famous method

Training may need weeks

Recognition is very fast, e.g. real-time for

digital cameras. Techniques

Integral image for feature extraction

Ada-Boost for feature selectionAttentional cascade for fast rejection of non-face

sub-windows

23


24/46

24

IMAGE FEATURES REF[3]Rectangle filters

Rectangle_Feature_value f=

(pixels in white area)

(pixels in shaded area)


25/46

EXERCISE Find the

Rectangle_Feature_val

ue(f) of the boxenclosed by the dottedline

Rectangle_Feature_value f=

(pixels in white area)

(pixels in shaded area)

f=(8+7)-(0+1) =15-1= 14

1 2 3 3

3 0 1 3

5 8 7 1

0 2 3 6

25


26/46

26

EXAMPLE: A SIMPLE FACE DETECTIONMETHOD USING ONE FEATURE

Result

This is a face:T he eye-part is dark,the nose-part is brightSo f is large, hence it is face

This is not a face.Because f is small

Rectangle_Feature_value ff= (pixels in white area) (pixels in shaded area)

If (f) is large it is face ,i.e.if (f)>threshold, then faceElse non-face


27/46

27

HOW TO FIND FEATURES FASTERINTEGRAL IMAGES FAST CALCULATION METHOD[LAZEBNIK09 ]

The integral image =

sum of allpixel valuesabove and to the left of(x,y)

Can be found veryquickly

(x,y)


28/46

28

COMPUTING THE INTEGRAL IMAGE[LAZEBNIK09 ]


29/46

29

COMPUTING THE INTEGRAL IMAGE[LAZEBNIK09 ]

Cumulative row sum: s(x, y) = s(x1, y) + i(x, y)

Integral image: ii(x, y) = ii(x, y1) + s(x, y)

ii(x, y-1)

s(x-1, y)i(x, y)


30/46

30

CALCULATE SUM WITHIN A RECTANGLE A,B,C,D are the values of the

integral images at the corners ofthe rectangle R.

The sum of image values inside Ris:

Area_R = A B C + D

If A,B,C,D are found , only 3additions are needed to findArea_R

D

C A

B

RectangleR

Area= Area_R


31/46

WHY DO WE NEED TO FIND PIXEL SUM OF RECTANGLES?ANSWER: WE WANT TO GET FACE FEATURES You may consider these

features as face features

Two eyes= (Area_A-Area_B)

Nose =(Area_C+Area_E-Area_D) Mouth =(Area_F+Area_H-Area_G)

They can be different sizes,

polarity and aspect ratios

31

AB

CDE

FGH


32/46

32

FACE FEATURE AND EXAMPLE-1

+2IntegralImage

A face

Shaded area

White areaF=Feat_val =pixel sum in shared area - pixel sum in whiteareaExample

Pixel sum in white area=216+102+78+129+210+111=846

Pixel sum in shared area=10+20+4+7+45+7=93

Feat_val=F=846-93If F>threshold,

feature=+1Else

feature=-1 End if;

We can choose threshold =768 , so feature is+1.

10 20 4

7 45 7

216 102 78

129 210 111

Pixel values insideThe areas


33/46

THE DETECTION CHALLENGE Use 24x24 base window

For y=1;y


34/46

SOLUTION TO MAKE IT EFFICIENT The whole 162,336 feature set is too large

Solution: select good features to make it moreefficient

Use: Boosting

Boosting

Combine many small weak classifiers to become

a strong classifier.Training is needed

34


35/46

35

BOOSTING FOR FACE DETECTION

Define weak learners based onrectangle features

otherwise0

)(if1)(

tttt

t

pxfpxh

window

value of rectangle feature

Pt=polarity{+1,-1}

threshold


36/46

36

AdaBoost training E.g. Collect 5000 faces, and 9400 non-faces.

Different scales.

Use AdaBoost for training to build a strongclassifier.

Pick suitable features of different scales andpositions, pick the best few. (Take months to do ,details is in [Viola 2004] paper)

Testing Scan through the image, pick a window and

rescale it to 24x24,

Pass it to the strong classifier for detection.

Report face, if the output is positive

FACE DETECTION USING ADABOOST


37/46

37

BOOSTING FOR FACE DETECTION [VIOLA2004] In the paper it shows that the following two features (obtained

after training) in cascaded picked by AdaBoost have 100%

detection rate and 50% false positive rate

But 50% false positive rate is not good enough

Approach [viola2004] :Attentional cascade

type2type3

Pick a window in theimage and rescale it to24x24 as image

I.e.H(face)=Sign{1h1(image)+2h2(image)}

H(face)=+1 faceH(face)=-1non-face

h1(image)h2(image)


38/46

38

BOOSTING FOR FACE DETECTION A 200-feature classifier can yield 95% detection rate and a false positive rate

of 1 in 14084 (Still not god enough) Recall: False positive rate

The detector output is positive but it is false (there is actually no face).Definition of False positive: A result that is erroneously positive when a

situation is normal. An example of a false positive: a particular test

designed to detect cancer of the toenail is positive but the person does nothave toenail cancer.(http://www.medterms.com/script/main/art.asp?articlekey=3377)

Still notgoodenough!

False positive rate

CorrectDetection

rate

X10-3


39/46

39

TO IMPROVE FALSE POSITIVE RATE:ATTENTIONAL CASCADE

Cascade of many AdaBoost strong classifiers Begin with with simple classifiers to reject many

negative sub-windows

Many non-faces are rejected at the first few

stages. Hence the system is efficient enough for real time

processing.

AdaboostClassifier1

AdaboostClassifier2

AdaboostClassifier3

True True TrueFacefound

Non-face Non-face

Input image

False False False

Non-face


40/46

AN EXAMPLEMore features for later stages in the

cascade [viola2004]

40

2 features 10 features 25 features 50 features

type3type2

AdaboostClassifier1

AdaboostClassifier2

AdaboostClassifier3


Non-face Non-face

Input image

False False False

Non-face


41/46

41

ATTENTIONAL CASCADEChain classifiers that are

progressively morecomplex and have lowerfalse positive rates: vs false neg determined by

% False Pos

%D

etection

0 50

0

100

Receiver operating

characteristic

AdaboostClassifier1

AdaboostClassifier2

AdaboostClassifier3


Non-face Non-face

Input image

False False False

Non-face

False positive rate


42/46

42

ATTENTIONAL CASCADE [VIOLA2004] Detection rate for each stage is 0.99 , for 10

stages, overall detection rate is 0.9910 0.9

False positive rate at each stage is 0.3, for 10stages

false positive rate =0.310 610-6)

AdaboostClassifier1

AdaboostClassifier2

AdaboostClassifier3


Non-face Non-face

Input image

False False False

Non-face

F R iti


43/46

DETECTION PROCESS IN PRACTICE[SMYTH2007]Use 24x24 sub-window Scaling

scale the detection (not the input image)

Features evaluated at scales by factors of 1.25at each level Location : move detector around the image (1

pixel increments)

Final detectionsA real face may result in multiple nearbydetections (merge them to become the finalresult)

43

Face RecognitionFace Recognition

FACE RECOGNITION


44/46

FACE RECOGNITION

Distinguishing a specific face from other faces

2009 TotallyLooksLike.com

FACE RECOGNITION:


45/46

FACE RECOGNITION:

APPLICATIONS Biometrics / access control

""Minority Report" 2002 20th Century Fox

Superbad" 2007 Columbia Pictures

Searching mugshotdatabases

Tagging photo albums Detecting fake ID cards

o no action requiredo scan many people at onceo places: airports, banks, safeso data: laptops, medical info


46/46

APPLICATIONS Security measure at

ATMs Digital Cameras

Public Surveillance

(CCTVs) atAirports, Hospitals, etc.

Televisions andcomputers can

save energy by reducingthe

brightness.

Face Detection&Rec

Documents