Page 1
10/22/18
1
TheAnalysisofFacesinBrainsandMachines
CS332VisualProcessinginComputerandBiologicalVisionSystems
HMAXmodel
PaulaJohnson
ElizabethWarren
Whyisfaceanalysisimportant?
Remember/recognizepeoplewe’veseenbefore
Categorization– e.g.gender,race,age,kinship
Socialcommunication– emotions/mood,intentions,trustworthiness,competenceorintelligence,attractiveness
Sceneunderstanding,e.g.directionofgazesuggestsfocusofattention
Page 2
10/22/18
2
Whyisfacerecognitionhard?
changingpose changingillumination
changingexpressionclutterocclusion
aging
Jenkins,White,VanMontfort &Burton,Cognition,2011
Howgoodareweatfacerecognition?
Page 3
10/22/18
3
Facerecognitionperformanceinhumans
chanceperformance
testmybrain.org
Wilmeretal.,2012Duchaine &Nakayama,2006
Page 4
10/22/18
4
Bruceetal.,1999
Facerecognitionperformanceinhumans
Whichofthe10photosonthebottomdepictsthetargetface?
Viewersare~70%correct
Performancedegradeswithchangesinpose,expression
Onlyslightimprovementwithshortvideoclipoftarget
Importanceoffamiliarvs.unfamiliarfacerecognition!
Howgoodarethebestmachines?Publicdatabasesoffaceimagesserveasbenchmarks:
LabeledFacesintheWild(LFW,http://vis-www.cs.umass.edu/lfw)>13,000imagesofcelebrities,5,749differentidentities
YouTubeFacesDatabase(YTF,http://www.cs.tau.ac.il/~wolf/ytfaces)3,425videos,1,595differentidentities
Privatefaceimagedatasets:
(Facebook)SocialFaceClassificationdataset4.4millionfacephotos,4,030differentidentities
(Google) 100-200millionfaceimages,~8milliondifferentidentities
LFW YTFFacebookDeepFace 97.4% 91.4%GoogleFaceNet 99.6% 95.1%Humanperformance 97.5% 89.7%
Page 5
10/22/18
5
Machinevisionapplicationsoffacerecognition
surveillance
accesscontrol
security,forensics
Moreapplicationsoffacerecognition
content-basedimageretrieval socialmedia
graphics,HCIhumanoidrobots
Page 6
10/22/18
6
Aspectsoffaceprocessing
Facedetection– findimageregionsthatcontainfaces
Faceidentification– whoistheperson?
Categorization– gender,age,race
Facialexpression– mood,emotion
Non-verbalsocialperceptionandcommunication
ItallbeganwithTakeoKanade (1973)…PhDthesis,PictureProcessingSystembyComputerComplexand
RecognitionofHumanFaces
• Specialpurposealgorithmstolocateeyes,nose,mouth,boundariesofface
• ~40geometricfeatures,e.g.ratiosofdistancesandanglesbetweenfeatures
Page 7
10/22/18
7
- FromtalkbyTakeoKanade,CBMMFaceIDChallengeWorkshop
Eigenfaces forrecognition(Turk&Pentland)PrincipalComponentsAnalysis(PCA)
Goal: reducethedimensionalityofthedatawhileretainingasmuchinformationaspossibleintheoriginaldataset
PCAallowsustocomputealineartransformationthatmapsdatafromahighdimensionalspacetoalowerdimensionalsubspace
Page 8
10/22/18
8
Typicalsampletrainingset…
Oneormoreimagesperperson
Aligned&croppedtocommonpose,size
Simplebackground
SampleimagesfromtheYalefacedatabase
Eigenfaces forrecognition(Turk&Pentland)
1-16
PerformPCA onalargesetoftrainingimages,tocreateasetofeigenfaces,Ei(x,y),thatspanthedataset
Firstcomponentscapturemostofthevariationacrossthedataset,latercomponentscapturesubtlevariations
EachfaceimageF(x,y)canbeexpressedasaweightedcombinationoftheeigenfaces Ei(x,y):
Ψ(x,y):averageface(acrossallfaces)
Ψ(x,y)
http://vismod.media.mit.edu/vismod/demos/facerec/basic.html
F(x,y)=Ψ(x,y)+Σi wi*Ei(x,y)
Page 9
10/22/18
9
RepresentingindividualfacesEachfaceimageF(x,y)canbeexpressedasaweightedcombinationoftheeigenfaces Ei(x,y):
Recognitionprocess:(1) Computeweightswi
fornovelfaceimage
(2) Findimageminfacedatabasewithmostsimilarweights,e.g.
min (wi −wim
i=1
k
∑ )2
F(x,y)=Ψ(x,y)+Σi wi*Ei(x,y)
Faceseverywhere...