1 Face Recognition: Motivation Overview: 1. Why faces? 2. Applications for Face Analysis Technology? 3. Faces and Human Perception. 2 Why Faces? Technology Perspective: • General challenge for Computer Vision − Faces are highly variable. − Geometry and appearance not too complicated, however, already difficult to describe with simple geometric basics or functions. • Many possible commercial applications. Human Perspective: • Face analysis is very easy for humans! -- Can't be difficult!? • Understanding the human visual system, might help to understand the human brain.
22
Embed
Face Recognition: Motivation · 2018-12-05 · Face Recognition Systems: Performance 14 Since the mid 90th there are several companies on the market and sell face recognition systems.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
1
Face Recognition: Motivation
Overview:
1. Why faces?
2. Applications for Face Analysis Technology?
3. Faces and Human Perception.
2
Why Faces?
Technology Perspective:
• General challenge for Computer Vision
− Faces are highly variable.
− Geometry and appearance not too complicated, however,
already difficult to describe with simple geometric basics or
functions.
• Many possible commercial applications.
Human Perspective:
• Face analysis is very easy for humans! -- Can't be difficult!?
• Understanding the human visual system, might help to
Security : Personal Device (Cell phone etc) Logon /
Medical Records / Internet Access
Law Enforcement Advanced Video Surveillance / CCTV Control
& Surveillance : Shoplifting / Drug Trafficking / Portal Control
4
7
The Face as Biometric Feature
Face recognition from different modalities:
• from single image.
• from two or more image, from video.
• from 3D data ( laser or structured light technology).
Face recognition covers different tasks:
• Face verification
• Face identification
• Expression and emotion recognition
• Age analysis
• Lip reading
• ….
8
Face Verification versus Identification
Face Verification
Is this the person,
the person claims to be?
Face Identification
Who is this person?
Face identification is the more difficult task! Current commercial systems are mostly limited to the verification task.
e.g. the ‘SmartGate’ installation at Sydney’s airport for crew members utilizes software from Cognitec. The system compares the face with stored images of the person matching the identity as claimed in the passport (passport picture not used).
An Example: Prof. Dr. Antonio Loprieno, Former rector of the University of Basel. The picture was taken a few years ago.
5
9
The machine readable biometric Passport
Germany : mandatory
Switzerland: voluntary!?
In a machine readable part at minimum the following information is stored:
• name, family name,
• county, passport number
• gender, date of birth
• date of expiration
In the RFID-Chip additional biometric information is stored:
• passport photograph
• two fingerprints ( Germany since 2007 )
10
How to generate a valid passport photo I
From: “Deutsche Bundesdruckerei”
6
11
How to generate a valid passport photo II
From: “Deutsche Bundesdruckerei”
12
Face Recognition at the Train Station in Mainz
At the main train station in Mainz the German Bundes Kriminalamt tested several commercial face recognition systems for their practicability (2006).
200 people equipped with an RIFD chip pass every day together with 20000 other persons the setup.
Controversial results!
7
13
Basic Face Recognition System
Face Detection
Feature Extraction
Face Recognition
Identification / Verification
Input Image / Video Related Applications
• Face Tracking • Pose Estimation • HCI Systems
Related Applications
• Gaze Tracking • Emotion Recognition • HCI Systems
Approach
• Holistic Templates • Features / Geometry • Hybrid
14
Face Recognition Systems: Performance
Since the mid 90th there are several companies on the market and sell face recognition systems.
Is face recognition solved?
How to evaluate recognition systems?
There is no general standardized test, however, a series of tests have been performed in the past.
1. FRVT Face Recognition Vendor Tests: NIST & DARPA
http://www.frvt.org
2. M2VTS, XM2VTS, BANCA: EU-sponsored research projects
http://www.ee.surrey.ac.uk/Research/VSSP/xm2vtsdb
http://banca.ee.surrey.ac.uk.
3. Colorade State University Web Site: DARPA
http://www.cs.colostate.edu/evalfacerec/
8
15
FRVT
organized by Dr. Jonathon Phillips
NIST (& DARPA)
http://www.frvt.org
“ Face Recognition Vendor Tests (FRVT) provide independent government evaluations of commercially available and prototype face recognition technologies. These evaluations are
designed to provide U.S. Government and law enforcement agencies with information to assist them in determining where and how facial recognition technology can best be deployed. In
addition, FRVT results help identify future research directions for the face recognition community.”
The evaluation is open to mature prototypes or commercial systems from academia and industry.
16
FRVT History
Since 1993 a series of test have been performed funded though various US government agencies ( NIST, DARPA, DoD).
Comment: This section on “human face perception” does not try to be comprehensive, it’s a simple attempt to convey a first impression on the research done in this field.
14
28 Human Face Perception:
What do we know – What can we learn?
Idea: First, investigate how the human brain solves the face recognition
task and second, transform this findings in computer algorithms!
If that is not directly possible, do it iteratively. 1.) Implement some first ideas
2.) Compare with human performance and behavior
3.) Implement better algorithms
…. and so on
1
29
Methods
Modeling Experiment
Psychophysics
&
fMRI
Computational
Face Model
Architecture & Development
Performance & Behavior
Investigation of Higher Cognitive Functions!?
15
30
Human Face Perception:
A “ Facespace ” was created using a morphing tool!
From a set of example faces the average face was computed.
Then the morphing tool was used to generate “morphs” between the original and the average and also extrapolations beyond the average. This extrapolations we call “anti-faces”.
An example of an experiment:
Prototype-referenced shape encoding revealed by high-level aftereffects.
David Leopold, Alice J. O'Toole, Thomas Vetter, & Volker Blanz Nature Neuroscience vol.4 no.1 (2001) 89-94.
31
Face Space
The experiment: Stimuli
0.25 0.50 0.75 1.00
16
32
Henry
Jim
John
Adam
(button 3)
(button 1)
(button 4)
(button 2)
ORIGINAL FACE ANTI-FACE
The experiment!
33
Experiment: A ‘Naming Task’, one out of four!
0.25 JOHN
Test 200 ms
RESPONSE
time
Identity Strength
Corr
ect Id
en
tifica
tio
ns
MG,MA,A
F,UH,
JP,ML,MS
0.00
0.50
1.00
0.75
0.25
-0.2 0.0 0.2 0.4
17
34
0.25 JOHN
Test 200 ms
RESPONSE
time
Identity Strength
Corr
ect Id
en
tifica
tio
ns
MG,MA,A
F,UH,
JP,ML,MS
0.00
0.50
1.00
0.75
0.25
-0.2 0.0 0.2 0.4
Adaptation 5 s
ANTI-JOHN
Experiment: A ‘Naming Task’, one out of four!
35
• Average face is special.
• The human brain is adaptive within seconds.
• “Morphs” between the average and an individual
code for the same identity.
• Aftereffect not only in topographic visual areas.
• …….
The experiment: Conclusions
18
36
Facial Attributes I: Gender
feminine masculine original
male
female
37
Experiment I: Hypotheses
H1 - Subjects rate the leadership aptitude of …
a) a man higher than of a woman.
b) a masculine person higher than of a feminine person.
H2 - Subjects rate the social competence of...
a) a woman higher than of a man.
b) a feminine person higher than of a masculine person.
Not only the gender but also the facial features of a person
affect gender-stereotypic attributions.
19
38
Experiment I: Results
feminine masculine
male
female 4.7 4.09*
Mean SC Mean LA
Mean SC 4.77* 4.58*
Mean LA 4.25 4.32
4.66 4.48*
40
Some other findings and experiments
Examples from:
Face Recognition by Humans: Nineteen Results Researchers Should Know About. Pawan Sinha et al., Proceedings of the IEEE Vol. 94, No. 11, November 2006
Example 1:
Fig. 1. Unlike current machine-based systems, human observers are able to handle significant degradations in face images. For instance, subjects are able to recognize more than half of all familiar faces shown to them at the resolution depicted here. Individuals shown in order are: Michael Jordan, Woody Allen, Goldie Hawn, Bill Clinton, Tom Hanks, Saddam Hussein, Elvis Presley, Jay Leno, Dustin Hoffman, Prince Charles, Cher, and Richard Nixon.
20
41
some other findings …..
Example 2:
Fig. 3. Images which contain exclusively contour information are very difficult to recognize, suggesting that high-spatial frequency information, by itself, is not an adequate cue for human face recognition processes. Shown here are Jim Carrey (left) and Kevin Costner.
Example 3:
Fig. 4. Try to name the famous faces depicted in the two halves of the left image. Now try the right image. Subjects find it much more difficult to perform this task when the halves are aligned (left) compared to misaligned halves (right), presumably because holistic processing interacts (and in this case, interferes) with feature-based processing. The two individuals shown here are Woody Allen and Oprah Winfrey.
42
more findings …..
Example 4:
Fig. 5. Sample stimuli from Sadr et al.’s [70] experiment assessing the contribution of eyebrows to face recognition: original images of President Richard M. Nixon and actor Winona Ryder, along with modified versions lacking either eyebrows or eyes.
Example 5:
Fig. 6. Even drastic compressions of faces do not render them unrecognizable. Here, celebrity faces have been compressed to 25% of their original width. Yet, recognition performance with this set is the same as that obtained with the original faces.
21
43
more findings …..
Example 6:
Fig. 15. (a) Newborns preferentially orient their gaze to face-like pattern on top, rather than one shown on bottom, suggesting some innately specified representation for faces (from [36]). (b) As a counterpoint to idea of innate preferences for faces, Simion et al. [73] have shown that newborns consistently prefer top-heavy patterns (left column) over bottom-heavy ones (right column). It is unclear whether this is the same preference exhibited in earlier work, and if it is, whether it is face-specific or some other general-purpose or artifactual preference.
Example 7:
Fig. 17. Upper left, an example of FFA (fusiform face area) in one subject, showing right-hemisphere lateralization. Also included here are example stimuli from Tong et al. [80], together with amount of percent signal change observed in FFA for each type of image. Photographs of human and animal faces elicit strong responses, while schematic faces and objects do not. This response profile may place important constraints on the selectivity and generality of artificial recognition systems.
44
Some Illusions: Thatcher Illusion
Thatcher Illusion
Rotate each image by 180
22
45
Some Illusions: Mask Illusion
46
– What can we learn?
We have seen some phenomena of human face perception, now how to start to implement a face recognition algorithm?
The results – an incomplete summary:
1. Human system extremely robust, however not perfect.
2. Fast adaptation but also very stable.
3. There exist top down mechanisms.
4. ……
Why are these findings so difficult to exploit for engineers?
• Mostly behavioral results.
• Only global input-output relations, difficult to isolate subsystems.
• No technology available to observe the brain on a neuronal level in a wide range simultaneously.
• No direct information on an algorithm or an architecture.