Introduction to Computational and Biological Vision Liron Michaely Alon Grubshtein

The Face Recognition ProblemThe Face Recognition Problem

Is a single image database sufficient for face Is a single image database sufficient for face recognition?recognition?

A Psychological ExperimentA Psychological Experiment

Introduction to Computational and Biological Introduction to Computational and Biological VisionVision

Liron MichaelyLiron MichaelyAlon GrubshteinAlon Grubshtein

IntroductionIntroductionIn recent years facial recognition has become a very In recent years facial recognition has become a very popular area of research in computer vision.popular area of research in computer vision.

Progress in the field of computer vision may require Progress in the field of computer vision may require deep physiological understanding, and in turn may deep physiological understanding, and in turn may advance certain theories in the field of neuroscience.advance certain theories in the field of neuroscience.

The Face Recognition Problem:The Face Recognition Problem:

Given an image of a scene, identify or verify the person Given an image of a scene, identify or verify the person

in the scene in the scene using a stored database of facesusing a stored database of faces..

Introduction (contd.)Introduction (contd.)

There are different approaches to understanding the way There are different approaches to understanding the way

we see things, which in turn inspire different algorithmswe see things, which in turn inspire different algorithms..Global ApproachGlobal Approach - a single feature vector that - a single feature vector that represents the whole face image is used as input to a represents the whole face image is used as input to a classifier. Works well for classifying frontal views of classifier. Works well for classifying frontal views of faces. Corresponds with the psychological theory known faces. Corresponds with the psychological theory known as the as the Template TheoryTemplate Theory..

Component-Based approachComponent-Based approach – This approach – This approach classifies local facial component. Corresponds with the classifies local facial component. Corresponds with the psychological psychological Feature theory.Feature theory.

Introduction (contd.)Introduction (contd.)

Since both solutions are based on Since both solutions are based on comparisons, efficiency is based on both comparisons, efficiency is based on both the time required for computation but also the time required for computation but also on the size of the database required.on the size of the database required.

In our experiment we want to examine the In our experiment we want to examine the minimum size of database required for minimum size of database required for successful face recognition successful face recognition

Our assumption:Our assumption: The human The human visual system is capable of visual system is capable of

successful facial recognition based successful facial recognition based on a single image, even under on a single image, even under

extreme conditions.extreme conditions.

Experimental ProcedureExperimental ProcedureThe experiment we conducted was based upon an The experiment we conducted was based upon an application we wrote .application we wrote .

Subjects were exposed to a single frontal image of a Subjects were exposed to a single frontal image of a face (the target), and were tested on their ability to face (the target), and were tested on their ability to distinguish the target from different other faces (the distinguish the target from different other faces (the distracters). distracters).

Our target was a young man, Chen Michaely. As Our target was a young man, Chen Michaely. As distracters we used a group of eight young men with distracters we used a group of eight young men with more or less similar prominent facial features more or less similar prominent facial features

Experiment (contd.)Experiment (contd.)

In the first stage of the experiment we presented the In the first stage of the experiment we presented the target to our test subjects a written description of him target to our test subjects a written description of him (his habits, details about his lifestyle, etc’…), and then (his habits, details about his lifestyle, etc’…), and then his 0 degree image.his 0 degree image.

Second stage was a series of seemingly random Second stage was a series of seemingly random pictures of both target and distracters from different pictures of both target and distracters from different angles. Subjects were asked to press a ‘Yes’ button angles. Subjects were asked to press a ‘Yes’ button

upon identifying the target.upon identifying the target. Response time was also kept.Response time was also kept.

Experiment (contd.)Experiment (contd.)

Implemented as a C++ program.Implemented as a C++ program.

Results saved as a CSV file.Results saved as a CSV file.

Experiment (contd.)Experiment (contd.)

Consistency precautions:Consistency precautions: Angle accuracyAngle accuracy Increased SimilarityIncreased Similarity Control imageControl image Image OrderImage Order

ResultsResults

Several definitions:Several definitions:

A correct “yes” answer is called a “Hit” (H).A correct “yes” answer is called a “Hit” (H).

A correct “no” answer is called a “Correct A correct “no” answer is called a “Correct Rejection” (CR).Rejection” (CR).

A wrong “yes” answer is called a “False A wrong “yes” answer is called a “False Alarm” (FA).Alarm” (FA).

A wrong “no” answer is called a “Miss” (M).A wrong “no” answer is called a “Miss” (M).

Results from 40 people were analyzed.Results from 40 people were analyzed.

Results (contd.)Results (contd.)

Answer Distribution

96%

4%

% Correct Answers

% Wrong Ansers

Results (contd.)Results (contd.)

No wrong answers in Control Image.No wrong answers in Control Image. Most FA mistakes occurred in two specific Most FA mistakes occurred in two specific

images of the same distracter.images of the same distracter.

Results (contd.)Results (contd.)

Distribution of Wrong Answers

7%

93%

%M

%FA

Normalized

34%

66%

%M

%FA

Results (contd.)Results (contd.)

Number of Mistakes as a function of Angle

0

5

10

15

20

-90 deg -45 deg 0 deg 45 deg 90 deg

Num M Num FA

Results (contd.)Results (contd.)

0

0.5

1

1.5

2

2.5

3

0 1-3 3-7 >7

Average Time per number of Mistakes

Results (contd.)Results (contd.)

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

-90 -45 0 +45 +90

Average Time Per Angle

Results (contd.)Results (contd.)

If we consider mistakes and response time If we consider mistakes and response time as indicators to difficult images, then the as indicators to difficult images, then the direction of face does not necessarily direction of face does not necessarily predict the difficulty level of the required predict the difficulty level of the required analysis. analysis.

ConclusionsConclusions

The HVS is capable of processing an The HVS is capable of processing an image, in such a way that allows it to image, in such a way that allows it to identify a face from a new point of view, identify a face from a new point of view, not previously in its database, and relying not previously in its database, and relying on a single frontal image.on a single frontal image.

DiscussionDiscussion

Main question: how is this done? Main question: how is this done? A Computer Science approach: Create a 3D A Computer Science approach: Create a 3D morphable model from three pictures. Use morphable model from three pictures. Use model to calculate new synthetic images, and model to calculate new synthetic images, and compare using component based algorithm.compare using component based algorithm.90% accuracy.90% accuracy.Is this also what we do?Is this also what we do?

non existing relation between the performance level and the non existing relation between the performance level and the angle of the face.angle of the face.

The HVS produces significantly lower results in the case of The HVS produces significantly lower results in the case of upside down faces.upside down faces.

BibliographyBibliography [1]. Bernd Heisele, Purdy Ho, Jane Wu, and Tomaso Poggio (2003)[1]. Bernd Heisele, Purdy Ho, Jane Wu, and Tomaso Poggio (2003)

“Face recognition: component-based versus global approaches”“Face recognition: component-based versus global approaches” . . Computer Vision and Image Understanding 91 (2003) 6–21Computer Vision and Image Understanding 91 (2003) 6–21

[2]. Karl Haberlandt[2]. Karl Haberlandt.” Cognitive Psychology”, .” Cognitive Psychology”, 2nd –Ed, Trinity 2nd –Ed, Trinity college. college.

[3]. [3]. Jennifer Huang, Bernd Heisele, and Volker Blanz (2003). Jennifer Huang, Bernd Heisele, and Volker Blanz (2003). “Component-based Face Recognition with 3D Morphable Models”“Component-based Face Recognition with 3D Morphable Models”..

[4]. Yin, R.K. (1969)[4]. Yin, R.K. (1969) “Looking at upside-down faces”. “Looking at upside-down faces”. J. Exp. J. Exp. Psychology 81, 141–145Psychology 81, 141–145

[5]. Pawan Sinha, Benjamin Balas, Yuri Ostrovsky, Richard Russell. [5]. Pawan Sinha, Benjamin Balas, Yuri Ostrovsky, Richard Russell. ““Face Recognition by Humans: 20 Results all Computer Vision Face Recognition by Humans: 20 Results all Computer Vision Researchers Should Know About”.Researchers Should Know About”.

AcknowledgmentAcknowledgment