Psychology of Face Recognition Brief Introduction 2ndedition

THE PSYCHOLOGY OF FACE RECOGNITION: A BRIEF INTRODUCTION (2nd Edition)

KEVIN BREWER ISBN: 978-1-904542-56-8 Orsett Psychological Services, PO Box 179, Grays, Essex RM16 3EW UK [email protected] COPYRIGHT Kevin Brewer 2010 COPYRIGHT NOTICE All rights reserved. Apart from any use for th e purposes of research or private study, or criticism or review, this publication may not be reproduced, sto red or transmitted in any form or by any means, without pr ior permission in writing of the publishers. In the cas e of reprographic reproduction only in accordance with t he terms of the licences issued by the Copyright Licen sing Agency in the UK, or in accordance with the terms o f licences issued by the appropriate organization out side the UK.

The Psychology of Face Recognition: A Brief Introduction (2nd edition) Kevin Brewer; 2010; ISBN: 978-1-904542-56-8 2

Contents Page Nu mber INTRODUCTION 3 CONFIGURAL PROCESSING OF FACE 4 Evidence For 4 Problems With 8 Applying Theories of Pattern Recognition to Face Recognition 8 FEATURE DETECTION THEORIES 10 Evidence For 10 Evidence Against 10 INFORMATION PROCESSING MODEL 11 Evidence For 12 Problems With 17 PHYSIOLOGICAL STUDIES AND FACE RECOGNITION 18 CULTURE AND FACE RECOGNITION 22 COMPUTER MODELLING AND FACE RECOGNITION 23 APPENDIX A - INTERNAL AND EXTERNAL FEATURES 23 APPENDIX B - PROSOPAGNOSIA 25 APPENDIX C - CAPGRAS DELUSION 26 APPENDIX D - RECOGNISING A FACE 26 REFERENCES 31


Introduction In terms of recognition, faces of different pe ople all have similar features in roughly the same configuration, yet we are able to distinguish a fam iliar face in the crowd. This ability, which takes a frac tion of a second, is far beyond the equivalent perceptio n of objects and patterns. Most people are not able to distinguish one tree from a similar group or one pe nguin in a massive crowd, for example. So face recognitio n and perception are unique types of visual processing. O r is it the same process of object recognition but more finely tuned for faces (Bublitz 2008)? Face recognition is the situation of using the face to identify a familiar individual. It is different to face perception, which includes the perception of emotions from facial expressions, and the perceptio n of unfamiliar faces (Roth and Bruce 1995). While face detection is the ability to discriminate a face fro m other objects. There is also face identification (naming the person) and face recall (describing the face from m emory) (Cohen 1989). The main question is whether faces are recogni sed by features (eg: hair, nose) or in a configural (whole ) way. Bruce and Young (1986) proposed that seven typ es of information are derived from seeing a face: � Pictorial - Basic visual information about the face ;

eg: lighting, facial expression. A simple yes/no recognition is possible.

� Structural - Information about the face's structure ;

eg: head shape, details of features. This allows th e recognition of faces when head angles or facial expressions change.

� Visually derived semantic - Judgments beyond visual

information; eg: age, sex, attributes like honesty. � Identity-specific semantic - Information about fami liar

people; eg: occupation, where encountered. � Name - Where this is known as there as acquaintance s

(familiar faces) whose name not known. � Expression - Information related to meaning of faci al

expressions. � Facial speech - Whether the individual is speaking or


not. These last two types of information are not important in face recognition. There are three main theoretical approaches to recognition of familiar faces: i) Configural processing of faces ii) Feature Detection Theories iii) Information Processing Model.

Configural Processing of Faces This approach argues that faces are recognised as a whole configuration (holistically), and features ar e not analysed separately. A configural way can include the spatial relationship between features on the face, or how t he features interact (eg: the shape of the mouth affec ts the perception of the shape of the nose) (Rakover 2002) . EVIDENCE FOR CONFIGURAL PROCESSING OF FACES 1. Recognition of upside-down faces Farah et al (1998) observed that "whereas most objects are somewhat harder to recognise upside dow n than rightside up, inversion makes faces dramatically ha rder to recognise" (p482). Researchers have found that faces are harder t o recognise upside-down than other objects, so it can not be the features only that are important (Yin 1969). Yi n found that students recognised photographs and draw ings of famous faces better than those of airplanes, hou ses, and cartoon figures without distinct faces. But the opposite was true if the photographs were presented upside down. This is the face-inversion effect. Also when a "grotesque" face is presented, the unusual features are not noticed upside-down (eg: "Thatcher illusion"; Thompson 1980 1) (figure 1). The "Thatcher illusion" is a picture of Margar et Thatcher where the mouth and eyes have been turned upside-down. Normally this looks "grotesque", but u pside-down there appears to be nothing wrong. The relationship between the eyes, nose and mo uth

1 Also called "Thompson illusion".


(ie: configuration) is harder to perceive in the up side-down face, and the "grotesque" features are not see n (Roth and Bruce 1995).

(Source: Anynobody; in public domain)

Figure 1 - The "Thatcher illusion". Yin (1970) argued that the inverted face is al so harder to recognise because it is more difficult to recognise the facial expression of such a face. But Leder and Bruce (1998) altered the spacing between features in photographs of faces (eg: dista nce


between eyes) which made them more distinctive. The se faces were easier to recognise than normal faces, b ut harder when upside down. Face recognition involves processing the individual features and their configuration, which is disrupted when upside down. Leder and Bruce (2000) asked participants to identify faces by unique combinations of features o r by distinctive spacing between features. Upside down photographs were harder to recognise based on spaci ng than combination of features or both aspects togeth er. 2. Composite faces Young et al (1987) combined the top and bottom halves of two famous faces of the time of the exper iment (politicians - Margaret Thatcher and Shirley Willia ms). Participants were asked to name the face by the top half, and they were unable to do this. If face recognitio n was based on features, then this should not be the case . The researchers argued that the newly-combined face is a new configuration. Individuals were slower to recognise the faces of two different halves together than if the two halve s were misaligned. Trying to recognise one half of the fac e together is limited by interference from the other half as perception of the whole face is automatic. 3. Part-whole effect Tanaka and Farah (1993) found that individuals were better at recognising two face parts as one face th an in isolation. This is because the brain is automatical ly better at processing whole faces. 4. Experimental evidence Tanaka and Sengco (1997) taught participants t o recognise a set of faces each given a name (eg: "Bo b"). Then parts of the face were presented and participa nts were asked to recognise them (eg: Is that Bob's nos e?"). The parts of the face were presented in isolation, in the context of the original face (old configuration condition), or in the original face that had been c hanged (eg: eyes moved further apart) (new configuration condition). If faces are recognised by individual features, then there should be little difference in recognition of face parts between the different con texts. However, participants were best at recognising face parts in the original context, then in an altered f ace, and lastly, in isolation. There was no difference w hen


the same procedure used house pictures and recognit ion of house parts (table 1).

Table 1 - Percentage correct recognition of feature s in Tanaka and Sengco (1997). Farah et al (1998) used the same-different mat ching paradigm with twenty-four undergraduates, who were presented with two faces for one second and then as ked if one feature (eye, nose or mouth) or all were simila r. Six faces were used that were similar on one, two or th ree (all) features (table 2). FACES A & B FAC ES A & B Relevant feature: Nose = same Nos e = same Irrelevant feature: Mouth = same Eye s = different Condition: Compatible Inc ompatible Table 2 - Examples of design in Farah et al (1998). If feature detection is used, participants sho uld be unaffected at rating the similarity of individual features irrelevant of other features. But if there is contamination by other features, this is evidence f or holistic face perception. The results were seen as supporting the latter . Participants achieved an average rate of 74.6% corr ect for similarity or difference of individual features when irrelevant features were compatible (ie: relevant f eature same/irrelevant feature same, relevant feature different/irrelevant feature different) and 61.8% f or incompatible (ie: relevant feature same/irrelevant feature different, relevant feature different/irrel evant feature same) (table 3). The irrelevant features ha d affected perception of the relevant features sugges ted face perception as a whole.

CONDITION FACIAL FEATURES DIFFERENCES IN RECOGNITION

HOUSE FEATURES

Isolation 65 Mouth best, eyes worse

84

Old configuration

77 Mouth best, nose worse

83

New configuration

72 Mouth best, nose worse

84


(After Farah et al 1998)

Table 3 - Percentage correct relevant feature match ing in Farah et al (1998). PROBLEMS WITH CONFIGURAL PROCESSING OF FACES i) Some facial features are more important tha n others. For unfamiliar faces, recognition depends o n external features of the face (eg: face outline, hairstyle), but internal features are more importan t for familiar faces (eg: eyes, nose) (Ellis et al 1979) (appendix A). The most important internal feature is the are a around the eyes, and the area around the nose is th e least important (Roberts and Bruce 1988). Sadr et al (2003) showed that the eyebrows wer e the most important for familiar faces. Participants wer e shown fifty Western celebrity faces (25 of each sex ) to recognise in three conditions - unaltered, with eye s missing, or with eyebrows lacking. Performance in t he last condition was significantly worse (46.3% corre ct) compared to no eyes (55.8% correct) and unaltered ( over 60% correct). ii) Most of the research is based around recog nition of faces in photographs (ie: 2D stimuli), when, in real life, face recognition is of a 3D stimuli (Eysenck and Keane 1995). APPLYING THE THEORIES OF PATTERN RECOGNITION TO FACE RECOGNITION The configural processing of faces is similar to how patterns and objects are recognised by the Template Matching Hypothesis and Prototype Theories. Template Matching Hypothesis One possibility is that individuals store a fi xed set of views of faces they have learnt.

Are relevant features:

IRRELEVANT FEATURE COMPATIBLE

IRRELEVANT FEATURE INCOMPATIBLE

SAME 91.5 71.6

DIFFERENT 57.6 51.9


EVIDENCE FOR: EVIDENCE AGAINST: It is difficult to There would need to be a recognise the same template for each view of face that has changed face: front and s ide, and this (eg: with/without beard). would require a m assive memory A change in wig reduced face capacity recognition to 50%, while a wig and beard change reduced accuracy to 30% (Patterson and Baddeley 1997 quoted in Brewer 2000) Prototype Theories Prototypes are not individual faces, but a sum mary of the main features of faces. General prototypes o f the face are stored in the memory, and individual faces are linked to them. This is a norm-based system for processing faces (Bublitz 2008). The process works in two ways - typical faces are quicker to recognise as faces compared to other obj ects, but distinctive faces (ie: those different to the t ypical face) are easier to recognise for individual faces. Valentine and Bruce (1986) have shown these processes experimentally. Participants were asked t o rate the familiarity of a famous person, and the reactio n time to answer was measured. The average time taken was 661 ms for distinctive faces as compared to 707 ms for typ ical ones. When participants were shown jumbled faces an d asked if it was a face or a non-face, recognition f or typical faces took 561 ms on average and distinctiv e faces 608 ms. Valentine (1991) sees distinctive faces as sto red in "face space" where few others are stored ("face-spa ce hypothesis"). Known faces are stored on dimensions of the space which represent the dimensions used to distin guish between the faces, and typical faces are clustered close together at the average. Distinctive faces are at t he extremes of the dimensions. Thus identification is much easier with less competing information in the memor y store. While Bruce et al (1994) found that distinct f aces are different in measurements of features, like nos e width, than faces rated as typical. Also caricature s of famous faces, which exaggerate the person's face, m ake them more distinctive, and easier to recognise (Val entine 1996). Leopold et al (2006) presented an "average" hu man face (based on merging different faces) or caricatu res to rhesus monkeys while measuring activity in face-det ecting


cells in the inferior temporal cortex. Activity was greater for the caricatures as the faces became mor e distinctive. Rhodes and Jeffrey (2006) suggested that pairs of groups of neurons are tuned to above-average and be low-average aspects of the face (eg: hair width, inter- eye distance) along dimensions in the face space. The problem with this approach is the inabilit y to explain the exact nature of prototypes.

Feature Detection Theories Individuals focus upon features (eg: hair, eye s) of the face, and build up a picture of the whole face to recognise. EVIDENCE FOR FEATURE DETECTION THEORIES Participants asked to describe unfamiliar fac es shown for a brief period used particular features. Hair was mentioned most often, then eyes, nose, mouth, eyebrows, chin and forehead (in that order) (Shephe rd et al 1981). Bradshaw and Wallace (1971) argue that facial features are processed independently, and in a part icular sequence. Using Identikit faces, the researchers sh owed participants pairs of faces that differed by featur es (either on 2,4 or 7 features), and asked them to sa y if it was the same person. The more features that were different, the qui cker were participants to answer as not familiar. With m any differences, the participants would encounter this quicker in their comparison of the features. EVIDENCE AGAINST FEATURE DETECTION THEORIES Sergent (1984) showed that faces with the same features but combined in different ways will not be recognised. In other words, the whole of the face m ust be taken into account. This research combined in eight faces, two dif ferent chins, two different eye colours, and two different arrangements of space on the face (eg: eyes and nos e high or low on face). The more features that differed, t he quicker the "different" response by participants. T he difference in chin produced a quick response, while different chins and one other feature produced the quickest response.


Further evidence against Feature Detection The ories comes from Tanaka and Farah (1993). Their aim was t o test the recall for specific features of the face. Participants were asked to recognise a particular f eature (Larry's nose) when presented in different faces, scrambled faces or in isolation. The Feature Detection Theories predict recogni tion of the feature irrelevant of the context. The resul ts of the research were that recognition was poorer in isolation or in a different context to learning, an d better in the original context learnt (around 70% accuracy). Thus face recognition is more than just the features separately 2. Rhodes (1988) distinguished between first orde r features (eg: eyes, nose) (feature detection) and s econd order features (eg: spatial relations between featu res) (configuration) as both involved in face perception .

Information Processing Model Bruce and Young (1986) argued that face recogn ition can be seen as involving three stages: i) The face is compared with a set of stored descriptions called "face recognition units" (FRU), and this produces a feeling of familiarity or not; ii) The memory is activated to recall facts ab out the person if familiar; iii) The retrieval of the name. This model of face recognition is based upon s tages that progress in a particular order (known as seria l processing). The order of the stages cannot be chan ged. This model makes use of different modules in the br ain - visual recognition (ie: FRUs) and semantic memory ( figure 2).

2 Police forces are now using with witnesses face reconstruction systems that computer generate whole face images (eg: EvoFIT; Lander 2002) rather than the individual features of Photofit systems.


FACE ↓ Stage 1 VISUAL ENCODING (ie: seeing face - structurall y encoding) ↓ Stage 2 MATCHING PROCESS (to Face Recognition Units - produces feeling of famili arity) ↓ Stage 3 SEMANTIC INFORMATION (recall facts about perso n - person identity nodes) ↓ Stage 4 NAME RETRIEVAL (recall memory - name generatio n) Figure 2 - Information Processing Model of face recognition. EVIDENCE FOR INFORMATION PROCESSING MODEL 1. Face recognition error studies This type of study focuses upon situations whe n face recognition fails. These are occasions when individ uals cannot recognise a familiar face, or they recognise the face but cannot recall information about the face ( like the person's name). Young, Hay and Ellis (1985) asked twenty-two volunteers (11 male, 11 female) at Lancaster Univer sity to keep a eight-week self-reported diary (table 4) of the times they could not recognise a famous or familiar face, or could not remember information or the name of th e person (face recognition errors). The participants were asked to record, as soon as possible after it happened, details of any errors o r difficulties in recognising/identifying another per son under the following headings: � Type of incident; � Source - Information, like facial features, availab le

at the time of the incident; � General details - eg: person in mass media, state o f

participant at the time of the incident; � Person involved - How well the person known on a sc ale

of 1 (unknown) to 5 (very well known); � Way incident ended - ie: able to recognise person


eventually or not; � Person details available - Information that could n ot

be recalled about the person. The first week of the study was treated as tra ining and the 140 records collected were not analysed. An example of diary event was this error: I just thought the person looked familiar, as she waved, and I thought it was at me. I waved bac k, then realised I didn't know her. She was wavin g at someone else (Young et al 1985 p508). ADVANTAGES DISADVANTAGES - Rich data about - Bias in what is recorded experience eg: for getting events - Record at time overcomes - Reaction to kno wing someone memory problems will re ad it - Study of areas that - Dependent on le vel of detail would be difficult by provided by parti cipants researcher - Useful to study - No independent way of infrequent behaviours verify self-reports Table 4 - Advantages and disadvantages of self-repo rted diary studies. The study produced 1008 such incidents (922 completed records), which were analysed for the typ e of face recognition error. Table 5 lists the categories of face recogniti on errors found by the researchers. The types of errors can be divided into five g roups (three were evident and two were not found) to supp ort the Information Processing Model: i) A failure to recognise familiar faces becau se, for example, the appearance has changed (eg: walkin g past a person and not recognising them, but told about i t later). This was due to a failure at point A in fig ure 3. This could include "highly familiar" faces (42% of these errors). ii) Recognition of the face leading to a feeli ng of familiarity only (eg: couldn't remember where met before). This can be seen as problem at point B in figure 3.


1. Person Unrecognised 2. Person Mis-Identified a. Unfamiliar person mis-identified as familia r person (usually viewing conditions poor) b. One familiar person mis-identified as anoth er (usuall y celebrities) 3. Person seemed familiar only a. Familiar person successfully identified (eg: acquaintance seen in unfamiliar context) b. Familiar person not identified c. Person found to be unfamiliar (viewing cond itions poor) 4. Difficulty of Retrieving Full Details of Pe rson a. Difficulty successfully resolved b. Difficulty not resolved Table 5 - Types of face recognition errors categori sed by Young et al (1985). FACE ↓ VISUAL ENCODING ↓ A MATCHING PROCESS ↓ B SEMANTIC INFORMATION ↓ C NAME RETRIEVAL Figure 3 - Blockages or errors in the Information Processing Model of face recognition. iii) Recognition of the face, feeling of familiarity, and only information about the person recalled, not their name (eg: famous person on television). This is a blockage at point C in figur e 3. iv) There were no cases of recognition and nam e retrieval without semantic information. This suppor ts the model because individuals cannot go to stage 4 with out passing through stage 3 in figure 2. It is not poss ible to recall the name without any information about th e person. v) There were no cases of name recall without feelings of familiarity or semantic information abo ut the


person. Again this supports the model. 2. Recognition reaction time studies This type of study measures the reaction time of participants answering questions about famous faces . Three types of questions are asked: a) Do you recognise the face? b) What information can you recall about them? c) What is their name? Each question will take slightly longer to ans wer because of the stages involved in finding the information. Question (a) involves stages 1 and 2 i n figure 2, question (b) stages 1, 2 and 3, and (c) a ll 4 stages. Young et al (1986) found the following average reaction times to answer the three questions: quest ion (a) 775 ms, (b) 931 ms, and (c) 1255 ms 3. 3. Case studies of brain-injured patients Situations will occur where individuals have s ome kind of injury (eg: accident or stroke) which leads to minor brain damage. The abilities that the individu als lose can help psychologists to understand how the b rain works. However, these are individual cases, and generalisation of the findings is not possible (tab le 6). Brain-injured patients are often studied throu gh the forced-choice test. The task is to say which one of the pair of photographs is familiar. One photograph is a famous face or an individual known to the participa nt, and the other photograph is a complete stranger. Evidence from Case Studies: a) Damage to FRUs (point A on figure 3) "PH" (De Haan et al 1987), injured in a car accident, was able to recognise familiar names, but not familiar faces. The inability to recognise faces is known as prosopagnosia 4 (appendix B).

3 Interestingly, recall of names of celebrities is more accurate if the celebrities were associated with a particular role (eg: James Bond; Roger Moore) (Bredart 1993). 4 The sufferer is unable to recognise familiar famous faces, individuals known to them, or even themselves in photographs or in the mirror. However, other object and pattern recognition abilities are


ADVANTAGES DISADVANTAGES - The loss of brain functions - Only unusual in dividual cases show how the brain works, and and results may n ot be which areas involved general isable - Better than deliberately - Usually no reco rd of damaging brain of animals, behaviour pre-inj ury for in terms of ethics, comparison applicability of animal models to humans, and the participants - Only shows corr elation can talk about their problems between damaged a rea of brain and pro blems - Easier to test than with animals - Modern brain-scanning techniques can pinpoint exact area of brain damaged Table 6 - Advantages and disadvantages of using bra in-injured patients in research. "PH" was presented with pairs of names (one familiar, one not) and was asked which was familiar : "PH" achieved 118 of 128 correct. Thus there was no dama ge to the semantic memory. When presented with pairs of f aces, "PH" got 51% correct (this is the same as guessing) . b) Problems with semantic information retrieva l (point B on figure 3) "KS" (Ellis et al 1989) sustained damage to hi s right temporal lobe during an operation on the brai n to deal with epilepsy. Only the long-term memory for information about people was impaired, not the gene ral long-term memory for number and words. c) Problems with name retrieval (point C on fi gure 3) "EST" (Flude et al 1989) was able to recognise familiar faces and recall semantic information abou t the person, but was poor at name retrieval. "EST" was a lso poor at naming objects, but had no problems with familiar name recognition. This suggests that name recognition is different to name retrieval for face s.

not affected. This suggests that there are different processes in recognising faces and non-faces.


Another brain condition that affects recogniti on is the Capgras delusion (first noted in 1923). It invo lves the sufferer believing that a familiar person is an impostor. The sufferer can recognise the face, but it does not "feel" like that person to them. Joseph (1 986) suggested that the cause was the failure of integra tion of visual information from the right and left hemispheres. While Cutting (1990) explained the del usion as due to the lack of usual right hemisphere advant age for face processing (appendix C). PROBLEMS WITH INFORMATION PROCESSING MODEL 1. The sequences of familiar face recognition are t oo rigid. A direct challenge to the sequences comes from "ME" (De Haan et al 1991), an amnesiac, who could match faces and names for famous people to 90% accuracy, but wo uld recall no semantic information about them. In figur e 2, this is going from stage 2 to 4 and missing out 3. 2. The Information Processing Model is also challen ged by "covert recognition". This is the correct recogniti on of faces without any conscious awareness of the recogn ition process. Individuals with brain-injury which leads to prosopagnosia are presented with photographs of fac es. One face is familiar and the other is not. The task is to say which is the familiar face. Researchers found that individuals will choose correctly even though they claim to have no conscio us recognition. In other words, they say that they are guessing, but they guess right nearly every time. G etting half right would be predicted by chance. De Haan et al (1987) showed covert recognition in prosopagnosia with the interference effect. Individ uals learn the name of a face and occupation together. T hen they are shown the names and faces next to the wron g occupations. It takes longer to read the name for n ormal participants because there is interference from the false occupation information. Prosopagnosia sufferers als o show interference when they should not (unless covert recognition has occurred) as they report not consci ous recognition of face. Some researchers have argued that there are tw o routes to face recognition: primary and secondary r outes. The former route is conscious, while the latter is at an emotional or unconscious level. Secondary processin g links to the idea of the feeling of familiarity. No rmally these two routes match (Hayden Ellis 1997).


3. Recognition of a face is linked to where the fac e was encoded. In other words, a person met in one situation/context is easier to recall in that situation/context (eg: at school), but harder to re call in another context (eg: in the street). 4. Face recognition can also be affected by differe nces in the situation between the encoding and the recal l situations. For example, research shows that differ ent lighting can influence face recognition. Participan ts had to match the photographs of ten men with video clip s in different lighting. There was a 79% accuracy for th e full-face, and 70% for the head with a 30 degree an gle change (Bruce et al 1999). 5. The exact function and processes of the model ar e too vaguely specified (Eysenck and Flanagan 2001). Burton et al (1990) adapted the Information Processing Model to accept that the process is bi-directional between the semantic information store (which now contains the name of the individual) and the Fa ce Recognition Units (FRU) (figure 4). The feeling of familiarity now takes place at the person identity nodes (PIN); ie: the person is recognised rather than the face. The new model is called an interactive activat ion and competition model (Burton and Bruce 1993). FACE ↓ VISUAL ENCODING ↓ MATCHING PROCESS (FRUs) ↑ ↓ SEMANTIC INFORMATION/NAME GENERATION (PINs) Figure 4 - Adapted Information Processing Model by Burton et al (1990).

Physiological Studies and Face Recognition In terms of physiology, the question is whethe r a particular area of the brain is involved in face perception, and this is because face perception is different to other forms of visual perception (tabl e 7).


� Brain injured patients who have impairment to one a bility (eg: face recognition) but not another (eg: object perce ption).

� Single cell recording in monkeys. � Newborns respond to faces and not other objects (eg : 30-minute

olds track a moving face more than patterns and sha pes; Farah et al 1998).

� Face inversion effect. Table 7 - Evidence for face perception as special ability. Electrical recording of brain activity with pa tients undergoing brain surgery for relief of epilepsy pro duced interesting findings. For example, Allison et al (1 994) found hemispheric differences in N 200 waves. The right hemisphere produced a larger response to upright fa ces compared to the left hemisphere, and there were differences between upright and inverted faces in t he right hemisphere response. Kreiman et al (2000) implanted electrodes in t he anterior region of the right hemisphere of such pat ients. Cells responded to famous faces, but not to emotion al faces of unknown actors 5. Neuroimaging techniques have shown that a part icular area of the brain is active in face perception - th e fusiform face area (FFA) in the fusiform gyrus 6. Kanwisher et al (1997) used functional magnetic res onance imaging (fMRI) on fifteen participants while viewin g full or scrambled faces, full-front views of faces or ho uses, and three-quarter views of faces or human hands. Th e FFA was active in response to full faces. Hasson et al (2001) presented a combined image which showed a face to one eye and a non-face to the othe r eye. The FFA was active on the side of the brain respond ing to the eye shown the face. Recently, Parr et al (2009) tested five adult chimpanzees' ability to recognise photographs of unfamiliar chimpanzees or non-face images while undergoing a PET scan. Specific areas of the brain were active during face processing and recognition that were not active during non-face image processing and recognition. The active areas were similar to those in humans during face recognition tasks, and this prov ided

5 The idea that a small set of cells fire in response to a particular visual object is called the "grandmother cell theory" (Barlow 1985). It tends not to be accepted now as a diffuse cortical network is involved in object recognition (Banich 2004). 6 The inferior occipital gyrus (occipital face area; OFA) is also involved, and both areas are more active in the right hemisphere (Schiltz et al 2006). Also activity in superior temporal sulcus (Tsao and Livingstone 2008).


evidence for faces as a special class of visual sti mulus. Jiang et al (2006) designed a computer model b ased on feature detection cells in the brain. The resear chers believed that face perception is the same as object ion perception generally, but that more cells are devot ed to it based on learning (expertise). The more cells de voted to a familiar face means better recognition, but on ly if face not inverted. This idea is supported by physiological eviden ce that the FFA is larger in adults relative to the wh ole brain than in children (Golari et al 2007). Individuals can also learn to distinguish obje cts to a fine degree, which supports the idea that face an d object perception are the same process. For example , Diamond and Carey (1986) found that judges at dog s hows could recognise individual dogs, but not if the photograph inverted. But physiological studies have shown that such expertise does not occur in the FFA. Among butterfl y experts, recognition of butterflies produced activi ty in an area close, but separate, to the FFA (Rhodes et al 2004). Among individuals taught to distinguish cars , neurons were active in the lateral occipital cortex during the recognition task (Jiang et al 2007). It is disputed as to whether the FFA is only f or face recognition. For example, Gauthier et al (1999 ) found activity in the FFA among individuals taught to distinguish bizarre computer-generated shapes (call ed "greebles"). One issue is whether face detection and identification are carried out by the same set of " face-selection cells" 7 or involve different cells. Tsao et al (2003) found evidence for the latter in macaques. Kobatake and Tanoka (1994) recorded cell activ ity in the inferotemporal cortex in monkeys. The cells res ponded to the face of a toy monkey, and to simplified face shapes, but not non-face shapes (figure 5). Other evidence for face processing as distinct to non-face processing comes from brain-injured patien ts. Moscovitch et al (1997) reported the case of "C.K" who was severely impaired at object recognition, but unaffected for face recognition. For example, shown as picture of a face made up of vegetables, he saw the face but not the vegetables. However, he does perform wo rse than controls for inverted faces. "His pattern of deficits indicated that face processing is not simp ly a

7 Also called gnostic units (or grandmother cells) - hypothetical cells that respond exclusively to a single high-level stimulus (Tsao and Livingstone 2008).


(Numbers in brackets = activity in face cells in re sponse to stimulus. Higher number = more activity) (After Tsao and Livingstone 2008)

Figure 5 - Examples of stimuli presented to monkeys by Kobatake and Tanaka (1994). final stage tacked onto the end of the non-face obj ect recognition pathway but rather a completely differe nt pathway that branches away from object recognition early in the visual hierarchy, and it is this branching o ff that we propose to equate with the detection proces s" (Tsao and Livingstone 2008 p420). Two other interesting examples come from study ing individuals with prosopagnosia. "R.M" (Sergent and Signoret 1992), who was a car expert, could still identify accurately information about cars after th e brain injury. 210 pictures of cars were shown and t he task was to identify the car's make, model, and yea r of production (to within two years). All three pieces of information were correct for 172 pictures. Of the remaining 38, R.M was correct on make for thirty-on e and model for twenty-two pictures. McNeil and Warrington (1993) reported the case of a farmer who after prosopagnosia was able to recognis e individual faces of his flock of sheep (and better than other farmers). Face processing can appear special (eg: innate ) when it is not for two reasons (Medin et al 2001). First ly, humans have greater experience with faces (and thei r importance from birth) than other objects. Secondly , face processing often means one particular face rather t han


the general category, which may mean more developme nt in he brain for it. For example, a desire to drink fro m a cup requires perception of any cup, not necessarily a specific one, but speaking to a close friend requir es perception of that face not just any face.

Culture and Face Recognition Yarbus (1965) was the first to monitor eye mov ements as individuals looked at faces. A systematic triang ular sequence of eye fixations on the eyes (most importa nt) and the mouth has been found. These findings have b een based on adults in the West. Blais et al (2008) reported cultural differenc es in the extraction of visual information from faces. Fo urteen Western Caucasian (WC) and 14 East Asian (EA) (8 Ch inese, 8 Japanese) students at the University of Glasgow, Scotland were recruited for the experiments. Pictur es of faces from both groups were presented individually, and the participants' eye movements and fixations were recorded by a head-mounted eye-tracker. The partici pants were asked to memorise fourteen faces and then pick ed them out from a choice of 28 as quickly as possible . This was the face recognition task. In the face categori sation task, participants were asked to categorised 112 fa ces as WC or EA. The participants were slightly better at recog nising faces of individuals from their own race in the fac e recognition task, but there was no differences in t he face categorisation task. The eye fixations varied with WC students having significantly more fixations on the eyes (and mouth) and EA students on the nose and ce ntral region of the face. Blais et al (2008) proposed that this differen ce was due to the fact that "direct or excessive eye conta ct may be considered rude in East Asian cultures.. and thi s social norm might have determined gaze avoidance in East Asian observers" (pp5-6). Furthermore, they concluded: Psychologists and philosophers have long assu med that while culture impacts on the way we thin k about the world, basic perceptual mechanisms are common among humans... We provide evidence th at social experience and cultural factors shape human e ye movements for processing faces, which contrad icts the view that face processing is universally achi eved (p6).


Computer Modelling and Face Recognition Face detection involves spotting the similarit ies in faces as opposed to non-faces, while identification focuses upon the distinctiveness of individual face s. In computational modelling terms, a "good detector sho uld be poor at individual recognition and vice versa" (Tsa o and Livingstone 2008). Yet the human brain can do both. Artificial face-detection systems designed for face detection often use template matching, and for face identification both feature detection and template matching can be used. Artificial systems using temp late matching can be slow, and dependent on the face bei ng accurately aligned. Humans can recognise faces as i n caricature. APPENDIX A - INTERNAL AND EXTERNAL FEATURES Face recognition ability varies between unfami liar and familiar faces. Unfamiliar faces are those seen once, and their memory is fragile being easily influenced by factors like lighting, viewing angle, and facial expression. Familiar faces tend not to be affected by such factors, and misidentification is very low (Fr owd et al 2007). Ellis et al (1979) reported that familiar face s are more accurately recognised using internal features (eyes, brows, nose, and mouth), and unfamiliar faces by ex ternal features (head shape, hair, and ears). Frowd et al (2007) investigated this finding i n relation to how eye witnesses are required to produ ce a memory of a face by the police. Experiment 1 Thirty staff and students at the University of Stirling, Scotland were asked to view ten facial photographs of celebrities (six actors/four pop sin gers), and then had to match the faces with forty composit es. The target photographs were always present as this experiment was testing the perceived accuracy of th e composites and not memory. The composites were constructed in three diffe rent ways and participants were randomly allocated to on e set. i) Complete composites constructed using techniques common to police work. For example, E-Fit (a comput erised version of Photo-FIT) where individual features of the faces (eg: different eyebrows) are combined on a template.


ii) Internal composites highlighting the internal features (ie: external features like hair removed). iii) External composites containing no internal fea tures. The whole and external composites were matched better than the internal composites (mean accuracy of 33%, 32.8% and 19.5% respectively). Experiment 2 Forty-eight undergraduates from the same unive rsity were asked to match celebrity faces to composites o f external or internal features, but distractor faces were added (ie: celebrities not in the target photograph s). The distractor faces were either similar to targets (hard condition) or dis-similar (easy condition). The aim was to replicate the police line-up. Matching based on external features was more accurate, and in the easy condition (ie: when distr actor faces dis-similar to targets) (figure 6).

0

10

20

30

40

50

Easy Hard

External Internal

Figure 6 - Mean accuracy (%) in matching target fac es and composites. Experiment 3 This experiment varied the familiarity of the target faces by using photographs of staff at the universi ty from the Computer Science and Psychology department s. Participants of a wide age range were recruite d from within and outside the University of Stirling. The composites were as in experiment 1. Complete and ex ternal composites were matched more accurately. Individual s familiar with the faces did not use internal featur es.


The researchers felt that the construction of composites as used by the police emphasised externa l features at the expense of internal ones. APPENDIX B - PROSOPAGNOSIA Schiltz et al (2006) presented the case of "P. S" who had acquired prosopagnosia after damage to the righ t hemisphere (inferior occipital cortex). She was 42 years old at the time of a closed head injury in 1992. Her performance on certain tasks was compared to seven age-matched women. The tasks involved the rea ction time to answer and accuracy in recognising pictures of either faces, cars, chairs, boats, or birds. P.S's scores were similar to the controls for identification of items in the between-category condition. This is where a photograph of a car, for example, just shown for 10 00ms is presented with a photograph of a bird (not seen before). In the within-category condition, P.S took longer and made more errors with faces than control s. This condition involved the pairing of a face (or n ay category) just seen with another from the same cate gory (table 8). So P.S was able to detect a face, but no t identify it.

Table 8 - Tasks used with P.S. Interestingly, individuals with prosopagnosia can do better than healthy controls at recognising inverte d faces (Farah et al 1995). This can be explained as individuals with prosopagnosia using the non-face processing system to recognise faces (because of da mage to the face processing system), and the face proces sing system is poor at recognising upside down faces in healthy controls as it was not "designed" for that (Tsao and Livingstone 2008).

CONDITION PROCEDURE ABILITY TESTED P.S

Between-category

Shown category A picture (eg: face), then A paired with category B picture (eg: bird)

Face detection Comparable to controls

Within-category

Shown category A picture 1 (eg: face 1), then A1 paired with another A (eg; face 2)

Face identification/recognition

Significantly slower reaction time (mean 794ms); significantly more errors (17% vs 4%)


APPENDIX C - CAPGRAS DELUSION Ellis et al (1993) tested the two explanations for Capgras delusion - no integration of visual informa tion from both hemispheres (Joseph 1986), and no usual r ight hemisphere advantage in face processing (Cutting 19 90). Participants were asked to fixate on a small c ross in the centre of the computer screen and two images were presented for 200 ms. The images varied in presenta tion (ie: both in one eye or in separate eyes) to avoid anticipation. The task was to say if the two images (either line drawings of common objects or adult ma le faces) were the same or different. Reaction times a nd errors made were recorded. The participants were three Greek males with a history of the Capgras delusion (table 9) and three sex, age and socio-economic matched sufferers of paranoi d schizophrenia. For the line drawings of objects, there was no significant differences between the two groups of participants in reaction time or number of errors. With the faces, the schizophrenics showed a right hemisp here advantage (ie: significantly quicker for faces pres ented in the left visual field compared to the right 8), while the Capgras delusion sufferers showed a left hemisp here advantage. The findings suggested that the Capgras delusi on is related to problems in the face processing centres of the right hemisphere, and provided some support for Cut ting (1990). � Participant 1 - Believed father impostor and that s ister was

someone else who looked her. � Participant 2 - Believed mother not his real parent but "hostile

substitute", and that father had been replaced. � Participant 3 - Believed mother substituted by "a h ostile person". (After Ellis et al 1993)

Table 9 - Details of Capgras delusions. APPENDIX D - RECOGNISING A FACE Here is a photograph of the actress, Heather Langenkamp (figure 7) which has been changed in way s used in face recognition experiments:

8 Information from the left visual field goes to the right hemisphere and vice versa.


� Eyes only (figure 8) � Face with no eyes (figure 9) � Scrambled face (figure 10) � Distorted face (figure 11) � Caricature (figure 12)

(Source: Photograph in public domain from http://commons.wikimedia.org/wiki/Main_Page )

Figure 7 - Face of Heather Langenkamp.

(Source: Photograph in public domain from http://commons.wikimedia.org/wiki/Main_Page ; my adaptation using image editor at http://www.pixlr.com/editor ) Figure 8 - Eyes only of Heather Langenkamp.


(Source: Photograph in public domain from http://commons.wikimedia.org/wiki/Main_Page ; my adaptation using image editor at http://www.pixlr.com/editor ) Figure 9 - Face of Heather Langenkamp with no eyes.


(Source: Photograph in public domain from http://commons.wikimedia.org/wiki/Main_Page ; my adaptation using image editor at http://www.pixlr.com/editor )

Figure 10 - Scrambled face of Heather Langenkamp.

(Source: Photograph in public domain from http://commons.wikimedia.org/wiki/Main_Page ; my adaptation using image editor at http://www.pixlr.com/editor )

Figure 11 - Distorted face of Heather Langekamp.


(Source: Photograph in public domain from http://commons.wikimedia.org/wiki/Main_Page ; my adaptation using image editor at http://www.pixlr.com/editor ) Figure 12 - Caricature face of Heather Langenkamp.


References Allison, T et al (1994) Human extrastriate v isual cortex and the perception of faces, words, numbers, and colours Ce rebral Cortex 5, 544-554 Banich, M.T (2004) Cognitive Neuroscience an d Neuropsychology (2nd ed) Boston: Houghton Mifflin Co Barlow, H.B (1995) The 12th Bartlett Memoria l Lecture: The role of single neurons in the psychology of perception Quar terly Journal of Experimental Psychology 37A, 121-145 Blais, C et al (2008) Culture shapes how we look at faces PLoS ONE 3, 8, e3022 (Freely available at http://www.plosone.org/article/info:doi%2F10.1371%2 Fjournal.pone.0003022 ) Bradshaw, J & Wallace, G (1971) Models for the processing and identification of faces Perception and Psychophysic s 9, 443-443 Bredart, S (1993) Retrieval failures in face n aming Memory 1, 351-366 Brewer, K (2000) Psychology and Crime Oxford: Heinemann Bruce, V & Young, A (1986) Understanding face recognition British Journal of Psychology 77, 305-327 Bruce, V; Burton, A & Dench, N (1994) What's d istinctive about a distinctive face? Quarterly Journal of Experimental Psychology 47A, 119-142 Bruce, V et al (1999) Verification of face ide ntities from images captured on video Journal of Experimental Psycholog y: Applied 5, 4, 339-360 Bublitz, N (2008) A face in the crowd Scient ific American Mind April/May, 59-65 Burton, A.M & Bruce, V (1993) Naming faces and naming names: exploring an interactive activation model of person recogniti on Memory 1, 457-480 Burton, A.M; Bruce, V & Johnston, R.A (1990) U nderstanding face recognition with an interactive activation model Br itish Journal of Psychology 81, 361-380 Cohen, G (1989) Memory in the Real World Hove: Lawrence Erlbaum Cutting, J (1990) The Right Cerebral Hemisph ere and Psychiatric Disorders Oxford: Oxford University Press De Haan, E.H; Young, A & Newcombe, F (1987) Fa ce recognition without awareness Cognitive Neuropsychology 4, 385-415 De Haan, E.H et al (1987) Faces interfere wi th name classification in a prosopagnosic patient Cortex 23, 309-316 De Haan, E.H; Young, A & Newcombe, F (1991) A dissociation between the sense of familiarity and access to semantic informa tion concerning familiar people European Journal of Cognitive Psychology 3, 51-67 Diamond, R & Carey, S (1986) Why faces are a nd are not special: An effect of expertise Journal of Experimental Psychol ogy: General 115,107-117 Ellis, A.W; Young, A.W & Critchley, E.M.R (198 9) Loss of memory for people following temporal lobe damage Brain 112, 1466-1483 Ellis, H.D et al (1979) Identification of fa miliar and unfamiliar faces from internal and external features: Some imp lications for theories of face recognition Perception 8, 431-439 Ellis, H.D et al (1993) Responses to facial and non-facial stimuli presented tachistoscopically in either or both visu al fields by patients with the Capgras delusion and paranoid schizophreni cs Journal of Neurology,


Neurosurgery and Psychiatry 56, 215-219 Eysenck, M.W & Flanagan, C (2001) Psychology f or A2 Level Hove: Psychology Press Eysenck, M.W & Keane, M.T (1995) Cognitive Psy chology: A Student's Handbook (3rd ed) Hove: Psychology Press Farah, M.J et al (1995) The inverted face in version effect in prosopagnosia: Evidence for mandatory, face-specifi c perceptual mechanisms Vision Research 35, 2089-2093 Farah, M.J et al (1998) What is "special" ab out face perception? Psychological Review 105, 3, 482-498 Flude, B; Ellis, A & Kay, J (1989) Face proces sing and retrieval in an anomic aphasia: names are stored separately from se mantic information about people Brain and Cognition 11, 60-72 Frowd, C et al (2007) The relative importanc e of external and internal features of facial composites British Journal of Ps ychology 98, 61-77 Gauthier, I et al (1999) Activation of the m iddle fusiform "face area" increases with expertise in recognising novel objec ts Nature Neuroscience 2, 6, 568-573 Golari, G et al (2007) Differential developm ent of high-level visual cortex correlates with category-specific recognitio n memory Nature Neuroscience 10, 4, 512-522 Hasson, U et al (2001) Vase or face? A neura l correlate of shape-selective grouping processes in the human brain Jou rnal of Cognitive Neuroscience 13, 744-753 Hayden Ellis (1997) speaking on Science Now BBC Radio 4 Hay, D & Young, A (1982) The human face. In El lis, A (ed) Normality and Pathology in Cognitive Functions New York: Academic Press Jiang, X et al (2006) Evaluation of a shape- based model of human face discrimination using functional magnetic resonance imaging and behavioural techniques Neuron 50, 1, 159-172 Jiang, X et al (2007) Categorisation trainin g results in shape- and category-selective human neural plasticity Neuron 53, 6, 891-903 Joseph, A.B (1986) Focal central nervous sys tem abnormalities in patients with misidentification syndrome Bibliothec a Psychiatrica 164, 68-79 Kanwisher, N et al (1997) The fusiform face area: A module in human extrastriate cortex specialised for face perception Journal of Neuroscience 17, 4302-4311 Kobatake, E & Tanaka, K (1994) Neuronal sele ctivities to complex object features in the ventral visual pathway of th e macaque cerebral cortex Journal of Neurophysiology 71, 856-867 Kreiman, G et al (2000) Category-specific vi sual responses of single neurons in the human medial temporal lobe Nature Ne uroscience 3, 946-953 Lander, K (2002) Memory for faces: If the face fits? Psychology Review September, 10-13 Leder, H & Bruce, V (1998) Local and relatio nal aspects of face distinctiveness Quarterly Journal of Experimental P sychology 51, 3, 449-473 Leder, H & Bruce, V (2000) When inverted fac es are recognised: The role of configural information in face recognition Quarterly Journal of Experimental Psychology 53, 2, 513-536 Leopold, D.A et al (2006) Norm-based face en coding by single neurons


in the monkey infero-temporal cortex Nature 442, 572-575 McNeil, J.E & Warrington, E.K (1993) Prosopa gnosia: A face-specific disorder Quarterly Journal of Experimental Psycholo gy 46A, 1-10 Medin, D.L et al (2001) Cognitive Psychology (3rd ed) Fort Worth, TX: Harcourt Moscovitch, W et al (1997) What is special a bout face recognition? Nineteen experiments on a person with visual object agnosia and dyslexia but normal face recognition Journal of Cognitive Neuros cience 9, 555-604 Parr, L et al (2009) Face processing in the chimpanzee brain Current Biology 19, 50-53 Rakover, S.S (2002) Featural and configuration al information in faces: a conceptual and empirical analysis British Journal of Psychology 93, 1-30 Rhodes, G (1988) Looking at faces: First-ord er and second-order features as determinants of facial appearance Perce ption 17, 43-63 Rhodes, G & Jeffrey, L (2006) Adaptive norm- based coding of facial identity Vision Research 46, 2977-2987 Rhodes, G et al (2004) Is the fusiform face area specialised for faces, individuation, or expert individuation? Jour nal of Cognitive Neuroscience 16, 2, 189-203 Roberts, A & Bruce, V (1988) Feature saliency in judging the sex and familiarity of faces Perception 17, 475-481 Roth, I & Bruce, V (1995) Perception and Repre sentation: Current Issues (2nd ed) Buckingham: Open University Press Sadr, J et al (2003) The role of eyebrows in face recognition Perception 32, 3, 285-293 Schiltz, C et al (2006) Impaired face discri mination in acquired prosopagnosia is associated with abnormal response to individual faces in the right middle fusiform gyrus Cerebral Cortex 16, 574-586 Sergent, J (1984) An investigation into compon ent and configurational processes underlying face perception British Journa l of Psychology 75, 221-242 Sergent, J & Signoret, J-L (1992) Varieties of functional deficits in prosopagnosia Cerebral Cortex 2, 375-388 Shepherd, J.W; Davies, G.M & Ellis, H.D (1981) Studies of cue saliency. In Davies, G.M; Ellis, H & Shepherd, J (eds) Percei ving and Remembering Faces New York: Academic Press Tanaka, J.W & Farah, M.J (1993) Parts and whol es in face recognition Quarterly Journal of Experimental Psychology 46A, 225-246 Tanaka, J.W & Sengco, J.A (1997) Features an d their configuration in face recognition Memory and Cognition 25, 5, 583-592 Thompson, P (1980) Margaret Thatcher - a new i llusion Perception 9, 483-484 Tsao, D.Y & Livingstone, M.S (2008) Mechanis ms of face perception Annual Review of Neuroscience 31, 411-437 Tsao, D.Y et al (2003) Faces and objects in macaque cerebral cortex Nature Neuroscience 6, 989-995 Valentine, T (1991) A unified account of the e ffects of distinctiveness, inversion and race in face recogni tion Quarterly Journal of Experimental Psychology 43A, 161-204


Valentine, T (1996) Recognising faces and reme mbering names. In Hatcher, D (ed) 1996 Annual Conference of Associati on for the Teaching of Psychology Leicester: ATP Valentine, T & Bruce, V (1986) The effect of d istinctiveness in recognising and classifying faces Perception 15, 525-535 Yarbus, A.L (1965) Role of Eye Movements in the Visual Process Moscow: Nauka Yin, R.K (1969) Looking at upside-down faces J ournal of Experimental Psychology 81, 141-145 Yin, R.K (1970) Face recognition: a dissociabl e ability? Neuropsychology 23, 395-402 Young, A; Hay, D & Ellis, A (1985) The face th at launched a thousand slips: everyday difficulties and errors in recognis ing people British Journal of Psychology 76, 495-523 Young, A; Hellawell, D & Hay, D (1987) Configur ational information in face perception Perception 16, 747-759 Young, A; McWeeny, K; Hay, D & Ellis, A (1986) Matching familiar and unfamiliar faces on identity and expression Psychol ogical Research 48, 63-68

Psychology of Face Recognition Brief Introduction 2ndedition

Documents