Nancy Kanwisher, MIT...Brain Specializations of Social Perception Infants Adults Daniel Harari, Tao Gao (+T1, Tenenbaum) Tao Gao, Harari (+T1, Tenenbaum) Lindsey Powell (Saxe) Peterson,

Nancy Kanwisher, MIT http://nancysbraintalks.mit.edu

My Interests: What is the architecture of the human mind (fundamental components?) How does this structure arise over development and over evolution? What are the computations and representations in each region? How do these parts work together to make us smart? What is “special” about the human brain that enables uniquely human cognition?

my part of CBMM

1

Image removed due to copyrightrestrictions. Please see the video.

Thrust 4: Social Intelligence

Goal: To understand the cognitive, computational, and neural basis of social perception.

PIs: Kanwisher, Nakayama, Tenenbaum, Saxe, Spelke

Grad Students & Postdocs:

Why social intelligence?

2

Images removed due to copyrightrestrictions. Please see the video.

– The source of much of human intelligence

– A major driver of evolution of the brain

– A large percent of human cognition (in minutes & cortical area)

• Social cognition is the crux of human intelligence

Why Social Intelligence?

-- Greatest feats of the human intellect are products of groups of people working together

Thrust four: social perception. What is that?

3

© Source Unknown. All rights reserved. This content is excluded from our Creative Commons license. For more information, see

https://ocw.mit.edu/help/faq-fair-use/.

© Getty Images. All rights reserved. This content is excludedfrom our Creative Commons license. For more information,see https://ocw.mit.edu/help/faq-fair-use/.

4

© Associated Press. All rights reserved. This content is excluded from our CreativeCommons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.

5

© Getty Images. All rights reserved. This content is excluded from our CreativeCommons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.

6

© Source Unknown. All rights reserved. This content is excluded from our CreativeCommons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.

7

© Reuters. All rights reserved. This content is excluded from our Creative Commonslicense. For more information, see https://ocw.mit.edu/help/faq-fair-use/.

8

-- Who is this?

– What are they attending to?

– What are they doing?

– What will happen next?

– What are they feeling?

– What is their character?

– Are they interacting with someone else?

– What is the nature of the interaction?

Turing++ Questions for Social Perception

Brain Specializations for Social Perception Infants Adults

Thrust 4 Approach Study these abilities in the system that does them best: the human mind and brain. Roadmap: 1. Psychophysics Characterize and quantify each ability: how good are we? what cues? characterize the input 2. Computational modeling 3. Discover brain basis with fMRI ECOG NIRs

some specific projects….. 9

-- Who is there?

– What are they attending to?

– What are they doing?

– What will happen next?

– What are they feeling?

– What is their character?

– Are they interacting with someone else?

– What is the nature of the interaction?

Brain Specializations of Social Perception Infants Adults

Lindsey Powell (+T1, Saxe)

Peterson: Face Recognition in Real-World Viewing

Turing++ Questions for Social Perception

. s s . s s

s s . ss . . s .

10

-- Who is there?

– What are they attending to?

– What are they doing?

– What will happen next?

– What are they feeling?

– What is their character?

– Are they interacting with someone else?

– What is the nature of the interaction?

Brain Specializations for Social Perception Infants Adults

Tao Gao, Harari (+T1, Tenenbaum)

Psychophysics and Modeling of reach target discrimination.

Daniel Harari, Tao Gao (+T1, Tenenbaum)

Gaze perception and the acuity of joint attention Peterson: Face Recognition in Real-World

Turing++ Questions for Social Perception

. s s . s s s s .

s s . . s .

11

-- Who is there?

– What are they attending to?

– What are they doing?

– What will happen next?

– What are they feeling?

– What is their character?

– Are they interacting with someone else?

– What is the nature of the interaction?

Brain Specializations of Social Perception Infants Adults

Daniel Harari, Tao Gao (+T1, Tenenbaum)

Tao Gao, Harari (+T1, Tenenbaum)

Peterson: Face Recognition in Real-World Viewing

Maryam Vaziri-Pashkam & Nakayama

Maryam Vaziri-Pashkam

Ultrafast Blocking movements (150ms, faster than choice RT) Predictive information exists prior to finger movement Information distributed over body Humans can extract this information without learning probably implicitly Shows stunning action prediction Next: machine learning on stimuli to discover cues

Turing++ Questions for Social Perception

. s s . s s

s s . ss . . s .

12

-- Who is there?

– What are they attending to?

– What are they doing?

– What will happen next?

– What are they feeling?

– What is their character?

– Are they interacting with someone else?

– What is the nature of the interaction?

Brain Specializations of Social Perception Infants Adults

Daniel Harari, Tao Gao (+T1, Tenenbaum)

Tao Gao, Harari (+T1, Tenenbaum)

Peterson: Face Recognition in Real-World Viewing

Maryam Vaziri-Pashkam & Nakayama What will happen next?

What are they feeling?Maryam Vaziri-Pashkam & Nakayama

Elinor McKone: Virtually entire literature on perc

of facial emotion uses posed expressions!

McKone & Dawel

Psychophysics fMRI (&MEG?) decoding

of: real versus posed real E1 vs real E2 static vs dynamic

Turing++ Questions for Social Perception

13

© Springer. All rights reserved. This content isexcluded from our Creative Commons license.For more information, seehttps://ocw.mit.edu/help/faq-fair-use/.Source: Dawel, A. et al. "Perceived emotiongenuineness: normative ratings for popular facial expression stimuli and the development ofperceived-as-genuine and perceived-as-fakesets." Behavior Research Methods (2016): 1-24. DOI 10.3758/s13428-016-0813-2.

-- Who is there?

– What are they attending to?

– What are they doing?

– What will happen next?

– What are they feeling?

– What is their character?

– Are they interacting with someone else?

– What is the nature of the interaction?

Brain Specializations of Social Perception Infants Adults

Daniel Harari, Tao Gao (+T1, Tenenbaum)

Tao Gao, Harari (+T1, Tenenbaum)

Peterson: Face Recognition in Real-World Viewing

Maryam Vaziri-Pashkam & Nakayama

McKone & Dawel

Decoding neural data (ECOG, fMRI, and MEG) of people watching movies:

identity (face/voice/body) actions good guy/bad guy interactions (pos/neg) etc.

Are they interacting with someone else?

Decoding neural data

identity (face/voice/body)

Leyla Isik

Isik, Kreiman, Kanwisher neural decoding

Isik, Kreiman, Kanwisher neural decoding

Isik, Kreiman, Kanwisher neural decoding

Turing++ Questions for Social Perception

14

-- Who is there?

– What are they attending to?

– What are they doing?

– What will happen next?

– What are they feeling?

– What is their character?

– Are they interacting with someone else?

– What is the nature of the interaction?

Brain Specializations of Social Perception Infants Adults

Daniel Harari, Tao Gao (+T1, Tenenbaum)

Tao Gao, Harari (+T1, Tenenbaum)

Peterson: Face Recognition in Real-World Viewing

Maryam Vaziri-Pashkam & Nakayama

McKone & Dawel

Isik, Kreiman, Kanwisher neural decoding

Isik, Kreiman, Kanwisher neural decoding

Isik, Kreiman, Kanwisher neural decoding

Lindsey Powell (Saxe)

Brain Specializations of Social Perception Poster: Using fNIRS to Map Functional Specificity in the Infant Brain

Turing++ Questions for Social Perception

15

-- Who is there?

– What are they attending to?

– What are they doing?

– What will happen next?

– What are they feeling?

– What is their character?

– Are they interacting with someone else?

– What is the nature of the interaction?

Brain Specializations of Social Perception Infants Adults

Daniel Harari, Tao Gao (+T1, Tenenbaum)

Tao Gao, Harari (+T1, Tenenbaum)

Lindsey Powell (Saxe)

Peterson, Ullman (+T3): Face Recognition in Real-World Viewing

Maryam Vaziri-Pashkam & Nakayama

McKone & Dawel

Ben Deen, Rebecca Saxe

The human STS comprises multiple functionally specific subregions for different aspects of social cognition.

Isik, Kanwisher, Kreiman neural decoding

Isik, Kanwisher, Kreiman neural decoding

Isik, Kanwisher, Kreiman neural decoding

Faces + Voices

Biological motion

Voice perception Language + ToM

Theory of mind

Turing++ Questions for Social Perception

. s s . s s

s s . ss . . s .

16

Courtesy of Elsevier, Inc., http:http://ocw.mit.edu/termse

MIT OpenCourseWarehttps://ocw.mit.edu

Resource: Brains, Minds and Machines Summer CourseTomaso Poggio and Gabriel Kreiman

The following may not correspond to a p articular c ou rse o n MIT OpenCourseWare, but has beenprovided by the author as an individual learning resource.

For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.