Neural Mechanisms of Object Perception

Neural Mechanisms of Object Perception

Zhiyong Yang

Brain and Behavior Discovery InstituteJames and Jean Culver Vision Discovery Institute

Department of OphthalmologyGeorgia Regents University

April 4, 2013

Outline1.A model of pattern recognition

2. An updated view of the ventral pathway

3. Neural codes for object perception 3.1. V1 and V2

3.2. V4

3.3. IT

4. A perspective based on untangling object manifolds

A Model of Pattern Recognition

• Features

• Probability distributions

• Decision rule

The Ventral Pathway

Kravitz et. al., 2013

Occipitotemporal network

Output pathways

Three Cortico-subcortical Output Pathways

1. Occipitotemporo-neostriatal pathway

reinforcement learning

2. Occipitotemporo-ventral striatum pathway

value

3. Occipitotemporo-amygdaloid pathway

emotion

Three Cortico-corticalOutput Pathways

1. Occipitotemporo-medial temporal pathway

long-term memory2. Occipitotemporo-orbitofrontal pathway

reward

3. Occipitotemporo-ventrolateral pathway

working memory and executive function

Neural codes for object perception

Neural Codes in V1

1.Responses selectively to a full range of visual features

orientation, direction, disparity, speed, luminance,

contrast, color, and spatial frequency

2. Functional maps

retinotopic map, orientation map, ocular dominance

map

3. Contextual modulation

4. Adaptive

5. Sparse and decorrelated relative to inputs

Orientation Selectivity

Hubel & Wiesel, 1968

Orientation Map

Nauhaus et. al., 2008

LNL Models of V1 Neurons

simple cells complex cells

Shape Codes in V2

1.Responses to single orientation

2. Responses to multiple orientations

3. Responses to shapes of intermediate

complexity

Anzai et. al. 2007

Stimulus sets

Grating stimuli

Contour stimuli

Hegde & Van Essen, 2007

Response profiles of exemplar V4 and V2 cells

Shape Codes in V4

1.Responses selectively to curvature, orientation, and object-relative position

2. Evidence for a sparse coding scheme

Pasupathy & Connor 2002

Carlson et. al., 2011

Sparseness Index = 0.80

Sparseness Index = 0.36

Sparseness Index = 0.22

Sparseness Index = 0.11

Neural Codes in IT

1. Structural, configurational, and

compositional for both 2D and 3D objects

2. Position, orientation, curvature

3. Skeletal shape and boundary shape

3. Structural and holistic

4. Categorical clustering

Brincat & Connor, 2004

2D contour shapes

Brincat & Connor, 2004

2D contour shapes

Yamane et. al., 2008

3D shapes

Yamane et. al., 2008

3D shapes

Categorical Coding

Kriegeskorte et. al., 2008

A perspective based on untangling object manifolds

1. Core object recognition and IT codes2. Untangling object manifolds and a proposal3. Open questions

DiCarlo et. al., 2012

Core Object Recognition1.Discriminate a visual object from all other possible visual objects within <200 ms.2.Discount changes due to changes in illumination, object position, size, scale, viewpoint, and visual context, and other structural variations.3.Comprise between invariance and generalization. 4. There are ~30,000 natural objects.5. Current models approach at best ~5% of human object perception.

Untangling Object Representations

The Ventral Visual PathwayEach area proportional to cortical surface area. Total number of neurons. Dimensionality of each representationPortion (color) dedicated to processing the central 10 deg of the visual fieldMedian response latency

IT Neural Codes

1. Spike counts in ~50 ms convey information object identity

2. Object identity information is available ~100 ms after

presentation

3. IT population presentation is untangled and object

identity can be decoded by weighted summation codes.

4. These codes are quite general.

IT Single-Unit Properties and Their Relationship to Population Performance

Abstraction Layers and Their Potential Links

Serial-Chain DiscriminativeModels of Object Recognition

A Neural Network Model of Object Recognition

Serre et. al., 2007

A Model of Object Recognition