Neural basis of human visual motion perception · 2006-10-02 · Neural basis of human visual motion perception Alex Huk Neurobiology & Center for Perceptual Systems UT-Austin. Neural

Neural basis of human visualmotion perception

Alex HukNeurobiology &Center for Perceptual SystemsUT-Austin

Neural basis of motion perception

CognitiveHow is this sensory evidence accumulated, remembered,and combined with other information to guide behavior?

SensoryHow does the brain represent sensory stimuli?

2) Direction selectivity• Neurons sensitive to direction in human visual cortex

1) Motion aftereffect • Neural basis of a direction-selective visual illusion

3) Direction of moving objects• Neurons that represent direction of object motion

Goals

fMRI

Resolution • spatially coarse, temporally sluggishbut• proportional to average spike rate (Rees et al, 2000; Heeger, Huk, Geisler, Albrecht, 2000; Logothetis et al, 2001)

BOLD signal• neural activity oxygen demand proportion of deoxygenated hemoglobin MRI signal

Average spiking activity • space: across each visual cortical area• time: across several seconds

Human visual cortex

V1

V2

V2

V3

V3

V3a

V4v

Motion responses in human visual cortex

moving stationary

vs

MT+

1Neural basis of the motion aftereffect

Motion aftereffect

Addams, 1834

*

Motion aftereffect

Motion aftereffect

Motion aftereffect

Motion aftereffect

Motion aftereffect

Motion aftereffect

MAE MAE control control MAEcontrol

fMRI experiment

MT+ responses on MAE trials

MAE MAE control control MAEcontrol

fMRI experiment

Average MAE time series

MAE MAE control control MAEcontrol

Adapt Blank TestMT+ responses on MAE trials

fMRI experiment

Motion aftereffect

Difference in MT+ activity during MAE vs control?

Motion aftereffect

MAE > control

Tootell et al, Nature (1995)He et al, Curr Bio (1998) Culham et al, J Neurophys (1999)Taylor et al, Neuroimage (2000)Hautzel et al, Brain Res (2001)Berman & Colby, Cog Br Res (2001)

Huk, Ress, & Heeger, Neuron (2001)

L

R

Direction-selective neurons

Monkey MT, MST:> 90% of neurons aredirection-selective

stationary

neuralresponse

perceiveddirection

stimulus

L R

NONE

Motion aftereffect: Theory

rightwardstationary

L R L R

NONE RIGHT

neuralresponse

perceiveddirection

stimulus

Motion aftereffect: Theory

stationary

L R L R

NONE RIGHT

adaptationneuralresponse

perceiveddirection

stimulus rightward

Motion aftereffect: Theory

stationary

L R L R

NONE RIGHT

adaptationneuralresponse

perceiveddirection

stimulus rightward

Motion aftereffect: Theory

stationary

L R L R

NONE RIGHT

adaptationneuralresponse

perceiveddirection

stimulus rightward

Motion aftereffect: Theory

stationarystationary

L R L R

NONE RIGHT LEFT

L Rneuralresponse

perceiveddirection

stimulus rightward

Motion aftereffect: Theory

Motion aftereffect: Theory

Direction-selective reduction in response

stationary

LEFT

L R

Cat Primary Visual CortexGiaschi et al 1993Hammond et al 1985,1986,1988Marlin et al 1988Saul & Cynader 1989Vautin & Berkeley 1877von der Heydt et al 1973

Monkey MTPetersen et al 1985van Wezel & Britten, 2001,2002

MAEillusory motion

controlno motion

attention increases MT+ responsese.g., Beauchamp et al 1997 Treue & Martinez Trujillo 1999 Huk & Heeger 2000

fMRInet increase ? theory + single neurons

direction-selective decrease

Motion aftereffect: Puzzle

+ attention

Equating attention during test period

equate attention

task

5 sec test

motion task:which patch movedoutward faster?

• performed on both MAE and control trials

• equal, threshold difficulty

Equating attention during test period

Motion aftereffect

Difference in MT+ activity during MAE vs controlwhen attention is equal?

Motion aftereffect with controlled attention

Huk, Ress, & Heeger, Neuron (2001)

MAE = control

Summary: Motion aftereffect

MT+ responses are affected by attention to MAE

Motion aftereffect does not necessarily depend on anet increase in MT+ response

Direction-selectivity?

2Direction-selectivity inhuman visual cortex

Direction-selective adaptation

Direction-selective response decrease• Compare Repeated vs Mixed

(e.g.,Grill-Spector et al, 1999;Kourtzi & Kanwisher, 2001)

X 12

X 12

Mixed direction

Direction-selective adaptation

Adapted direction

Direction-selective adaptation

X 12

Mixed direction

Adapted direction

Direction-selective adaptation

X 12

X 12

Mixed direction

Adapted direction

Direction-selective adaptation

X 12

X 12

Mixed direction

Adapted direction

Direction-selective adaptation

X 12

X 12

Mixed direction

Adapted direction

Control attentionspeed task

mixeddirection

adapteddirection

mixeddirection

adapteddirection 6

adaptationresponseamplitude

time (sec)

0 18 36 54 72

fMRI experiment

time (sec)

mixeddirection

adapteddirection

mixeddirection

adapteddirection 6

adaptationresponse

mea

nfMRI response(% BOLD change)

0 18 36

Average fMRI response

Direction-selectiveadaptation?

Direction-selective adaptation

Huk, Ress, & Heeger, Neuron (2001)

fMRI response(% BOLD change)

adaptation baseline response

Direction-selectivity in human visual cortex

Huk, Ress, & Heeger, Neuron (2001)

direction-selectivity index(adaptation response /

baseline response)

Summary: Direction-selectivity

Direction-selective neurons,increasing across visual cortex,

strongest in MT+

Direction of moving objects?

3Direction of moving objects

+ =

Gratings and plaids

componentgrating 1

componentgrating 2

plaidpattern

≠

component-motion cells direction of local components

Component and pattern motion cells

grating component moving strong response

=

pattern-motion cellsdirection of objects and patterns

=

component-motion cellsdirection of local components

Component and pattern motion cells

grating component moving strong response

=

pattern moving strong response

Movshon et al, 1986Albright, 1984

pattern-motion cellsdirection of objects and patterns

?

Human cortexcomponent-motion cells pattern-motion cells

+

Selective adaptation protocol

Adapted direction

Mixed direction

Selective adaptation protocol

Adapted direction

Mixed direction

Selective adaptation protocol

Adapted direction

Mixed direction

…

Selective adaptation protocol

Adapted direction

Mixed direction

…

Selective adaptation protocol

Adapted direction

Mixed direction

…

Selective adaptation protocol

Adapted direction

Mixed direction

…

Selective adaptation protocol

Adapted direction

Mixed direction

…

Selective adaptation protocol

Adapted direction

Mixed direction

…

Selective adaptation protocol

Adapted direction

Mixed direction

…

…

Selective adaptation protocol

Adapted direction

Mixed direction

1.3 sec 0.7 sec

plaid motion

Trials

1.3 sec 0.7 sec

plaid motion

response 8

Trials

time (sec)

0 16 32 48

mixeddirection

adapteddirection

mixeddirection

adapteddirection 6

fMRI experiment

64

time (sec)

mixeddirection

adapteddirection

mixeddirection

adapteddirection 6

adaptationresponse

mea

nfMRI response(% BOLD change)

0 16 32

Average fMRI response

mea

n

mixeddirection

adapteddirection

time (s)0 16 32

fMRI response(% BOLD

signal change)

Modulationof pattern-motion

response

Plaid direction

Predictions: Pattern-motion activity

mixeddirection

adapteddirection

mea

n

time (s)0 16 32

fMRI response(% BOLD

signal change)

No modulationof component-motion

response

Component direction

Predictions: Component motion activity

0

0.2

-0.2

time (s)0 16 32

fMRI response(% BOLD

signal change)

mixeddirection

adapteddirection

Pattern motion adaptation

Modulation of patternmotion response?

0

0.2

-0.2

time (s)0 16 32

fMRI response(% BOLD

signal change)

mixeddirection

adapteddirection

Pattern-motion adaptation

Pattern-motion response

⇓

MT+

Pattern motion adaptation

Huk & Heeger, Nature Neurosci (2002)

0

0.2

-0.2

time (s)0 16 32

fMRI response(% BOLD

signal change)

mixeddirection

adapteddirection

MT+

Pattern motion adaptation

Huk & Heeger, Nature Neurosci (2002)

no adaptation when plaiddirection varies (p = .35 - .74)

pattern-motion

0

0.1

0.2

V1 V2 V3 V3A V4v MT+

visual area

Pattern motion selectivity across visual cortex

pattern-motiondirection-selectivity index

(adaptation response / baseline response)

Huk & Heeger, Nature Neurosci (2002)

+ =strong

pattern-motionpercept

Pattern motion perception

+ =

+ =

strongpattern-motion

percept

weakpattern-motion

percept

Pattern motion perception

Adelson & Movshon, 1982

Pattern motion perception

Adapted direction

Mixed direction

Pattern motion perception

Adapted direction

Mixed direction

…

…

Adapted direction

Mixed direction

Pattern motion perception

0

0.2

-0.2

time (s)0 16 32

mixeddirection

adapteddirection

fMRI response(% BOLD

signal change)

MT+

Pattern motion perception

weak pattern-motion response

weak percept

⇓

Huk & Heeger, Nature Neurosci (2002)

strong percept,strong adaptation

0

0.2

-0.2

time (s)0 16 32

mixeddirection

adapteddirection

fMRI response(% BOLD

signal change)

MT+

Pattern motion perception

weak percept,weak pattern motion

response

Huk & Heeger, Nature Neurosci (2002)

equally strong adaptationwith high + low SF plaids

(p = .93)

0

0.2

-0.2

time (s)0 16 32

mixeddirection

adapteddirection

fMRI response(% BOLD

signal change)

MT+

Pattern motion perception

Huk & Heeger, Nature Neurosci (2002)

0

0.2

-0.2

time (s)0 16 32

mixeddirection

adapteddirection

fMRI response(% BOLD

signal change)

MT+

Pattern motion perception

Huk & Heeger, Nature Neurosci (2002)

strong pattern-motion perceptweak pattern-motion percept

0

0.1

0.2

V1 V2 V3 V3A V4v MT+

visual area

patternmotionpercept

Relation between fMRI response and percept

pattern-motiondirection-selectivity index

(adaptation response / baseline response)

Huk & Heeger, Nature Neurosci (2002)

Summary: Pattern motion

Pattern-motion selectivity in human visual cortex

Early visual areas represent component motions,later areas (MT+) represent pattern motion

Strength of pattern-motion activitycorresponds to strength of pattern-motion percept

Conclusions

Neural basis of human motion perception

• Neurons: Direction-selectivity in human cortex

• Processing: Component-motion pattern-motion

• Perception: Relative response strengths correspond to motion percepts

Conclusions

Human / monkey homology• Strong direction-selectivity, pattern-motion in:

human MT+ ⇔ monkey MT/MST

• Human MT+ subdivisible into MT and MST (Huk, Dougherty, & Heeger, J Neurosci, 2002)

Passive viewing vs controlled attention

passive viewing controlled attention

Response to moving test stimulus 70% larger (p ~0) Responses were not saturated during task performance

trial type viewing condition, p = 0.001

Future directions: Decision

Decision [LIP]Sensory [MT]

Accumulation of evidence

?

Future directions: Decision

Combination of sensory evidence and other knowledge

Decision [LIP]

Future directions: Decision

Decision [LIP]

?Combination of sensory evidence and other knowledge

Bias

Future directions: Decision

?

1) start point: boost baseline?Decision [LIP]

Combination of sensory evidence and other knowledge

Bias

Future directions: Decision

?

1) start point: boost baseline?2) weighting: steepen slope?

Decision [LIP]

Combination of sensory evidence and other knowledge

Bias

Future directions: Decision

?

1) start point: boost baseline?2) weighting: steepen slope?3) threshold: lower maximum?

Decision [LIP]

Combination of sensory evidence and other knowledge

Bias

Collaborators

Monkey single-units and psychophysicsMike Shadlen, John Palmer

Retinotopy, gray matter segmentation & flatteningBrian Wandell, Alex Wade, Alyssa Brewer

MR Physics Gary Glover

fMRIDavid HeegerDavid RessBob Dougherty Geoff Boynton

x

y t

Future directions: Sensory

Speed

spacetime

selective adaptation: separate sensitivity to speed from temporal and spatial frequencies

The aperture problem

The aperture problem

The aperture problem

“Component motion” cells aperture problem

“Pattern motion” cells object motion

Future directions: DecisionAccumulation of evidence

?

Functional subdivision of humanareas MT and MST

Monkey MT / MST complex

MT translationsmaller receptive fieldsretinotopic organization

MST optic flowlarger receptive fieldsno clear retinotopy

MTMST

Strategy: MT+ subdivision

Localizer [MT+]• all motion responsive neurons

Retinotopy [MT]• small receptive fields• retinotopic organization

Ipsilateral [MST]• large receptive fields• no retinotopy

Localizing MT+

0 0.4 0.8correlation

vs

Retinotopy stimulus

Wedge of moving dots

motion-wedge rotates slowly through visual field

Retinotopy and receptive field size

small RF[MT]

large RF[MST]

Ipsilateral stimulus

10 deg

Ipsilateral motion

Ipsilateral stationary

Ipsilateral stimulus and receptive fieldsize

small RF[MT]

no response

large RF[MST]

ipsilateral response

Strategy: MT+ subdivision

Localizer [MT+]• all motion responsive neurons

Retinotopy [MT]• small receptive fields• retinotopic organization

Ipsilateral [MST]• large receptive fields• no retinotopy

Subdividing human MT+

Localizer [MT+] Retinotopy [MT] Ipsilateral [MST]

Subdividing human MT+

Localizer [MT+] Retinotopy [MT] Ipsilateral [MST]r rθ

Summary: Subdivision of MT+

• Human MT+ is subdivisible into MT, MST• MT contains a retinotopic map of motion direction• MST neurons summate motion over larger regions of space

Direction-selective adaptation

Direction-selective adaptation

Direction-selective adaptation

Speed task: control attention

Direction-selective adaptation

Direction-selective adaptation

Speed task: control attention

X 3

X 3

oppositedirection

adapteddirection

oppositedirection

adapteddirection 6

adaptationresponseamplitude

time (sec)

0 16 32 48 54

fMRI experiment

Direction-selective adaptation

Huk, Ress, & Heeger, Neuron (2001)

Direction-selectiveadaptation

Direction-selectivity⇓

Opposite-direction > Adapted-direction

MT+

Pattern-motion controls

Weak effects just due to SFs? strong

weak

Pattern-motion controls

Weak effects just due to SFs? No. Strong adaptation with

coherent high and low-SF plaids.(F1,55 = 0.07, p = .93)

strong

weak

Pattern-motion controls

Weak effects just due to SFs? No. Strong adaptation with

coherent high and low-SF plaids.(F1,55 = 0.07, p = .93)

Were responses really due todirection-selective adaptation?

Pattern-motion controls

Yes. Rotating plaid direction from trial to trial produced no response modulation.

(p = .35 - .74)

Weak effects just due to SFs? No. Strong adaptation with

coherent high and low-SF plaids.(F1,55 = 0.07, p = .93)

Were responses really due todirection-selective adaptation?

Conclusions

Adaptation

• Characterize neural response properties

• Quantify selectivity across visual areas

• Selectively isolate hierarchical stages of processing