Neural mechanisms of feature- based attention Taosheng Liu.

Neural mechanisms of feature-based attention

Taosheng Liu

What is attention?

“Everyone knows what attention is. It is the taking possession by the mind in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought.”

-William James (1890)

• Types of visual attention– Overt attention– Covert attention

• Spatial

• Feature-based

• Object-based

Attention and the brain

• Effects vs. control

Outline

• The effect of feature-based attention on visual cortex– How does attention modulate sensory

representations?

• The control of feature-based attention– What is the source of control and how is control

implemented?

• Attention and object recognition

The effect of FB attention to motion

Treue & Martinez-Trujillo, 1999, Nature

Respons

e

Atte

nd ‘u

p’At

tend

‘dow

n’

Questions:• Does feature-based

attention modulate neuronal subpopulations in the attended location?

• If so, how does it correlate with behavior?

MT

Respons

eAt

tend

‘up’

Atte

nd ‘d

own’

upward preferring units

More adaptation for a upward test stimulus when attending ‘up’ vs. ‘down’

Respons

edownward preferring units

Use adaptation to assess feature selectivity

fMRI adaptation

• A voxel contains many neurons.

• fMRI adaptation can assess feature selectivity within a voxel.

Adapting stimulus

play demo

0 1 2 3 4

Time (s)

0

1

2

3

Spati

alfr

equ

e ncy

(cp

d)

+20°-20°

Behavior: tilt aftereffect (n=8)

+20°-20°

0°

. . . . .

Adapter (4 s) Test (0.5 s)1 s…

Pre-adaptation (40 s)

Attend +20 Attend -20 Attend +20 Attend -20

fMRI adaptation protocol

. .. ..

Adapter (4 s) Test (1 s)

1 s 1.2 s

Attended

Unattended

Blank

…Pre-adaptation (40 s)

…

Task inside the scanner: report the orientation of the test stimulus.

fMRI details

• Siemens 3T Allegra • Surface coil• 21 coronal/oblique slices • 3 mm isotropic voxels• TE = 30 ms, FA = 75º• TR = 1.2 s• Bite bar to minimize head

motion

Surface reconstruction and retinotopic mapping

QuickTime and aﾪMicrosoft Video 1 decompressorare needed to see this picture.

Retinotopic mapping and localizer

QuickTime and aﾪYUV420 codec decompressor

are needed to see this picture.

real data (TL)

fMRI response to the test stimulus

Unattended

Attended

-0.4

-0.2

0

0.2

0.4

0.6

0.8.V1

0 5 10 15

Time (s)

fMR

I response (%)

V2

0 5 10 15

Time (s)

adapter test

0 5 10 15-0.4

-0.2

0

0.2

0.4

0.60.8

V3A/B

0 5 10 15

V7

U nattended

Attended

-0.4

-0.2

0

0.2

0.4

0.6

0.8 V3 hV4

-0.4

-0.2

0

0.2

0.4

0.6

0.8 LO1 LO 2

Time (s)

fMR

I resp

onse (%

)

Attention modulation index

Rattn – Runattn

Rattn + Runattn

Correlation between behavioral and imaging results

A model relating psychophysical and imaging data

neutralattended

-90 -45 0 45 900

0.2

0.4

0.6

0.8

1

Neu

ral resp

on

se

Preferred orientation (deg)

-90 -45 0 45 900

0.2

0.4

0.6

0.8

1

Preferred orientation (deg)

-90 -45 0 45 90-10

-5

0

5

10

Sh

ift in p

refe

rred

o

rien

tatio

n

Preferred orientation (deg)

Dragoi et al, 2000, 2001

-90 -45 0 45 900

0.2

0.4

0.6

0.8

1

Preferred orientation (deg)

Psychophysics

fMRI

Ne

ura

l re

sp

on

se

Ne

ura

l re

sp

on

se

Summary & conclusion

• Feature-based attention enhances activity of neuronal subpopulations when the attended and unattended features are processed in the same retinotopic region.

– Attentional modulation of orientation-selective fMRI response adaptation.

– Attentional modulation constant across visual areas, suggesting a feed-forward mechanism.

– Significant correlation between TAE and AMI only in V1.

Liu etal, 2007, Neuron

The control of feature-based attention

• Components of attentional control– Disengage/shift– Engage/maintain

• Goal:– Separate different

components– Feature-based

attention

Task and design

responseMotionColorinstruction

button2Green‘hold’

button1Red‘shift’

Sustained effect for motion

R SPL/IPL

-5 0 5 10 15

% sig

nal ch

ang

e

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

Time (sec)

color to motion

MT+: effects of attention for motion.

FEF, SPL/IPL: sustained attentional control for motion.

Time (sec)-5 0 5 10 15

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20L ITG (MT+)

motion to color

hold motion

hold color

Transient shift activity

color to motion

motion to color

hold motion

hold color

Precu, IPS, PCG: transient control of attention shift.

Summary

• Effects of attention: – MT+ (motion) and V4 (color)

• Attentional control: – Transient control: disengage/shift (superior parietal lobule, left

intra-parietal sulcus, left pre-central gyrus).

– Sustained control: engage/maintain (frontal eye fields, superior-inferior parietal lobule for motion; superior frontal gyrus for color).

Liu etal, 2003, Cerebral Cortex

Current and future plans

• Attentional control within feature dimensions– What are the ‘shift’ regions?– What are the ‘hold’ regions?--attentional priority

The representation of attentional priority

• Spatial attention– Higher areas with a

spatiotopic map send feedback signals

• Feature-based attention– Are there neurons that

encode the attended direction in higher areas?

FEFLIP

Decoding of brain activity Kamitani & Tong (2007)

• Classifier scheme

• Classifier can reliably decode orientation information in early visual cortex

Learning sequence of views of three-dimensional objects:

The effect of temporal coherence on object memory

How do we recognize shapes?

Temporal association: object views appearing close in time are associated.Wallis & Bulthoff (1999)

Harman & Humphrey (1999)

7 views x 1 s/view x 3 repeats

No accuracy effects

Attention? Effort? ???

Exp 1 - replication

stimuli

Exp 1 - method

Exp 1 - results

Exp 2: test novel viewsTest views 1,3,5,7

Exp 3 – method

Exp 3 - results

Exp 4

Encoding task: preference rating

“rate how much you like each sequence on a 3-point scale”

Exp 4 - results

Summary

• RR always the worst– temporal association works

• SS never exceeds SR– temporal vs. spatiotemporal coherence

• SS depends on study time and intention– potential confound