Modulation of human extrastriate visual processing by - Brain

Brain (1998),121,1357–1368

Modulation of human extrastriate visual processingby selective attention to colours and wordsAnna C. Nobre,1,2 Truett Allison3 and Gregory McCarthy2,*

1Neuropsychology Laboratory, West Haven Veterans Correspondence to: Dr A. C. Nobre, University of Oxford,Administration Medical Center and Departments of Department of Experimental Psychology, South Parks2Neurosurgery and3Neurology, Yale University Medical Road, Oxford OX1 3UD, UK.School, New Haven, USA E-mail: [email protected]

*Present address: Department of Radiology, Box 3808,Duke University Medical Center, Durham, NC 277710,USA

SummaryThe present study investigated the effect of visual selectiveattention upon neural processing within functionallyspecialized regions of the human extrastriate visual cortex.Field potentials were recorded directly from the inferiorsurface of the temporal lobes in subjects with epilepsy.The experimental task required subjects to focus attentionon words from one of two competing texts. Words werepresented individually and foveally. Texts were interleavedrandomly and were distinguishable on the basis of wordcolour. Focal field potentials were evoked by words in theposterior part of the fusiform gyrus. Selective attentionstrongly modulated long-latency potentials evoked by

Keywords: visual selective attention; event-related potentials; extrastriate cortex; top-down modulation

Abbreviations: ERP5 event-related potential; fMRI5 functional MRI

IntroductionThe brain is capable of shifting the focus of attentionselectively and dynamically based upon what is relevant tothe organism. Behavioural studies indicate that objects that arethe focus of attention are better perceived and discriminated(Posner, 1980). Studies of patients with focal brain lesionsand non-human primates have indicated that regions withinthe parietal, frontal and cingulate cortices form a corticalnetwork that controls the direction of attention in space(Mesulam, 1990). PET (Corbettaet al., 1993; Nobreet al.,1997) and functional MRI (fMRI) (Gitelmanet al., 1996;Nobreet al., 1996) have confirmed the basic architecture forthe visuospatial attention system. However, basic issuesremain regarding how the attention network modulatesstimulus processing. In this study we examined the effect ofselective attention to colour and words upon brain activitywithin regions of the human visual cortex specialized forprocessing objects and words.

© Oxford University Press 1998

words. The attention effect co-localized with word-relatedpotentials in the posterior fusiform gyrus, and wasindependent of stimulus colour. The results demonstratedthat stimuli receive differential processing withinspecialized regions of the extrastriate cortex as a functionof attention. The late onset of the attention effect and itsco-localization with letter string-related potentials butnot with colour-related potentials recorded from nearbyregions of the fusiform gyrus suggest that the attentioneffect is due to top-down influences from downstreamregions involved in word processing.

Functional specialization of human extrastriatecortexStudies of primates (Felleman and Van Essen, 1991) andhumans with brain lesions (Mesulam, 1994) have shown thatthe transduction of the external visual world involves manyfunctionally specialized brain regions. These regions havebeen shown to cluster along two main stimulus-processingstreams. One stream includes visual regions situated moredorsally, and is concerned with the position or motion ofobjects in space. In contrast, a ventral stream contains regionsspecialized for object recognition (Ungerleider and Mishkin,1982; Ungerleider, 1995). Neuroimaging methods havegreatly facilitated the localization of functionally specializedvisual cortical regions in humans. PET and fMRI haveidentified areas contributing to the perception of simplefeatures, such as colour (Luecket al., 1989) and motion(Watsonet al, 1993; McCarthyet al., 1995; Tootellet al.,1995), and to the perception of complex objects and faces

1358 A. C. Nobreet al.

(Malach et al., 1995; Puceet al., 1995, 1996; Kanwisheret al., 1997). Recordings of event-related potentials (ERPs)directly from the cortical surface have characterized thefunctional and temporal dynamics of visual areas within theventral visual stream (Allisonet al., 1994a, b). Of particularinterest to this report was the description of areas specializedfor processing colour and letter strings in the inferior temporallobe. Colour-sensitive potentials were recorded at 200–300 msfrom the posterior part of the lingual and fusiform gyri(Allison et al., 1993). Potentials evoked by letter strings, butnot by other salient or meaningful visual stimuli, wererecorded at 150–200 ms (N200) from the posterior fusiformgyrus laterally to colour-sensitive potentials (Nobreet al.,1994).

Modulation of visual processing by attentionSingle-cell recordings in monkeys have shown that the firingpatterns of visual extrastriate neurons are sensitive to thetask-relevance of the stimulus. For example, there is adynamic expansion or contraction of receptive fields in cellsin area V4 and the inferior temporal cortex during presentationof attended or ignored stimuli (Braitman, 1984; Moran andDesimone, 1985; Haennyet al., 1988; Haenny and Schiller,1988; Chelazziet al., 1993; Treue and Maunsell, 1996).

ERP studies in humans have also suggested thatvisuospatial attention can modulate processing in theextrastriate cortex. ERPs evoked by visual stimuli diverge atan early latency depending on whether the stimulus appearswithin the relevant or irrelevant location in tasks involvingsustained attention and competing sources of stimuli(VanVoorhis and Hillyard, 1977) or tasks involving shifts ofattention directed by cueing stimuli (Mangun and Hillyard,1987, 1990; Anllo-Vento, 1995). In addition, ERPs linked tosemantic processing are also significantly modulated byselective attention when word stimuli are used (McCarthyand Nobre, 1993; Bentinet al., 1995).

The ability of non-spatial visual features to direct attentionhas been less well explored. A few PET studies havedemonstrated that blood flow in selective extrastriate regionsis increased when attention is focused upon the relevantvisual feature (Corbettaet al, 1991; Haxbyet al., 1991).ERP correlates of non-spatial selective attention have alsobeen described, but these have typically been contingent onprior selection of the relevant location (e.g. Harter and Guido,1980; Harter and Aine, 1984; Hillyard and Munte, 1984;Kenemanset al., 1993; Anllo-Vento, 1995). The neuralgenerators of these non-spatial attention effects cannot bededuced from the scalp recordings to date, but have beenproposed to reflect facilitation of feature-specific channels(Harter and Aine, 1984; Aine and Harter, 1986). In thepresent study, the ability of non-spatial attention to modulatefunctionally selective areas of the visual cortex was examineddirectly with intracranial ERPs. Stimuli in a sustainedselective attention task were distinguishable on the basis ofcolour and of higher-order language-related attributes. The

task addressed the ability of these perceptual and cognitivecues to direct attention, and to modulate processing withinvisual extrastriate areas.

MethodSubjectsNineteen individuals with electrodes placed on the posteriorinferior surface of the temporal lobe participated. All werepatients of the Yale–West Haven Veterans AdministrationMedical Center epilepsy–surgery programme and hadmedically intractable epilepsy. Electrodes were implanted tolocate the seizure focus in order to evaluate the option ofneurosurgery. Experimental recordings were obtainedconcurrently with clinical monitoring 3–14 days after implantsurgery. The protocols used were approved by the Yale andVeterans Administration Medical Center HumanInvestigations Committees. Informed consent was obtained.Subjects volunteered to participate and understood theexperimental nature of the task.

Subjects were nine women and 10 men, and averaged31 years of age (range 19–44 years). Their verbal IQaveraged 91 (range 75–124) and performance IQ averaged96 (range 79–123). Most of the subjects studied were righthanded (15/19). Left-handedness is noted in relevant figurelegends. Results from the amobarbital test, which assesseshemispheric dominance for language and memory (reviewedby Sasset al., 1991), showed that the left hemisphere wasdominant for language in 18 cases. In one case the righthemisphere was dominant. Information regarding the side oftemporal lobe neurosurgery was available in 15 cases. Sevenof the 15 resections were in the left hemisphere and eightwere in the right hemisphere.

Experimental taskThe experimental task is illustrated in Fig. 1. Subjects werepresented with two randomly intermixed texts, in either redor green words. Words appeared briefly (100 ms duration)and singly in the centre of a computer monitor at a rapidpace (500–700 ms stimulus onset asynchrony). The monitorwas placed ~70 cm from the subject, and the average word(five letters) subtended 2.1° horizontally.

Texts were selected from books and periodicals targetedto the sixth-grade reading level, and contained 250–350words. Pairs of texts with similar word length were randomlyintermixed. Word order within each text was maintained, andrandomization was constrained so that no more than threesuccessive words came from a given text. Twenty textcombinations were formed from 40 texts. Subjects weretypically tested with five combinations of text. Onecombination was presented for practice and four combinationswere used during data collection. The texts used and the textcolour to be read within each pairing were randomizedacross subjects.

Attention modulation of extrastriate cortex 1359

Fig. 1 Schematic of experimental task. Words from twocompeting tasks were presented foveally (average word size 2.1°).Words appeared briefly (100 ms exposure) at a rapid pace (500–700 ms stimulus onset asynchrony). Words from the contributingtasks were interleaved randomly. Word order within each text wasmaintained and no more than three words from either textappeared in direct succession. The words from each text werediscriminable on the basis of colour. Words from one text werepresented in red letters and words from the other were presentedin green letters. Subjects read silently one of the texts forcomprehension. Texts narrated stories or discussed issues.Performance was assessed by multiple-choice questions. Correctanswers depended on reading texts.

The selective attention demands in this task had beenvalidated in behavioural studies using normal volunteersin a previous study (Nobre, 1993). Normal subjects weresignificantly more accurate in answering questions pertainingto tasks under selective attention conditions than in conditionsin which they were required to divide their attention and readboth texts simultaneously. Recordings from scalp electrodesrevealed that the ERPs elicited by identical physical stimulidiffered as a function of the focus of attention. Equivalentresults were obtained when texts and questions were selectedfrom standardized college entrance scholastic tests orperiodicals targeted towards the sixth-grade reading level.

Colour information and text cohesiveness were both foundto contribute to the behavioural and ERP selective-attentioneffects.

The experiment took place in the subject’s hospital room.Lighting and noise were reduced as much as possible duringrecording. The EEG was recorded from 32 or 64 electrodessimultaneously, filtered with a bandpass of 0.1–100 Hz, anddigitized at 170–250 Hz. Codes indicating the onset and thetype of each stimulus were incorporated into the data streamfor off-line analysis. Reference and ground electrodes wereplaced on surface locations, typically the mastoid. Electrodesexhibiting abnormal seizure-related activity were excludedfrom the sites chosen for recording. It was not necessary toapply artefact rejection procedures. Common artefacts whichoccur in scalp recordings, such as those which result fromeye blinks or muscle contraction, do not impose problems inintracranial recordings.

Subjects were seated upright in an adjustable bed andviewed the monitor, which was placed perpendicular to theirgaze. They were instructed to read silently for comprehensioneither the red (the ‘attend-red condition’) or the green (the‘attend-green condition’) text. Subjects had at least onepractice block with the task. A minimum of four experimentalblocks followed, two in each experimental condition. Eachblock lasted 5–7 min. At the conclusion of each block subjectsanswered multiple-choice questions about the relevant text.The questions were text-specific and could not have beenanswered from general knowledge.

Subjects often participated in other visual tasks whichinvolved coloured stimuli or words. Seven subjects alsoparticipated in an anomalous-sentence task, in whichsentences that either ended normally or with a semanticallyanomalous word were viewed one word at a time (Kutas andHillyard, 1980; McCarthyet al., 1995; Nobre and McCarthy,1995). Nine subjects participated in a colour-adaptation task(Allison et al., 1993) in which red and blue chequerboardswere presented successively as pairs, in either repeated ornon-repeated fashion.

Electrode placement and localizationThis report focuses on recordings made from subduralelectrodes (2.2 mm diameter and 1.0 cm inter-electrodedistance) over the posterior part of the inferior temporal lobein the 19 subjects. The recording sites considered were asubset of the total sites available, and included only electrodeslocated posteriorly to the posterior commissure line andoverlying the inferior occipital, lingual, fusiform or inferiortemporal gyri, as well as the intermediate sulci. In total,recordings from 135 sites were analysed: 76 over the lefthemisphere and 59 over the right hemisphere. Strips of 8–12 stainless-steel electrodes were placed subdurally throughburr holes in the skull during implant surgery. Grids placedfollowing craniotomy contained a maximum of 64 electrodesin an 8 3 8 matrix. Electrodes were localized in MRimages obtained following implantation, and locations were


translated into the standardized coordinate system of Talairachand Tournoux (1988). Three-dimensional digital images ofstructural MRI data were used for electrode localization. TheTalairach coordinates were computed automatically followingidentification of major landmarks in the MR images: theanterior and posterior commissures, and the most posterior,anterior, superior and inferior cortical points. Thestandardized and normalized Talairach space was used tocharacterize the electrode location along they-axis (anterior–posterior). In thex-axis (medial–lateral) the use of Talairachcoordinates would often lead to incorrect anatomicalcharacterization due to individual variation in theconfiguration of gyri and sulci. Therefore, in the medial–lateral dimension each electrode was located relative to majorsurrounding sulci. Coronal images and images reconstructedin the plane accompanying the course of the electrodestrips were used for anatomical inspection. For example, anelectrode located on the lateral fusiform gyrus, midwaybetween the midfusiform sulcus and the occipitotemporalsulcus, is so depicted in the figures. Thus, even though aschematic brain is used for illustrative purposes, each locationis anatomically correct. Electrodes were placed bilaterally innine subjects, in the left hemisphere only in six, and in theright hemisphere only in four.

ResultsBehavioural performanceSubjects exhibited a high degree of comprehension of therelevant (attended) texts. The score averaged 90% and rangedbetween 67 and 100%. It was not possible to query thesubjects’ comprehension of the irrelevant texts systematically,since this would have altered the ‘ignore’ condition wewere attempting to impose. During debriefing following theexperiment, however, subjects indicated ignorance of thecontents of the irrelevant text.

The efficacy of this selective attention condition has beenvalidated in studies of normal volunteers, in whom it waspossible to do more extensive experimentation and to usestandardized texts and scoring procedures. In such studiescomprehension scores were significantly higher when subjectsfocused attention on one text compared with when attentionwas divided across both texts simultaneously (Nobre, 1993).

Modulation of focal field potentials in theinferior temporal lobeFocal effects of selective attention were observed consistentlyon ERPs elicited by words recorded from the posteriorportion of the fusiform gyrus. Figure 2 shows characteristiceffects recorded from five subjects. The waveforms are shownin Fig. 2A. In one case two waveforms are presented fromone subject (waveforms 1 and 2). Recordings from the fourword conditions in the experiment are shown in each case.Solid waveforms represent ERPs elicited by words that were

attended, while dashed waveforms were elicited by ignoredwords. Black and grey waveforms represent ERPs elicitedby red and green words respectively. In all cases, the selectiveattention effect was largely independent of word colour. Thewaveforms diverged as a function of whether the text wasattended or ignored. The onset of the attention effect wastypically late (400–500 ms). Words that appeared in theignored text elicited waveforms characterized by a largenegative-going deflection between 350 and 800 ms (closedarrows), whereas attended words remained more positiveduring the later portion of the waveform.

Most waveforms elicited a pronounced earlier sharpnegative potential peaking around 150 ms (N200, openarrows). However, the selective-attention effect was alsorecorded from locations adjacent to the maximal N200 peak.Note that the late attention effect is similar at sites 1 and 2despite considerable differences in earlier ERPs at these sites.However, in no case have we recorded a clear attention effectin the absence of earlier ERPs evoked by words. The N200was not consistently affected by selective attention.

The electrode locations for the five subjects are shown ona schematic brain in Fig. 2B. The Talairach coordinate wasused to display the electrode along they-axis (anterior–posterior) of the brain. However, the correct anatomicallocation of the electrodes relative to the major gyri and sulciin the medial–lateral dimension was preserved, by locatingthe electrode locations on coronal MRI sections (Fig. 2C).All of the electrodes in this case were located on thefusiform gyrus.

Fifteen of the 19 subjects had electrodes over the posteriorfusiform gyrus or occipitotemporal sulcus. Of these, fourteenexhibited ERPs that were modulated by attention. Theproportion of effects obtained in the other anatomicallocations sampled was much lower. Table 1 summarizes thedistribution of the attentional effects observed. ERPs werealways compared across stimuli with identical physicalcharacteristics. Attention effects were defined as a cleardeviation between ERPs elicited by attended and ignoredwords that was consistent across colours. The attention effectwas therefore independent of physical differences betweenstimuli, and the two colours served as a within-subjectreplication. These attention effects were not subtle, andseparate scoring by two of the investigators (A.C.N. andG.McC.) was completely convergent.

Focal ERPs and their modulation by attention wererecorded in both hemispheres. ERPs modulated by attentionwere obtained in 10 of 11 subjects with posterior fusiformelectrodes on the left hemisphere, and in seven of eightsubjects with posterior fusiform electrodes on the righthemisphere.

Figure 3 illustrates the focal nature of the selective-attention effect in one additional subject. The main effectcan be seen at electrode 2, located on the posterolateralfusiform gyrus. Electrode 1, located on the inferior occipitalgyrus, also recorded a large ERP, but the magnitude of theattention effect was greatly attenuated. Electrode 3, located


Fig. 2 Examples of focal effects of selective attention in five right-handed subjects. (A) Typical averaged potentials evoked by each wordtype (n 5 600) in the stories across runs. Positive polarity is plotted upwards in this and all figures. Solid black lines indicate thepotentials elicited by the red words when subjects attended to the red stories. Dashed dark lines indicate the potentials elicited by the redwords when subjects ignored the red stories. Similarly, light solid lines indicate the potentials elicited by green words when subjectsattended the green stories, while light dashed lines indicate potentials elicited by green words that were ignored. This convention isfollowed in all figures (solid5 attended words). The most prominent attention effect occurred during the later part of the waveforms,where ignored words elicited a large late negative potential at 500–700 ms (closed arrows). Words often elicited a characteristic negativepotential around 150 ms (N200, open arrows, sites 2 and 3) at focal locations of the posterior inferior temporal lobe. Waveformsrecorded from sites 1 and 2 are from the same subject. Each of the other sets of waveforms comes from a different subject. (B)Illustration of the corresponding electrode locations on the inferior surface of the brain. The relevant anatomical landmarks are labelledon the right hemisphere of Fig. 3A. The anterior–posterior location of the electrode was depicted using the Talairachy coordinate. Thelevels of the anterior (y 5 0) and posterior commissures (AC and PC) are marked on the schematic view of the inferior surface of thebrain, as well as the anterior–posterior extents at each 20 mm. The medial–lateral location of the electrode was labelled according toactual anatomical location relative to the major sulci and gyri seen on coronal MRI sections. (C) Tracing of a coronal section used todetermine the medial–lateral location of a sample electrode location (example 5, filled circle). The major relevant sulci and gyri aremarked, from medially to laterally: CS5 collateral sulcus; FG5 fusiform gyrus; ITG5 inferior temporal gyrus; LG5 lingual gyrus;OTS 5 occipitotemporal sulcus.

on the posterior part of the inferior temporal gyrus, recordeda much smaller ERP and a smaller attention effect. Theselective attention effect at electrode 2 was similar to theexamples in Fig. 2. The effect was independent of wordcolour. In the cases of red texts (Fig. 3B) and green texts(Fig. 3C), the waveforms differed according to whether thetext was attended or ignored. The onset of the attention effectwas relatively late (400–500 ms), occurring after the positivepotential peaking around 250 ms (P250, asterisk). Words thatappeared in the ignored text elicited waveforms characterized

by a large negative-going deflection between 350 and 800 ms(closed arrowheads), whereas words that appeared in theattended text elicited waveforms characterized by lessnegativity or a positivity. The pronounced earlier negativepotential peaking around 150 ms (N200, open arrowheads)was not affected by selective attention.

Figure 4 shows examples of attention effects recordedfrom both hemispheres of an additional subject. As in Figs2 and 3, the effect was independent of colour. The attentioneffect was centred in the posterior fusiform gyrus (Fig. 4B).


Table 1 Distribution of attention effect in the posteriorinferior temporal lobe

Anatomical region Effects by Effects bysubject electrodes

Left occipital inferior gyrus 1/4 25% 1/8 13%Right occipital inferior gyrus 1/3 33% 1/4 25%Occipital inferior gyrus bilaterally 1/5 20% 2/12 16%

Left lingual gyrus 0/7 0% 0/11 0%Right lingual gyrus 0/5 0% 0/10 0%Lingual gyrus bilaterally 0/11 0% 0/21 0%

Left fusiform gyrus 10/11 91% 22/33 67%Right fusiform gyrus 7/8 86% 7/19 37%Fusiform gyrus bilaterally 14/15 93% 29/52 56%

Left inferior temporal gyrus 4/13 31% 4/27 15%Right inferior temporal gyrus 1/11 9% 1/23 4%Inferior temporal gyrus bilaterally 5/16 31% 5/50 10%

Proportion of attention effects by the total recording sites indifferent regions of the posterior inferior temporal lobe. Resultsare summarized by the number of subjects containing electrodesat the relevant location, and by the total number of recordingsmade across subjects from each anatomical region. Both thefraction and the percentage of positive findings are presented.Because some effects were very focal, the proportion of effectscalculated with respect to the total recording sites is generallylower than the proportions calculated with respect to subjects.Electrodes overlying sulci are included with the region medial toit (i.e. the lingual gyrus region includes electrodes on thecollateral sulcus and the fusiform gyrus region includes electrodeson the occipitotemporal sulcus as well as on the mid-fusiformsulcus).

Location 3 in the right hemisphere (Fig. 4A) and locations2 and 5 of the left hemisphere (Fig. 4C) registered the largesteffects (closed arrowheads). The effect had a relatively lateonset, about 500 ms, and did not modulate the earlier N200(open arrowheads) and P250 (asterisks) potentials. Attentioninduced changes in waveform morphology that were similarto those in Figs 2 and 3. Attended words elicited ERPs thatremained positive during the later part of the waveform,while ignored words evoked a large negative deflection.

Polarity inversion of ERP attention effectswithin the fusiform gyrusThe change in polarity of potential from the region ofnegative current sinks to the region of positive current sourcesis referred to as a polarity inversion (reviewed by Allisonet al., 1986). Polarity inversions combined with a steepvoltage gradient indicate proximity to the active tissue. Apolarity inversion from the surface cortex to the underlyingwhite matter is indicative of a local generator in that regionof cortex, while a polarity inversion recorded across a sulcuslocates the active tissue to cortex within the sulcus (e.g.Allison et al., 1991).

In these recordings we sometimes observed polarityinversions of the late negative ERP evoked in the ignore

condition. An example is shown in Fig. 5. This recordingwas made in a subject who had a grid of electrodes on thefusiform and inferior temporal gyri. From the lateral portionof the fusiform gyrus (locations 2, 4 and 7) the typical broadnegativity was evoked in the ignore condition (closed arrows).This negativity inverted in polarity to a positivity at electrodes3 and 6, located in the medial fusiform gyrus (openarrowheads). The inversion is best seen in the differencewaveform shown in Fig. 5C, and primarily occurred betweenlocations in the medial and lateral fusiform gyrus.

Figure 6 summarizes similar polarity inversions of theignore condition ERP in all additional cases in which it wasobserved. In most cases the polarity inversion occurredbetween electrodes on either side of the occipitotemporalsulcus, or between electrodes on either side of a sulcusseparating the medial and lateral parts of the fusiform gyrus(the midfusiform sulcus). The filled black symbols denoteelectrodes at which attended words elicited more positiveERPs than ignored words (as in Figs 2–4). Non-filled symbolsare electrodes at which the opposite pattern was observed:ERPs elicited by ignored words were more positive. Asbefore, the effects were invariant with respect to word colour.Lines connect electrode locations showing the reversalswithin individual subjects. Data from one subject, who wasstudied on two separate occasions (1 year apart), are shownusing circles instead of squares.

Relationship of colour processing to theattention effectNine of the 19 subjects studied also participated in a studyof visual areas sensitive to colour (Allisonet al., 1993).Colour-sensitive potentials were recorded in seven of thesesubjects, and were located over the posterior part of thelingual and fusiform gyri, usually medial to sites exhibitinglarge attention effects. At these locations, colour affected theshape of the ERP elicited by red and green words. ERPsrecorded at colour-sensitive locations did not exhibit thelarge ignore-condition negativity. Attention effects wereoccasionally observed, but were small and not always reliableacross colours.

Relationship of word-related processing to theattention effectSeven subjects also participated in tasks investigating wordprocessing in sentences. Focal attention effects were observedin all seven cases. Overall, there was good correspondencebetween the shape of the ERPs in the present task and in theother word tasks. Waveforms often contained prominentnegative potentials, similar to the previously described letterstring-specific N200 (Nobreet al., 1994) and late negativedeflections. It was not possible to confirm that the N200swere letter string-specific for this subset of subjects, however,because they did not partake in a sufficient number of other


Fig. 3 Example of focal effect of selective attention in one additional right-handed subject. (A) Illustration of the electrode locations onthe inferior surface of the brain. The relevant anatomical landmarks are labelled on the right hemisphere. OIG5 inferior occipital gyrus.See Fig. 2 legend for other abbreviations. (B) Averaged potentials evoked by all red words in the stories across runs. Solid lines indicatethe potentials elicited by the red words when subjects attended to the red stories. Dashed lines indicate the potentials elicited by the redwords when subjects ignored the red stories. Words elicited a negative potential around 150 ms (N200, open arrowhead) and a positivepotential around 250 (P250, asterisk) at focal locations of the posterior inferior temporal lobe. In addition, ignored words elicited a largelate negative potential at 500–700 ms (closed arrowhead). (C) Equivalent averaged potentials elicited by all green words across runs.

Fig. 4 Examples of focal effects of selective attention in the two hemispheres of another subject (left-handed). (A) Averaged potentialsevoked by all green words in the right hemisphere. The largest ERP features (N200, open arrowhead; P250, asterisk) co-localized withthe largest attention effect (late negativity to ignored words, closed arrow) and occurred in the posterior lateral portion of the fusiformgyrus (electrode 3). (B) Diagram of electrode locations on the inferior surface of both hemispheres. (C) Averaged potentials evoked byall green words in the left hemisphere. The largest ERPs (open arrowheads, asterisks) and attention effects (closed arrowheads) occurredin the posterior portion of the fusiform gyrus.


Fig

.5

Exa

mpl

esof

reco

rdin

gsfr

oma

grid

ofsu

bcor

tical

elec

trod

esov

erth

ein

feri

orsu

rfac

eof

the

post

erio

rte

mpo

ral

lobe

inon

ead

ditio

nal

subj

ect

(lef

t-ha

nded

).(A

)L

ocat

ion

ofel

ectr

odes

over

the

left

post

erio

rin

feri

orte

mpo

ral

lobe

.Fi

lled

circ

les

repr

esen

tel

ectr

ode

loca

tions

whe

reig

nore

dw

ords

elic

ited

ala

rge

late

nega

tive

pote

ntia

l,as

inpr

evio

usfig

ures

.O

pen

circ

les

are

loca

tions

whe

reth

epo

lari

tyof

the

atte

ntio

nef

fect

was

reve

rsed

orin

cons

iste

nt.

(B)

Ave

rage

dpo

tent

ials

elic

ited

byal

lre

dw

ords

.W

avef

orm

sar

esh

own

inth

esa

me

layo

utas

the

elec

trod

elo

catio

nsin

A.

Lar

gene

gativ

ela

tepo

tent

ials

elic

ited

byig

nore

dw

ords

are

labe

lled

with

clos

edar

row

head

s.R

ever

sed

atte

ntio

nef

fect

sar

ela

belle

dw

ithop

enar

row

head

s.(C

)Su

btra

ctio

nw

avef

orm

sis

olat

ing

the

atte

ntio

nef

fect

.W

avef

orm

sel

icite

dby

igno

red

wor

dsw

ere

subt

ract

edfr

omw

avef

orm

sel

icite

dby

atte

nded

wor

ds.

The

resu

lting

pola

rity

ofth

em

ore

typi

cal

atte

ntio

nef

fect

,w

hen

igno

red

wor

dsel

icit

larg

erne

gativ

ityth

anat

tend

edw

ords

,is

posi

tive

(clo

sed

arro

whe

ads)

.T

here

vers

alof

the

effe

ctre

sults

ina

nega

tive

pote

ntia

lin

the

subt

ract

ion

wav

efor

ms

(ope

nar

row

head

s).


Fig. 6 Summary of polarity inversions of the ignore-condition ERP in all additional cases in which itwas observed (n 5 8). The filled black symbols denote electrodes at which attended words elicitedmore positive ERPs than ignored words. Non-filled symbols are electrodes at which the opposite patternwas observed: ERPs elicited by ignored words were the more positive. The effects were invariant withrespect to word colour. Lines connect electrode locations showing the reversals within individualsubjects. Data from one subject, who was studied on two separate occasions 1 year apart, are shownusing circles instead of squares.

visual recognition tasks. When present, N200 potentials werenot systematically modulated by attention, and were notaffected by the semantic content or context of words inother word tasks. The latency and duration of the negativedeflections were similar across tasks despite large differencesin stimulus onset asynchrony, suggesting that the time-courseof the ERPs was not determined by the onset of the followingstimulus. The later portions of the waveform showed nosystematic effects of word content or context in the otherword or sentence tasks.

DiscussionThe present study has demonstrated that neuronal activitywithin specialized regions of the extrastriate visual cortexcan be altered by selective attention independent of thephysical attributes of the stimulus. Attention is thereforecapable of affecting visual processing not only across visualareas but also within cortical areas that are specialized forprocessing particular attributes. Identical physical stimulireceived differential processing within these visual regionsas a function of their task relevance. These findings supportand extend the results from previous neuroimaging andneurophysiological studies of selective attention. Thedifferential activation between areas of the extrastriate cortexhas been seen when subjects selectively attended to colour,form or speed of motion (Corbettaet al., 1990, 1991). Thedirect demonstration of attention effects within extrastriateregions supports the interpretations from related scalp ERPexperiments. The earliest effects of visual spatial selectiveattention in tasks using simple stimuli have been ascribedto modulation of visual extrastriate regions (Mangun andHillyard, 1991; Mangun,1995). These conclusions werestrengthened by dipole localization of the ERP effects and

coregistration with PET results (Heinzeet al., 1994; Woldorffet al., 1996).

The present study also indicated that modulation of visualprocessing within extrastriate regions can occur in the absenceof any spatial cues. To date, ERP effects of selective attentionto non-spatial features have been contingent upon priorselection of spatial location (e.g. Harter and Guido, 1980;Harter and Aine, 1984; Hillyard and Munte, 1984; Kenemanset al., 1993; Anllo-Vento, 1995).

Local generation of the selective-attention effectThe evidence suggests that the potentials recorded from theinferior temporal lobe, and their modulation by attention,were locally generated in the posterior half of the fusiformgyrus and surrounding sulci. First, these potentials were quitefocal and exhibited steep voltage gradients. Secondly, thepolarity of the late attention effect sometimes changed acrosspairs of electrodes, with one or both electrodes situated inthe posterior fusiform gyrus. Polarity reversals coupled withsteep voltage gradients are generally taken as evidence forlocal generation of neuronal activity (e.g. Schlag, 1973;Creutzfeldt, 1974; reviewed by Allisonet al., 1986). In thecase of active cortex situated within a sulcus, the polarityinversion occurs from one side of the sulcus to the other.A well-studied example occurs in the hand area of thesomatosensory cortex in monkeys (McCarthyet al., 1991)and humans (Allisonet al., 1989, 1991). Analogousrecordings were obtained in this study (Figs 5 and 6), andsuggest active cortex within the occipitotemporal sulcus orwithin the midfusiform sulcus. In addition, when therecordings were made from the posterior fusiform withoutcrossing a sulcus, polarity inversions were not seen. Rather,the attention effect was seen as a relative positivity in the


500–800 ms latency range (e.g. Figs 2–4). This is consistentwith active generators in the surface cortex of the fusiformgyrus, as well as along sulci. Thus these recordings stronglysuggest that the attention effect occurs primarily in thesurface cortex of the fusiform gyrus or in the cortex withinthe midfusiform or occipitotemporal sulci.

Functional specialization of visual areasmodulated by attentionThe results suggest that the attention effect occurred primarilyin the parts of the fusiform gyrus specialized for processingwords rather than colours. The functional specialization ofthe cortex most strongly modulated by attention could notbe inferred from the present study alone. However, the dataare most consistent with the interpretation that these regionswere equivalent or adjacent to regions that respondpreferentially to letter strings (Nobreet al., 1994). The largestERPs and attention effects were recorded from the same areawhere letter string N200s were described previously—theposterior fusiform gyrus. Waveforms often contained an earlynegative potential, and focal potentials were also elicited bywords in other tasks. Neither the shape of the ERPs nor theirmodulation by attention was affected by word colour. In asubset of subjects recordings were taken from sites wherecolour-specific potentials were identified. At these sites theERPs varied according to word colour, and attention effectswere smaller and less consistent.

The attention experiment was carried out before we begantesting face-specific processing sites in the fusiform gyrussystematically (Allisonet al., 1994a, b), hence the relationshipbetween word-attention and possible face-attention sites isunknown. However, as a rule, letter string N200 sites aresomewhat medial to face N200 sites (Allisonet al., 1994a,1996; Nobreet al., 1994). It is possible that face and wordattention effects may show some anatomical segregationrelated to the segregation of earlier perceptual processing ofthese stimulus categories.

Top-down modulation of extrastriate cortex byattentionThe late onset of the attention effect, presumably after mostcolour and word-form processing is completed, suggests top-down modulation of visual processing in a letter stringprocessing region. The attention effects in the present taskoccurred several hundred milliseconds after the letter stringN200s (Nobreet al., 1994). While letter string potentialstypically peaked around 150–200 ms, the attention effectsstarted around 350 ms and peaked later. Visual processinglinked to the N200, presumably related to word form, hadprobably been concluded at the onset of the selective-attention effect.

Two types of information were available that distinguishedrelevant and irrelevant stimuli in the task: colour and

language-related cues. Colour was the only distinguishingphysical feature. Other visual features, such as word form,did not differ across attention conditions. Other than colour,only higher-order constraints related to language wereavailable, such as semantic priming and syntactic constraints.

Top-down modulation of extrastriate processing of letterstrings is the most parsimonious description of the selective-attention effect. Higher-order language constraints may havecontributed to extrastriate modulation, either directly orindirectly through communication with brain regions in anexecutive attention system (Mesulam, 1990; Posner andRaichle, 1994). Colour information may also have contributedto the modulation of visual processing in colour-insensitiveextrastriate regions. In order to assess the relativecontributions of colour and language-related information,further studies would be required in which each dimensionis manipulated in turn. Such manipulations have been carriedout using scalp recordings. Both colour and language-relatedinformation modified the scalp selective-attention effect(Nobre, 1993). Intracranial studies will be required, however,to assess the effect on specific extrastriate regions.

It is difficult to know what such late differential activitycould contribute toward selection of word reading. Somepossibilities include the integration of word meaning withthe ongoing context of the relevant story (or inhibition ofintegration of processing linked to irrelevant stimuli), signalsrelated to working memory of the relevant text which serveto prime visual areas for differential treatment of subsequentstimuli, or internal rehearsal strategies. It is tempting tointerpret the large negative deflection evoked by ignoredwords as an active filtering process. However, the polarityof a potential does not specify whether the underlyingneuronal process is inhibitory or excitatory (Allisonet al.,1986). In addition, terminal words during the anomaloussentence task, which were presumably attended, also exhibitedlarge negative deflections. The alternative interpretation isjust as likely: that the processing of attended words isenhanced and is reflected electrophysiologically by anincreased positive ERP. Finally, filtering of irrelevant stimuliand enhancement of relevant stimuli are not mutuallyexclusive.

Lateralization of the selective-attention effectNo strong evidence of hemispheric lateralization was obtainedfor the attention effect. Focal ERPs and attention effectswere observed in the left and right hemispheres with similarprobability and magnitude. Polarity inversions were observedmore frequently in the left hemisphere (six out of eightcases), but the significance of this finding is uncertain.Attention effects upon visual areas specialized for processingletter strings might have been expected to show stronger left-hemisphere lateralization. However, intracranial ERPs linkedto letter strings have been recorded bilaterally and didnot exhibit consistent lateralization (Nobreet al., 1994).Interpreting lateralization of intracranial ERP studies in


general is problematic because electrode placement is oftenasymmetrical, and functional reorganization cannot be ruledout in patients. Lateralization of word-specific processingand attention would be better investigated with methods thatenable more symmetrical sampling. Scalp recordings of word-specific potentials (Nobre and McCarthy, 1994; Breyer andNobre, 1996) and fMRI studies (Puceet al., 1996) haveshown stronger left lateralization of word processing in theextrastriate cortex.

ReferencesAine CJ, Harter MR. Visual event related potentials to colouredpatterns and colour names: attention to features and dimension.Electroencephalogr Clin Neurophysiol 1986; 64: 228–45.

Allison T, Wood CC, McCarthy G. The central nervous system. In:Coles MGH, Donchin E, Porges SW, editors. Psychophysiology.New York: Guilford Press; 1986. p. 5–25.

Allison T, McCarthy G, Wood CC, Darcey TM, Spencer DD,Williamson PD. Human cortical potentials evoked by stimulationof the median nerve. II: Cytoarchitectonic areas generating short-latency activity. J Neurophysiol 1989; 62: 694–710.

Allison T, McCarthy G, Wood CC, Jones SJ. Potentials evoked inhuman and monkey cerebral cortex by stimulation of the mediannerve. [Review]. Brain 1991; 114: 2465–503.

Allison T, Begleiter A, McCarthy G, Roessler E, Nobre AC, SpencerDD. Electrophysiological studies of color processing in humanvisual cortex. Electroencephalogr Clin Neurophysiol 1993; 88:343–55.

Allison T, Ginter H, McCarthy G, Nobre AC, Puce A, Luby M,et al. Face recognition in human extrastriate cortex. J Neurophysiol1994a; 71: 821–5.

Allison T, McCarthy G, Nobre A, Puce A, Belger A. Humanextrastriate visual cortex and the perception of faces, words,numbers, and colors. [Review]. Cereb Cortex 1994b; 4: 544–54.

Allison T, Puce A, Nobre A, Spencer D, McCarthy G.Electrophysiological evidence for anatomical segregation of regionsof extrastriate visual cortex responsive to faces, letter strings, andcolors. Neuroimage 1996; 3 (3 Pt 2): S263.

Anllo-Vento L. Shifting attention in visual space: the effects ofperipheral cueing on brain cortical potentials. Int J Neurosci 1995;80: 353–70.

Bentin S, Kutas M, Hillyard SA. Semantic processing and memoryfor attended and unattended words in dichotic listening: behavioraland electrophysiological evidence. J Exp Psychol: Hum PerceptPerform 1995; 21: 54–67.

Braitman DJ. Activity of neurons in monkey posterior temporalcortex during multidimensional visual discrimination tasks. BrainRes 1984; 307: 17–28.

Breyer NB, Nobre AC. Neurophysiological studies of semanticorganisation in the human brain. Neuroimage 1996; 3 (3 Pt 2): S430.

Corbetta M, Miezin FM, Dobmeyer S, Shulman GL, Petersen SE.Attentional modulation of neural processing of shape, color, andvelocity in humans. Science 1990; 248: 1556–9.

Corbetta M, Miezin FM, Dobmeyer S, Shulman GL, Petersen SE.Selective and divided attention during visual discrimination ofshape, color, and speed: functional anatomy by positron emissiontomography. J Neurosci 1991; 11: 2383–402.

Corbetta M, Miezin F, Shulman G, Petersen SE. A PET study ofvisuospatial attention. J Neurosci 1993; 13: 1202–26.

Creutzfeldt O. Neuronal basis of EEG waves. In: Re´mond A, editor.Handbook of electroencephalography and clinical neurophysiology,Vol. 2, Pt C. Amsterdam: Elsevier; 1974. p. 2C-5–2C-55.

Felleman DJ, Van-Essen DC. Distributed hierarchical processing inthe primate cerebral cortex. Cereb Cortex 1991; 1: 1–47.

Gitelman DR, Nobre AC, Meyer JR, Parrish TB, Callahan C,Russell EJ, et al. Functional magnetic resonance imaging of covertspatial attention. Neuroimage 1996; 3 (3 Pt 2): S180.

Haenny PE, Schiller PH. State dependent activity in monkey visualcortex. I. Single cell activity in VI and V4 on visual tasks. ExpBrain Res 1988; 69: 225–44.

Haenny PE, Maunsell JH, Schiller PH. State dependent activity inmonkey visual cortex. II. Retinal and extraretinal factors in V4.Exp Brain Res 1988; 69: 245–59.

Harter MR, Aine CJ. Brain mechanisms of visual selective attention.In: Parasuraman R, Davies DR, editors. Varieties of attention.Orlando: Academic Press; 1984. p. 293–321.

Harter MR, Guido W. Attention to pattern orientation: negativecortical potentials, reaction time, and the selection process.Electroencephalogr Clin Neurophysiol 1980; 49: 461–75.

Haxby JV, Grady DC, Horwitz B, Ungerleider LG, Mishkin M,Carson RE, et al. Dissociation of object and spatial visual processingpathways in human extrastriate cortex. Proc Natl Acad Sci USA1991; 88: 1621–5.

Heinze HJ, Mangun GR, Burchert W, Hinrichs H, Scholz M, MuenteTF, et al. Combined spatial and temporal imaging of brain activityduring visual selective attention in humans. Nature 1994; 372: 543–6.

Hillyard SA, Munte TF. Selective attention to color and location:an analysis with event-related brain potentials. Percept Psychophys1984; 36: 185–98.

Kanwisher N, McDermott J, Chun MM. The fusiform face area: Amodule in human extrastriate cortex specialized for face perception.J Neurosci 1997; 17: 4302–11.

Kenemans JL, Kok A, Smulders FT. Event-related potentials toconjunctions of spatial frequency and orientation as a function ofstimulus parameters and response requirements. ElectroencephalogrClin Neurophysiol 1993; 88: 51–63.

Kutas M, Hillyard SA. Reading senseless sentences: brain potentialsreflect semantic incongruity. Science 1980; 207: 203–5.

Lueck CJ, Zeki S, Friston KJ, Deiber MP, Cope P, CunninghamVJ, et al. The colour centre in the cerebral cortex of man. Nature1989; 340: 386–9.

Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, KennedyWA, et al. Object-related activity revealed by functional magneticresonance imaging in human occipital cortex. Proc Natl Acad SciUSA 1995; 29: 8135–9.


Mangun GR. Neural mechanisms of visual selective attention.[Review]. Psychophysiology 1995; 32: 4–18.

Mangun GR, Hillyard SA. The spatial allocation of visual attentionindexed by event-related brain potentials. Hum Factors 1987; 29:195–211.

Mangun GR, Hillyard SA. Allocation of visual attention to spatiallocations: tradeoff functions for event-related brain potentials anddetection performance. Percept Psychophys 1990; 47: 532–50.

Mangun GR, Hillyard SA. Modulations of sensory-evoked brainpotentials indicate changes in perceptual processing during visual–spatial priming. J Exp Psychol Hum Percept Perform 1991; 17:1057–74.

McCarthy G, Nobre AC. Modulation of semantic processing byspatial selective attention. Electroencephalogr Clin Neurophysiol1993; 88: 210–29.

McCarthy G, Wood CC, Allison T. Cortical somatosensory evokedpotentials. I. Recordings in the monkey Macaca fascicularis. JNeurophysiol 1991; 66: 53–63.

McCarthy G, Spicer M, Adrignolo A, Luby M, Gore J, Allison T.Brain activations associated with visual motion studied by functionalmagnetic resonance imaging in humans. Hum Brain Mapp 1995a;2: 234–43.

McCarthy G, Nobre AC, Bentin S, Spencer DD. Language-relatedfield potentials in the anterior–medial temporal lobe: I. Intracranialdistribution and neural generators. J Neurosci 1995b; 15: 1080–9.

Mesulam M-M. Large scale neurocognitive networks and distributedprocessing for attention, language and memory. [Review]. AnnNeurol 1990; 28: 597–613.

Mesulam M-M. Higher visual functions of the cerebral cortex andtheir disruption in clinical practice. In: Albert DM, Jacobiec FA,editors. The principles and practice of ophthalmology: the Harvardsystem. Philadelphia: WB Saunders, 1994: p. 2640–53.

Moran J, Desimone R. Selective attention gates visual processingin the extrastriate cortex. Science 1985; 229: 782–4.

Nobre AC. Linguistic processing in the human brain:neurophysiological studies of neural organization and selectiveattention. [PhD thesis]. New Haven: Department of Psychology,Yale University; 1993.

Nobre AC, McCarthy G. Language-related ERPs: scalp distributionsand modulation by word type and semantic priming. J Cogn Neurosci1994; 6: 233–55.

Nobre AC, McCarthy G. Language-related field potentials in theanterior–medial temporal lobe: II. Effects of modulation by wordtype and semantic priming. J Neurosci 1995; 15: 1090–8.

Nobre AC, Allison T, McCarthy G. Word recognition in the humaninferior temporal lobe. Nature 1994; 372: 260–3.

Nobre AC, Mesulam MM, Sebestyen GN, Frackowiak RSJ, FrithCD. Functional localisation of the neural network for visual spatial

attention by positron-emission tomography. Neuroimage 1996; 3 (3Pt 2): S192.

Nobre AC, Sebestyen GN, Gitelman DR, Mesulam M-M,Frackowiak RS, Frith CD. Functional localization of the systemvisuospatial attention using positron-emission tomography. Brain1997; 120: 515–33.

Posner MI. Orienting of attention. QJ Exp Psychol 1980; 32: 3–25.

Posner MI, Raichle ME. Images of mind. New York: ScientificAmerican Library; 1994.

Puce A, Allison T, Gore JC, McCarthy G. Face-sensitive regions inhuman extrastriate cortex studied by functional MRI. J Neurophysiol1995; 74: 1192–9.

Puce A, Allison T, Asgari M, Gore JC, McCarthy G. Differentialsensitivity of human visual cortex to faces, letter strings, andtextures: a functional magnetic resonance imaging study. J Neurosci1996; 16: 5205–15.

Sass KJ, Lencz T, Westerveld M, Novelly RA, Spencer DD, KimJH. The neural substrate of memory impairment demonstrated bythe intracarotid amobarbital procedure. Arch Neurol 1991; 48:48–52.

Schlag J. Generation of brain evoked potentials. In: Thompson RF,Patterson MM, editors. Bioelectric recording techniques, Part A:Cellular processes and brain potentials. New York: Academic Press;1973: p. 273–316.

Talairach J, Tournoux P. Co-planar stereotaxic atlas of the humanbrain. Stuttgart: Thieme; 1988.

Tootell RB, Reppas JB, Dale AM, Look RB, Sereno MI, MalachR, et al. Visual motion after effect in human cortical area MTrevealed by functional magnetic resonance imaging [see comments].Nature 1995; 375: 139–41. Comment in: Nature 1995; 375: 106–7.

Treue S, Maunsell JH. Attentional modulation of visual motionprocessing in cortical areas MT and MST. Nature 1996; 382: 539–41.

Ungerleider LG. Functional brain imaging studies of corticalmechanisms for memory. [Review]. Science 1995; 270: 769–75.

Ungerleider LG, Mishkin M. Two cortical visual systems. In: IngleDJ, Goodale MA, Mansfield RJW, editors. Analysis of visualbehavior. Cambridge (MA): MIT Press; 1982: p. 549–86.

VanVoorhis S, Hillyard SA. Visual evoked potentials and selectiveattention to points in space. Percept Psychophys 1977; 22: 54–62.

Watson JDG, Myers R, Frackowiak RS, Hajnal JV, Woods RP,Mazziotta JC, et al. Area V5 of the human brain: evidence from acombined study using positron emission tomography and magneticresonance imaging. Cereb Cortex 1993; 3: 79–94.

Woldorff MG, Fox PT, Matzke M, Lancaster JL, Veeraswamy S,Zamarripa F, et al. Visual spatial attention: integration of PET andERP data. Neuroimage 1996; 3 (3 Pt 2): S242.

Received September 25, 1997. Revised January 15, 1998.Accepted February 16, 1998.

Modulation of human extrastriate visual processing by - Brain

Documents