Associative knowledge controls deployment of visual selective attention

182 nature neuroscience • volume 6 no 2 • february 2003

articles

Selection of incoming sensory information can be guided bymany criteria. For example, an observer can direct attentionto a restricted portion of the visual field1 or to stimuli pos-sessing a specific property such as a specific color2.

A variety of research methodologies have led to numeroustheories of visual selective attention2–7. One such theory, the‘biased competition’ model7–12, proposes that objects in thescene activate corresponding representations in the brain,which then compete for perceptual awareness and motorbehavior. Competition among different representations can bebiased by both bottom-up and top-down factors7,11–13. Whencompetition is under top-down control, selection of a relevantobject is the consequence of an internal template priming itsrepresentation at the disadvantage of the representation ofcompeting objects. It has been suggested that top-down con-trol signals arise from working memory7–12. In its present for-mulation, however, the theory does not take into accountanother potentially important determinant of selective atten-tion: the existence of associative links among central repre-sentations of visual objects.

It has long been recognized that long-term memories,including those of visual objects, are associated with one anoth-er through links of variable strength14–18. The strength of theselinks may depend on many factors, such as pictorial similarityor a relationship at the semantic or episodic level19. We spec-ulate that these associative links are important in visual selec-tive attention. For example, when asked to look for a glass ina cluttered scene, an individual might select objects related tothe glass, such as a bottle, in addition to or instead of the glass

Associative knowledge controlsdeployment of visual selectiveattention

Elisabeth Moores1,2, Liana Laiti1 and Leonardo Chelazzi1

1 Department of Neurological and Vision Sciences, Section of Physiology, University of Verona, Strada Le Grazie 8, 37134 Verona, Italy2 Neurosciences Research Institute, Aston University, Birmingham B4 7ET, UK

Correspondence should be addressed to L.C. ([email protected])

Published online 6 January 2003; doi:10.1038/nn996

According to some models of visual selective attention, objects in a scene activate correspondingneural representations, which compete for perceptual awareness and motor behavior. During avisual search for a target object, top-down control exerted by working memory representationsof the target’s defining properties resolves competition in favor of the target. These models,however, ignore the existence of associative links among object representations. Here we showthat such associations can strongly influence deployment of attention in humans. In the contextof visual search, objects associated with the target were both recalled more often andrecognized more accurately than unrelated distractors. Notably, both target and associatedobjects competitively weakened recognition of unrelated distractors and slowed responses to aluminance probe. Moreover, in a speeded search protocol, associated objects rendered searchboth slower and less accurate. Finally, the first saccades after onset of the stimulus array weremore often directed toward associated than control items.

itself. Indeed, it has been shown that increasing the visual sim-ilarity between target and distractors increases attention to thedistractors20 and that items capture attention in proportion totheir similarity to the contents of working memory20–23.

Here we extend this notion to include associative linksbetween objects and argue that objects associated with the tar-get in long-term memory tend to attract attention in a com-parable way to objects with visual similarities to the target. Weused an adapted visual search paradigm to investigate thishypothesis and found that during search performance, associ-ations between objects affected recall and recognition of dis-tractor items, competition between items, accuracy of reportof a target’s presence or absence and eye movements.

RESULTSIn experiment 1, participants performed a rapid-presentation

visual search task on arrays containing differing combi-nations of target items, target-related items and control items,followed by free recall (by verbal report) of the array items(Figs. 1a and 2).

For the search task, accuracy and reaction times (RTs) inthe presence versus absence of an associated item did not differsignificantly. Overall accuracy was high: 95% in target-absentconditions (94% with and 96% without the associated item)and 76% in target-present conditions (75% with and 77%without the associated item). A two-factor analysis of variance(ANOVA) on accuracy scores showed no effect of the relatedobject overall (F1,14 = 0.28) and no significant interactionbetween target and related object presence (F1,14 = 1.50),

©20

03 N

atu

re P

ub

lish

ing

Gro

up

h

ttp

://w

ww

.nat

ure

.co

m/n

atu

ren

euro

scie

nce

articles

nature neuroscience • volume 6 no 2 • february 2003 183

but a highly significant effect of target presence (F1,14 = 20.41,P < 0.001). Mean RTs were very long (2,124 ms versus 2,101 msfor target-absent trials in which the associated item was pre-sent or absent, respectively, and 2,098 ms versus 1,981 ms forequivalent target-present conditions), suggesting that despiteinstructions to respond as quickly as possible in the search task,participants were attempting to memorize objects beforeresponding. A two-factor ANOVA showed no significant maineffects or interactions (all Fs < 1).

For this experiment, our main interest was in recall per-formance. The probability of correctly recalling a given objectwas compared when it was presented as associated versus con-trol (Fig. 1b). For both target-absent and target-present con-ditions, ANOVA analyses showed that associated objects werereported more frequently than control objects (52% versus38%, F1,14 = 62.50, P < 0.001; and 52% versus 36%, F1,14 = 23.43,P < 0.001, respectively). A two-factor ANOVA investigating the effect of target presence on recall of associated versus con-trol stimuli showed a main effect of object type (F1,14 = 62.82,

P < 0.001), no main effect of target presence (F1,14 = 0.20) andno interaction between the two variables (F1,14 = 0.21).

Participants virtually never reported items associated withthe target but not present in the search array (0.97% of thetotal reports compared to 0.48% for control items; P > 0.05). Ad′ analysis showed that sensitivity for reporting related itemswas significantly greater than for reporting control items (3.39versus 2.40, P < 0.01).

These results suggest that associated objects tend to grabattention, allowing easier access to perceptual awareness andworking memory24,25. It is possible, however, that associatedobjects are simply remembered more easily, or conversely, thatnon-associated objects might be forgotten more rapidly26.

Therefore, to minimize the memory requirements of thetask, we carried out a second experiment using a recency judg-ment task (experiment 2). Participants performed a search taskfollowed by a recency judgment—they indicated which of twoobjects they had just seen in the previous search array (Fig. 3a).The choices were ‘old’ or ‘new’.

For the target-absent conditions, target-related objectsafforded greater accuracy than control objects in being select-ed as ‘old’ (79% versus 63%; F1,24 = 15.31, P < 0.001; Fig. 3b),suggesting that associated objects may grab attention.

For the target-present conditions, there was no significantdifference between associated and control objects (69% versus71%; F1,24 = 0.59). This lack of an effect in the target-presentcondition is presumably because targets themselves were select-ed by attention, suggesting competition between target and asso-ciated objects. This contrasts with findings from experiment 1,in which observers may have attempted a more thorough sam-pling of the array, given the longer average array duration andthe requirement of recalling as many objects as possible. A two-factor ANOVA investigating the effect of target presence onrecognition accuracy of associated versus control items showeda main effect of object type (F1,24 = 7.49, P < 0.01), no maineffect of target presence (F1,24 = 0.10) and a significant interac-tion between the two variables (F1,24 = 9.16, P < 0.01).

Given the different performance for the associated versuscontrol objects in the target-absent conditions of experiment 2,we used two of the remaining target-absent conditions to testwhether there was a bias toward choosing an associated objectas ‘old’ even when it had not been presented in the search array.The comparison showed no significant effect.

We also analyzed conditions in which the recency judge-ment was between two distractors. First, accuracy in distrac-tor recognition was higher (68%) when the search arraycontained only distractors relative to when it also containedthe target (61%, F1,24 = 5.23, P < 0.05), suggesting that the tar-get took away resources from distractor processing. A similar,though only marginally significant, effect was obtained by com-paring target-absent conditions with or without an associatedobject (63% versus 68%; F1,24 = 3.24, P = 0.08). Such a decre-ment in distractor recognition due to the associated stimuluswas not observed in target-present conditions (64% versus61%, respectively, with and without the associated object; F1,24= 1.53), again probably reflecting competition between the tar-get and the associated object. Finally, target presence did notyield a further decrement in distractor recognition compared towhen related and control objects were present alone (63% ver-sus 64%, as above). In summary, it appeared that the presenceof either target or related items resulted in decreased distractorrecognition. However, the presence of both did not result in asummed cost.

Fig. 1. Verbal report experiment. (a) Following a visual search task inwhich they were instructed to detect the presence of a pre-specifiedtarget, participants were asked to recall as many objects as they couldfrom the search array. A target was present on 50% of the trials. Target,target-related and control items (Fig. 2) were each presented indepen-dently in 50% of the trials, giving rise to 8 different conditions. Twelvetarget objects were used once in each of the 8 conditions, for a total of96 trials. These 8 conditions allowed us to compare recall probabilitiesof associated versus control items, both in the presence and in theabsence of the target. A staircase calibration procedure increased ordecreased presentation time, respectively, if less or more than twoobjects were reported correctly. Across participants, mean duration ofthe search array was 126 ms, and mean number of items reported was1.39. (b) The graph shows the difference in frequency of report in target-present and target-absent conditions when each potentially associatedobject was presented as an associated (rather than control) object. Thepair number indication refers to the associated object because when theassociated object acted as a control, its target object would vary. The y-axis refers to the percentage increase in report when the given objectwas associated with the target.

a

b

©20

03 N

atu

re P

ub

lish

ing

Gro

up

h

ttp

://w

ww

.nat

ure

.co

m/n

atu

ren

euro

scie

nce

184 nature neuroscience • volume 6 no 2 • february 2003

Finally, we analyzed the effect of the presence or absence ofassociated items on the accuracy and speed of the search task.For target-absent conditions, there was a significant differencein accuracy between associated and distractor conditions (86%versus 89%; F1,24 = 7.87, P = 0.01). Associated objects some-times led to incorrect decisions that the target was present, pre-sumably because they were more difficult to reject as candidatetargets. For the target-present conditions of the task, accura-cy in the two conditions was statistically indistinguishable (72%versus 73%, F1,24 = 0.06). A two-factor ANOVA showed a high-ly significant effect of target presence (F1,24 = 59.32, P < 0.001),no effect of related object presence (F1,24 = 1.11), but a mar-ginally significant interaction between the two (F1,24 = 3.72, P = 0.06). No significant effects were found for correct RTs.

In summary, experiment 2 showed that: (i) objects associ-ated with a target are recognized better than control objects,but only in the absence of the target, (ii) distractor objects areless easily recognized in the presence of the associated objectand particularly in the presence of the target itself, attestingcompetition for processing and memory storage, and (iii) intarget-absent conditions, the presence of an associated objecttends to impair search performance, perhaps because associ-ated objects are treated as potential targets. Together, thesefindings strongly indicate that more attention was given toassociated objects in the search array.

The above results led to two subsequent experiments. First,the improved recognition of the related objects raised the pos-sibility that better processing of the related object entailed thefocusing of attention at its location. Alternatively, attentionalselection of the related object might take place at the level ofabstract and spatially-invariant object representations, with noinvolvement of spatial attention.

Experiment 3 tested these possibilities by means of a luminance-probe detection protocol. The task was similar to those alreadydescribed, except that after the array had been on for 50 ms, asmall probe was presented in the center of one of the fourobjects. The participants’ first task was to respond to the probeas fast as possible, then they indicated whether the target waspresent or absent.

If spatial attention were mediating selection of associatedobjects (and targets), then one might predict faster responses toprobes appearing on either target or associated objects in com-parison to control and distractor objects. Contrary to this pre-diction, however, mean probe RTs were not significantly fasteron related versus control items in either target-absent (826 msversus 850 ms; F1,17=0.916) or target-present conditions (871 msversus 877 ms; F1,17 = 0.03). Similarly, probe accuracy was not significantly higher when it was on related versus control

items in either target-absent (85% versus 80%; F1,17 = 2.48) ortarget-present conditions (84% versus 85%; F1,17 = 0.08). How-ever, this lack of a local benefit was not limited to the relateditem, but extended to the target object. In the target-presentconditions, mean RTs were slightly longer for probes appearingon the target object itself than on distractors, both in the pres-ence (878 ms versus 847 ms, respectively; F1,17 = 2.34, n.s.) andabsence (847 ms versus 827 ms, respectively; F1,17 = 1.81, n.s.)of the related item. Mean accuracy was indistinguishablebetween probes appearing on target and distractor items, eitherin the presence (84% versus 85%, F1,17 = 0.04) or absence (89%versus 86%, F1,17 = 1.26) of the related item. These results,which we have confirmed with other probing procedures, sug-gest that our search paradigm does not engage focal attentionat the location of either the target or the related item.

Instead, further analyses showed a non-spatially specific andhighly reliable general cost in performing the probe task wheneither target or related objects were present. There was a signif-icant RT difference between the distractors-only condition andthe target-only condition (793 ms versus 832 ms, F1,17 = 5.66,P < 0.05). This presumably indicates that the target capturesattentional resources, thus leaving fewer available for process-ing the probe—an effect akin to that exerted by the target on

articles

Fig. 2. One exemplar of all object pairs used in the five experiments.Stimuli consisted of digital photographs of everyday objects, animals,vegetables, etc. Pilot testing revealed that all stimuli could be resolved atthe presentation eccentricity used. Stimuli were either 100 × 67 pixelsor 67 × 100 pixels in size (approximately 3 × 2 cm). Four versions ofeach object were used to preclude the possibility of participants search-ing for objects on the basis of local features. Furthermore, to avoideffects of visual similarity, we chose photographs of associated objectsbearing as little resemblance as possible to any potential exemplar of thepaired object. For each target object, a pair associate was chosen, and avariety of associations were represented among the object pairs. Each ofthe 12 related items also served as a control in trials in which partici-pants searched for a different target. This negated the possibility thatrelated objects happened to be more visible than other distractors.

©20

03 N

atu

re P

ub

lish

ing

Gro

up

h

ttp

://w

ww

.nat

ure

.co

m/n

atu

ren

euro

scie

nce

distractor recognition accuracy in experiment 2. Furthermore, intarget-absent conditions, probe responses were significantly slow-er in the presence of the related object (827 ms versus 793 ms,F1,17 = 8.51, P < 0.01). In target-present conditions, the pres-ence of the related object in addition to the target resulted infurther slowing (872 ms versus 832 ms, F1,17 = 8.89, P < 0.01). Atwo-factor ANOVA investigating probe RTs showed main effectsof both target presence or absence (F1,17 = 11.69, P < 0.01) andassociated object presence or absence (F1,17 = 7.49, P < 0.01) andno interaction between the two (F1,17 = 0.08).

It could be argued that such general costs in probe respons-es engendered by both the target and the related object are dueto decision mechanisms rather than selective attention. Forexample, if it takes longer on target-absent trials to reject anassociated item as a potential target because of decision-leveleffects, then one might expect delayed responses to the probe.We find this unlikely for two reasons. First, a similar cost inprobe responses was produced both by the target and by therelated object. As it seems unreasonable to invoke two sepa-rate mechanisms to explain each effect, and because there areno grounds to suggest that deciding that a target is presentinvolves a more costly process than deciding that a target isabsent, it becomes far less attractive to account for the costproduced by the related object in terms of decision-level effects(alone). Second, because participants were instructed first andforemost to respond as quickly as possible to the probe, with-out regard to the search task (which was non-speeded), it isnot obvious that probe RTs should be affected by decisionprocesses concerning target presence/absence. Instead, we sug-gest that this general cost arises because the target and any asso-ciated object capture attentional resources and act as strongcompetitors, thus reducing available resources to process theluminance probe.

This idea might seem at odds with the lack of spatially spe-cific processing benefits at the location of either the target or

the related object. These two observations can be reconciled,however, according to the following logic. First, the local ben-efit may have been overshadowed by the general competitiveeffect exerted by the target and related objects against theprobe. Note that, although no spatially specific effect was foundfor the related items, no effect was found for the target itemseither, and probes on targets actually yielded the very slowestresponses despite the fact that target objects must have beenselected to guide the behavioral response. As suggested previ-ously, a related possibility is that target and related objects didattract attentional resources, but this did not entail focusingspatial attention at their location. It should be emphasized thateffects due to semantic associations are likely occurring at thelevel of spatially invariant, prototypical representations ofobjects. It is nonetheless conceivable that, whereas selection ofrelated objects (and of the target) initially occurs at the levelof spatially-invariant, abstract object representations, it mightlater affect their spatially-specific, low-level descriptions too.It is also conceivable that early focusing of spatial attention atthe location of target or related objects would take place underdifferent testing conditions, for example in a search paradigmrequiring target localization.

We also analyzed performance in the search task. Two sub-jects were eliminated from these analyses because they had mis-understood the instructions for this task. For target-absentconditions, there was a trend toward associated conditions beingless accurate than distractors-only conditions (75% versus 79%;F1,15 = 3.51, P = 0.08). Consistent with experiment 2, in target-present conditions, the presence of the related object had noeffect (67% versus 66%; F1,15 = 0.60). A two-factor ANOVAfound a significant effect of target presence (F1,15 = 9.85, P < 0.01),no main effect of related object presence (F1,15 = 0.71) and anon-significant interaction between the two (F1,15 = 2.42).

In experiment 4, we sought to replicate the effect of the relat-ed object on the search performance itself, with no other taskrequirement, while controlling for the frequency and pairingof the target and associated items (Fig. 4a). It might be arguedthat the repeated presentation of two stimuli together, or thepresentation of one particular stimulus when the other is theobject of search, renders the paired object more visible becauseof expectation and imagery effects21–23, without the implica-tion of long-term associations. Thus, we included two condi-tions in which an unrelated distractor appeared exactly the same

articles

nature neuroscience • volume 6 no 2 • february 2003 185

Fig. 3. Forced-choice recency judgement experiment. (a) Following avisual search task, participants were shown two objects and asked todistinguish the previously seen object (‘old’) from the new one (‘new’).In the initial search task, there were four conditions, obtained bycrossing target presence/absence with related object presence/absence. Whenever an associated object was present, a control objectwas also present. For the recency judgement task, each object couldbe either an associated, control or distractor object in the trial(whether presented or not in the search array), bringing the totalnumber of conditions to 12. The participants’ second task was topress one of two keys, depending on whether the right or left picturewas recognized from the previous array. Frequency of occurrence ofthe different conditions was such that paying attention to any of thefour objects in the array was equally helpful for the second task.There were 192 trials. (b) The graph shows the difference in accuracyin target-present and target-absent conditions when each object waspresented as an associated (rather than control) object. The pair num-ber indication refers to the associated object. Positive numbers alongthe y-axis refer to improved performance when the object was pre-sented as associated.

a

b

©20

03 N

atu

re P

ub

lish

ing

Gro

up

h

ttp

://w

ww

.nat

ure

.co

m/n

atu

ren

euro

scie

nce

186 nature neuroscience • volume 6 no 2 • february 2003

number of times and in exactly the same context as the associ-ated item (target-present and target-absent, control conditions).

ANOVA analyses for the target-absent condition showedthat differences between associated and distractor conditionswere significant for both accuracy and correct RTs (F1,15 = 8.61,P < 0.01 and F1,15 = 13.21, P < 0.002, respectively; Fig. 4b andc). These results, which we have now replicated numeroustimes, indicate that the presence of an associated object in target-absent conditions renders performance slower and less accu-rate than when only unrelated distractors are present. Theassociated object made it more likely that participants wouldincorrectly claim to have seen the target (82% versus 90% accu-racy) even if it was not there, while also lengthening the timerequired to make a correct rejection (867 ms versus 801 ms).Importantly, we did not find a similar difference between the con-trol and distractor-absent conditions (F1,15 = 2.11 and F1,15 = 0.39for accuracy and RTs, respectively). Finally, the differencebetween control-absent and associated conditions was highlysignificant (F1,15 = 25.26, P < 0.001 and F1,15 = 9.07, P < 0.01,for accuracy and RTs, respectively), showing that the more fre-quent appearance of an associated object when searching fora particular target cannot account for the difference betweendistractor and associated conditions (control-absent condi-tions had 92% accuracy and 813 ms mean RT).

When the target was present, there was no significant dif-ference in terms of accuracy or RT between target-present andtarget-present + associated conditions (80% versus 81%, F1,15= 0.41; 641 ms versus 655 ms, F1,15 = 0.71, respectively). A two-factor ANOVA on the accuracy data showed significant maineffects of target presence (F1,15 = 7.39, P < 0.05) and presenceof the related object (F1,15 = 5.05, P < 0.05), and a significantinteraction between the two (F1,15 = 6.25, P < 0.05).

Importantly, these results do not seem to be due to partic-ipants’ confusion over target identity. Results suggested verylittle confusion, and confusion between a target and its asso-ciated object never occurred.

Thus, in the absence of the target, it appears that associateddistractors were treated, at least transiently, as potential targets,presumably because they tended to be selected by attention andattained a higher degree of activation than other distractors.

In a final experiment, we tested whether objects associatedwith the target are more potent attractors of gaze as comparedto unrelated distractors by measuring eye movements duringsearch performance.

No clear-cut picture emerged from looking at saccadic laten-cies from the onset of the search array for the different condi-tions and object types. Overall, mean latency of first saccadesacross participants was 306 ms. In contrast, the landing of sac-cades was significantly affected by both condition and objecttype. In the distractors-only condition, 69.9% of first saccadeswere directed to distractors, whereas 30.1% went to parts of thescreen not occupied by any object. Thus, any single distractorhad a mean probability of 17.5% to be targeted by the first sac-cade after array onset. In the target-only condition, most initialsaccades (47.2%) were directed to the target, with a significant-ly lower proportion of saccades directed to any of the three dis-tractors (9.3% each, F1,19 = 87.44, P < 0.001). In the associatedcondition, the associated object, in the absence of the target,behaved somewhat like the target itself, receiving 23.1% of initialsaccades, whereas 16.7% of initial saccades went to controlobjects and 17.0% went to each distractor object. Importantly,the difference between associated and control objects was sig-nificant (F1,19 = 9.18, P < 0.01). The presentation of both asso-

ciated and target objects together dramatically reduced the num-ber of saccades toward the associated object in comparison withthe associated condition (12.2%, F1,19 = 38.10, P < 0.001), but italso tended to reduce the number of saccades to the target relativeto the target-only condition (40.9%, F1,19 = 4.09, P = 0.057). Sim-ilar to the target-only condition, only 8.3% of initial saccadeswere directed to individual distractors in the target + associatedcondition, and only 9.4% went to the control object. A two-factorANOVA investigating the effect of target presence on the per-centage of saccades made to related versus control objects showedmain effects of both target presence (F1,19 = 38.55, P < 0.001)and object type (F1,19 = 7.29, P < 0.05) but no significant inter-action effect (F1,19 = 2.04).

In terms of fixation durations, no significant differenceswere found between related and control objects (165 versus154 ms, F1,18 = 1.54, n.s.), whereas targets were fixated formuch longer than other objects (258 ms).

articles

Fig. 4. Rapid-presentation visual search experiment. (a) In experi-ment 4, the four conditions obtained by crossing targetpresence/absence with related item presence/absence were aug-mented by two more control conditions (as detailed in the results sec-tion), for a total of 288 trials. To render the search task highlydemanding, exposure duration was calibrated for each participantthrough a staircase procedure aimed at maintaining correct perfor-mance across conditions between 70–90%. This resulted in a meanpresentation time across subjects of only 73 ms (range 47–97 ms).Feedback on accuracy was given in an attempt to counteract biastoward responding that the target was absent. At the end of each trialblock, participants were asked to write down the target they had justbeen looking for, as a check for confusion. Graphs show (b) accuracyand (c) RT differences between conditions when an associated objectto the target was present versus when only random distractors werepresent under both target-present and target-absent conditions. In(b), positive numbers along the y-axis indicate better performancewhen the associated object was absent. In (c), positive numbers indi-cate slower performance when the associated object was present.

a

b

c

©20

03 N

atu

re P

ub

lish

ing

Gro

up

h

ttp

://w

ww

.nat

ure

.co

m/n

atu

ren

euro

scie

nce

Analyses of the key-press responses made by the partici-pants replicated the results of experiment 4. Accuracy was sig-nificantly lower in target-absent conditions whenever anassociated object was presented (96% versus 98%; F1,19 = 5.37,P < 0.05), and average reaction times were slower (790 ms ver-sus 758 ms; F1,19 = 12.10, P < 0.005).

Thus, in the absence of the target, associated distractorswere treated, at least transiently, as potential targets. Initial sac-cades had a greater probability of landing on associated thancontrol items. This may be akin to the higher frequency of sac-cades made to a distractor that is visually similar to the targetcompared to a dissimilar distractor27. Furthermore, when tar-get and associated items were presented together, both sufferedsome competitive cost, although this cost was larger for asso-ciated than target objects.

These results have a clear element of similarity to those ofexperiment 3. In experiment 3, both the target and the relat-ed item slowed responses to the probe, suggesting that theywere acting as strong competitors against the probe stimulus.Similarly, in experiment 5, both the target and the related itemacted as strong competitors, capturing gaze at the disadvan-tage of control and distractor objects.

Two considerations might help explain why in experiment5, but not in experiment 3, we found evidence of a local captureof attention at the position of the related item and of the target,as reflected by the pattern of eye-gaze responses. First, in exper-iment 5, unlike all others, the search array remained visible fora full second, and this might have allowed enough time for thetransition from a first stage of selection at the abstract level ofobject representation to a second stage of selection involvingspatially specific descriptions. It should be noted that eye move-ments to the related and target objects had an average latencyof >300 ms, again raising the possibility that a spatial code forthe selected object might not have been available immediatelyafter array onset, but only a few hundred milliseconds later.

DISCUSSIONOur results support the idea that target-related objects affectdeployment of attention during visual search and extend theprevious finding that contents of working memory can influ-ence deployment of attention21–22. We also extend the findingthat distractors with visual similarities to the target attractattention20,27 to include distractors with semantic and asso-ciative links to the target. To some extent, results from the freerecall and recency judgment experiments might be explained interms of better memory of associated objects rather than interms of privileged selection of their central representations.However, this argument applies more to the free recall exper-iment than to the recency judgment task, as the latter did notdepend on explicit memory of the objects and could have beenperformed on the basis of low-level pattern analysis. In addi-tion, responses to the search array in the recency judgmentexperiment, eye movement measurements and results fromthe calibrated-exposure visual search and probe detection taskstogether suggest that objects associated with the target aremore likely to be selected by attention, often at the disadvan-tage of unrelated distractors. Associated objects tended to grabattention and gaze and were apparently treated as potentialtargets by the decision mechanisms, sometimes leading toincorrect responses. We propose that the degree of activationof their central representation is intermediate between that ofa true target and that of an unrelated distractor. In general,results from these experiments are consistent with the claim

that semantic information is accessed very early in the courseof processing28.

In several of our experiments, the associated object engen-dered competitive costs for unrelated objects, similar to thecosts engendered by the target itself. This was clearly the case inthe recency judgment, the probe detection and the eye move-ment tasks. This is crucial evidence attesting the attentionalnature of the effects exerted by semantic associations. Althoughbenefits for the associated object could be generally account-ed for in terms of conceptual priming alone (see below), con-current costs for unrelated objects must involve an attentionalprocess, namely competition for a common pool of resources.

To our knowledge, these findings are the first to illustratethat associations in long-term memory can have a bearing onhow we allocate attention. Previous research has shown thatcontextual information acquired through implicit learning canimprove performance when searching stable arrays for non-sense objects or letters29,30. These effects have been attributedto the ability of the visual system to learn quickly about statis-tical regularities of the visual environment, such as consistentlayout information and object covariations. In contrast, ourexperiments assessed the influence of conceptual associationsdeposited in long-term memory and acquired over the years.

It has long been known that real-world scene contexts canfacilitate object recognition31–36 (but see ref. 37). Here we usedstimulus objects that stood alone against a white background.Thus instead of indicating improved processing of objectssemantically consistent with their scene context, our resultsshow a competitive advantage of related objects against unre-lated distractors in capturing selective attention. In general,however, our results are concordant with the claim that objectmemory and semantic knowledge can influence percep-tion38–43, thus arguing against a strictly serial processingscheme44 and a strictly modular view of brain function45.

The present results prompt an extension of models of atten-tion such as the biased competition model7–12, which suggestthat top-down control signals arising from working memorycan bias the interactions among the representations of stimulicompeting for attention. Top-down control signals not onlyraise the activity of the target’s representation, but this activa-tion also spreads to associated representations, making themmore likely to be selected. In addition, increased activation ofthe associated object may in turn spread to the target’s repre-sentation, even when this is not driven by the retinal image,thus making observers more likely to detect the target objectin its absence. The latter effect may be akin to the phenome-non of false memories—studying lists of associated wordsmakes subjects more likely to falsely recall or recognize a relat-ed word that was not actually on the lists46.

It should be noted that the findings presented here havestrong links to semantic priming26,47,48. In addition to theaccepted properties and mechanisms of this phenomenon,however, we also propose that conceptual priming may act tobias attention toward primed objects. In the presence of com-peting objects, this bias makes primed objects more likely toaccess perceptual awareness and to control decision mecha-nisms. Whereas in our first two experiments, priming alonemight explain improved recall and recognition for associateditems, the data from experiments 3, 4 and 5 are more difficultto explain by semantic priming alone. If the related item wereprocessed more quickly and efficiently, then, unless anotherattention-like mechanism sensitive to the strength and/or speedof activation is assumed, accuracy (as in experiment 4) would

articles

nature neuroscience • volume 6 no 2 • february 2003 187

©20

03 N

atu

re P

ub

lish

ing

Gro

up

h

ttp

://w

ww

.nat

ure

.co

m/n

atu

ren

euro

scie

nce

188 nature neuroscience • volume 6 no 2 • february 2003

surely be expected to be higher and not lower in the presence ofassociated objects. In addition, if associated objects wereprocessed more easily, then they should require fewer andshorter fixations to decide that they were not the target.

In conclusion, we propose that the activation of object rep-resentations in working memory primes associated represen-tations, which either makes them receive more attention in atop-down manner or affords them more rapid and efficientprocessing, which subsequently attracts attention.

METHODSFor a fully detailed description of the methods, see SupplementaryMethods online.

Participants. For experiments 1–5, there were 15, 25, 18, 16 and 20participants (6, 9, 3, 8 and 12 males), respectively. Most were studentsat either Verona or Aston University and were naive as to the purpos-es of the experiments. They all gave informed written consent to par-ticipate. Ages ranged between 18–40.

Software and set-up. Experiments were programmed using E-PrimeBeta software running on a PC. Participants sat at a viewing distance of~57 cm from the computer monitor in a dimly lit room. In experi-ments 1, 4 and 5, the four pictures were presented in the four quad-rants of the visual field (Figs. 1a and 4a), with the center of eachpicture approximately 6 cm from fixation. In experiments 2 and 3, thefour pictures were presented approximately 6 cm above, below, to theleft and to the right of fixation (Fig. 3a). Responses, except verbalreports in experiment 1, were made with the computer keyboard.

Design. All experiments included a visual search task in which a targetword was presented, followed by a fixation point and a stimulus array.A target was presented on 50% of the trials, and participants wereasked to detect its presence. For details of experiments 1, 2 and 4, seeFigs. 1a, 3a and 4a. Target and associated objects appeared in the stim-ulus array independently (50% probability each), creating the mainfour conditions used in all experiments. In experiment 1, a controlobject was also presented independently, creating four additional con-ditions. In experiments 2, 3 and 5, the control object was always pre-sented together with the associated object.

In experiment 3, the search array and conditions were the same as inexperiment 2, except that participants’ first task was to respond to asmall probe appearing 50 ms after array onset in the center of one ofthe pictures (and remaining a further 50 ms until array offset, followedby a mask). The probe was a 1° × 1° white square, with a 0.7° × 0.7°red square inside. In 240 trials, participants responded as quickly aspossible whether the probe was on the left or right-hand side of thescreen before responding whether the target was present or absent.Timings were as for experiment 2, except that the target word was pre-sented for 1,000 ms.

In Experiment 5, the four conditions of the task (targetpresent/absent, related item present/absent) were administered over192 trials. The target word was presented for 1,000 ms, fixation crossfor 800 ms and stimulus array for 1,000 ms or until response (whichev-er was sooner). The only requirement was to detect the target, with noexplicit instructions regarding eye movements (except to fixate cen-trally at the start of the trial). Eye movements were recorded (250 Hz,0.1° spatial resolution) with the aid of a video-based system (Eyelink,SensoMotoric Instruments, GmbH), with only data from the right eyeanalyzed. Trials where fixation was not within a virtual box of 2° aroundthe fixation point when the array appeared and up until 100 ms afterits onset were excluded from further analysis (19.3%). Latency andlanding position of saccades were classified according to whether theeye position fell within one of four virtual boxes (approximately 6° ×6°) centered on the locations of the array items or anywhere else on thescreen. This information was then integrated with the informationregarding the actual objects on the screen in any given trial.

Note: Supplementary information is available on the Nature Neuroscience website.

AcknowledgmentsFinancial support for this project was provided by grants to L.C. from the

Human Frontier Science Program, the McDonnell-Pew Foundation, the

Italian Ministero dell’Università e della Ricerca Scientifica e Tecnologica

(MURST) and the Italian Consiglio Nazionale delle Ricerche (CNR). E.M.

was supported by a Marie Curie Fellowship of the European Community

programme “Improving Human Research Potential and the Socio-economic

Knowledge Base” under contract number HPMFCT-2000-00562. We thank

J. Duncan and M. Peterson for helpful comments on preliminary versions of

this paper and M. Veronese for preparing the figures.

Competing interests statementThe authors declare that they have no competing financial interests.

RECEIVED 29 OCTOBER; ACCEPTED 10 DECEMBER 2002

1. Posner, M.I. Orienting of attention. Q. J. Exp. Psychol. 32, 3–25 (1980).2. Bundesen, C. A theory of visual attention. Psychol. Rev. 97, 523–547

(1990).3. Treisman, A. & Gelade, G. A feature integration theory of attention.

Cognit. Psychol. 12, 97–136 (1980).4. Posner, M.I. & Deheane, S. Attentional networks. Trends Neurosci. 7, 75–79

(1994).5. Van der Heiden, A.H.C. Selective Attention in Vision (Routledge, London,

1992).6. LaBerge, D. Attentional Processing (Harvard Univ. Press, Cambridge,

1995).7. Desimone, R. & Duncan, J. Neural mechanisms of selective visual

attention. Annu. Rev. Neurosci. 18, 193–222 (1995).8. Chelazzi, L., Miller, E.K., Duncan, J. & Desimone, R. A neural basis for

visual search in inferior temporal cortex. Nature 363, 345–347 (1993).9. Luck, S.J., Chelazzi, L., Hillyard, S.A. & Desimone, R. Neural mechanisms

of visual selective attention in in areas V1, V2 and V4 of macaque visualcortex. J. Neurophysiol. 77, 24–42 (1997).

10. Chelazzi, L., Duncan, J., Miller, E.K. & Desimone, R. Responses of neuronsin inferior temporal cortex during memory-guided visual search. J. Neurophysiol. 80, 2918–2940 (1998).

11. Reynolds, J.H., Chelazzi, L. & Desimone, R. Competitive mechanismssubserve attention in macaque areas V2 and V4. J. Neurosci. 19, 1736–1753(1999).

12. Reynolds, J.H., Pasternak, T. & Desimone, R. Attention increasessensitivity of V4 neurons. Neuron 26, 703–714 (2000).

13. Yantis, S. Control of visual attention. in Attention (ed. Pashler, H.) 223–256(Univ. College London Press, London, 1998).

14. Hebb, D.O. The Organization of Behavior (Wiley, New York, 1949).15. Deese, J. The Structure of Associations in Language and Thought (Johns

Hopkins Press, Baltimore, 1965).16. Anderson, J.R. & Bower, G.H. Human Associative Memory (V.H. Winston,

Washington, DC, 1973).17. Collins, A.M. & Loftus, E.F. A spreading-activation theory of semantic

processing. Psychol. Rev. 82, 407–428 (1975).18. McClelland, J.L. Connectionists models of memory. in The Oxford

Handbook of Memory (eds. Tulving, E. & Craik, F.I.M.) 583–596 (OxfordUniv. Press, Oxford, UK, 2000).

19. Estes, W.K. Classification and Cognition (Oxford Univ. Press, New York,1994).

20. Duncan, J. & Humphreys, G. Visual search and stimulus similarity. Psychol.Rev. 96, 433–458 (1989).

21. Downing, P.E. Interactions between visual working memory and selectiveattention. Psychol. Sci. 11, 467–473 (2000).

22. Awh, E., Jonides, J. & Reuter-Lorenz, P. Rehearsal in spatial workingmemory. J. Exp. Psychol. Hum. Percept. Perform. 24, 780–790 (1998).

23. Ishai, A. & Sagi, D. Common mechanisms of visual imagery andperception. Science 268, 1712–1774 (1995).

24. Wolfe, J.M. Inattentional amnesia. in Fleeting Memories (ed. Coltheart, V.)(MIT Press, Cambridge, Massachusetts, 1999).

25. Sperling, G. The information available in brief visual presentations.Psychological Monographs 74, 1–29 (1960).

26. Tulving, E. & Schacter, D.L. Priming and human memory systems. Science247, 301–306 (1990).

27. Findlay, J.M. Saccade target selection during visual search. Vision Res. 37,617–631 (1997).

28. Thorpe, S., Fize, D. & Marlot, C. Speed of processing in the human visualsystem. Nature 381, 520–522 (1996).

29. Chun, M.M. Contextual cueing of visual attention. Trends Cogn. Sci. 4,170–178 (2000).

30. Chun, M.M. & Jiang, Y. Top-down attentional guidance based on implicitlearning of visual covariation. Psychol. Sci. 10, 360–365 (1999).

articles©

2003

Nat

ure

Pu

blis

hin

g G

rou

p

htt

p:/

/ww

w.n

atu

re.c

om

/nat

ure

neu

rosc

ien

ce

articles

nature neuroscience • volume 6 no 2 • february 2003 189

31. Henderson, J.M. & Hollingworth, A. High-level scene perception. Annu. Rev.Psychol. 50, 243–271 (1999).

32. Biederman, I., Glass, A.L. & Stacy, E.W. Searching for objects in real-worldscenes. J. Exp. Psychol. 97, 22–27 (1973).

33. Palmer, S.E. The effects of contextual scenes on the identification of objects.Mem. Cognit. 3, 519–526 (1975).

34. Friedman, A. Framing pictures: the role of knowledge in automatizedencoding and memory for gist. J. Exp. Psychol. Gen. 108, 316–355 (1979).

35. Biederman, I., Mezzanotte, R.J. & Rabinowitz, J.C. Scene perception: detectingand judging objects undergoing relational violations. Cognit. Psychol. 14,143–177 (1982).

36. Boyce, S.J., Pollastek, A. & Rayner, K. Effect of background information onobject identification. J. Exp. Psychol. Hum. Percept. Perform. 15, 556–566(1989).

37. Hollingworth, A. & Henderson, J.M. Does consistent scene context facilitateobject perception? J. Exp. Psychol. Gen. 127, 398–415 (1998).

38. Humphreys, G.W., Riddoch, M.J. & Price, C.J. Top-down processes in objectidentification: evidence from experimental psychology, neuropsychology andfunctional anatomy. Philos. Trans. R. Soc. Lond. B Biol. Sci. 352, 1275–1282(1997).

39. Parasuraman, R. & Martin, A. Interaction of semantic and perceptual

processes in repetition blindness. Vis. Cognit. 8, 103–118 (2001).40. Butter, C.M. & Goodale, M.A. Visual search selectively enhances recognition

of the search target. Vis. Cognit. 7, 769–784 (2000).41. Peterson, M.A. & Gibson, B.S. Shape recognition inputs to figure-ground

organization in three-dimensional displays. Cognit. Psychol. 25, 383–429(1993).

42. Peterson, M.A. & Gibson, B.S. Object recognition contributions to figure-ground organization: operations on outlines and subjective contours. Percept.Psychophys. 56, 551–564 (1994).

43. Peterson, M.A. & Kim, J.H. On what is bound in figures and grounds. Vis.Cognit. 8, 329–348 (2001).

44. Glaser, W.R. Picture naming. Cognition 42, 61–105 (1992).45. Fodor, J.A. Modularity of Mind (MIT Press, Cambridge, Massachusetts, 1983).46. Roediger, H.L. III & McDermott, K.B. Creating false memories: remembering

words not presented in lists. J. Exp. Psychol. Learn. Mem. Cogn. 21, 803–814(1995).

47. Roediger, H.L. & McDermott, K.B. Implicit memory in normal humansubjects. in Handbook of Neuropsychology Vol. 8 (eds. Boller, F. & Grafman, J.)63–131 (Elsevier, Amsterdam, 1993).

48. Ochsner, K.N., Chiu, C.Y. & Schacter, D.L. Varieties of priming. Curr. Opin.Neurobiol. 4, 189–194 (1994).

©20

03 N

atu

re P

ub

lish

ing

Gro

up

h

ttp

://w

ww

.nat

ure

.co

m/n

atu

ren

euro

scie

nce