Multiple Visual Memory Phenomena In a Memory Search Task

Multiple visual memory phenomena in a memory

search task

MONICA FABIANI,a JONATHAN HO,b ALEX STINARD,c AND GABRIELE GRATTONa

aBeckman Institute, University of Illinois at Urbana-Champaign , Urbana, Illinois, USAbPsychology Department, Columbia University, New York, New York, USAcDepartment of Psychology, University of Missouri–Columbia, Columbia, Missouri, USA

Abstract

This paper reports evidence of the existence ofmultiple and distinct visualmemory processes in amemory search task in

which a divided field stimulus presentation was used at study (Experiments 1–3) and either a foveal (Experiments 1 and

2) or a lateralized (Experiment 3) stimulus presentation was used at test. These memory processes can be distinguished

on the basis of (1) whether or not they are hemispherically organized; and (2) the locus of their underlying brain

activity, as evidenced by the scalp distribution of the event-related brain potentials and by the localization of the event-

related optical signal that accompany them. These memory effects are discussed in the context of visual formmemory.

Descriptors: Hemispheric organization, Encoding-related lateralization, Sensory signature, Visual working memory,

Memory search task, Event-related brain potentials (ERPs), Event-related optical signal (EROS)

A substantial body of literature reports the effects of previous

stimulus exposure in early visual processing, from passive

phenomena such as habituation (e.g., Colon, Boumen-Van den

Eerden, & Cuyten, 1983; Megela & Teyler, 1979) to memory

effects such as visual form and repetition priming (e.g., Doyle,

Rugg, & Wells, 1996; Paller & Gross, 1998; Paller, Kutas, &

McIsaac, 1998; Rugg, 1985; Walsh, Le Mare, Blaimire, &

Cowey, 2000). In addition, a recent series of experiments, using a

divided-field presentation at study and a foveal presentation at

test, has shown that some visual memory processes are

hemispherically organized, in that the behavior and/or the brain

activity at test maintains a ‘‘signature’’ of the lateralized

encoding condition (Fabiani, Stadler, & Wessels, 2000; Gratton,

Corballis, & Jain, 1997; Gratton, Fabiani, Goodman-Wood, &

DeSoto, 1998; Talsma, Wijers, Klaver, & Mulder, 2001). In this

paper we present three experiments using the polarity and scalp

distribution of event-related brain potentials (ERPs; e.g.,

Fabiani, Gratton, & Coles, 2000), the localization and time

course of the event-related optical signal (EROS; Gratton,

Corballis, Cho, Fabiani, & Hood, 1995; Gratton & Fabiani,

1998, 2001), and the presence of hemispheric organization to

discriminate among these early visualmemory effects. The results

suggest that at least two distinct types of visual memory effects

are present, one of which is hemispherically organized whereas

the other is not.

Several investigators have proposed that the successful

retrieval of amemory trace involves an overlap between encoding

and retrieval processes, and possibly the reactivation of the same

brain areas that were used at encoding (e.g., Kosslyn, 1980;

Roediger, Weldon, & Challis, 1989). Recently, Gratton and

colleagues have suggested that, because the visual processing

system is contralaterally organized, the memory traces left by

laterally presented visual stimuli should maintain a ‘‘sensory

signature’’ of the hemifield of initial stimulus presentation, in the

form of a performance advantage and/or of the existence of

differential brain activity (Gratton et al., 1997; see also Fabiani,

Stadler, et al., 2000; Gratton, 1998). This hypothesis has been

tested in recognition paradigms using verbal (Fabiani, Stadler,

et al., 2000) and nonverbal (Gratton et al., 1997) visual stimuli.

The results indicate that, for both verbal and nonverbal stimuli,

there is evidence of encoding-related lateralizations in the ERPs

elicited by test stimuli presented foveally. Note that this

lateralized activity observed at test switches sides according to

the hemifield stimulated at encoding, and that the test stimuli

vary only with respect to encoding side and are identical in every

other way (including the fact that they require the same manual

response). Thus, these data suggest that memory traces may

retain some of the information contained in the sensory world,

and that, if this information implies a differential early

involvement of the two cerebral hemispheres, the retrieval

activity will be hemispherically organized (for related work, see

also Senkfor, Van Petten, & Kutas, 2002).

The ERP experiments mentioned above have investigated

the hemispheric organization of visual memory in recognition

paradigms in which study and test are separated by several

This research was supported by NIMHGrantMH 57125 to GabrieleGratton, and by McDonnell-Pew Grant 97-32 to Monica Fabiani. Wethank Marsha Goodman-Wood, Ted Moallem, and M. CatherineDeSoto for help with some of the data collection, and the Max PlanckInstitute for Cognitive Neuroscience and the University of Leipzig,Germany, for hosting us during the preparation of this article.

Address reprint requests to: Monica Fabiani, University of Illinois,Beckman Institute, 405 North Mathews Avenue, Urbana, IL, 61801,USA. E-mail: [email protected].

Psychophysiology, 40 (2003), 472–485. Blackwell Publishing Inc. Printed in the USA.Copyright r 2003 Society for Psychophysiological Research

472

minutes, and stimuli are unique. Thus, the encoding-related

lateralization effects investigated in these experiments suggest

that the underlying memory phenomena have a fairly long-time

constant. Other studies, however, suggest that effects of the

hemispheric organization of memory can also be observed in

short-term memory paradigms, where stimuli are repeated, both

within and across conditions, and where the time interval

between study and test is of the order of seconds (see Gratton

et al., 1998; Talsma et al., 2001). In this paper we report three

experiments using a short-term memory paradigm, in which the

encoding-related lateralization effects are compared to other

memory phenomena (such as the early old–new effect, with a

latency of less than 400 ms) occurring within the same latency

range.

A limitation of the ERP data described above is that they do

not provide much information about the neuronal generators of

the lateralized activity. However, given the sensory nature of the

processes involved, it is likely that at least some of the lateralized

activity may originate in visual cortex or elsewhere in the ventral

visual processing stream. To test this hypothesis, Gratton et al.

(1998) ran an optical imaging study using a divided-field

memory-search paradigm (Sternberg, 1966). On each trial,

subjects studied a set of two items (the memory set), which were

presented to the left and right of a fixation cross. They were then

shown one test item presented foveally, and were asked to

indicate whether or not it belonged to the preceding memory set.

The EROS activity elicited by the centrally presented test stimuli

was recorded from a number of locations over occipital cortex.

The results indicated that test stimuli that belonged to the

memory set (i.e., ‘‘old’’ stimuli) elicited an EROS response in

medial occipital cortex, with a latency of 50–150ms, which was

larger in the hemisphere ipsilateral to the encoding side (i.e., the

unexposed hemisphere). ‘‘New’’ test stimuli elicited bilateral

EROS activity.

These data demonstrated the existence of a type of brain

activity that (a) is influenced by previous exposure to the stimuli,

and (b) is hemispherically organized. However, the relationship

between this type of lateralized activity and previous reports of

other ERP differences between ‘‘old’’ and ‘‘new’’ stimuli is

unclear. In fact, it has long been known that, in the Sternberg

paradigm, the ERP activity elicited by correctly identified test

stimuli is generally more positive when they belong to the

memory set (‘‘old’’ or ‘‘yes’’ responses) than when they do not

(‘‘new’’ or ‘‘no’’ responses; e.g., Ford, Pfefferbaum, Tinklen-

berg, & Kopell, 1982). An increased positivity for previously

presented items has also been observed in repetition priming

paradigms, in whichmemory is not explicitly tested (e.g., Besson,

Kutas, & Van Petten, 1991; Hamberger & Friedman, 1992;

Paller & Gross, 1998; Rugg, 1990; Van Petten & Senkfor, 1996;

for a review, see Rugg, 1995).

Although the Sternberg paradigm and the repetition priming

task differ along a number of dimensions (such as whether or not

subjects are performing an explicit memory task), the old–new

ERP effects observed in these conditions show substantial

similarities, at least at short latencies after stimulation (i.e., less

than 400msFearly old–new effect). This may be due to the fact

that both types of tasks involve a preliminary ‘‘familiarity’’

judgment, which occurs early and relatively automatically.

The existence of an early familiarity judgment is hypothesized

by several two-step models of recognition memory (e.g.,

Atkinson & Juola, 1973; Eriksen, Eriksen, & Hoffman, 1986;

Jacoby, 1991).

There are at least two accounts for the old–new ERP

difference observed at early latency, which are not necessarily

mutually exclusive. First, the difference can be due to an

increased negativity in response to new items. Wijers, Mulder,

Okita, and Mulder (1989; see also Friedman, 1990), using a

combined visual/memory search paradigm, described a negative

fronto-central activity, whose amplitude was proportional to the

amount ofmemory search required (expected to bemaximum for

new items). Other investigators (e.g., Mecklinger, Kramer, &

Strayer, 1992; see also Rugg, 1995), in paradigms using word

stimuli, also proposed that new items elicit an increased negative

activity, and interpreted this finding as an increased N400 to

nonrepeated items. Finally, it is possible that part of the old–new

ERP differences could be explained by an increased P300 to old

items (e.g., Friedman, 1990; Karis, Fabiani, & Donchin, 1984;

Mecklinger & Meinshausen, 1998; see also Mecklinger, 2000).

This may be due to the fact that repeated (old) items, which are

identified as such on the basis of an early familiarity judgment,

may involuntarily capture the subject’s attention. The possible

implications of such a phenomenon have been exploited by

researchers interested in the ‘‘guilty knowledge’’ paradigm (e.g.,

Farwell & Donchin, 1991; Johnson & Rosenfeld, 1992).

In the three experiments reported in this paperwe investigated

whether the encoding-related lateralizations elicited by old

stimuli are distinguishable from the more widely studied early

old–new ERP effect. We used the divided-field Sternberg

paradigm described byGratton et al. (1998), as previous research

has shown that both types of effects can be elicited in this

paradigm. Experiments 1 and 3 involved an analysis of the ERPs

elicited by the test stimuli. Data obtained in these studies provide

information about the relationship between the encoding-related

lateralizations and the old–new ERP effect. The results suggest

the existence of polarity and scalp distribution differences

between these two types of activities. Experiment 2 is based on

a new analysis of the data reported by Gratton et al. (1998), in

which we examined the EROS response elicited in occipital areas

by the test stimuli, at the same latency as the ERP effects found in

Experiments 1 and 3. These data support the existence of

separate loci of activity for the lateralization and old–new effects,

at least in occipital cortex.

EXPERIMENT 1

Methods

Participants

Seven participants (2 women, age range 20–29 years) were run

for two sessions. All participants reported themselves to be right-

handed, in good health, had normal or corrected-to-normal

vision, and signed informed consent prior to participation.

Stimuli and Procedures

The 26 letters comprising the English alphabet were used as

stimuli. The characters were capitalized and presented in a sans-

serif font. Each trial began with the simultaneous presentation of

two letters (memory set) displayed, respectively, 2.51 to the left

and right of a central fixation cross. Each letter subtended

approximately 0.51 of visual angle and was displayed for 200 ms.

After an interval of 2,000ms, another letter was presented in the

center of the computer screen, immediately above the fixation

cross (test letter). This letter could be the one previously

Multiple visual memory phenomena 473

presented to the left (old-left condition), the one presented to the

right (old-right condition), or a new letter. Subjects had to

indicate whether or not each test letter belonged to the memory

set by pressing one of two buttons (see Figure 1). Response-hand

assignments were fixed across participants, with the right hand

used to indicate ‘‘old’’ responses, and the left hand used to

indicate ‘‘new’’ responses. It is important to note, however, that

no discrimination was required of the participants between the

two ‘‘old’’ conditions, which, therefore, did not differ in terms of

response assignment. The three test-letter conditions were

equiprobable. The order of the stimuli and stimulus conditions

was randomized (varied mapping), with the constraint that the

two letters in any given memory set could not be identical. The

first session was used for practice, and included 300 trials. In the

second session, during which ERPs were recorded, participants

completed a total of 600 trials.

ERP Recording

The electroencephalogram was recorded from 19 scalp locations

(10–20 Electrode System; Jasper, 1958) by means of an electrode

cap (ElectroCap International, Inc.). The left mastoid was used

as an on-line reference, and an average mastoid reference was

derived off-line. The recording locations included three midline

sites (Fz, Cz, and Pz), eight sites to the left of themidline (F7, F3,

T3, C3, T5, P3, O1, and leftmastoid), and their homologous sites

to the right of the midline. The vertical and horizontal

electrooculogram (EOG) was recorded bipolarly, and ocular

artifacts were removed from the EEG according to a procedure

described byGratton, Coles, andDonchin (1983). A 0.01–35-Hz

band-pass was used for all electrophysiological recordings.

Impedance was kept below 20KO. EEG and EOGwere sampled

at 200Hz for 1,200ms starting 100ms before the presentation of

each test stimulus.

Results

Behavior

The average accuracy for each condition is reported in Table 1.

Reaction time (RT) was faster for old than for new items,

t(6)5 26.56, po.0001, but accuracy did not differ between these

two conditions, t(6)5 0.59, n.s. Although no difference was

expected between the old-left and old-right conditions, old-left

stimuli had, in this case, slower RT, t(6)5 2.98, po.05, and

lower accuracy, t(6)5 3.23, po.05.

ERPs

The focus of this study was on the ERPwaveforms elicited by the

test stimuli. These data are analyzed and discussed in two ways.

The first is the comparison of the waveforms elicited by the old

stimuli (i.e., by both the old-left and old-right stimuli combined)

and the new stimuli (the old–new effect). The second comparison

is between the waveforms elicited in the old-left and old-right

conditions. This comparison is aimed at examining encoding-

related lateralization effects. Only correct trials were included in

the ERP analyses described below.

Early old–new effect. The grand average ERP waveforms for

old and new stimuli are shown in Figure 2. A difference between

old and new items was evident at a latency of 250–350ms. This

early effect, visible atmost electrode sites, could be due to either a

reduced or delayed P300 component for the new items

(consistent with the longer RT for this condition), to the presence

of a negativity, which would be more prominent for the new

items, or to a combination of both effects.

This effect was quantified by measuring the average voltage

amplitude in a time window from 250 to 350ms poststimulus,

separately for each subject, electrode, and condition. The data

were then analyzed at the midline electrodes with a repeated

measure ANOVA with two factors: stimulus (old or new) and

electrode (Fz, Cz, and Pz). There was a main effect of stimulus,

474 M. Fabiani et al.

Memory Set

Brain

Representation

Encoding-related

Lateralization

Test Stimulus

X + Y

X

Old Left

X + Y

Old Right New

Y X Y X Y X

X + Y

Y Z

Y X Y X Y X

Figure 1. Schematic representation of the experimental rationale and paradigm, indicating how stimuli presented to the left and right

of fixation (X and Y) may be expected to leavememory traces that are different in the two hemispheres, and how the presentation of

a test stimulus in the center may lead to differential response in the two hemispheres.

Table 1. Behavioral Results for Experiment 1

Test stimulus Correct RT Accuracy

Old left 497 (100) 0.965 (0.018)Old right 471 (107) 0.986 (0.007)Old 484 (103) 0.975 (0.011)New 576 (99) 0.972 (0.022)

Note: Standard deviations are in parentheses.

with the old stimulus waveforms being more positive than the

new, F(1,6)5 6.04, po.05, and a main effect of electrode,

F(2,12)5 64.74, po.001, e5 0.85), with activity at Pz being

more positive than that at Cz and Fz.

To assess whether there was any systematic lateralization of

the old–new effect, the data were further analyzed with an

ANOVA including three factors: stimulus (old or new), electrode

pair (F7-F8, F3-F4, T3-T4, C3-C4, T5-T6, P3-P4, O1-O2), and

electrode side (left or right). There were, as in the previous

analysis, a main effect of stimulus, F(1,6)5 7.33, po.05, and a

main effect of electrode, F(6,36)5 11.48, po.005, e5 0.28.

There was also an interaction between stimulus type and

electrode side, F(1,6)5 9.65, po.05, indicating that the early

old–new effect was larger over the right hemisphere. This

difference was characterized by an increased frontal negativity

for new items, particularly evident over the right hemisphere, as

indicated by a significant three-way interaction between stimu-

lus, electrode pair, and electrode side, F(6,36)5 3.93, po.05,

e5 0.43. A problem in interpreting the right/left hemisphere

differences in this experiment, however, is that subjects were all

required to use the right response for old items, and the left

response for new items. Thus the hemispheric differences might

be due to motor potentials overlapping over the memory effects.

One of the purposes of Experiment 3 (see below) is to control for

this problem.

Encoding-related lateralization effects. This analysis was

aimed at establishing whether any systematic lateralization could

be detected in the waveforms elicited by the ‘‘old’’ test stimuli, as

a function of the hemifield at which they were presented at study.

Note that, at test (a) the stimuli were all presented centrally;

(b) old-right and old-left stimuli required the same response; and

(c) old-right and old-left stimuli consisted, on average, of the

same letters (as the assignment of letters to the various conditions

was randomized). Therefore, any systematic difference of

lateralization between old-left and old-right stimuli should be

attributed to the encoding conditions, that is, to a differential

memory trace depending on the hemisphere of first exposure

and/or processing. Grand average lateralization waveforms are

shown in Figure 3. These waveforms represent the difference

between the contralateral (i.e., right electrodes in the old-left

condition, and left electrodes in the old-right condition) and

ipsilateral conditions (i.e., left electrodes in the old-left condition,

and right electrodes in the old-right condition) for the old items,

and are derived in a manner analogous to the calculation of the

lateralized readiness potential (LRP; Gratton, Coles, Sirevaag,

Eriksen, & Donchin, 1988).

The analysis of the encoding-related lateralization was based

on the same quantification procedures described above. The data

set was limited to the old-left and old-right conditions, and the

data were entered in a repeated measure ANOVA with three

factors: stimulus (old-left or old-right), electrode pair (F7-F8,

F3-F4, T3-T4, C3-C4, T5-T6, P3-P4, O1-O2), and electrode side

(left or right). The critical result was the presence of a significant

interaction between stimulus type and electrode side, F(1,6)5

10.72, po.05. Note that this interaction is formally equivalent to

the calculation of an LRP (Gratton et al., 1988; Fabiani, Stadler,

et al., 2000). When separate analyses were performed for the

different electrode pairs, the interaction was significant at the P3-

P4 pair, F(1,6)5 12.40, po.05. In fact, all subjects showed the

same effect at this electrode pair. Note that this effect indicates

the existence of more negative activity over the contralateral

(exposed) side. This is in apparent contradiction with the greater

positivity observed for old items in the old–new effect.

Summary and Discussion

The ERPs recorded to test stimuli obtained in this study revealed

the following effects in a latency range between 250 and 350ms

after the test stimulus. First, there was an early old–new effect,


Figure 2.Grandaverage ERPwaveforms at all electrode locations recorded during the test phase for new stimuli (dotted line), stimuli

previously studied in the right hemifield (dashed line), and stimuli previously studied in the left hemifield (solid line). The vertical

arrows indicate the time of stimulus presentation. The shaded areas indicate the interval used for the measurements.

characterized by more positive-going activity to old (repeated)

stimuli, which was consistent with previous data reported in the

literature obtained with both explicit and implicit memory

paradigms (e.g., Mecklinger, 2000; Rugg, 1995). Second, there

was a systematic lateralization as a function of the encoding

hemisphere, analogous to that described in long-term recognition

paradigms (Fabiani, Stadler, et al., 2000; Gratton et al., 1997).

The polarity and scalp distribution of these two effects were

different. In fact, the old–new effect consisted of larger positive

activity for old (repeated) items, whereas the encoding-

lateralization effect involved an increased positivity at

ipsilateral scalp electrodes (i.e., located over the nonexposed

hemisphere).

As noted above, these effects are in apparent contradiction.

However, it is difficult, on the basis of the ERP data alone, to

determine whether these two effects are completely independent

(i.e., whether they correspond to nonoverlapping neural

substrates). The contralateral control double-subtraction proce-

dure used to derive the encoding-related lateralizations (Gratton,

1998) may in fact artificially alter the scalp distribution of the

effects. For example, when this procedure is applied to move-

ment-related potentials obtained from hands and feet, the two

resulting lateralizations are of opposite polarity because of the

relative orientation of their underlying generators (see Brunia &

Vingerhoets, 1981). Thus, in the present study, the opposite

polarity of the old–new and encoding-related lateralization

effects cannot be used in a conclusive manner to determine

whether they differ.

Gratton et al. (1998) reported EROS data obtained in this

paradigm showing increased brain activity to test stimuli in the

hemisphere not previously exposed to the stimuli. However, this

activity occurred at very early latencies (50–150ms) and in

medial (possibly primary) visual cortex (see also Badgaiyan &

Posner, 1997; Talsma et al., 2001). Given the latency of the early

old–new and of the encoding-related ERP effects described here

(250–350ms), it is unlikely that they can correspond to the same

functional phenomena. Gratton et al. (1998) had also recorded

EROS data for an extended period of time after stimulation, and

from more lateral recording sites than those included in their

paper. Therefore, in Experiment 2, we present new analyses of

that experiment, which involve the data not previously con-

sidered. For these analyses we focused on the same latency range

as that of the ERP effects (250–350ms) presented for Experiment 1.

Note that these data, differently from those obtained with the

lateralized ERP waveforms, are localized, and thus provide both

temporal and spatial information independently for the two

hemispheres.

EXPERIMENT 2

Methods

Participants

Four healthy young adults (3 women, age 24–28) signed

informed consent before participating in the study. Three of

the participants were right-handed and all had normal or

corrected-to-normal vision.

Procedures

Six-hundred trials were presented during each of 2 practice and

28 experimental sessions, each run on a different day, to allow for

the recording of EROS fromdifferent scalp locations (see below).

The paradigm was identical to that described for Experiment 1,

with the exception that response-hand assignments were

randomized across subjects.

EROS Recording and Analysis

The event-related optical signal is a non invasive brain imaging

method based on the measurement of changes in the diffusion of

near-infrared light through brain tissue as a consequence of

activity (Gratton & Fabiani, 2001). During the experimental

sessions, EROS was recorded from 28 different locations (one

per session) over the occipital area using a single-channel

frequency-domain optical system (Gratton et al., 1990; Gratton

& Fabiani, 1998, 2001). The source was a 715-nm LED

(powero1mW) modulated at 112MHz (cross-correlation

frequency5 5 kHz). Estimates of the phase delay (EROS)

were obtained every 20ms. The detector was a 3-mm fiber

optic bundle connected to a photomultiplier tube. The recording

locationsmapped a strip 7.2 cmwide and 1.5 cmhigh centered on

the midline, 1.75 cm above the inion. The distance between

adjacent recording locations was 0.5 cm along the vertical

dimension and 1.2 cm along the horizontal dimension.

The source-detector distance was 2.5 cm, allowing for the


Figure 3. Grand average lateralization waveforms for each lateral electrode pair for test stimuli sorted according to the side of

lateralization during study. Negative potentials (upward deflections) indicate greater negativity at electrode locations contralateral

to the side of presentation during study (exposed side). The vertical arrows indicate the time of stimulus presentation.

investigation of a brain area up to 3 cm deep (Gratton, Fabiani,

et al., 1995; Gratton, Sarno, Maclin, Corballis, & Fabiani,

2000), expected to encompass primary and extrastriate visual

areas (Homan, Herman, & Purdy, 1987).

The pulsation artifact in the EROS measures was compen-

sated for off-line with a procedure described by Gratton and

Corballis (1995). Trials with recording artifacts (less than 10%),

mostly due tomovements, were discarded. A 100-ms prestimulus

baseline was subtracted from the raw phase-delay data, which

were than averaged across trials separately for each subject,

recording location, and condition. The averages were filtered

using a 5-point boxcar filter.

Our optical instrument allows us to derive various measures

related to the migration of photons in active brain areas, which

refer to the photons that are back-scattered from the tissue to the

detector. One of the measures we derive reflects the relative time

of arrival of the photons (photon delay). Because our instrument

uses sources of light that are modulated at 110MHz, the time of

arrival is measured as a phase shift of the photon density wave

arriving to the detector. Changes in phase shifts reflect changes

in the back-scattering of the photons. Specifically, neuronal

depolarization is thought to cause a swelling of the dendritic part

of the neurons (because of ion and water movement), resulting in

a drop in the back-scattering of near-infrared photons. This

allows photons to penetrate more deeply into the cortex, and

effectively increases the phase delay parameters, with changes of

the order of 1 ps (10�12 s), or about 0.051 of phase. Phase

measures are localized (with no cross-talk across locations at

separations of 1.5 cm or more; Maclin, Gratton, & Fabiani, in

press) and track the time course of neuronal activity (for a review,

see Gratton & Fabiani, 2001).

Results

Behavior

The average accuracy for each condition is reported in Table 2.

Reaction time was faster for old than for new items, t(3)5 5.15,

po.05, but accuracy did not differ between these two conditions,

t(3)5 1.91, n.s. There were no differences between the old-left

and old-right conditions in this study, either in RT, t(3)5 0.20,

n.s., or accuracy, t(3)5 0.96, n.s.

EROS Data

We focused on the EROS data for the interval between 250 and

350ms after stimulus, corresponding to the latency at which the

ERP effects were observed in Experiment 1. To account for

individual differences in functional neuroanatomy, the data for

each subject at this response latency were aligned on the basis of

the location showing the maximum response averaged across

conditions. Note that the focus of this study is the difference

between conditions; therefore, this approach cannot bias the

measurements. This alignment procedure allowed us to compute

maps across subjects, separately for each experimental condition

(old-left, old-right, and new). The maps were obtained by

computing the average value in the chosen interval across

subjects for each recording location (aligned across subjects

according to the procedure described above), and produced using

the Quattro Pros graphingmodule. Thesemaps are presented in

Figure 4. They indicate that the location of maximum EROS

activity for the old-left and old-right conditions differed from the

location ofmaximum response for the New condition, in that the

responses for the old conditions were more lateral than that for

the new condition. This is also evident from Figure 5, in

which the data from left and right recording locations were

collapsed as a function of the hemisphere contralateral and

ipsilateral to the encoding side for old stimuli. For the new

stimuli, there was no encoding side, so the left and right sides of

the map are symmetrical.

Figure 6 shows the 90% confidence interval for the location

showing the maximum response for the old conditions (collapsed

together) compared to that of the new condition. Ninety percent

confidence intervals were used because, if confidence intervals

are not overlapping, the probability that the difference is due to

chance could not exceed 1/10� 1/10, or .01. In fact, the actual

probability is likely to be even smaller because, given that a

repeated measure design was used, at least some of the variance

ought to be attributable to individual differences.

For this figure, left and right hemisphere responses were

averaged together. For comparison purposes, the figure also

includes the location of the medial response reported at earlier

latencies by Gratton et al. (1998), and a reference map of the

surface projection of visual cortex derived fromMaier, Dagnelie,

Spekreijse, and Van Dijk (1987). This figure shows that there was

no overlap between the confidence intervals for the locations of

the maximum activity elicited by old and new items in lateral

occipital cortex at a latency of 250–350ms after stimulus. For

this reason, separate EROS time courses and amplitude estimates

were obtained for these two locations. The grand average EROS

time courses are shown in Figures 7 and 8 for the lateral (old) and




Old left 453 (68) 0.978 (0.010)Old right 495 (51) 0.964 (0.022)Old 473 (54) 0.971 (0.011)New 548 (48) 0.945 (0.044)


Figure 4. Grand average surface EROS maps (latency 250–350ms) for

the three experimental conditions (old left, old right, and new). The

coordinates of the maps are in centimeters with respect to the inion. The

maps of individual subjects were aligned on the location of maximum

response (across conditions) for each hemisphere before computation of

the grand average maps.

medial (new) locations, respectively. For comparison purposes,

the time course of the EROS elicited by the presentation of the

memory set items at these two locations is also shown.

The statistical analysis was conducted by computing

the average phase value during the 250–350-ms interval,

separately for each subject, experimental condition, and location.

These average values are plotted in Figure 9, with the data

collapsed across left and right recording sides. Note that, for the

old stimuli, the data are presented on the basis of whether the

recording locationwas contralateral or ipsilateral to the encoding

hemifield.

A planned comparison approach was used, in which three

comparisons (not statistically independent from each other)

were set up: The first was between the old and new items;

the second was restricted to old items, and involved data from

the hemisphere previously exposed to the stimulus (i.e., the

contralateral hemisphere) and the nonexposed hemisphere

(i.e., the ipsilateral hemisphere). The third comparison was

between data from the exposed (the contralateral-old condition)

and non exposed (the ipsilateral-old and the new condition)

locations. These analyses were performed separately for the

locations at which the old items elicited the maximum response

and for the locations at which the new items elicited the

maximum response. The location at which the new response was

maximum showed a differentiation between old and new items,

t(3)5 � 3.64, po.05, but no contralateral–ipsilateral difference,

t(3)5 � 0.54, n.s., and only a marginal effect of previous

exposure to the stimulus, t(3)5 � 2.66, po.08. Presentation of

the memory set stimuli (at study) also elicited significant activity

at this location, although at a shorter latency (150 ms),

t(3)5 2.41, po.05. The location at which the old response was

maximum did not show either a statistically significant old–new

difference, t(3)5 1.45, n.s., or a statistically significant contral-

ateral–ipsilateral difference, t(3)5 1.01, n.s., or a response to the

memory set stimuli, to1 at all latencies. However, it did show a

significant difference between exposed and nonexposed condi-

tions, t(3)5 10.28, po.01. An examination of Figure 4 suggests

that this response may be greater on the exposed side

(contralateral old condition), in particular for the right hemi-

sphere. In fact, the difference between old-left (contralateral) and

old-right (ipsilateral) stimuli was significant in the right


Figure 5. Grand average surface EROS maps (latency 250–350ms) for

old and new test items. Data are plotted as a function of whether they

were recorded from locations contralateral and ipsilateral to the side of

stimulus presentation at study. The coordinates of the maps are in

centimeters with respect to the inion. The maps of individual subjects

were aligned on the location ofmaximumresponse (across conditions) for

each hemisphere before computation of the grand average maps.

Figure 6. Ninety percent confidence intervals for the locations of maximum activity for old and new items (top). Location of the

medial occipital cortex activity (latency 50–150ms, reported in Gratton et al., 1998) and a reference map of the surface projection of

visual cortex derived fromMaier et al. (1987, bottom) are also included as references. The shaded rectangle traced over the surface

projection map corresponds to the area plotted in the top graph.

hemisphere, t(3)5 3.40, po.05, but not in the left hemisphere,

t(3)5 � 0.54, n.s. This suggests that the old-left condition may

lead to a bilateral representation of the stimulus in this cortical

area, whereas the old-right condition may lead to a contralateral

representation only.


The EROS data reported here indicated that two different

locations within occipital cortex showed activity in response to

the presentation of old and new items. Interestingly, the area

responding to new items also appeared to respond tomemory-set

items (which could be considered ‘‘new’’ for the subject; see

Figure 8). For the area responding to old items, the EROS

activity was limited to the hemisphere previously exposed to the

stimulus, at least for the old-right condition. Note that in the old-

right condition early visual processing is expected to occur in the

left hemisphere. Because the stimuli were verbalizable letters, in

this condition there may be a reduced need for right hemisphere

involvement, leading to lateralized activity. However, for old-left

letter stimuli, the first visual processing is expected to occur in the

right hemisphere, which is nondominant for language. Thus, a

recruiting of homologous left-hemisphere areas may be needed,

leading to bilateral activation.

EXPERIMENT 3

Introduction

The results of Experiments 1 and 2 indicate that brain activity at

time of test is lateralized as a function of the side of encoding.

Furthermore, the data suggest that old and new items are

characterized by different brain activity. However, the inter-

pretation of these data is limited by some aspects of the

experimental designs. First, in Experiment 1, all subjects were

given the same response assignments. Although this reduced the

error variance in the analyses of old-left versus old-right effects, it

introduced a confound in the comparison between old and new

items. Second, in both experiments, it is difficult to determine the

functional significance of the lateralized brain potentials because

no systematic behavioral differences were observed. In fact, the

accuracy at test was very high in all subjects, and because the test

stimulus was always presented in a foveal position, it was not

possible to determine whether a match between the encoding and

retrieval hemisphere would facilitate performance (see Gratton

et al., 1997). Finally, new items were less frequent in the test

phase than the two types of old items combined, thus creating an

unbalance in probability that could affect the difference between

old and new activity.

For these reasons we ran a third study, in which these issues

were addressed directly. This experiment was an ERP study

similar to Experiment 1, with the following exceptions: (1)

instead of using letters, we used graphic characters as stimuli,


Figure 8.Time course of the activity for the location atwhich the response

was maximum for the ‘‘new’’ stimuli, collapsed across left and right

hemispheres. The time course of the activity elicited by memory set

stimuli at the same location is reported as a reference.

Figure 7. Time course of the activity for the location at which the EROS

response was maximum for the ‘‘old’’ stimuli, collapsed across left and

right hemispheres. The time course of the activity elicited by memory set

stimuli at the same location is reported as a reference.

Figure 9. Bar graphs summarizing the activity for the locations of

maximumresponse for new andold stimuli, respectively. The bars labeled

‘‘C’’ refer to data for ‘‘old’’ stimuli recorded from locations contralateral

to the side of their initial encoding. The bars labeled ‘‘I’’ refer to data

for ‘‘old’’ stimuli recorded from locations ipsilateral to the side of their

initial encoding; old ipsilateral. The bars labeled ‘‘N’’ refer to ‘‘new’’ test

stimuli.

because these stimuli cannot be easily verbalized and therefore

are more likely to be processed on the basis of visual memory

rather than verbal/auditory coding; (2) the test stimuli were

presented laterally, instead of centrally, although never in the

same position used at encoding (thus allowing us to compare

same- anddifferent-hemifield conditions between study and test);

(3) stimulus-response assignments were counterbalanced across

subjects; and (4) old and new items were equiprobable at test.

Note that the changes in stimulus material and presentation

locations (which might require the subject to operate covert

attention shifts) are expected to lead to a longer encoding time.

Therefore we expect all effects in this study to be delayed with

respect to those observed in Experiments 1 and 3.

Methods

Participants

Ten participants (4 women, age range 20–57 years) were run for

one session. All participants reported themselves being right-

handed, in good health, had normal or corrected-to-normal

vision, and signed informed consent prior to participation.

Stimuli and Procedures

Only procedures that differ from those reported for Experiment 1

are described here. Twenty-six ASCII characters (from ASCII

#180 to 205 included; e.g., ) were used as stimuli instead

of letters. As in Experiment 1, each trial began with the

simultaneous presentation of two characters (memory set)

displayed, respectively, 2.51 to the left and right, and 2.51 above

a central fixation cross. After an interval of 1,600ms, another

character was presented (test stimulus) either 2.51 to the left or

right of the fixation cross, but always 2.51 below it. Thus, these

generated a 2� 3 design, with hemifield of test (left or right) and

test stimulus type (old left, old right, and new) as factors.

Response-hand assignments for ‘‘old’’ and ‘‘new’’ stimuli were

counterbalanced across participants. Old and new conditions

were equiprobable, with 320 trials each.

ERP Recording

The electroencephalogram was recorded from 22 scalp locations

(electrodes FP1 and FP2 were addedwith respect to Experiment 1).

The digitizing rate was 200Hz, and a baseline of 80ms was used.

All other aspects of ERP recording and analysis were identical to

those used for Experiment 1.

Results

Behavior

The average RTand accuracy for each condition are reported in

Figure 10. As can be seen from this figure, performance was

lower and reaction times longer in this study than in Experiments

1 and 2. This was most likely due to the fact that the graphic

stimuli were not familiar or easily verbalizable and were

also presented peripherally at test. There were no significant

differences between old and new items either in RTor accuracy,

and no significant differences between old left and old right

conditions, all Fo1. There were also no significant differences as

a function of test side, Fo1. However, as predicted, there were

significant interactions between the encoding and retrieval

side for both RT, F(1,9)5 13.05, po.01, and accuracy,

F(1,9)5 9.26, po.05, for old items. These data indicate that

recognition performance is higher when stimuli are presented in

the same hemifield at study and test (see Gratton et al., 1997).

Note that the stimulus position was never the same at study and

test, thus ruling out location-specific priming effects.

ERPs

Only correct trials were included in the ERP analyses described

below.

Early old–new effect. The grand average ERP waveforms for

old and new stimuli are shown in Figure 11. In this figure, to

render its interpretation clearer, the data across different testing

conditions (left and right) were collapsed together. In particular,

three types of averages are presented: ‘‘old’’ trials in which the

side of presentation of the test stimulus matched that used at

encoding, ‘‘old’’ trials in which the side of presentation of the test

stimulus mismatched that used at encoding, and ‘‘new’’ trials.

As for Experiment 1, a difference between old and new items was

evident beginning at a latency of 250–350ms. This effect was

most clearly visible at frontal sites and was quantified by

measuring the average voltage amplitude in a time window

from 250 to 350ms poststimulus, separately for each subject,


Figure 10.RT and accuracy results for Experiment 3. Dark-shaded bars refer to test stimuli displayed to the left of fixation, and light-

shaded bars refer to test stimuli displayed to the right of fixation.

electrode, and condition. The data were then analyzed at the

midline electrodes with a repeated measure ANOVA with three

factors: encoding condition (old-left, old-right, or new), test

hemifield (left or right), and electrode (Fz, Cz, and Pz). There

was a main effect of electrode, F(2,18)5 4.38, po.05, e5 0.76),

with activity at Pz being more positive than that at Cz and Fz.

However, the difference between old and new items (Encoding

Condition�Electrode interaction) was larger at frontal than

parietal locations, F(4,36)5 13.93, po.0001, e5 0.61), with the

new items beingmore negative than the old items at Fz, but not at

the other two midline electrode sites. This suggests that the old–

new effect is not due solely to a delayed P300 for the new items,

because if that were the case, this effect would be greater, or at

least equally large, at parietal electrodes. Furthermore the old–

new difference appeared to extend further in time, and to reverse

polaritymuch later than in Experiment 1. This again is consistent

with the idea that encoding processes lasts significantly longer in

Experiment 3 than in Experiment 1. Finally, the right-lateraliza-

tion of the old–new effect present in Experiment 1 was no longer

visible in this study, suggesting that it may at least in part be due

to response requirements.

Encoding-related lateralization effects. As in Experiment 1,

this analysis was aimed at testing systematic encoding-related

lateralizations. Testing hemifield was added as a factor in this

study. The prediction was that lateralization effects observed at

test would vary as a function of the encoding hemifield,

independently of the testing hemifield. Grand average lateraliza-

tion waveforms are shown in Figure 12. These waveforms

represent the difference between the contralateral (i.e., right

electrodes in the old-left condition, and left electrodes in the old-

right condition) and ipsilateral conditions (i.e., left electrodes in

the old-left condition, and right electrodes in the old-right

condition) for the old items, collapsed across testing hemifield


Figure 11.Grand average ERPwaveforms at midline electrode locations recorded during the test phase for new stimuli (dashed line),

stimuli for which study and test hemifield matched (solid line), and stimuli for which study and test hemifield mismatched (dotted

line). The vertical arrows indicate the time of stimulus presentation. The shaded area indicates the interval used for themeasurement.

Figure 12. Grand average lateralization waveforms for each lateral electrode pair for test stimuli sorted according to the side of

lateralization during study. Negative potentials (upward deflections) indicate greater negativity at electrode locations contralateral

to the side of presentation during study (exposed side). The vertical arrows indicate the time of stimulus presentation.

conditions. These results largely replicate those obtained in

Experiment 1 (see Table 3), with the exception of the longer

latency of the lateralization effects. In both studies the largest

effects were visible at the P3/P4 electrode pair, reflecting a

posterior distribution. The delayed latency of the lateralization

effect in Experiment 3 than Experiment 1 appears to correspond

to the increased task difficulty (which is also reflected by the

longer reaction times for all stimuli), and in particular to the

lesser familiarity of the stimuli used in this study. A further

reason for encoding delay is that the test stimuli were presented in

an eccentric (and variable) location, thus requiring the subjects to

shift their covert attention for accurate stimulus encoding (overt

attention shiftsFi.e., gaze shiftsFwere prevented by the short

stimulus presentation time). Because reaction times were delayed

by more than 150 ms in Experiment 3 than in Experiments 1

and 2 (a phenomenon that we attribute to delayed stimulus

encoding), we correspondingly delayed the time window used for

measurement to one between 400 and 600 ms after the onset of

the test stimulus.

The data were entered in a repeated measures ANOVA with

four factors: stimulus (old-left or old-right), testing hemifield

(left or right), electrode pair (Fp1-Fp2, F7-F8, F3-F4, T3-T4,

C3-C4, T5-T6, P3-P4, O1-O2), and electrode side (left or right).

The critical results were (1) a significant interaction between

stimulus type and electrode side, F(1,9)5 6.82, po.05, support-

ing the existence of an encoding-related lateralization effect; and

(2) a significant three -way interaction between stimulus type,

electrode pair, and electrode side, indicating that the encoding-

related lateralization effect was largest at P3/P4, F(7,63)5 5.03,

po.05, e5 0.30). In addition, there was also a significant

Encoding�Test interaction, indicating that potentials were

more positive when encoding and test sides matched than when

they did not, irrespective of electrode location or hemisphere,

F(1,9)5 25.42, po.001.


The results of this study largely replicated and extended those

obtained in Experiments 1 and 2, although, as predicted, delays

in reaction time and latency of ERP responses were observed.

First, they confirmed the occurrence of an early old–new effect,

consisting of a negativity that differentiates new from old items at

frontal sites, under conditions in which the hand of response was

counterbalanced across subjects, and old and new stimuli were

equiprobable. In this study, this is unlikely to be due to a delayed

P300 for new items because (a) the scalp distribution of the effect

is frontal rather than parietal, as it would be expected if the effect

was due to a delayed P300; (b) the probability of new and old

items was matched in this study, and there was no evidence of a

delayed P300 for the new items (see Figure 11). Rather,

the waveforms show two distinct positive peaks, most evident

at Fz.

Second, the results supported the occurrence of an encoding-

related lateralization effect, with a stimulus set that was

unfamiliar and could not be easily verbalized. In addition, by

using a lateralized presentation at test, we showed a same-

hemifield advantage in performance (see also Gratton et al.,

1997). The same-hemifield conditions also elicited more positive

waveforms than the mismatch conditions. Finally, in this study

the latencies of the early old–new effect and that of the encoding-

related lateralization effect differed, adding a further element of

dissociation between the two. In this respect, it is noteworthy that

the right frontal lateralization of the old/new effect observed in

Experiment 1 was not replicated in Experiment 3, suggesting that

that lateralization may, in part, reflect response requirements.

General Discussion

In all experiments reported here we obtained evidence that the

brain activity recorded at test bears a ‘‘sensory signature’’ of

the hemifield at which the stimulus was first encoded. This was

achieved by comparing test stimuli that had been studied to the

left and right of fixation (the ‘‘old left’’ and ‘‘old right’’

conditions). Note that these two conditions are identical in all

respects, including preparatory processes, hand of response, and

probability of occurrence. Thus, any systematic relationship

between the way in which the brain activity at test is lateralized

and the side of encoding can be unambiguously attributed to

encoding-related phenomena. The results support the existence

of a phasic lateralized response with these properties at posterior

electrode sites in the ERP waveforms and in occipital areas in the

EROS data.

The lateralization data presented in this paper differ from

those reported by Gratton et al. (1997) and Fabiani, Stadler,

et al. (2000) in that they were obtained using a working-memory

paradigm instead of a long-term recognition procedure. Note

that, in a working-memory paradigm such as the modified

Sternberg task described here, the same characters are used as

both old and new stimuli on different trials. Given the

large number of trials involved, each stimulus is likely to be

presented many times in each hemifield and condition. As a

consequence, any long-term memory effect is likely to cancel out

when the different conditions are compared to each other, with

only short-term, more phasic effects remaining evident. In fact,

the lateralization waveforms presented by Gratton et al. (1997)

and Fabiani, Stadler, et al. (2000) show more sustained effects

than those reported here. Further, the memory set used in the

Sternberg paradigm is much smaller than the word lists used by

Fabiani, Stadler, et al. (2000) and the series of pictorial stimuli

used by Gratton et al. (1997). The use of a small memory set may

allow subjects to activate specific templates for matching with

possible test stimuli, which may contribute to both the ERP and

EROS phenomena described in this paper.

Differences between the brain activity elicited by old and new

stimuli were also observed in all experiments at a latency of

250–350ms. In the ERP experiments (1 and 3) this early old–new




Old leftTest left 686 (92) 0.902 (0.112)Test right 721 (70) 0.876 (0.108)Old rightTest left 722 (100) 0.908 (0.139)Test right 689 (92) 0.936 (0.069)Old (all)Test left 704 (83) 0.905 (0.115)Test right 705 (74) 0.906 (0.065)NewTest left 710 (63) 0.924 (0.068)Test right 708 (62) 0.919 (0.066)


effect was characterized by a widely distributed larger positivity

to old stimuli with respect to new stimuli, most evident over the

right hemisphere. The early old–new effects obtained with EROS

were characterized by a difference in the location of maximum

response for these two conditions in visual cortex.

The early old–new ERP data were consistent with previous

reports of increased positive response to the old items. However,

this difference could also be characterized as an increased

negativity elicited by the new items (see Mecklinger, 2000; Rugg,

1995). This negativity wasmost evident at anterior electrode sites

and could be related to the ‘‘search negativity’’ described by

Wijers et al. (1989), although it may also be taken as an N400-

like response to new items, reflecting the lack of familiarity for

them (Curran, 2000; Mecklinger et al., 1992). According to this

interpretation, the presence of the ‘‘search negativity’’ would

suggest that the processing of the new stimuli is, on average, more

prolonged than that of the old stimuli because subjects need

to exclude that the test item is part of the memory set before

responding.

One of the central issues addressed by this paper was whether

the differences between old and new stimuli reflected by the

encoding-related lateralizations and by the early old–new effect

are manifestations of the same brain phenomena. The ERP data

reported in Experiments 1 and 3 suggest that these two effects are

in fact separable on the basis of their polarity and scalp

distribution (as well as latency in Experiment 3). However, a

stronger conclusion cannot be drawn because it is, in principle,

possible to have overlapping brain generators that would

produce the observed results. The EROS data, however, given

their greater spatial specificity, corroborate the ERP findings.

These data indicate that two different brain locations are

activated for old and new stimuli in occipital cortex. In fact,

only the activity observed at the location of maximum response

to ‘‘old’’ stimuli appears to be hemispherically organized, at least

for stimuli that were presented in the right hemifield at study.

Experiment 3 differed in several respects from Experiments 1

and 2. First, it included stimulus material that was more difficult

to encode and verbalize. The test stimuli were also presented

eccentrically, requiring some form of covert orienting (overt

orientingFi.e., fixation shiftsFwas prevented by the short

stimulus presentation time). Predictably, these changes resulted

in slower reaction times, and delayed ERP responses (by ap-

proximately 150 ms). Thus, the latency of the encoding-related

lateralization in this study was approximately 400–500ms.

However, as in Experiment 1, the lateralization effect was

maximum at the P3/P4 electrode pair.

Experiment 3 showed conclusively that the lateralization

effects are not linked to a particular response hand (as hand of

response was counterbalanced across subjects) nor to a foveal

presentation of the test stimulus, possibilities that were left open

after Experiment 1. Experiment 3 also revealed that re-presenting

the test stimulus on the samehemifield used at encoding improves

memory performance with respect to the ‘‘across-hemispheres’’

condition. This ‘‘same-hemisphere’’ advantage lends support to

the idea that the lateralization effects may reflect a form of visual

memory that may be useful for performance, especially in

instances in which verbal rehearsal is not available. In other

words, the use of stimuli difficult to verbalize is an important

feature of this experiment because it makes visual memory more

significant for performance, reducing the extent towhich subjects

can rely on verbal memory (which may not be supported by a

similarly contralaterally organized system). The similarity of the

lateralization effects observed in Experiments 1 and 3 makes it

possible to link the ERP effects observed in these two studies with

the optical effects observed in Experiment 2 (which was based on

the same experimental conditions used in Experiment 1). This

suggests that the encoding-related lateralizations observed in the

ERPs are at least in part associated with phenomena occurring in

medial and lateral occipital cortical areas (as revealed by the

optical measures). It should be clear, of course, that the ERP and

EROS effects are not necessarily manifestations of the same

neural activity, especially because of the limited recording area

used for the EROS study. It is likely that brain generators in

other areas (including parietal and frontal cortex) may provide a

substantial contribution to the scalp recorded ERPs. Rather, the

data suggest that the EROS and ERP effects are functionally

similar and that they may be reflections of neural circuits

influenced by the same experimental manipulations.

Brain imaging techniques, such as positron emission tomo-

graphy (PET) and functional magnetic resonance imaging

(fMRI), have been used to investigate the brain areas associated

with priming and recognition effects. In general, these studies

have indicated that repetition and other priming effects are

associated with reduced activity in response to repeated stimuli in

early processing areas for each sensory modality, including

striate and extrastriate areas in the case of visual stimuli

(e.g., Reber, Stark, & Squire, 1998; Schacter & Buckner,

1998). Frontal areas (particularly in the right hemisphere) have

been associated with effortful and intentional retrieval processes,

usually involved in explicit tasks (Buckner & Koutstaal, 1998).

Further, medial temporal lobe structures have also been

implicated in retrieval processes, although in a less consistent

manner (e.g., Schacter & Buckner, 1998). The EROS data

reported here are, by and large, consistent with the hypothesis

that new stimuli elicit a stronger response than old stimuli, at

least in some locations of visual cortex.

In the present study we also found locations in visual cortex

that respond more to old stimuli than to new stimuli (see also

Gratton et al., 1997, for EROS activity at earlier latencies).

Similar results were obtained with fMRI by Reber et al. (1998;

for similar effects in parietal cortex, see also Henson, Rugg,

Shallice, Josephs, & Dolan, 1999; Wiggs, Weisberg, & Martin,

1999) in a recognition task (but not when the subjects were

engaged in categorization). In addition, fMRI (D’Esposito et al.,

1997), PET (Kosslyn et al., 1993), and magneto-encephalo-

graphic (MEG; Raij, 1999) data suggest that visual imagery may

correspond to activation of occipital cortical areas.

A number of issues remain to be addressed in future research.

First, the exact localization of the EROS activity remains to be

established. This can be addressed by coregistering the EROS

data with the structural MR images of individual subjects.

Second, the relationship between the EROS and ERP effects

needs to be clarified. This could be achieved by using more

extensive EROS montages as well as by seeding ERP source

localizationmethods on the basis of the EROS results. Third, the

exact functional significance of the effects described here needs

to be elucidated further, although Experiment 3 indicates that

there are performance advantages when study and test hemifield

match. This also suggests that hemispherically organized

memory representations can be used to perform the task, at least

in cases in which a verbal representation is not readily available.

In conclusion, the EROS data reported by Gratton et al.

(1998) at earlier latencies and the EROS and ERP data described

in this paper involving a latency window between 250 and 350ms


after stimulus onset indicate that there are multiple memory

effects in early visual processing, some of which are located in

closely spaced cortical areas. Some of these effects are

hemispherically organized, whereas others appear to be bilateral.

These effects can be distinguished on the basis of a combination

of evidence provided by ERP and EROS data.

REFERENCES

Atkinson, R. C., & Juola, J. F. (1973). Factors influencing speed andaccuracy of word recognition. In S. Kornblum (Ed.), Attention andperformance (Vol. IV, pp. 583–612). New York: Academic Press.

Badgaiyan, R. D., & Posner, M. I. (1997). Time course of corticalactivations in implicit and explicit recall. Journal of Neuroscience, 17,4904–4913.

Besson, M., Kutas, M., & Van Petten, C. (1991). ERP signs of semanticcongruity and word repetition in sentences. In C. H. M. Brunia, G.Mulder, & M. N. Verbaten (Eds.), Event-related brain research (EEGSuppl. 42; pp. 259–262). Amsterdam, The Netherlands: Elsevier.

Brunia, C. H. M., & Vingerhoets, A. J. J. M. (1981). Oppositehemisphere differences in movement related potentials preceding footand finger flections. Biological Psychology, 13, 261–269.

Buckner, R. L., & Koutstaal, W. (1998). Functional neuroimagingstudies of encoding, priming, and explicit memory retrieval. Proceed-ings of the National Academy of Sciences, USA, 95, 891–898.

Colon, E. J., Boumen-Van den Eerden, C. A., & Cuyten, M. W. (1983).Randomized long interstimuli interval flash-VEP and SSEP. ActaNeurologica Belgica, 83, 177–183.

Curran, T. (2000). Brain potentials of recollection and familiarity.Memory & Cognition, 28, 923–938.

D’Esposito, M., Detre, J. A., Aguirre, G. K., Stallcup, M., Alsop, D. C.,Tippet, L. J., & Farah, M. J. (1997). A functional MRI study ofmental image generation. Neuropsychologia, 35, 725–730.

Doyle, M. C., Rugg, M. D., & Wells, T. (1996). A comparison ofthe electrophysiological effects of formal and repetition priming.Psychophysiology, 33, 132–147.

Eriksen, B. A., Eriksen, C. W., & Hoffman, J. E. (1986). Recognitionmemory and attentional selection: Serial scanning is not enough.Journal of Experimental Psychology: Human Perception and Perfor-mance, 12, 476–483.

Fabiani, M., Gratton, G., & Coles, M. G. H. (2000). Event-related brainpotentials: Methods, theory and applications. In J. Cacioppo,L. Tassinary, & G. Berntson (Eds.), Handbook of psychophysiology(pp. 53–84). New York: Cambridge University Press.

Fabiani, M., Stadler, M. A., &Wessels, P. M. (2000). True memories butnot false ones produce a sensory signature in human lateralized brainpotentials. Journal of Cognitive Neuroscience, 12, 941–949.

Farwell, L. A., & Donchin, E. (1991). The truth will out: Interrogativepoligraphy (‘‘Lie detection’’) with event-related brain potentials.Psychophysiology, 28, 531–547.

Ford, J. M., Pfefferbaum, A., Tinklenberg, J. R., & Kopell, B. S. (1982).Effects of perceptual and cognitive difficulty on P3 and RT in youngand old adults. Electroencephalography & Clinical Neurophysiology,54, 311–321.

Friedman, D. (1990). ERPs during continuous recognition memory forwords. Biological Psychology, 30, 61–87.

Gratton, E., Mantulin, W. W., van de Ven, M. J., Fishkin, J. B., Maris,M. B., & Chance, B. (1990). The possibility of a near-infrared opticalimaging system using frequency domain methods. Proceedings of theThird International Conference for Peace throughMind/Brain Science,Hamamatsu City, Japan, pp. 183–189.

Gratton, G. (1998). The contralateral organization of visual memory: Atheoretical concept and a research tool. Psychophysiology, 35,638–647.

Gratton, G., Coles, M. G. H., & Donchin, E. (1983). A new method foroff-line removal of ocular artifact. Electroencephalography andClinical Neurophysiology, 55, 468–484.

Gratton, G., Coles, M. G. H., Sirevaag, E., Eriksen, C. W., & Donchin,E. (1988). Pre- and poststimulus activation of response channels:A psychophysiological analysis. Journal of Experimental Psychology:Human Perception and Performance, 11, 331–344.

Gratton, G., & Corballis, P. M. (1995). Removing the heart from thebrain: Compensation for the pulse artifact in the photon migrationsignal. Psychophysiology, 32, 292–299.

Gratton, G., Corballis, P. M., Cho, E., Fabiani, M., & Hood, D. (1995).Shades of gray matter: Noninvasive optical images of human

brain responses during visual stimulation. Psychophysiology, 32,505–509.

Gratton, G., Corballis, P. M., & Jain, S. (1997). Hemisphericorganization of visual memories. Journal of Cognitive Neuroscience,9, 92–104.

Gratton, G., & Fabiani, M. (1998). Dynamic brain imaging: Event-related optical signal (EROS) measures of the time course andlocalization of cognitive-related activity. Psychonomic Bulletin &Review, 5, 535–563.

Gratton, G., &Fabiani,M. (2001). Shedding light onbrain function: Theevent-related optical signal. Trends in Cognitive Science, 5, 357–363.

Gratton, G., Fabiani, M., Friedman, D., Franceschini, M. A., Fantini,S., Corballis, P., & Gratton, E. (1995). Rapid changes of opticalparameters in the human brain during a tapping task. Journal ofCognitive Neuroscience, 7, 446–456.

Gratton, G., Fabiani, M., Goodman-Wood, M. R., & DeSoto, M. C.(1998). Memory-driven processing in human medial occipital cortex:An event-related optical signal (EROS) study. Psychophysiology, 35,348–351.

Gratton, G., Sarno, A. J., Maclin, E., Corballis, P. M., & Fabiani, M.(2000). Toward non-invasive 3-D imaging of the time course ofcortical activity: Investigation of the depth of the event-related opticalsignal (EROS). NeuroImage, 11, 491–504.

Hamberger, M., & Friedman, D. (1992). Event-related potentialcorrelates of repetition priming and stimulus classification in young,middle-aged and older adults. Journal of Gerontology: PsychologicalSciences, 47, 395–405.

Henson, R. N. A., Rugg,M. D., Shallice, T., Josephs, O., &Dolan, R. J.(1999). Recollection and familiarity in recognitionmemory: An event-related functional magnetic resonance imaging study. The Journal ofNeuroscience, 19, 3962–3972.

Homan, R. W., Herman, J., & Purdy, P. (1987). Cerebral location ofinternational 10-20 system electrode placement. Electroencephal-ography and Clinical Neurophysiology, 66, 376–382.

Jacoby, L. L. (1991). Process dissociation framework: Separatingautomatic from intentional uses of memory. Journal of Memory andLanguage, 30, 513–541.

Jasper, H. H. (1958). The ten-twenty electrode system of the Inter-national Federation. Electroencephalography and Clinical Neuro-physiology, 10, 371–375.

Johnson, M. M., & Rosenfeld, J. P. (1992). Oddball-evoked P300-basedmethod of deception detection in the laboratory. II: Utilization ofnon-selective activation of relevant knowledge. International Journalof Psychophysiology, 12, 289–306.

Karis, D., Fabiani, M., & Donchin, E. (1984). P300 and memory:Individual differences in the von Restorff effect. Cognitive Psychol-ogy, 16, 177–216.

Kosslyn, S. M. (1980). Image and mind. Cambridge, MA: HarvardUniversity Press.

Kosslyn, S. M., Alpert, N. M., Thompson, W. L., Maljkovic, V., Weise,S. B., Chabris, C. F., Hamilton, S. E., Rauch, S. L., & Buonanno, F. S.(1993). Visual mental imagery activates topographically organizedvisual cortex: PET investigations. Journal of Cognitive Neuroscience,5, 263–287.

Maclin, E., Gratton, G., & Fabiani, M. (in press). Optimum filtering forEROS measurements. Psychophysiology.

Maier, J., Dagnelie, G., Spekreijse, H., & Van Dijk, B. W. (1987).Principal component analysis for source localization of VEPs in man.Vision Research, 27, 165–177.

Mecklinger, A. (2000). Interfacing mind and brain: A neurocognitivemodel of recognition memory. Psychophysiology, 37, 1–18.

Mecklinger, A., Kramer, A. F., & Strayer, D. L. (1992). Event relatedpotentials and EEG components in a semantic memory search task.Psychophysiology, 29, 104–119.

Mecklinger, A., &Meinshausen, R. M. (1998). Recognition memory forobject form and object location: An event-related potential study.Memory and Cognition, 26, 1068–1088.


Megela, A. L., & Teyler, T. J. (1979). Habituation and the human evokedpotential. Journal of Comparative and Physiological Psychology, 93,1154–1170.

Paller, K. A., & Gross, M. (1998). Brain potentials associated withperceptual priming versus explicit remembering during the repetitionof visual word-form. Neuropsychologia, 36, 559–571.

Paller, K. A., Kutas, M., & McIsaac, H. K. (1998). An electrophysio-logical measure of priming of visual word-form. Consciousness &Cognition, 7, 54–66.

Raij, T. (1999). Patterns of brain activity during visual imagery of letters.Journal of Cognitive Neuroscience, 11, 282–299.

Reber, P. J., Stark, C. E., & Squire, L. R. (1998). Contrasting corticalactivity associated with category memory and recognition memory.Learning & Memory, 5, 420–428.

Roediger, H. L. III, Weldon, M. S., & Challis, B. H. (1989). Explainingdissociations between implicit and explicit measures of retention: Aprocessing account. In H. L. Roediger, III & F. I. M. Craik (Eds.),Varieties of memory and consciousness: Essays in honor of EndelTulving (pp. 3–41). Hillsdale, NJ: Erlbaum.

Rugg, M. D. (1985). The effects of semantic priming and word repetitionon event-related potentials. Psychophysiology, 22, 642–647.

Rugg, M. D. (1990). Event-related brain potentials dissociate repetitioneffects of high- and low-frequency words. Memory & Cognition, 18,367–379.

Rugg,M. D. (1995). ERP studies of memory. InM.D. Rugg &M.G.H.Coles (Eds.), Electrophysiology of mind: Event-related brain potentialsand cognition (Vol. 25, pp. 133–170). Oxford, England: OxfordUniversity Press.

Schacter, D. L., & Buckner, R. L. (1998). On the relations amongpriming, conscious recollection, and intentional retrieval: Evidencefrom neuroimaging research. Neurobiology of Learning and Memory,70(1–2), 284–303.

Senkfor, A. J., Van Petten, C., & Kutas, M. (2002). Episodic actionmemory for real objects: An ERP investigation with Perform, Watch,and Imagine action encoding tasks versus a non-action encoding task.Journal of Cognitive Neuroscience, 14, 402–419.

Sternberg, S. (1966). High-speed scanning in human memory. Science,153, 652–654.

Talsma, D., Wijers, A. A., Klaver, P., & Mulder, G. (2001). Workingmemory processes show different degrees of lateralization: Evidencefrom event-related potentials. Psychophysiology, 38, 425–439.

Van Petten, C., & Senkfor, A. J. (1996). Memory for words and novelvisual patterns: Repetition, recognition, and encoding effects in theevent-related brain potential. Psychophysiology, 33, 491–506.

Walsh, V., Le Mare, C., Blaimire, A., & Cowey, A. (2000). Normaldiscrimination performance accompanied by priming deficits inmonkeys with V4 or TEO lesions. Neuroreport, 11, 1459–1462.

Wiggs, C. L., Weisberg, J., & Martin, A. (1999). Neural correlates ofsemantic and episodic memory retrieval. Neuropsychologia, 37,103–118.

Wijers, A. A., Mulder, G., Okita, T., & Mulder, L. J. (1989). Event-related potentials during memory search and selective attention toletter size and conjunctions of letter size and color. Psychophysiology,26, 529–547.

(Received August 30, 2000; Accepted December 31, 2002)


Multiple Visual Memory Phenomena In a Memory Search Task

Documents