Classification images of two right hemisphere patients: A window into the attentional mechanisms of spatial neglect

B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

ava i l ab l e a t www.sc i enced i rec t . com

www.e l sev i e r. com/ loca te /b ra in res

Research Report

Classification images of two right hemisphere patients: Awindow into the attentional mechanisms of spatial neglect

Steven Shimozakia,⁎, Alan Kingstoneb, Bettina Olkc, Robert Stowed, Miguel Ecksteina

aDepartment of Psychology, University of California, Santa Barbara, CA 93106, USAbDepartment of Psychology, University of British Columbia, Vancouver, British Columbia, CanadacSchool of Humanities and Social Sciences, International University Bremen, Bremen, GermanydDepartments of Psychiatry and Neurology, University of British Columbia School of Medicine and Riverview Hospital, Canada

A R T I C L E I N F O

⁎ Corresponding author. Fax: +1 805 893 4303.E-mail address: [email protected]: http://www.psych.ucsb.edu/~eckstein

0006-8993/$ – see front matter © 2006 Elsevidoi:10.1016/j.brainres.2006.01.033

A B S T R A C T

Article history:Accepted 9 January 2006Available online 21 February 2006

While spatial neglect most commonly occurs after right hemisphere lesions, damage todiverse areas within the right hemisphere may lead to neglect, possibly through differentmechanisms. To identify potentially different causes of neglect, the visual information used(the ‘perceptual template’) in a cueing task was estimated with a novel technique known as‘classification images’ for five normal observers and two male patients with right-hemisphere lesions and previous histories of spatial neglect (CM, age 85; HL, age 69).Observersmade a yes/no decision on the presence of a ‘White X’ checkerboard signal (1.5°) atone of two locations, with trial-to-trial stimulus noise added to the 9 checkerboard squares.Prior to the stimulus, a peripheral precue (140 ms) indicated the signal location with 80%validity. The cueing effects and estimated perceptual templates for the normal observersshowed no visual field differences. Consistent with previous studies of spatial neglect, bothpatients had difficulty with left (contralesional) signals when preceded by a right(ipsilesional) cue. Despite similar behavioral results, the patients' estimated perceptualtemplates in the left field suggested two different types of attentional deficits. For CM, theleft template matched the signal with left-sided cues but was opposite in sign to the signalwith right-sided cues, suggesting a severely disrupted selective attentional strategy. For HL,the left templates indicated a general uncertainty in localizing the signal regardless of thecue's field. In conclusion, the classification images suggested different underlyingmechanisms of neglect for these two patients with similar behavioral results and holdpromise in further elucidating the underlying attentional mechanisms of spatial neglect.

© 2006 Elsevier B.V. All rights reserved.

Keywords:Visual attentionSpatial neglectCueingClassification imageBayesian observer

1. Introduction

1.1. Spatial neglect and cueing

Spatial neglect can occur after a unilateral brain injury anddescribes a syndrome in which the patient ignores visual

(S. Shimozaki)./lab/steve5.html (S. Shim

er B.V. All rights reserved

information in the hemifield opposite to the side of the lesion(for a review, see Rafal, 1994; Lezak, 1995; Gazzaniga, 1998). It isnot a purely visual deficit as it can be dissociated from visualfield loss (hemianopia), and it is commonly assumed thathemineglect reflects an attentional deficit (although manyhave argued that hemineglect also reflects a deficit in the

ozaki).

.

27B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

representation of space, see Bisiach, 1993; Bisiach andLuzzatti, 1978). Aside from its clinical implications, manycognitive neuroscientists view neglect as an excellent modelfor examining the brainmechanisms that mediate attentionalorienting. While neglect may result from a brain injury to oneof many regions, the most common and most severe cases ofhemineglect result from brain injuries to the right parietallobe. This site coincides with a number of primate single-cellrecording studies (Mountcastle et al., 1975; Robinson et al.,1995; Colby et al., 1995, 1996; Andersen, 1995) and brainimaging studies on normal human observers (Darby et al.,1996; Corbetta et al., 1978; Courtney et al., 1996) suggestingthat a primary responsibility of the parietal cortex is therepresentation and integration of visuo-spatial information.Typically, neglect is most severe at the onset of injury, and,over the course of a year, the patient recovers to a chronic levelof deficit (which can approach normal levels of functionality).

A significant paradigm in the general study of attention isknown as the cueing task (Posner, 1980). In this task, observersdetect a target stimulus that could appear at one of two ormore locations. Before the stimulus appears, a precue appearsthat indicates the probable location of the forthcoming target.Trials in which the target appears at the cued location areknown as valid trials, while those in which the target appearsat an uncued location are known as invalid trials. There mayalso be a neutral cue trial type, in which no reliableinformation is given about where a target may appear. Thetypical finding for these different precue conditions is thatwith normal observers performance is fastest and/or mostaccurate when a target appears at a validly cued location,worst when a target appears at an invalid (uncued) location,and intermediate for neutral cue trials.

Fig. 1 – Trial types in the cued yes/no contrast discrimination tapresence of a high contrast checkerboard ‘white X’ (140 ms, eithtrials. A precue (2.5° square, 140 ms) indicated the signal location

Aside from studies on normal observers, the cueing taskhas been applied in both the diagnosis and study of neglect. Ina now classic investigation, Posner et al. (1984) found thathemineglect patients were severely impaired when the precueappeared in the ipsilesional (good) visual field and the targetappeared in the contralesional (bad) visual field. Thus,hemineglect patients, unlike normal observers, manifest asignificantly larger cueing effect when the precue appears inthe ipsilesional visual field than when it appears at thecontralesional visual field. From this study and subsequentinvestigations like it, Posner and colleagues developed atheory of attention in which the parietal cortex is responsiblefor disengaging attention from a selected spatial location.Hence, when the parietal cortex is lesioned, as it is for mostneglect patients, performance is severely compromised whenattention must be disengaged from an ipsilesional cue andshifted to a contralesional target. While this “disengagedeficit” provides an accurate qualitative characterization ofthe attentional deficit that characterizes neglect patients, itfails to provide a quantitative description of the deficit. Theaim of the present study is to begin to address thisshortcoming.

1.2. Description of cueing task

Fig. 1 depicts the cueing task used in the current study.Observers performed a yes/no contrast discrimination of a3 × 3 checkerboard pattern configured as a ‘white X’. Onhalf the trials, the signal appeared for 40 ms (normals) or140 ms (patients) at either one of two locations (left andright), with a 140 ms precue indicating the probablelocation of the signal with 80% validity. Observers had to

sk, right-sided cues only. Observers judged upon theer 2.5° left or right of central fixation) appearing on half thewith 80% validity on signal present trials.

28 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

judge if the signal appeared (and not where) during aparticular trial. Thus, there were three trial types, validsignal present (upper), invalid signal present (center), andsignal absent (lower), as depicted in Fig. 1. One might viewthis task as the equivalent to the original cueing taskperformed by Posner (1980), except measuring accuracy asopposed to reaction time. As depicted in Fig. 2, theluminances of each of the 9 squares comprising thecheckerboards were not constant but had a differentsample of random noise added for each trial. This wasnecessary to compute the classification images describedlater. The classification image technique developed byAhumada and Lovell (1971) gives a direct estimate of theperceptual filter (template) of an observer based on how anobserver's judgments correspond to the image noise that isadded to the stimulus.

1.3. A quantitative model of the cueing task

Recently, investigators have applied quantitative tools andcomputational modeling that have been successfully usedto study basic visual phenomena to investigate themechanisms mediating visual attention (see Dosher andLu, 2000a,b; Eckstein et al., 1997). For the Posner cueingtask, one starting point within this general approach is toconsider the theoretical framework of the ideal Bayesianobserver (Eckstein et al., 2002; Shimozaki et al., 2003, seeKersten et al., 2004 for a review). Ideal observer analysisdetermines the algorithm that achieves the best possibleperformance for a given task, and it is a potentiallypowerful technique to analyze performance in visualtasks generally (Barlow, 1978; Burgess et al., 1981; Kersten,1984; Pelli, 1985; Liu et al., 1995; Tjan et al., 1995; Bennettet al., 1999; Gold et al., 1999) and visual attention tasksspecifically (Eckstein et al., 2002; Shimozaki et al., 2003).First, ideal observers can be used as an absolute standardof performance that can be compared to human perfor-mance. Second, ideal observers can also serve as a startingpoint for quantitative modeling. As real observers, humanscannot match the performance of the ideal observer, andthe difference between human and ideal observers can bemodeled with respect to the ideal observer as various

Fig. 2 – The signal with and without noise. On each trial, arandom luminance value was added independently to eachsquare of the checkerboard (represented on the right); thiswas necessary to compute the classification images. Forvisibility, the signal in the figure has a higher contrast thanthat used in the experiment.

sources of inefficiency of human perceptual and decisionprocessing.

Fig. 3 presents a quantitative model of the Posner cueingtask based on the ideal observer known as the WeightedLikelihood model (Eckstein et al., 2002; Shimozaki et al., 2003;also see Appendix B). The responses of the WeightedLikelihood model (xc and xuc) are based on the match(correlation) between a template at each location and thestimuli at each of the two possible signal locations. From theseresponses, the model calculates the probability of signalpresence at each location (the likelihoods). These likelihoodsare then multiplied by separate scalar values, or weighted,with the weight for the cued location (wc) typically being largerthan that for the uncued location (wuc). Finally, the twoweighted likelihoods are added together and compared to acriterion to make a ‘yes’ or ‘no’ decision upon signal presence.

The Weighted Likelihood model is equivalent to the idealobserver under certain conditions (see Appendix B), and wewill focus on two of these conditions in this study. The firstprimary condition is that the weights (wc andwuc) are equal tothe cue validities (as the cue validities correspond to the priorprobabilities of the signal presence, given the cue location).Wecan assess this condition by comparing the ideal weights(which are wc = 0.80 and wuc = 0.20 in this experiment) to thebest estimate of the human observers' weights. Note that, forthe ideal observer, the differential weighting is the onlymechanism of attention, thus the ideal observer is sometimescalled a purely ‘selective’ model of attention. The secondprimary condition is that the templates at each location are anideal shape, which in this task is simply the checkerboardsignal. We assessed the template shapes in this studywith theclassification image technique.

As mentioned earlier, human observers (both normal andpatient) may be modeled within the framework of an idealobserver as losses of efficiency compared to the ideal observer.For the Weighted Likelihood model of the cueing task, theseinclude: inadequate weighting in the integration of informa-tion from cued and uncued location (the first condition above),templates that do not match the signal (the second conditionabove), noise internal to the observer, and uncertainty aboutthe signal location leading to multiple templates at differentlocations instead of one template (Barlow, 1978; Burgess et al.,1981; Pelli, 1985; Eckstein et al., 2002; Shimozaki et al., 2003).

1.4. Basics of the classification image technique

Recently, we have assessed a quantitative distinction betweenthesemechanisms using a technique known as “classificationimages” (Eckstein et al., 2002). The classification imagetechnique combines the noise samples in the visual stimuliacross particular trial outcomes (such as false alarms in a yes/no task) to estimate the spatial profile of the information usedin a visual task, or roughly speaking, the shape of theunderlying perceptual templates (Ahumada and Lovell,1971). It has been used successfully to study a number ofdifferent visual phenomena, including Vernier acuity (Ahu-mada, 1996; Beard and Ahumada, 1998) and other positiondiscrimination judgments (Levi and Klein, 2002; Li et al., 2004),depth (Neri et al., 1999), and Kaniza squares and other illusorycontours (Gold et al., 2000).

Fig. 3 – Schematic of theWeighted Likelihoodmodel in the cueing task. Themodel response of a valid signal present trial on theright is depicted. See Appendix B for a description of the model.

29B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

Furthermore, we have used classification images previous-ly to assess visual attention in normal observers (Eckstein etal., 2002, 2004) and have found that the technique can be usedto test predictions of different types of attentional mechan-isms that are otherwise indistinguishable by measuringbehavioral performance. Here, we propose to use the classi-fication image technique to identify the mechanisms mediat-ing the attentional loss in hemineglect patients as assessed bythe cueing paradigm. Specifically, we assess the attentionaldisruption mediating the larger cueing effect when the cueappears at the ipsilesional visual field.

Fig. 4 illustrates the calculation of the classification imagefor the simple cueing task used in this study. Assume that wehave selected those trials in which the cue only appeared onthe right (upper box). Also assume that the second linerepresents a sequence of stimuli for a number of trials in thebasic cueing task, with each trial presented with an indepen-

Fig. 4 – Calculating classification images for the cue validity taswhich the cue appears on the right for a hypothetical observer. Thpresented with added luminance noise in an actual experiment (sfields resulting in a false alarm (a ‘signal present’ response to a scalculated, one for the cued location and one for the uncued locatiofor trials in which the cue appears on the left.

dent sample of image noise (for presentation purposes, thesamples of noise are not shown). In the figure, a hypotheticalobserver's response to the trial appears under each stimulus.The first step in calculating classification images is selectingindividual trials by the type of trial, defined by both stimuluspresence and the observer's response. For example, the figuredepicts choosing those trials in which no signal appeared, butthe observer made an error and responded ‘yes’ (a ‘falsealarm’). Then, all the noise fields leading to a false alarmdecision are averaged to create the classification image.Intuitively, an error in a false alarm trial indicates that somepart of the noise field (which varies from trial to trial) led theobserver to believe the signal was present; in other words, theimage noise on that trial mimicked an aspect of the patternthat the observer considers to be the signal. Thus, theclassification image (for false alarms) is an average of thespatial information used by an observer to judge signal

k. The figure represents a series of 7 trials and responses ine stimuli are presented without noise for clarity but would beee Fig. 2). The classification image is themean of all the noiseignal absent stimulus). Two classification images may ben; also, there are two classification images (cued and uncued)

1 It should be noted that a lower amplitude classification imagein this case could have two causes. The first is an underweightingof the likelihoods, expressed as wc and wuc in Fig. 3. The second isthat the classification image might be subject to more noiseinherent to the observer (internal noise), as noted by Ahumada(2002).

30 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

presence, or, in other words, the ‘perceptual template’ of theobserver.

It should be noted that there are two separate classificationimages in Fig. 4, one for the valid-cued location (in this case,the right ipsilesional field) and one for the invalid-uncuedlocation (the left contralesional field). We may also calculateclassification images for trials in which the cue appeared onthe left for the two locations. Finally, it should be noted thatclassification images can be calculated from the otherresponse outcomes in this task other than the False Alarms(Correct Rejection, Valid Hit, Valid Miss, Invalid Hit, InvalidMiss) and that the classification images across these differentoutcomes may be combined into more general classificationimages (Beard and Ahumada, 1998; Ahumada, 2002; Murray etal., 2002). If we consider the classification images across thedifferent outcomes, a final division is whether the checker-board signal appeared at the location on that trial (signal) ornot (no-signal). The no-signal classification images wouldinclude trials in which the signal appeared in the otherlocation and trials in which the signal did not appear at all.Thus, for each location (left and right), we may define sixclassification images: cued, further divided into valid cued-signal and valid cued-no signal, and invalid uncued, furtherdivided into an invalid uncued-signal and invalid uncued-nosignal.

1.5. Possible classification images for different attentionaldisruption mechanisms in neglect patients

Fig. 5 gives some hypothetical classification images, the lefthalves of the figures giving the 2-dimensional classificationimages and the right halves of the figures give the sameclassification images presented as one-dimensional plots. Theplots present the nine values of the classification images inserial reading order, starting at the upper left and going left toright and top to down. Fig. 5A depicts the checkerboard signalitself, which, in the context of this experiment, is also the bestpossible template to maximize performance; intuitively, thistemplate uses all the information available and thus corre-sponds to the ideal observer template. Figs. 5B through Fdepict possible classification images for some different typesof attentional deficits. In the case of right hemisphere damage,these classification images would be expected for the locationin the left visual field. Fig. 5B depicts the case in which noclassification image is found or zero values are found for theentire checkerboard. Such a classification image indicates thatthe observer did not or could not use any of the informationpresented at that location. Fig. 5C depicts a classificationimage in which the observer uses only the central square ofthe checkerboard. Fig. 5D presents the special case of objectneglect, in which the observer does not use information fromthe left side of the checkerboard (e.g., Behrmann and Tipper,1994; Tipper and Behrmann, 1996). In this special case,potentially, this pattern of neglect might be expected fromeither the left or right visual field.

In the cases represented in Figs. 5B through D, the observerdoes not use all or part of the information available.Previously, we have characterized this type of deficit as theobserver having a template or filter of the incorrect shape (ascompared to an ideal observer). The last two cases presented

in Figs. 5E and F represent a different type of deficit, one inwhich the observer uses a template of the correct shape, but ofthe wrong value or amplitude. The ideal observer can bedescribed as having two components, a template of the correctshape and a correct strategy of the use of the information inthe final decision. This strategy is expressed as the weightgiven to the information at each location, which is repre-sented as the amplitude of the template (relative to theamplitude of the template at the other location). Roughly, itrepresents how much importance to give the information ateach location. Fig. 5E represents a case in which not enoughweight is given to the information at the cued location.1 Fig. 5Frepresents an extreme case in which an opposite weighting isgiven to the information at the cued location. This last casedescribes a situation in which the observer takes informationin favor of target presence as evidence against target presence.

We assessed the behavioral performance (accuracy) andclassification images of 5 normal and 2 patient observers inthe cueing task described above. Both patients had suffered aright hemisphere stroke at least 1 year prior to this study;these injuries resulted in neglect for both patients. At the timeof this study, assessment by standard neuropsychologicalexamination indicated that both patients' neglect had im-proved since their injuries, with one patient testing as fullyrecovered. Despite the neuropsychological results, this studyfound clear attentional deficits for both patients. In addition,even though the two patients had similar behavioral effects,the classification images clearly indicate that different under-lying attentional mechanisms are responsible for the patients'performances.

2. Results and discussion

Fig. 6 depicts a schematic of all conditions in the simple cueingtask, divided by cue location and visual field, with each celldepicting only the precue display and the stimulus display.Table 1 presents the hit and false alarm rates for the observers,again divided by cue location and visual field. Note thatsignals on the left were valid trials for left cues and invalidtrials for right cues, and vice versa for signals on the right. Forboth Table 1 (hit and false alarm rates) and Table 2 (cueingeffects), the means and standard errors for each measure arecomputed over the values calculated for each individualsession. All observers were considerably above chance inthis task (chance performance would be indicated by equal hitand false alarm rates), but they were not close to perfectperformance (hit rates at 100% and false alarm rates at 0%).

We defined cueing effects for this task as the differencebetween the hit rate for validly cued signals minus the hit ratefor invalidly cued signals (Eckstein et al., 2002; Shimozaki etal., 2003). The cueing effects were also divided by cue side, sothat the cueing effect for left cues was defined as hit rate (cell

Fig. 5 – Hypothesized classification images. For each figure, the left shows the 2-dimensional classification image and the rightshows a 1-dimensional representation of the same classification image, in which each of the 9 squares of the checkerboard arerepresented in ‘reading’ order, going from left to right and from up to down, starting at the upper left. (A) The ideal observer,which is the signal itself. The ideal observer is represented as the dashed line in parts B through F. (B) No classification image(suggesting that the observer does not use any information from that location). (C) The observer uses only the center point(center only). (D) The observer cannot use information from the left side of the stimulus (local neglect). (E) The observer has theideal observer template shape but does not give the information from that location enough weight (under-weighting). (F) Theobserver has the ideal observer template shape but gives the information from that location a weight opposite to theappropriate weighting (opposite weighting).

31B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

a) − the hit rate (cell b), and the cueing effect for right cues wasdefined as hit rate (cell e) − the hit rate (cell d). Table 2 and Fig.7 present the cueing effects by cue side for all observers(Column 1—cueing effect, Column 2—standard error acrosssessions). Columns 3, 4, and 5 present the results of tests ofsignificant (nonzero) cueing effects, assessed by single-samplet statistics. Columns 6, 7, and 8 present results of the fits of theobserver results to the Weighted Likelihood model (seeAppendix B). The three parameters from the fits were theweight for the cued location (wc), a measure of sensitivity, ordifficulty of the task for the observer (d′), and log(crit), ameasure of decision bias. The ideal observer has a weightequal to the cue validity (0.80); therefore, humans may beassessed by comparing their fitted weights to the ideal weight.This assessment is not absolute performance (which would bepercent correct or d′), but rather the use of the informationprovided by the cue and appropriate use of the combination ofinformation across the cued and uncued locations.

Two observers showed no (PC left cues, RP) or negative(PC, right cues) cueing effects, indicating a general lack ofthe use of the precue in the task. We (Shimozaki andEckstein, 2003) have found that normal observers (espe-cially naive observers, such as PC and RP) can vary in thesize of their cueing effects in these types of cueing tasks.The results for the other three normal observers generallyindicated modest overall cueing effects, with the exceptionof EM having a near-zero cueing effect for right-sidedcues. Furthermore, the fits to the Weighted Likelihoodmodel showed that all cueing effects for the normalobservers were less than or equal to the cueing effectsexpected from the ideal observer (as indicated by allweights being less than or equal to 0.80). Finally, ingeneral, the normal observers also showed a tendency forslightly larger cueing effects for left-sided cues. Analysesby two-sample t tests on left- and right-sided cueingeffects sampled by each individual session found that this

Fig. 6 – Schematic of all trial types in the cueing task, divided by visual field, with each cell depicting only the precue displayand the stimulus display.

32 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

tendency was not significant for the normal observers,except for PC (t(18) = 3.548, P = 0.0023).

From previous research (Posner et al., 1984), neglectpatients should suffer most severely when detecting a signalin the contralesional left field after receiving an invalidipsilesional right cue (Fig. 1, invalid signal present trial; Fig.6, cell d). Therefore, for the neglect patients, it is anticipatedthat the cueing effect with ipsilesional (right) cues (hit rate(cell e) − hit rate (cell d)) will be large. Conversely, the cueingeffect with contralesional (left) cues (hit rate (cell a) − hit rate(cell b)) should be comparable to normal observers. Asexpected, the patients did have a large cueing effect withright-sided cues, and the patients' cueing effects for left-sidedcues were smaller and comparable to the normal observers'cueing effects. Analyses on these cueing effects by two-sample t test analyses, sampled by calculating the cueingeffects for each individual session, found a significantdifference between left- and right-sided cues for both patients(HL, t(27) = 2.822, P = 0.0088; CM, t(55) = 2.184, P = 0.0333), unlikethe normal observers. Furthermore, the fits for the patientswith right-sided cues to the Weighted Likelihood modelindicated a greater weighting of the cued information thanthe ideal observer (HL = 0.98, CM = 0.97), while the same fits forleft-sided cues indicated that weights were slightly less thanthe ideal observer (HL = 0.72; CM = 0.68).

While some of the normal observerswere not optimal (withtwonormalobservershavingnoornegative cueing effects), thesuboptimal pattern of greater weighting of cued informationshown by the patients (with right-sided cues) was not seenwith any of the normal observers. Additionally, the patients'larger cueing effect for right-sided cues is opposite the generaltendency of the normal observers, who generally had slightlylarger cueing effects with left-sided cues. Finally, the similarsize of cueing effects for the patientswith left-sided cues to thenormal observers (for both left- and right-sided cues) suggeststhat the comparison between the patients and normalobservers in this study was reasonable.

Figs. 8–12 depict the normal observers' classificationimages, and Figs. 13 and 14 depict the patients' classificationimages, calculated from about 2500 trials for the normalobservers and 820 (HL) and 1641 (CM) total trials for thepatients. As mentioned earlier, classification images wereseparated by location (left or right), cue presence, and signalpresence. Each square of the checkerboard pattern is pre-sented along the x axis in ‘reading’ order, top left to bottomright. The ideal observer template for this task (the signalitself, Fig. 5A) is represented in Figs. 8–14 as the alternatingdashed line, consistent with the alternating white and blacksquares. For these figures, the amplitudes of the ideal observertemplate were chosen by the best χ2 (chi-square) fit to thehuman observers' templates (solid lines). We also assessedthe shape of the classification images by measuring thecorrelations of the classification images with the idealobserver template (the signal). Table 3a gives the correlationsof the normal observers' classification images with the idealobserver template, and Table 3b gives the correlations of thepatients' classification images with the ideal observertemplate.

As expected, in general, the normal observers' estimatedperceptual template shapes were well-matched to the idealobserver's template (the signal) across the types of classifica-tion images and showed no hemispheric differences. The oneexception is in the uncued-no signal classification imagesacross the normal observers. There were near-zero uncued-nosignal classification images on both sides for TB, and on theright side for EM, leading to low correlations with the idealobserver for these classification images. The other observersalso showed a decrease in the amplitude of the uncued-nosignal classification images, but not to thedegree of TBandEM.

The classification images of both patients indicated aneffect in the (left) contralesional field. Furthermore, despite thesimilarity of the behavioral (cueing) results between the twopatients, the pattern of effects shown by the classificationimages differed for the two patients. For HL, the classification

Table 1 – Hit and false alarm rates for the observers bycue side

Signal on left Signal on right Signal absent

Hitrate

SE hitrate

Hitrate

SE hitrate

FArate

SE FArate

Normal observersEM Left

cue0.775 0.017 0.607 0.054 0.309 0.016

Rightcue

0.677 0.052 0.720 0.017 0.303 0.019

PC Leftcue

0.804 0.018 0.789 0.037 0.489 0.030

Rightcue

0.882 0.028 0.708 0.020 0.457 0.030

RP Leftcue

0.699 0.020 0.724 0.040 0.306 0.033

Rightcue

0.670 0.044 0.661 0.021 0.270 0.034

SS Leftcue

0.783 0.028 0.604 0.063 0.315 0.028

Rightcue

0.616 0.034 0.737 0.021 0.220 0.014

TB Leftcue

0.734 0.022 0.553 0.062 0.217 0.025

Rightcue

0.586 0.051 0.756 0.027 0.231 0.026

PatientsHL Left

cue0.849 0.049 0.658 0.104 0.282 0.041

Rightcue

0.217 0.091 0.792 0.048 0.177 0.032

CM Leftcue

0.716 0.028 0.655 0.079 0.281 0.028

Rightcue

0.608 0.055 0.915 0.020 0.433 0.036

Hits = correct ‘signal present’ response in signal present trial. Falsealarm (FA) = incorrect ‘signal present’ response in signal absenttrial. Error bars are standard errors of cueing effects, over hit andfalse alarm rates calculated for each individual session.

33B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

images in the left hemifield were uncorrelated with the idealobserver when the signal was not present. On the other hand,regardless of whether the left or right side was cued, theclassification images were positively correlated with the idealobserver when the signal was present. This pattern wasindicated by a significant difference between the left signaland no-signal classification images, collapsed over cue pres-ence (2-sampleHotellingT2 = 18.492, F(9,386) = 2.013,P=0.0368).Ahumada (2002) found that this pattern of results is consistentwith a specific difficulty for stimuli with phase information,such as grating patterns or the checkerboard, such that theobserver was unable to determine the exact location of thepotential signal. This type of difficulty is called locationuncertainty (Pelli, 1985) and may be modeled within theframework of models like the Weighted Likelihood model asthe placement of multiple templates (instead of the onetemplate) in slightly different locations. In this framework,when the signal is present, only the template matched to thesignal location responds well, leading to a classification imagealso well-matched to the signal. However, when the signal isabsent, the use of the multiple templates leads to a classifica-

tion image that loses (or ‘blurs’) the phase properties of thesignal, giving a flat classification image.

For CM in the left (contralesional) field, the classificationimageswere positively correlatedwith the signal, exceptwhentherewas neither a signal nor a cue at the ipsilesional location.In this last case, the classification image was negativelycorrelated with the signal, suggesting that CM took evidenceconsistent with signal presence as evidence against signalpresence. In agreement with this pattern, there was asignificant difference between the left cued vs. uncued no-signal classification images (2-sample Hotelling T2 = 20.270, F(9,1160) = 2.237, P = 0.0178). Clearly, this represents a severelysuboptimal attentional strategy for CM, and it also suggests adifferent type of attentional loss than HL. Consistent with thissuggested difference between HL and CM, significant differ-ences were found between HL and CM for both the left cuedno-signal classification images (2-sampleHotellingT2 = 17.837,F(9,718) = 1.960, P = 0.0413) and the left uncued no-signalclassification images (2-sample Hotelling T2 = 17.445, F(9,1035) = 1.923, P = 0.0453).

Aside from the contralesional (left) classification images,the classification images in the ipsilesional (right) field for thetwo patients also suggested a potential difficulty in usinginformation in the right field. For HL, no classification imagewas found in the uncued trials overall, and, for CM, noclassification image was found in the uncued-no signalcondition. These results for the ipsilesional field seem to beconsistent with the suggestion of several authors (Làdavas etal., 1990; Olk et al., 2002; Rusconi et al., 2002;Wright et al., 2005)that the neglect syndrome is not restricted to hemifield andmay instead reflect a more general graded spatial deficitacross hemifields.

The last analyses assessed the relationship of the observedclassification images with the observed behavioral results. Inother words, can the classification images be used to predictthe hit and false alarm rates of the observers? As the observershad no a priori information regarding signal presence, werestricted our analysis to the classification images collapsedover signal presence/absence. These classification imagesmay be found at the tops of Figs. 8–14 and denoted as ‘All’.The analyses of cueing effects for left-sided cues included theCued Left (top left, Figs. 8–14A) classification images and theUncued Right (top right, Figs. 8–14B) classification images, andthe analyses of cueing effects for right-sided cues included theCued Right (top right, Figs. 8–14A) classification images andthe Uncued Left (top left, Figs. 8–14B) classification images. Inthe remaining figures, analyses for left-sided cues using theCued Left and Uncued Right classification images are denoted‘Cued L Unc R’ and analyses for right-sided cues using theCued Right and Uncued Left classification images are denoted‘Cued L Unc R’.

Fig. 15 depicts the correlations of the Cued Left/UncuedRight and the Cued Right/Uncued Left classification imageswith the ideal observer template (the signal itself); thesevalues are also presented in Table 3 under ‘Cued and Uncued’.We may use these correlations as an assessment of how well(or how efficient) the human observer could potentially usethe information at the cued and uncued locations, with highpositive correlations indicating good matches to the idealobserver. Fig. 16 depicts the correlations of the Cued Left with

Table 2 – Cueing effects for the observers by cue side

Cue Cueing SE T df P value wc d′ log(crit)

EM Left 0.169 0.057 2.977 9 0.0155 0.78 1.36 −0.12Right 0.044 0.057 0.771 9 0.4603 0.57 1.37 −0.04

PC Left 0.015 0.041 0.366 9 0.7228 0.53 1.12 −0.36Right −0.174 0.034 −5.118 9 0.0006 0.13 1.37 −0.62

RP Left −0.024 0.045 −0.533 9 0.6069 0.44 1.37 −0.02Right −0.009 0.049 −0.184 9 0.8581 0.50 1.30 0.06

SS Left 0.179 0.069 2.599 9 0.0288 0.82 1.37 −0.14Right 0.122 0.043 2.839 9 0.0194 0.70 1.55 0.10

TB Left 0.181 0.056 3.225 9 0.0104 0.79 1.49 0.12Right 0.170 0.066 2.594 9 0.0290 0.77 1.53 0.06

PatientsHL Left 0.225 0.091 2.488 13 0.0272 0.68 1.34 0.02

Right 0.575 0.085 6.793 14 0.0000 0.97 1.61 −0.84CM Left 0.088 0.081 1.078 26 0.2911 0.72 1.66 −0.22

Right 0.304 0.059 5.161 29 0.0000 0.98 1.67 −0.04

Column 1: cueing effect = hit rate for valid signal location − hit rate for invalid signal location for all cues on the same side. With respect to Fig. 6:left side cues, cueing = hit rate for cell e − hit rate for cell d; right side cues, cueing = hit rate for cell a − hit rate for cell b. Column 2: standard errorsof cueing effects, from cueing effects calculated for each individual session. Columns 3, 4, and 5: tests of significant (nonzero) cueing effects,single-sample t tests, degrees of freedom, and P values. Columns 6, 7, and 8: results of fits to theWeighted Likelihoodmodel,wc = weight for cuedlocation, d′ = measure of sensitivity/difficulty, crit = decision criterion. See Appendix B for details.

34 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

the Uncued Right classification images, and the correlations ofthe Cued Right with the Uncued Left classification images.These correlations indicate the similarity of the potential useof information at the cued and uncued locations.

Considering the correlations in both, Figs. 15 and 16 suggesttwo distinct patterns among the normal observers. The cuedand uncued classification images of PC, RP, and SS haveuniformly high correlations with the ideal observer (Fig. 15)and each other (Fig. 16). Thus, these three observers may becharacterized as having nearly ideal templates at both thecued and uncued locations for both left- and right-sided cues.

Fig. 7 –Cueing effects for all observers divided by cue side, meastandard errors over cueing effects calculated for each individualleft-sided vs. right-sided cues (P b 0.05).

For EM and TB, Fig. 15 indicates that the cued classificationimages had high correlations with the ideal observer, whilethe uncued classification images had lower correlations withthe ideal observer on the right for EM and on both sides for TB.Fig. 16 also shows that the Cued/opposite Uncued correlationswere correspondingly low in these conditions for these twoobservers. The pattern of correlations for the patients shownin Figs. 15 and 16 indicates distinct differences from thenormal observers and from each other. For HL, Fig. 15 showsthat the Cued Right classification image correlation with theideal observer was high and similar to the normal observers',

sured as valid hit rate − invalid hit rate. The error bars aresession. * — significant difference between cueing effects for

Fig. 8 – Classification images for EM, a normal observer. The classification images are presented as 1-dimensional plots,with each of the 9 squares of the checkerboard represented in ‘reading’ order, going from left to right and from up to down,starting at the upper left. The signal (ideal observer) is represented as the dashed line, with the amplitudes matched to the bestchi-square fit to the unsigned amplitudes of the human observers' templates (solid lines). The symbols correspond to the cue(squares), signal (X), and classification image (shaded region) locations relative to the fixation point for trials used for eachclassification image. The cell letters correspond to the cells in Fig. 6. (A) Cued (cue appeared on same side), cued-signal, cued-nosignal. (B) Uncued (cue appeared on same side), uncued-signal, uncued-no signal.

35B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

and his Cued Left classification image correlation with theideal observer was positive but somewhat lower than the CuedRight correlation. The uncued classification image correla-tions with the ideal observer, however, were near-zero forboth sides. As expected from these correlations with the idealobserver, Fig. 16 shows that both Cued/opposite Uncuedcorrelations were near zero for HL. For CM, Fig. 15 indicatesthat his cued classification image correlations with the idealobserver on both sides were high and similar to the normalobservers. The Uncued Right correlation with the idealobserver was low, while the Uncued Left correlation with theideal observer was negative (indicating the opposite weightingof information by CM). Correspondingly, Fig. 16 indicates thatCM's Cued Left/Uncued Right correlation was low and that hisCued Right/Uncued Left correlation was negative.

Tables 4–6 and Figs. 17–19 summarize the results ofsimulations of three models to predict human performancefrom the classification images. Cueing effects may be gener-ated within the constraints of the Weighted Likelihood modelin two ways (excluding internal noise added to the uncuedlocation1). The first assumes that that there is a differentialweighting of the cued and uncued locations, irrespective of the

templates at the cued and uncued locations. The idealobserver exhibits a cueing effect in this way as it weights thelikelihoods by cue validity and has the same (optimal)template at the cued and uncued locations (Eckstein et al.,2002; Shimozaki et al., 2003). The second way is a difference inthe template shapes such that the uncued template is a poorermatch to the ideal observer than the cued template, irrespec-tive of the weights. The goal of the simulations was to assessthe relative importance of weighting changes and any shapedifferences of the cued and uncued classification images inpredicting the observers' results.

The models were fit to the valid and invalid hit rates andthe false alarm rates of the observer, and these results arepresented in Tables 4–6. However, the fits to the false alarmrates were generally quite good, so that the overall fits may bedescribed as a fit to the cueing effect (valid hit rate − invalid hitrate), which are presented in Figs. 17–19. Table 4 and Fig. 17give the results of the simulations using the normalizedclassification images and equal weighting (wc = wuc = 0.50). Inthis case, if the classification images are the same at the twolocations, the model processes the cued and uncued locationsthe same as well, and no cueing effect is found. Thus, within

Fig. 9 – Classification images for PC, a normal observer. See Fig. 10 for details.

Fig. 10 – Classification images for RP, a normal observer. See Fig. 8 for details.

36 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

Fig. 11 – Classification images for SS, a normal observer. See Fig. 8 for details.

Fig. 12 – Classification images for TB, a normal observer. See Fig. 8 for details.

37B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

Fig. 13 – Classification images for HL, a patient observer. See Fig. 8 for details.

Fig. 14 – Classification images for CM, a patient observer. See Fig. 8 for details.

38 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

Table 3 – Correlations of normal observer and patientclassification images with the ideal observer

Left location Right location

Correlation SE Correlation SE

(a) Normal observersEM Cued 0.9566 0.0022 0.9106 0.0048

Signal 0.9008 0.0040 0.9054 0.0064No signal 0.9670 0.0036 0.8973 0.0074Uncued 0.8459 0.0115 0.5964 0.0193Signal 0.8387 0.0115 0.7341 0.0146No signal 0.7316 0.0181 0.3731 0.0244

PC Cued 0.9064 0.0052 0.8942 0.0060Signal 0.8551 0.0094 0.9008 0.0054No signal 0.7911 0.0077 0.7302 0.0155Uncued 0.8855 0.0059 0.7732 0.0093Signal 0.8574 0.0087 0.8556 0.0125No signal 0.8585 0.0096 0.7158 0.0138

RP Cued 0.9372 0.0033 0.9206 0.0050Signal 0.9617 0.0025 0.9625 0.0036No signal 0.7963 0.0152 0.7786 0.0148Uncued 0.8699 0.0071 0.9381 0.0062Signal 0.7121 0.0129 0.9031 0.0103No signal 0.8181 0.0107 0.9186 0.0109

SS Cued 0.9624 0.0030 0.9683 0.0020Signal 0.9266 0.0048 0.9477 0.0023No signal 0.9317 0.0055 0.8822 0.0060Uncued 0.8944 0.0096 0.9315 0.0067Signal 0.8737 0.0121 0.9072 0.0067No signal 0.8129 0.0163 0.8716 0.0139

TB Cued 0.9723 0.0025 0.9265 0.0044Signal 0.9098 0.0050 0.9003 0.0062No signal 0.9563 0.0050 0.8139 0.0097Uncued 0.2599 0.0239 0.5775 0.0185Signal 0.8489 0.0116 0.7368 0.0142No signal −0.1277 0.0283 0.3011 0.0241

(b) PatientsHL Cued 0.4914 0.0096 0.9215 0.0049

Signal 0.6972 0.0129 0.6324 0.0147No signal 0.1406 0.0148 0.8652 0.0054Uncued 0.0174 0.0236 0.0349 0.0269Signal 0.6616 0.0212 −0.1574 0.0158No signal −0.1681 0.0218 0.1272 0.0300

CM Cued 0.8326 0.0162 0.9062 0.0126Signal 0.8154 0.0155 0.6291 0.0200No signal 0.6833 0.0178 0.9457 0.0154Uncued −0.4114 0.0229 0.2443 0.0218Signal 0.7036 0.0193 0.7279 0.0222No signal −0.5311 0.0231 −0.0655 0.0266

Standard deviations were estimated by Monte Carlo resampling ofassumed independent Gaussian-distributed values for the classifi-cation images.

39B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

the constraints of the model, any cueing effects found in thissimulation reflect differences in the shapes of the classifica-tion images at the cued and uncued locations relative to thesignal, such that the template at the uncued location is apoorer match (assuming a positive cueing effect). Table 5 andFig. 18 give the results of the simulations using the idealtemplates at the cued and uncued locations and the optimalweighting (wc = 0.80). As there is no difference in thetemplates, this simulation estimated the cueing effect solelybased on the differential weighting of the cued and uncuedlocations and how well the observers' cueing effects matched

those predictions. Table 6 and Fig. 19 give the results ofsimulations using the normalized classification images andthe optimal weighting (wc = 0.80). In this case, both theweighting and any difference in shape in the classificationimagesmay contribute to the cueing effect. If the classificationimages were approximately the same at the cued and uncuedlocations, then the predicted cueing effects for this simulationshould be similar predicted cueing effects with the idealtemplates and the optimal weighting (wc = 0.80). Thus, withthe combined results across the simulations, one may assessthe relative importance of differential weighting and differ-ential shape at the cued and uncued locations in predictingthe observed cueing effects.

Table 7 summarizes the observed cueing effects, thecorrelations between the cued classification image and theuncued classification on the opposite side, and the P valuesfrom the chi-square goodness of fits across the simulations.For the normal observers with cueing effects near zero (EM,right cues; PC, left cues; RP, both cues), the 0.50 model hadgenerally good fits, while the other models with the 0.80weighting (with the observed classification images and theideal templates) had worse fits that were equally poor. This isan expected result from the lack of a cueing effect and also therelatively high correlations of the cued and uncued classifi-cation images with the ideal observer. For SS, his classifica-tion images had relatively high correlations with each otherand the ideal observer, and he showed a cueing effect. In hiscase, the 0.50 model fit poorly, as it should. The 0.80 modelshad better fits to SS's results for left-sided cues and slightlyworse fits for right-sided cues, and the fits were. These resultsare consistent with the model of cueing effects as differentialweighting, with the left-cue weight close to the optimal 0.80and the right-cue weight between 0.50 and 0.80. A separatesimulation with SS's classification images in which theweight was free to vary found that the best-fitting weightwas 0.65 (SNR = 1.665, log(crit) = 0.10, χ2(0) = 0.004). Asuboptimal weighting also describes PC's negative cueingeffect for right-sided cues, as her correlations of the CuedRight and Uncued Left classification images with the idealobserver and each other were relatively high and the fits tothe three models were poor, as expected. A simulation of herclassification images with the weights free to vary found thatPC's negative cueing effect corresponded best to a weight of0.13 (or 0.87 on the uncued side) (SNR = 1.530, log(crit) = −0.62,χ2(0) = 0.004).

EM and TB both had moderate cueing effects on the leftside, which were fit relatively well assuming 0.80 weights andthe ideal templates. Unlike the previous examples, lowercorrelations to the ideal template were found for the UncuedRight locations and therefore also for the Cued Left/UncuedRight correlations. Correspondingly, their cueing effects werenot fit well with the 0.80 weighting and the observedclassification images. Furthermore, the cueing effects werefit relatively well assuming equal weighting of the cued anduncued locations and the observed classification images.These results describe cueing effects that were determinedby a difference in shape only. TB's moderate cueing effect forright-sided cues were fit well by the 0.80 weighting and theideal templates. She had a low correlation for the Uncued Leftclassification imagewith the ideal template andwith the Cued

Fig. 15 – Correlations of the ideal observer template (the signal) with the classification images, collapsed over signalpresence. Cued L Unc R columns = Cued Left (top left, Figs. 8–14A), Uncued Right (top right, Figs. 8–14B). Cued R Unc Lcolumns = Cued Right (top right, Figs. 8–14A), Uncued Left (top left, Figs. 8–14B). Cued = filled squares, uncued = open circles.Error bars indicate the estimated standard errors from Monte Carlo resampling (see text for details).

Fig. 16 – Correlations of the cued and uncued classification images collapsed over signal presence. Cued L Unc Rcolumns = Cued Left (top left, Figs. 8–14A) correlated with Uncued Right (top right, Figs. 8–14B). Cued R Unc Lcolumns = Cued Right (top right, Figs. 8–14A) correlated with Uncued Left (top left, Figs. 8–14B). Cued = filled squares,uncued = open circles. Error bars indicate the estimated standard errors from Monte Carlo resampling (see text for details).

40 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

Table 4 – Weighted Likelihood model predictions of behavioral performance with classification images as templates, cuedweight (wc) = 0.50, by cue side

Cue side Predicted Hv Predicted Hi Predicted FA Predicted cueing SNR log(crit) χ2(1) P value

Normal observersEM Left 0.784 0.600 0.313 0.184 1.650 −0.06 0.110 0.7401

Right 0.723 0.693 0.300 0.031 1.530 −0.04 0.019 0.8904PC Left 0.807 0.765 0.488 0.042 1.305 −0.34 0.098 0.7542

Right 0.743 0.742 0.458 0.002 1.125 −0.22 4.521 0.0335RP Left 0.709 0.710 0.307 −0.001 1.425 −0.04 0.102 0.7494

Right 0.672 0.645 0.271 0.026 1.440 0.08 0.199 0.6555SS Left 0.746 0.731 0.307 0.016 1.500 −0.10 4.907 0.0267

Right 0.720 0.686 0.219 0.034 1.725 0.12 1.128 0.2882TB Left 0.750 0.505 0.219 0.245 1.785 0.16 0.468 0.4939

Right 0.860 0.385 0.253 0.475 2.160 −0.08 18.247 b0.0001

PatientsHL Left 1.000 0.714 0.281 0.286 32.820 −0.68 4.085 0.0433

Right 0.808 0.224 0.224 0.584 1.755 −0.04 1.411 0.2349CM Left 0.792 0.392 0.310 0.400 1.740 −0.06 11.492 0.0007

Right 0.884 0.646 0.395 0.238 2.040 −0.32 0.423 0.5154

Performance with left cues was predicted by using the Cued Left (top left, Figs. 8–14A) and Uncued Right (top right, Figs. 8–14B) classificationimages. Performance with right cues was predicted by using the Cued Right (top right, Figs. 8–14A) and Uncued Left (top left, Figs. 8–14B)classification images. Predicted Hv = predicted valid hit rate. Predicted Hi = predicted invalid hit rate. Predicted FA = predicted false alarmrate. Predicted cueing = predicted cueing effect (predicted valid hit rate − predicted invalid hit rate). SNR = signal-to-noise ratio of stimuli(free parameter). Log(crit) = natural logarithm of the criterion (free parameter). χ2(1), P value = chi-square goodness of fit statistic and P valueof the predicted and actual hit and false alarm rates, df = 1.

41B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

Right classification image, such that both the 0.50 and the 0.80weighting with her observed classification images overpre-dicted her cueing effect.

The primary result for the patients was a large cueing effectwith right-sided cues. For HL, his right-sided cueing effectcould be fit with the observed classification images and theequal (0.50) weighting, suggesting that his cueing effect couldbe described by the difference in shape. For CM, note that thenegative correlation of theUncued Left classification image ledto a negative weighting of the uncued location. His right-sidedcueing effect was slightly underpredicted with his observed

Table 5 – Weighted Likelihood model predictions of behaviora(wc) = 0.80, by cue side

Cue side Predicted Hv Predicted Hi Predicted FA

Normal observersEM Left 0.844 0.497 0.321

Right 0.788 0.573 0.299PC Left 0.861 0.685 0.493

Right 0.812 0.656 0.464RP Left 0.767 0.580 0.308

Right 0.731 0.508 0.276SS Left 0.789 0.589 0.309

Right 0.776 0.546 0.222TB Left 0.839 0.422 0.230

Right 0.983 0.527 0.236

PatientsHL Left 1.000 0.710 0.281

Right 0.804 0.222 0.222CM Left 0.728 0.357 0.346

Right 0.931 0.590 0.402

See Table 4 caption for details.

classification images and a weighting of 0.50 and slightlyoverpredicted with a weighting of 0.80. Thus, the difference inshape of the classification images was relatively successful incharacterizing the general attentional effect shown by neglectpatients.With left-sided cues, the low correlations for the rightuncued classification images led to poor fits of their modestcueing effects (similar to TB's right-sided cueing effects). The0.80 model with the ideal templates predicted HL's cueingeffect and overpredicted CM's cueing effect.

Except for TB right-sided cueing effects, it appeared thatassessing the shape and potential weights of the observed

l performance with ideal observer templates, cued weight

Predicted cueing SNR log(crit) χ2(1) P value

0.346 1.635 −0.28 6.397 0.01140.215 1.545 −0.12 6.314 0.01200.176 1.305 −0.48 3.784 0.05170.155 1.155 −0.32 17.776 b0.00010.187 1.395 −0.10 7.428 0.00640.223 1.410 0.02 9.321 0.00230.200 1.425 −0.14 0.009 0.92440.231 1.680 0.04 2.020 0.15520.416 1.875 −0.12 11.192 b0.00010.456 3.645 −1.04 27.551 b0.0001

0.290 32.025 −1.60 4.087 0.04320.581 1.740 −0.06 1.419 0.23360.372 1.200 −0.08 17.453 b0.00010.342 2.070 −0.76 0.210 0.6468

Table 6 – Weighted Likelihood model predictions of behavioral performance with classification images as templates, cuedweight (wc) = 0.80, by cue side

Cue side Predicted Hv Predicted Hi Predicted FA Predicted cueing SNR log(crit) χ2(1) P value

Normal observersEM Left 0.777 0.590 0.311 0.187 1.335 −0.12 0.123 0.7258

Right 0.770 0.586 0.298 0.184 1.365 −0.10 4.239 0.0395PC Left 0.838 0.702 0.488 0.136 1.110 −0.42 1.976 0.1598

Right 0.822 0.677 0.461 0.146 1.110 −0.36 17.043 b0.0001RP Left 0.765 0.577 0.308 0.188 1.320 −0.10 7.462 0.0063

Right 0.714 0.510 0.272 0.204 1.245 0.04 7.719 0.0055SS Left 0.781 0.594 0.310 0.187 1.350 −0.14 0.021 0.8848

Right 0.768 0.569 0.221 0.200 1.590 0.06 1.045 0.3067TB Left 0.739 0.530 0.215 0.208 1.515 0.14 0.036 0.8495

Right 0.760 0.561 0.230 0.198 1.545 0.06 0.096 0.7567

PatientsHL Left 0.848 0.698 0.283 0.150 1.725 −0.30 0.073 0.7870

Right 0.736 0.526 0.200 0.210 1.545 0.16 4.997 0.0254CM Left 0.752 0.555 0.282 0.198 1.335 −0.04 0.846 0.3577

Right 0.860 0.725 0.397 0.134 1.455 −0.48 2.476 0.1156

See Table 4 caption for details.

42 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

classification images could describe the cueing effects (orlack thereof) of the normal observers. A difference in shapein the classification images also could predict the largeright-sided cueing effects of the patients. In the cases of

Fig. 17 – Predicted cueing effects from the Weighted Likelihoodwc = 0.50. Cued L Unc R columns = cueing effects for left-sided cu(top right, Figs. 8–14B) classification images. Cued R Unc L columRight (top right, Figs. 8–14A) and Uncued Left (top left, Figs. 8–14B)as Fig. 8). Open circles—cue weight (wc) = 0.50.

modest cueing effects and large differences in shape (lowcorrelations of the uncued classification image with theideal template and the cued classification image), predic-tions were poor as the models overpredicted the cueing

model (Fig. 3) using the classification images as templates,es using the Cued Left (top left, Figs. 8–14A) and Uncued Rightns = cueing effects for right-sided cues using the Cuedclassification images. Black circles—behavioral results (same

Fig. 18 – Predicted cueing effects from the Weighted Likelihood model (Fig. 3) using the ideal templates, wc = 0.80. Cued LUnc R columns = cueing effects for left-sided cues using the Cued Left (top left, Figs. 8–14A) and Uncued Right (top right,Figs. 8–14B) classification images. Cued R Unc L columns = cueing effects for right-sided cues using the Cued Right (top right,Figs. 8–14A) and Uncued Left (top left, Figs. 8–14B) classification images. Black circles—behavioral results (same as Fig. 8). Graydiamonds—cue weight (wc) = 0.80.

43B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

effect (TB right cues, HL and CM left cues). Furthermore, inthese cases, the ideal weighting (0.80) with the idealtemplates could account for the cueing effects relativelywell.

Another aspect of the fits was that the template shapedifferences could account for cueing effects in two normalobservers (EM left cues, TB left cues). In a similar task tothe present study with 2-dimensional Gaussian (blurrydisk-like in white noise) stimuli, it was found that fourobservers had no changes in shape in their cued anduncued classification images (Eckstein et al., 2002). As inthe cases with the poor fits above (TB right cues, HL andCM left cues), the ideal weighting and template couldaccount the left-sided cueing effects of EM and TB. Onepossibility is that the correlations of the uncued classifi-cation images were underestimated in these cases (TB andEM). Differential weighting without changing the shape(such as the ideal observer) leads to lower amplitudeclassification images in the uncued location (Eckstein et al.,2002). Hence, the correlation of such a lower amplitudeclassification image with the ideal observer would be moresensitive to noise (both from the observer and the addedimage noise). Perhaps greater statistical power or highersignal contrasts would lead to better correspondence of theobserved classification images with the behavioral perfor-mances over all the observers.

3. General discussion and summary

Both patients had been assessed initially with a standardbattery of tests for hemineglect (Behavioral InattentionTest) and judged to be either partially (HL) or completely(CM) recovered. Despite their significant recoveries, both thebehavioral results and the classification images from thepresent study revealed clear effects consistent with atten-tional loss in the contralesional field. Thus, it seems thatthe methodology in this study may be more sensitive atdetecting residual attentional loss than the standardtesting.

There are several differences between the task used in thepresent study and the battery of tasks used in the BehavioralInattention Test, which include line bisection, various can-cellation tests, and drawing and copying tasks. One differenceis that the critical manipulation in the trials in the presentstudy occurred within tenths of seconds, whereas in thestandard neglect tests trials tend to be time-unlimited. Thus,it could be that the patients' attentional loss as measured inthe present study can only be revealed only under relativelysevere time constraints (see also Olk et al., 2002; Harvey et al.,2002). Furthermore, the patients may have been able to useresponse strategies to compensate for their attentional loss inthe standard neuropsychological tests, strategies that could

Fig. 19 – Predicted cueing effects from the Weighted Likelihood model (Fig. 3) using the classification images astemplates, wc = 0.50. Cued L Unc R columns = cueing effects for left-sided cues using the Cued Left (top left, Figs. 8–14A) andUncued Right (top right, Figs. 8–14B) classification images. Cued R Unc L columns = cueing effects for right-sided cues using theCued Right (top right, Figs. 8–14A) and Uncued Left (top left, Figs. 8–14B) classification images. Black circles—behavioral results(same as Fig. 10). Gray circles—cue weight (wc) = 0.80.

44 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

not be used in the cueing task in the present investigation.Our ability to control for compensatory response strategiesmight be attributed to the restricted time afforded to thepatients when performing our task, as well to the fact that thepresent task was relatively novel when compared to thestandard tests that the patients were possibly more familiarwith. Finally, the testing in this study was conducted over 3 to5 days and therefore represented a relatively intensive andextensive attentional examination when compared to thestandard BIT. Thus, the statistical power of our presentinvestigation may have been able to detect and measure theresidual attentional deficits that are undetected by moreroutine tasks that characterize the standardized BIT andeveryday life.

The behavioral results (cueing effects) of the two patientswere similar to each other and also consistent with thedisengagement deficit pattern that Posner et al. (1984) hadfound for neglect patients with parietal damage. However, twosubstantially different patterns of classification images werefound in the left hemisphere. When the signal was notpresent, HL's classification image had no correlation withthe signal (ideal observer template). Conversely, when thesignal was present, the classification images were positivelycorrelated with the signal. For signal patterns with phaseinformation (such as the checkerboard or Gabor patches),

Ahumada (2002) found that nonlinear detection mechanismsthat deviate from the assumption that a single perceptualtemplate lead to classification images can lead to this patternof results: classification images calculated from the signalpresent trials are biased in favor of the signal, whileclassification images calculated from the signal absent trialsdo not exist (i.e., are flat). This is known as locationuncertainty, in which the observer is uncertain of the exactlocation of the signal and can be modeled as the placement ofmultiple templates at different locations in the stimulus (Pelli,1985).

CM's classification image for the left hemisphere, on theother hand, indicated that he potentially could use theinformation at that location but appeared to use thatinformation inappropriately under certain conditions. Asdiscussed in Introduction, the ideal observer must havetwo components, the shape of the template at the locationand also the weight given to the information. When eitherthe cue or the signal appeared on the left, CM's classifi-cation image for the left had a high positive correlationwith the signal. This suggests that CM could use theinformation appropriately in this case, both in terms of theshape of the template and the weight given to thatinformation. However, when the cue appeared on theright side and the signal was not present on the left, the

Table 7 – Summary of model fits of the three models presented in Tables 4–6 and Figs. 7–10

Cueside

Cueingeffect

Correlation cued vs.opposite uncued

P values classificationimages wc = 0.50

P values idealtemplates wc = 0.80

P values classificationimages wc = 0.80

Normal observersEM Left 0.169 0.477 0.7401 0.7258 0.0114

Right 0.044 0.888 0.8904 0.0395 0.0120PC Left 0.015 0.737 0.7542 0.1598 0.0517

Right −0.174 0.746 0.0335 b0.0001 b0.0001RP Left −0.024 0.920 0.7494 0.0063 0.0064

Right −0.009 0.759 0.6555 0.0055 0.0023SS Left 0.179 0.951 0.0267 0.8848 0.9244

Right 0.122 0.819 0.2882 0.3067 0.1552TB Left 0.181 0.578 0.4939 0.8495 b0.0001

Right 0.170 0.328 b0.0001 0.7567 b0.0001

PatientsHL Left 0.225 −0.078 0.0433 0.7870 0.0432

Right 0.575 0.197 0.2349 0.0254 0.2336CM Left 0.088 0.251 0.0007 0.3577 b0.0001

Right 0.304 −0.370 0.5154 0.1156 0.6468

Column 1: cueing effect = valid hit rate − invalid hit rate (from Table 2). Column 2: correlations of the cued classification image with the oppositeuncued classification image (from Fig. 16). Column 3: P values from the chi-square goodness of fits for the model with 0.50 weighting and theobserved classification images (from Table 4 and Fig. 17). Column 4: P values from the chi-square goodness of fits for the model with 0.80weighting and the ideal templates (from Table 5 and Fig. 18). Column 5: P values from the chi-square goodness of fits for the model with 0.50weighting and the observed classification images (from Table 6 and Fig. 19).

45B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

correlation with the signal was negative and nearly thesame (unsigned) amplitude (see Fig. 5F) as the otherclassification images. In fact, the classification imagesuggests that CM took information from the left in favorof signal presence as evidence against signal presence.There are two interpretations to this result. The first isthat the high unsigned amplitude indicates that thepotential quality of information for the uncued-no signalclassification image was nearly as high as the otherclassification images, with the reversal in sign suggestingthat CM weighted that information entirely inappropriately.Alternatively, it might be said that CM has an entirelyinappropriate (opposite) template shape.

Thus, for HL and CM, the classification images suggest twodifferent causes of their neglect-type behavior. In general,when the cue was on the right side of space in this study, CMseemed to have the ability to use (perceive) the informationfrom the left side, but not the ability to give the informationthe proper importance. Conversely, HL appeared not to able tolocalize the exact expected signal location on the left side,regardless of cue location. This is perhaps not an unexpectedoutcome, as the areas of injury differed, with HL's injury moreposterior and dorsal to CM's. While neglect occurs morefrequently after right hemispheric brain injury, and particu-larly to the right parietal cortex, injuries to other areas of thebrain also lead to hemineglect. These areas include injury tothe frontal cortex (dorsolateral, Critchley, 1966; Heilman et al.,1970, 1983) and injury to subcortical areas, such as thethalamus (Watson and Heilman, 1979; Karussis et al., 2000)and the basal ganglia (Vallar and Perani, 1986; Damasio et al.,1980). Therefore, neglect may be more accurately described asan assembly of syndromes having a similar phenomenology,with possibly different mechanisms of disruption rather thana unitary syndrome (Harvey et al., 2002; Olk et al., 2001). Evenwithin these two patients, substantially differentmechanisms

of attentional loss appear to be expressed. Perhaps with thetesting of more patients, it would be possible to characterizedifferent types of attentional loss (that currently fall under thesame category as neglect) to specific areas of brain injury.

There is the question of whether the results of the twopatients might be due to their age. While the patients didexhibit a general deficit in performance, as indicated bythe longer stimulus durations and the higher stimuluscontrasts, we do not believe that age accounts for thehemispheric differences in the cueing effects and theclassification images. First, the patients' cueing effects forleft-sided cues matched well with both the normalobservers' cueing effects and those predicted by the idealobserver, suggesting that there was not a general discrep-ancy or loss of the use of the cue for the patients. Second,it seems unlikely that aging would produce the hemi-spheric asymmetries consistent with neglect that thepatients demonstrated in this study, as there has beenno previous suggestion of neglect-like symptoms in normalaged populations (e.g., Lezak, 1995; Gazzaniga, 1998).

In conclusion, we tested two patients and five normalobservers in a cueing task, a common test of visual attention.We found clear indications of attentional deficits for thepatients, even after more than a year after their injury andafter the patients had partially (HL) or fully (CM) recoveredwhen assessed by a standard neuropsychological examination(the Behavioral Inattention Test). Furthermore, the classifica-tion image technique indicated differing causes of neglect forthe two patients, even though they showed similar resultsfrom behavioral tests (i.e., the cueing effects). Like the studieswe have conducted with normal observers (Eckstein et al.,2002), the present study suggests that the research techniquepresented here can test for different types of mechanisms ofattentional loss that are otherwise indistinguishable by thestandard measures of behavioral performance. Finally, as

Fig. 20 – A series of horizontal CT image scans for HL showing his lesion.

46BR

AIN

RESEA

RC

H1080

(2006)

26–52

47B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

suggested by these two patients, the types of attentionaldeficits may differ from patient to patient, and it is hoped thatthe technique presented here could be used to characterizethese different types of deficits with regard to the specificbrain areas that are compromised.

4. Experimental procedures

We performed a study of classification images in a cueing taskon 5 normal observers and two parietal patients with a historyof neglect. Observers performed a yes/no contrast discrimina-tion of a signal appearing at one of two locations 2.5° to the leftand right of a central fixation point (Fig. 1). The signal was a1.5° 3 × 3 checkerboard pattern configured as a high contrast‘white X’ (see Fig. 2, left, for a larger view). As a lower contrast‘pedestal’ (contrast = 7.8%) checkerboard appeared in thelocations without the signal, this task may be viewed as adiscrimination of the lower (pedestal) and higher (signal)contrast checkerboards. The stimulus display appeared foreither 50 ms (normals) or 140 ms (patients), with the signalappearing on half the trials, and observers judged on each trialwhether the target was present. Immediately prior to thestimulus display, a 2.5° square precue (140 ms) appeared atone of the two possible signal locations; on signal presenttrials, the precue indicated the signal location with 80%validity. Gaussian image noise was added to the stimuli tocalculate classification images; this noisewas added to each ofthe 9 squares comprising the checkerboards (SD = 11.7%contrast).

The five normal observers were 4 females, ages 21 (PC,RP, EM) and 22 (TB), and 1male, age 39 (SS, an author), with noknown brain injury or deficit. Each normal observer partic-ipated in approximately 2500 trials/observer, and the con-trast for their signal was 19.5%, except for TB, who was testedon a slightly higher contrast, 21.9%. The normal observersviewed a CRT monitor in a darkened room with theluminance calibrated by the Dome Calibration system. Forthe patients, the signal contrasts were higher than those forthe normal observers (HL = 27.3%, CM = 23.4%), and thenumber of trials was less (HL = 820, CM = −1641). The highercontrasts and longer durations (140 ms vs. 50 ms) for thepatients were chosen so that overall performances wereapproximately equivalent to the normal observers. Further-more, stimuli were presented on a laptop computer with anLCD monitor (Toshiba Satellite 1805-S207), and luminanceand color calibrations were performed with the Optical 3.7(Pantone Colorvision) system.

4.1. Patients

Both patients were male (HL, age 69; CM, age 85) and hadsuffered a right hemispheric stroke at least a year prior totesting. Approximately 1 year and 9 months prior to testing,HL presented with left hemiparesis, left hemianopia, and leftneglect and extinction. CT examination (Fig. 20) indicated anextensive area of low density reflecting ischemic damage(without hemorrhage) in the territory of the right posteriorcerebral artery, especially the striate and peristriate cortices,

posterior temporal neocortex, and posterior periventricularwhite matter, with some extension into the inferoposteriorparietal area. HL also suffered a lacunar infarct in the rightcaudate, as well as some deep ischemic changes to whitematter. On the Behavioral Inattention Test (BIT, a battery ofcancellation, bisection, drawing, and copying tasks designedto assess neglect, with any score below 129 out of 146indicating neglect, Wilson et al., 1987), HL scored 100/146approximately 3 months after his injury, indicating a cleardeficit. Just prior to testing, his score on the BIT hadimproved to 126 but still indicated residual neglect at thetime of testing.

Approximately 1 year and 4 months prior to testing, CMpresented with a left hemiparesis, left neglect, and lefthemisensory deficit, including the left arm, hand, wrist, andleg. His MRI image scan immediately after the stroke indicatedan ischemic infarction of the right middle cerebral artery(MCA); also, an old right MCA territory infarction was found,apparently from a transient incident 2 years prior to the acutestroke and from which he apparently had made a fullfunctional recovery (see Fig. 21). At the time of testing, CM'sscore on the BIT was 141/146, suggesting a nearly full recoveryfrom neglect (as assessed by the BIT).

4.2. Calculation of classification images

Classification images were calculated for each side (left andright) based on cue presence on that side and were furtherdivided by signal presence on that side. Thus, for each side,there were six classification images: cued, cued-signal, cued-no signal, uncued, uncued-signal, and uncued-no signal. Theindividual noise fields were weighted by the observer'sresponse on that trial, either −1 for a ‘no’ response or +1 for a‘yes’ response. This weighting is modified from a simple rulefor combining noise fields across different response outcomeswhen calculating classification images (Beard and Ahumada,1998; Ahumada, 2002; Murray et al., 2002). The classificationimage for each of the six outcomes is the mean of theindividual noise fields (after weighting) corresponding to thatoutcome.

One-sample Hotelling T2 statistics were employed toassess whether the classification images were nonzero(different from a flat or zero value classification images)or differed from the ideal observer; two-sample Hotelling T2

statistics were employed to assess differences betweenclassification images. The Hotelling T2 statistic is themultivariate equivalent to the t statistic (see Appendix A).Correlations of the observed classification images with thesignal were calculated to assess the match of the observedtemplates to the ideal template in this task. As a method toassess the relationship between the classification imagesand the observed cueing effects, correlations were alsocalculated between the classification images when one sidewas cued and the classification images when the oppositeside was not cued. Standard errors of the correlations wereestimated by a simulation method known as ‘Monte Carloresampling’ (Efron and Tibshirani, 1993). The values of eachsquare of the classification images were assumed to beindependent and Gaussian-distributed with the samemeans and standard deviations as the empirical values.

48 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

49B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

Over 100 iterations, a sampled value was chosen for eachsquare of the classification image corresponding to itsparticular independent Gaussian distribution. The resultingclassification images were correlated (either with the signalor with another classification image similarly perturbed bynoise), and a standard error was estimated by computingthe standard error across the correlations computed foreach iteration.

To further assess the relationship of the classificationimages to the behavioral performance, Monte Carlo simula-tions were performed on each observer's classification imagesby inserting the classification images into the WeightedLikelihoodmodel as the templates (Fig. 3). Similar simulationswere performed in Eckstein et al. (2002). The model's freeparameters were found by the best fit (by chi-square goodnessof fit) to the observer's behavioral performance, which werethe valid and invalid hit rates (correct ‘signal present’responses for signal present trials) and false alarm rates(incorrect ‘signal present’ responses for signal absent trials)for left and right cues. The free parameters were the internalsignal-to-noise ratio (SNR), or the internal signal contrast of theobserver needed to best match the behavioral performance,and the decision criterion (crit). For the simulations, allclassification image amplitudes were normalized to haveequal contrast energy, eliminating differences in amplitudein the classification images. Simulations were run with 30,000trials for each trial type. Two general types of simulationswereperformed. The first fixed the weight applied to the cuedlocation (wc) (and also the uncued location, as wuc = 1 − wc). Ofparticular interest are the weights (wc = 0.50), which assumesequal weighting of the cued and uncued locations, and(wc = 0.80), which matches the cue validity and the weight ofthe cued likelihood for the ideal observer. In the second type ofsimulation, the weight (wc) was free to vary.

Acknowledgments

The authors thank Michael Miller for his assistance infacilitating this research, and patients CM and HL for theirefforts in participating in this study. Supported by awards toME from NSF-0135118, NIH-53455, and NASA NAG-1157; andawards to AK from the Human Frontier Science Program, theNatural Sciences and Engineering Research Council of Canada,the Michael Smith Foundation for Health Research, and theHuman Early Learning Partnerships.

Appendix A. Hotelling T2

Tests for nonzero classification images and differences fromthe ideal observer were performed with the single-sampleHotelling T2 statistic (Eq. (A1)); this is the multivariateequivalent of the univariate t statistic (Harris, 1985; seeAbbey and Eckstein, 2002; Eckstein et al., 2002, 2004). In Eq.

Fig. 21 – A series of horizontal MRI image scans for CM showingthe T1-weighted (T1W)MRI images corresponding to the same re

(A1), N = the number of observations (trials), x = a vectorcontaining the observed classification image, x0 = a vectorcontaining the null hypothesis classification image (either allzeroes or the ideal observer with the best-fitting amplitude),and K−1 = the inverse of the sample covariance matrix for theindividual noise samples for that particular classificationimage.

T2 ¼ N½x� x0�tK�1½x� x0� ðA1Þ

The single-sample Hotelling T2 statistic may be transformedinto an F statistic (Eq. (A2)), with p = the length of vectors x

and x0 (the number of points in the classification images,which is 9). The resulting F statistic has p degrees offreedom in the numerator, and N − p degrees of freedom inthe denominator.

F ¼ N� ppðN� 1ÞT

2: ðA2Þ

The independent two-sample Hotelling T2 statistic (Eq.(A3)) was used to assess differences between two observedclassification images, both scaled and unscaled. Eq. (A3) givesthe computation for the independent two-sample Hotelling T2,with N1 = the number of observations for the first observedclassification image, N2 = the number of observations for thesecond observed classification image, x1 = a vector containingthe first observed classification image, x2 = a vector containingthe second observed classification image, and K−1 = theinverse of the pooled covariance matrix combining the sumof square deviations and sum of squared products from bothsamples.

T2 ¼ N1N2

ðN1 þN2Þ x2 � x1�tK�1 x2 � x1½ �:� ðA3Þ

The independent two-sample Hotelling T2 may be trans-formed into an F statistic (Eq. (A4))with p = the lengthof vectorsx1 and x2 (the number of pixels in the classification images).The resulting F statistic has p degrees of freedom in thenumerator, and N1 + N2 − p − 1 degrees of freedom in thedenominator.

F ¼ N1 þN2 � p� 1pðN1 þ N2 � 2Þ T2: ðA4Þ

Appendix B. Weighted likelihood model

B.1. General description

One quantitative approach for analysis of the cueing task isto consider the theoretical framework of the Bayesian idealobserver (Eckstein et al., 2002; Shimozaki et al., 2003, seeKersten et al., 2004, for a review). The Weighted Likelihood

his lesion. Upper: T2-weighted (T2W) MRI images. Lower:gion as the T2-weighted images in the upper part of the figure.

50 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

model (Fig. 3) is based on the Bayesian ideal observer and,under specific circumstances, is equivalent to the idealobserver.

In general (see Fig. 3), the Weighted Likelihood model forthe Posner cueing task calculates the posterior probabilityof signal presence given the data (the stimuli presented), orthe likelihood, based on the prior probabilities by usingBayes theorem. It first computes the correlation between atemplate at each location and the stimuli (the ‘data’) ateach of the two possible signal locations (xc and xuc). Then,from these scalars, the likelihoods of signal presence at thetwo locations are calculated. To calculate the likelihoods,the scalars resulting from the correlation at each locationundergo a nonlinear (Gaussian) transformation to calculatethe prior probabilities of the data given signal presence ateach location. This result is weighted separately, dependingon cue location (by wc and wuc); normally, wc is larger thanwuc. Finally, the two weighted likelihoods are addedtogether, and this single value is compared to a criterion.A value larger than the criterion leads to a ‘yes’ response(signal present), and a value less than the criterion leads toa ‘no’ response (signal absent).

The model is equivalent to the ideal observer when thefollowing properties are met: Aa the weights by (wc and wuc)equal the cue validities (in this study, these were 0.80 and0.20). The cue validities represent the prior probabilities oftarget presence, given the cue at that location. (B) Bothtemplates at the cued and uncued locations have the idealshape. In this study, the ideal shape is the signal itself;intuitively, such a template uses all the possible informationavailable. (C) There is no internal noise. Note that, for theideal observer, attention is expressed only as a larger weightgiven to the cued location, or alternatively, attentionoperates only as a selective mechanism. This is counter tomany common models of attention, which propose a changein the detectability at the cued location (such as ‘enhance-ment’ (Posner, 1980) or tuning (Spitzer et al., 1988; Yeshurunand Carrasco, 1998, 1999)). We have found that the idealobserver shows a robust cueing effect, despite proposing onlya selective attention mechanism (Eckstein et al., 2002;Shimozaki et al., 2003; see also Kinchla, 1974; Kinchla et al.,1995 for a similar model to the ideal observer), bringing intoquestion the interpretation of the cueing effect in terms of achange of detectability or visual processing at the cuedlocation.

B.2. Mathematical description

This model is based on an optimal Bayesian observer (Greenand Swets, 1974) for the cueing task and can be found inEckstein et al. (2002) and Shimozaki et al. (2003).

xc = response at cued locationxuc = response at uncued locationsc = the event of a signal at the cued locationsuc = the event of a signal at the uncued locationnc = the event of a noise stimulus at the cued locationnuc = the event of a noise stimulus at the uncued locationwc = the weight for the cued locationwuc = 1 − wc = the weight for the uncued location

First, the model computes the cross-correlation of atemplate with stimuli at the cued and uncued locations,giving two responses (xc and xuc). The model calculates threelikelihoods from the responses xc and xuc.

(A) The likelihood of xc and xuc given sc and nuc (valid signaltrial).

Lsc ;nuc ¼ pðxc; xucjsc;nucÞ

(B) The likelihood of xc and xuc given suc and nc (invalidsignal trial).

Lnc ;suc ¼ pðxc; xucjnc; sucÞ

(C) The likelihood of xc and xuc given nc and nuc (signalabsent trial).

Lnc ;nuc ¼ pðxc; xucjnc;nucÞ:

Each likelihood is the joint probability of xc given thestimulus at the cued location and the probability of xuc giventhe stimulus at the uncued location.

Lsc ;nuc ¼ pðxc; xucjsc;nucÞ ¼ pðxcjscÞpðxucjnucÞ

Lnc ;suc ¼ pðxc; xucjnc; sucÞ ¼ pðxcjncÞpðxucjsucÞ

Lnc ;nuc ¼ pðxc; xucjnc;nucÞ ¼ pðxcjncÞpðxucjnucÞ:

The likelihoods are weighted (wc andwuc) and summed. Tofit the behavioral data to themodel, we allowwc to vary freely.

weighted Ls ¼ wcLsc ;nuc þwucLnc ;suc

¼ wcpðxcjscÞpðxucjnucÞ þwucpðxcjncÞpðxucjsucÞ:

The model calculates the weighted likelihood ratio of theresponses.

Ls=n ¼ weighted LsLn

¼ wcpðxcjscÞpðxucjnucÞ þwucpðxcjncÞpðxucjsucÞpðxcjncÞpðxucjnucÞ :

The model assumes that the prior probability densityfunctions of the response at each location are Gaussiandistributions of unit variance, one for the signal and one forthe noise. The mean of the noise distribution is zero, andsensitivity isdefinedas themeanof thesignal distribution (d′, orsignal-to-noise ratio (SNR)). This is a standard Signal DetectionTheory assumption (Green and Swets, 1974). Note that d′ is thesame regardless of whether the location is cued or uncued.

g xð Þ ¼ Gaussian function of unit variance l ¼ 0;r2 ¼ 1� �

¼ 1ffiffiffiffiffiffi2p

p e�12ðxÞ2

Then, for a noise stimulus,

p xjnð Þ ¼ g xð Þ ¼ 1ffiffiffiffiffiffi2p

p e�12ðxÞ2

For a signal stimulus

p xjsð Þ ¼ g x� lsð Þ ¼ g x� d Vð Þ ¼ 1ffiffiffiffiffiffi2p

p e�12ðx�d VÞ2 :

51B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

Substituting the Gaussian assumption into the weightedlikelihood ratio,

Ls=n ¼ wcpðxcjscÞpðxucjnucÞ þwucpðxcjncÞpðxucjsucÞpðxcjncÞpðxucjnucÞ

¼ wcgðxc � SNRÞgðxucÞ þwucgðxcÞgðxuc � SNRÞgðxcÞgðxucÞ

¼wc

1ffiffiffiffi2p

p e�12ðxc�SNRÞ2 1ffiffiffiffi

2pp e�1

2ðxucÞ2 þwuc1ffiffiffiffi2p

p e�12ðxcÞ2 1ffiffiffiffi

2pp e�1

2ðxuc�SNRÞ2

1ffiffiffiffi2p

p e�12ðxcÞ2 1ffiffiffiffi

2pp e�1

2ðxucÞ2

¼ wce�12 ðxc�SNRÞ2þx2ucð Þ þwuce�

12 x2cþðxuc�SNRÞ2ð Þ

e�12 x2cþx2ucð Þ : ðB1Þ

The performance of the model was estimated with MonteCarlo simulations of 10,000 trials for each d′ and was fit (usingχ2 goodness of fit) to the observers' invalid hit rates, valid hitrates, and false alarm rates. To find the predicted hit and falsealarm rates, the log likelihood ratio was compared to acriterion (log(crit)). Responses generating log likelihood ratiosgreater than log(crit) led to ‘signal present’ decisions for themodel. Therefore,

pHv = valid hit rate = p(log(Ls/n) N log(crit)|sc,nuc)pHi = invalid hit rate = p(log(Ls/n) N log(crit)|nc,suc)pFA = false alarm rate = p(log(Ls/n) N log(crit)|nc,nuc)

and there were three free parameters for each fit: d′, crit,and wc.

This model is equivalent to the Bayesian ideal observer if(a)wc = cue validity, (b) the template is ideal (the signal itself inthis case), and (c) there is no internal noise.

R E F E R E N C E S

Abbey, C.K., Eckstein, M.P., 2002. Classification image analysis:estimation and statistical inference for two-alternative forced-choice experiments. J. Vis. 2, 66–78.

Ahumada Jr., A., 1996. Perceptual classification images fromVernier acuity masked by noise. Perception, 26, ECVPSupplement, 18 (Abstract).

Ahumada Jr., A.J., 2002. Classification image weights and internalnoise level estimation. J. Vis. 2, 121–131.

Ahumada Jr., A.J., Lovell, J., 1971. Stimulus features in signaldetection. J. Acoust. Soc. Am. 49, 1751–1756.

Andersen, R.A., 1995. Encoding of intention and spatial location inthe posterior parietal cortex. Cereb. Cortex 5, 457–469.

Barlow, H.B., 1978. The efficiency of detecting changes in densityin random dot patterns. Vision Res. 18, 637–650.

Beard, B.L., Ahumada Jr., A., 1998. Technique to extract relevantimage features for visual tasks. In: Rogowitz, B.E., Pappas, T.N.(Eds.), Proceeding of SPIE, vol. 3299, pp. 79–85.

Behrmann, M., Tipper, S.P., 1994. Object-based attentionalmechanisms: evidence from patients with unilateral neglect.In: Umilta, C., Moscovitch, M. (Eds.), Attention and PerformanceXV: Conscious and Nonconscious Information Processing. MITPress, Cambridge, MA, pp. 351–375.

Bennett, P.J., Sekuler, A.B., Ozin, L., 1999. Effects of aging oncalculation efficiency and equivalent noise. J. Opt. Soc. Am. A16, 654–668.

Bisiach, E., 1993. Mental representation in unilateral neglect and

related disorders: the Twentieth Barlett Memorial Lecture. Q. J.Exp. Psychol. 46, 435–461.

Bisiach, E., Luzzatti, C., 1978. Unilateral neglect of representationalspace. Cortex 14, 129–133.

Burgess, R.F., Wagner, R., Jennings, J., Barlow, H.B., 1981. Efficiencyof human visual signal discrimination. Science 214, 93–94.

Colby, C.L., Duhamel, J.R., Goldberg, M.E., 1995. Oculocentricspatial representation in parietal cortex. Cereb. Cortex 5,470–481.

Colby, C.L., Duhamel, J.R., Goldberg, M.E., 1996. Visual, presaccadic,and cognitive activation of single neurons in monkey lateralintraparietal area. J. Neurophysiol. 76, 2841–2852.

Corbetta, M., Akbudak, E., Conturo, T.E., Snyder, A.Z., Ollinger, J.M.,Drury, H.A., Linenweber, M.R., Peterson, S.E., Raichle, M.E., VanEssen, D.C., Bisiach, E., Luzzatti, C., 1978. Unilateral neglect ofrepresentational space. Cortex 14, 129–133.

Courtney, S.M., Ungerleider, L.G., Keil, K., Haxby, J.V., 1996. Objectand spatial visual working memory activate separate neuralsystems in human cortex. Cereb. Cortex 6, 39–49.

Critchley, M., 1966. The Parietal Lobes. Hafner, New York.Damasio, A.R., Damasio, H., Chui, H.C., 1980. Neglect following

damage to the frontal lobe or basal ganglia. Neuropsychologia18, 123–132.

Darby, D.G., Nobre, A.C., Thangaraj, V., Edelman, R., Mesulam,M.-M., Warach, S., 1996. Cortical activation in the humanbrain during lateral saccades using EPISTAR functionalmagnetic resonance imaging. NeuroImage 3, 53–62.

Dosher, B.A., Lu, Z.L., 2000a. Mechanisms of perceptual attentionin precuing of location. Vision Res. 40, 1269–1292.

Dosher, B.A., Lu, Z.L., 2000b. Noise exclusion in spatial attention.Psychol. Sci. 11, 139–146.

Eckstein, M.P., Ahumada, A.J., Watson, A.B., 1997. Imagediscrimination models predict visual detection in naturalimage backgrounds. Proc. SPIE Electron. Imaging, Hum. Vis.Appl. 3016, 44–56.

Eckstein, M.P., Shimozaki, S.S., Abbey, C.K., 2002. The footprints ofvisual attention in the Posner cueing paradigm revealed byclassification images. J. Vis. 2 (1), 25–45.

Eckstein, M.P., Pham, B.T., Shimozaki, S.S., 2004. The footprints ofvisual attention during search with 100% valid and 100%invalid cues. Vision Res. 44, 1207–1993.

Efron, B., Tibshirani, R., 1993. An Introduction to the Bootstrap.Chapman and Hall, New York.

Gazzaniga, M.S., 1998. Attention and selective perception. In:Gazzaniga, M.S., Ivry, R.B., Mangun, G. (Eds.),Fundamentals of Cognitive Neuroscience. W.W. Norton,New York, pp. 207–245.

Gold, J., Bennett, P.J., Sekuler, A.B., 1999. Signal but not noisechanges with perceptual learning. Nature 402 (6758), 176–178.

Gold, J.M., Murray, R.F., Bennett, P.J., Sekuler, A.B., 2000. Derivingbehavioural receptive fields for visually completed contours.Curr. Biol. 10, 663–666.

Green, D.M., Swets, J.A., 1974. Signal Detection Theory andPsychophysics. Robert E. Kreiger Publishing Co., Inc,Melbourne, FL.

Harris, R.J., 1985. A primer of multivariate statistics. AcademicPress, Orlando, Florida, pp. 99–118.

Harvey, M., Olk, B., Muir, K., Gilchrist, I.D., 2002. Manualresponses and saccades in chronic and recovered hemispatialneglect: a study using visual search. Neuropsychologia 40,705–717.

Heilman, K.M., Pandya, D.N., Geschwind, N., 1970. Trimodalinattention following parietal lobe ablations. Trans. Am.Neurol. Ass. 95, 259–261.

Heilman, K.M., Valenstein, E., Watson, R.T., 1983. Localization ofneglect. In: Kertesz, A. (Ed.), Localization in Neurology.Academic Press, New York, pp. 471–492.

Lezak, M.D., 1995. Neuropsychological Assessment. Oxford Univ.Press, London.

52 B R A I N R E S E A R C H 1 0 8 0 ( 2 0 0 6 ) 2 6 – 5 2

Karussis, D., Leker, R.R., Abramsky, O., 2000. Cognitive dysfunctionfollowing thalamic stroke: a study of 16 cases and review of theliterature. J. Neurol. Sci. 172, 25–29.

Kersten, D., 1984. Spatial summation in visual noise. Vision Res.24, 1977–1990.

Kersten, D., Mamassian, P., Yuille, A., 2004. Object perception asBayesian inference. Annu. Rev. Psychol. 55, 271–304.

Kinchla, R.A., 1974. Detecting target elements in multi-elementarrays: a confusability model. Percept. Psychophys. 15,149–158.

Kinchla, R.A., Chen, Z., Evert, D., 1995. Precue effects in visualsearch: data or resource limited? Percept. Psychophys. 57,441–450.

Làdavas, E., Petronio, A., Umiltà, C., 1990. The deployment ofvisual-attention in the intact field of hemineglect patients.Cortex 26, 307–317.

Levi, D.M., Klein, S.A., 2002. Classification images for detection andposition discrimination in the fovea and parafovea. J. Vis. 2,46–65.

Li, R.W.-H., Levi, D.M., Klein, S.A., 2004. Perceptual learningimproves efficiency by re-tuning the decision ‘template’ forposition discrimination. Nat. Neurosci. 7, 178–183.

Liu, Z., Knill, D.C., Kersten, D., 1995. Object classification forhuman and ideal observers. Vision Res. 35, 549–568.

Mountcastle, V.B., Lynch, J.C., Georgopoulos, A., Sakata, H., Acuna,C., 1975. Posterior parietal association cortex of the monkey:command functions for operations within extrapersonalspace. J. Neurophysiol. 38, 871–905.

Murray, R.F., Bennett, P.J., Sekuler, A.B., 2002. Optimal methods forcalculating classification images: weighted sums. J. Vis. 2,79–104.

Neri, P., Parker, A.J., Blakemore, C., 1999. Probing the humanstereoscopic system with reverse correlation. Nature 401,695–698.

Olk, B., Harvey, M., Dow, L., Murphy, P.J.S., 2001. Illusion processingin hemispatial neglect. Neuropsychologia 39, 611–625.

Olk, B., Harvey, M., Gilchrist, I.D., 2002. First saccades reveal biasesin recovered neglect. Neurocase 8, 306–313.

Pelli, D.G., 1985. Uncertainty explains many aspects of visualcontrast detection and discrimination. J. Opt. Soc. Am. A 2,1508–1532.

Posner, M.I., 1980. Orienting of attention. Q. J. Exp. Psychol. 32,3–25.

Posner, M.I., Walker, J.A., Friedrich, F.J., Rafal, R.D., 1984. Effects ofparietal injury on covert orienting of attention. J. Neurosci. 4,1863–1874.

Rafal, R., 1994. Neglect. Curr. Opin. Neurobiol. 4, 231–236.Robinson, A., Bowman, E., Kertzman, R., 1995. Covert orienting of

attention in macaques. II. Contributions of parietal cortex. J.Neurophysiol. 74, 698–712.

Rusconi, M.L., Maravita, A., Bottini, G., Vallar, G., 2002. Is the intactside really intact? Perseverative responses in patients withunilateral neglect: a productive manifestation.Neuropsychologia 40, 594–604.

Shimozaki, S.S., Eckstein, M.P., 2003. The suboptimality ofattentional weighting across precue validities. EuropeanConference on Visual Perception, September, 2003.

Shimozaki, S.S., Eckstein, M.P., Abbey, C.K., 2003. Comparison oftwo weighted integration models for the cueing paradigm:linear and likelihood. J. Vis. 3 (3), 209–229.

Spitzer, H., Desimone, R., Moran, J., 1988. Increased attentionenhances both behavioral and neuronal performance. Science240 (April 15), 338–340.

Tipper, S.P., Behrmann, M., 1996. Object-centered not scene-basedvisual neglect. J. Exp. Psychol. Hum. Percept. Perform. 22,1261–1278.

Tjan, B.S., Braje, W.L., Legge, G.E., Kersten, D., 1995. Humanefficiency for recognizing 3-D objects in luminance noise.Vision Res. 35, 3053–3069.

Vallar, G., Perani, D., 1986. The anatomy of unilateral neglect afterright-hemisphere lesions: a clinical/CT-scan correlation studyin man. Neuropsychologia 24, 609–622.

Watson, R.T., Heilman, K.M., 1979. Thalamic neglect. Neurology29, 690–694.

Wilson, B., Cockburn, J., Halligan, P.W., 1987. BehaviouralInattention Test. Thames Valley Test Company, Titchfield,Hampshire.

Wright, A., Olk, B., Wee, J., Kingstone, A., 2005. The effect ofneglect on ipsilesional space: a case study. J. Cogn. Neurosci.,Suppl. 54.

Yeshurun, Y., Carrasco, M., 1998. Attention improves or impairsvisual performance by enhancing spatial resolution. Nature396, 72–75 (Nov 5).

Yeshurun, Y., Carrasco, M., 1999. Spatial attention improvesperformance in spatial resolution tasks. Vision Res. 39,293–306.