Kosslyn (1978) Visual images preserve metric spatial ...wexler.free.fr/library/files/kosslyn (1978) visual images...Journal of Experimental Psychology: Human Perception and Performance

Journal of Experimental Psychology:Human Perception and Performance1978, Vol. 4, No. 1, 47-60

Visual Images Preserve Metric Spatial Information:Evidence from Studies of Image Scanning

Stephen M. KosslynHarvard University

Brian J. ReiserNew York University

Thomas M. BallJohns Hopkins University

Four experiments demonstrated that more time is required to scan furtherdistances across visual images, even when the same amount of material fallsbetween the initial focus point and the target. Not only did times system-atically increase with distance but subjectively larger images required moretime to scan than did subjectively smaller ones. Finally, when subjects werenot asked to base all judgments on examination of their images, the distancebetween an initial focus point and a target did not affect reaction times.

Introspections about visual imagery veryoften include references to "scanning" acrossimages. Kosslyn (1973) attempted to dem-onstrate that scanning of images is a func-tional cognitive process, and his experimentindicated that more time was required totraverse greater distances across mentalimages. However, in the course of scanninglonger distances,1 people in Kosslyn's ex-periment also passed over more parts of theimaged object. For example, in scanningfrom the motor to the porthole of an imagedspeedboat, a person passed over the reardeck and part of the cabin; in scanning fromthe motor to the more distant anchor, onescanned over all of these parts plus thefront deck and bow. Given this confounding,then, we have no way of knowing whetherKosslyn's results were a consequence ofpeople actually scanning over a quasi-pic-torial, spatial image. One could argue thatthe image itself was epiphenomenal in thissituation and that the apparent effects of dis-tance actually were a consequence of how

This work was supported by National Instituteof Mental Health Grant 1 R03 MH 27012-01 andNational Science Foundation Grant BNS 76-16987awarded to the first author. We thank Phil Green-barg and Dan Estridge for technical assistance.

Requests for reprints should be sent to StephenM. Kosslyn, 1236 William James Hall, 33 Kirk-land Street, Cambridge, Massachusetts 02138.

people accessed some sort of underlying liststructure. Parts separated by greater dis-tances on the image might simply be sepa-rated by more entries in a list of parts ofthe object.

The notion that scanning corresponds toprocessing a list structure, and not thespatial "surface" image (see Kosslyn, 1975,1976; Kosslyn & Pomerantz, 1977), recentlyseemed to receive support from Lea (1975).In a typical experiment, people evaluated

1 We will use terms like distance and size in re-ferring to mental images, even though imagesthemselves—not being objects—do not have suchphysical dimensions. Nevertheless, we claim thatimages represent these dimensions in the same waythat they are encoded in the representations under-lying the experience of seeing during perception.Thus, we experience images as if we were seeinga large or small object, or one at a relatively nearor far distance from us. In addition, the apparentdistances between parts of an imaged object areexperienced in the same way that one would ex-perience apprehending the distances when seeingthe parts of the object. We will use the termquasi-pictorial in referring to these sorts of pic-torial properties of an image, because an image—not being an object—cannot have the physicalproperties of an actual picture. For convenience,we will refer to an imaged object that is experi-enced as being some subjective size as if theimage were that size, and we will refer to apparentdistances on an imaged object as if they weredistances on the image itself.

Copyright 1978 by the American Psychological Association, Inc. All rights of reproduction in any form reserved.

47

48 S. KOSSLYN, T. BALL, AND B. REISER

from memory the relative locations of ob-jects in a circular array. Lea asked his sub-jects to learn the array via imagery. Follow-ing this, they were given the name of oneobject and asked to name the first, second,or nth item in a given direction. Lea foundthat the time to respond depended on thenumber of intervening items between an ini-tial focus point and the target, but not onthe actual distance separating a pair of ob-jects in the array. The interpretation ofthese results is muddied, however, becauseLea never insisted that his subjects baseall judgments on actual processing of theimage itself. That is, subjects were not toldto count the items as they appeared in theirimage but only to count the appropriatenumber of steps to the target. It is reason-able to suppose that these people encodedthe circular array both as a list and as animage. Given that imagery tends to requiremore time to use in this sort of task thando nonimaginal representations (Kosslyn,1976), subjects may have actually arrivedat most judgments through processing non-imaginal list structures. If so, then it is notsurprising that actual distance separatingpairs did not affect retrieval times.

The present experiments, then, test theclaim that distance affects time to scan im-ages by removing the confounding betweendistance and the number of intervening itemsscanned across. If images really do preservemetric spatial information, and images them-selves can in fact be scanned, then actualdistance between parts of an imaged objectshould affect scanning time. If the apparenteffects of distance observed by Kosslyn(1973) were in fact due to accessing somesort of ordered list, however, then onlyordinal relations between parts—not actualinterval distances—should affect the timeneeded to shift one's attention from one partof an image to another.

Experiment 1

This experiment is an attempt to distin-guish between the effects of scanning differ-ent distances and scanning over differentnumbers of intervening items. The peoplewho participated in the experiment scanned

visual images of three letters arrayed on aline, "looking" for a named target. Uponmentally focusing on the target, the subjectclassified it according to whether it wasupper- or lowercase. In scanning to thetarget letter, one had to traverse one of threedistances and pass over zero, one, or twointervening letters; letter arrays were con-structed such that each distance appearedequally often with each number of interven-ing items, allowing us to consider each vari-able independently of the other.

The present claim is that distance per seaffects time to scan an image. However, wealso expect people to take more time inscanning over more items since each itempresumably must be "inspected" as it isscanned over, which requires an incrementof time. The present claim does not speak tothe issue of which factor affects image scan-ning more—distance or number of interven-ing items; we are primarily concerned withdemonstrating that effects of distance arenot simply an artifact of how many thingsmust be scanned over.

Method

Materials. We constructed two books of stimuli,each containing 36 arrays of letters. Each arrayconsisted of three letters spaced along a 20.32-cmlong line. Each array contained two letters of onecase, and one of the other; each case (upper andlower) was represented equally often across arrays.Target letters were placed 5.08, 10.16, and 15.24 cmfrom the point of focus (one of the two ends ofthe line), and zero, one, or two other letters in-tervened between the target and point of focus.Intervening items were spaced at equal intervalsbetween the target and focus point. The arrayswere constructed such that each distance occurredequally often with each number of interveningitems. Each of these nine conditions was repre-sented by 8 arrays, half of which had an upper-case letter as the target and half of which had alowercase letter as the target. Further, for halfof each target type in each condition, the focuspoint was specified as the left end of the line, andfor half it was the right end. We did not useletters whose upper- and lowercases seemed diffi-cult to distinguish (c, k, o, p, s, u, v, w, x, a).The remaining 16 letters of the alphabet were usedas targets and distractors. Each of these lettersappeared at least once as a target in each case,at each distance, and with each number of inter-vening items, but not with every possible combi-nation of these variables (this would have required

SCANNING DISTANCES ON IMAGES 49

far more trials than we used). The arrays wererandomly divided into two sets, which were placedin separate books, and the order of arrays wasrandomized within each book (with the constraintthat no more than three consecutive targets couldbe of the same case).

We also constructed a tape recording. The tapecontained 72 trials of the form "1 ... cover . . .left . . . A." Each trial was coordinated with anarray in the books. The first word named the trialnumber and was followed 5 sec later by the wordcover (which was the signal to conceal the arrayand to construct an image). Two seconds thereafterthe word left or right was heard (indicatingpoint of focus, each word appearing on half of thetrials, as noted above). Finally, 3 sec after this,the name of a letter in the corresponding arraywas heard. Presentation of the letter delivereda pulse to a voice-activated relay that started areaction time clock (which was stopped by thesubject's pressing either of two response buttons).A new number was presented 10 sec after the let-ter, and the sequence was repeated with a newtrial.

Procedure. Written instructions describing theexperimental procedure were given to the subjectand then were reviewed orally by the experimenter.It was emphasized that we were interested instudying how people process visual mental images,and therefore we wanted the subject always touse an image in performing this task—even if thisdid not seem the most efficient strategy. Thesegeneral instructions preceded every experiment re-ported in this article. Before we are willing tomake inferences about imagery from data, we wantto be sure that those data were in fact producedvia imagery processing.

The subject was told that he or she would soonsee simple arrays of letters. We explained that thetask was to study an array and then to shut one'seyes and mentally picture the array as it appearedon the page. We would next ask the subject tofocus on one end of the image and then to scanto a igiven letter in the array. As soon as thetarget letter was clearly in focus, we wanted thesubject to "look" at the letter: If it were upper-case, he or she should push one button; if it werelowercase, the other button should be pushed.

Following this, we explained the meanings ofthe tape-recorded cues that accompanied the arrays.Upon hearing a number, the subjects were to turnto the next page in the book in front of them,which would have that number at the top (pageswere numbered consecutively). They should studythis array until hearing the word cover, at whichpoint they should cover the array with a smallpiece of cardboard and mentally image the array.While visualizing the array, they then would hearthe word right or left, directing them to "mentallystare" at that end of the line. They should continueto focus at that end until hearing the next word,which would be the name of a letter in the array.At this point the subjects were to scan to the

named letter and classify it according to its case.Eight practice trials (half upper- and half lower-case, in a random order) preceded the actual testtrials. The subjects were questioned during thesetrials to ensure that they were performing the taskas instructed. The subjects were asked to performthe task as quickly as possible while keeping errorsto an absolute minimum.

This procedure, then, prevented the subjectsfrom initially encoding an array differently de-pending on the point of focus or the distance ofa target or the number of intervening letters. Theorder of the two books was counterbalanced oversubjects, as was the hand (dominant/nondominant)assigned for indicating each case. Each person wasinterviewed at the conclusion of the 20-min taperecording and was asked to estimate the per-centage of time he or she actually followed instruc-tions while performing the task. Further, we askedeach subject to attempt to discern the purposesand motivations of the present experiment.

Subjects. Twelve Johns Hopkins Universitystudents volunteered to participate as subjects tofulfill a course requirement. Although 2 of thesepeople reported noticing distance effects during thecourse of the experiment, and 2 people reportedobserving that it was easier when there were nointervening items, no subject reported noticing botheffects, and no subject deduced any part of thehypothesis independently of noticing his or herbehavior during the task. Data from 1 additionalpotential subject were discarded because she esti-mated complying with the imagery instructionsonly 60% of the time, and data from another po-tential subject were discarded because his meanreaction times were more than twice the meansof all the other subjects. The 12 remaining sub-jects reported complying with the instructions atleast 75% of the time.

Results

An analysis of variance was performed onthe data. Only reaction times from correctresponses were used, and errors and wildscores were replaced by the mean of theother scores in that condition for that sub-ject. A wild score was defined as one thatexceeded twice the mean of the other scoresin that cell for that subject; only one scoreper cell could be so denned, however. Be-cause we wished to generalize over both sub-jects and items, we used the Quasi F sta-tistic, F' (Clark, 1973).

As expected, scanning times increased assubjects had to scan further 'distances toreach the target letter, F'(2, 30) = 9,89,p < .01. In addition, times also increasedwhen subjects had to scan across more in-


2.6

2.4

2.2

! 2.0

1.8

1 .6

< 1 . 4UJIE

1.2

1.0

2 l.L.

I l.L.

OI.L.

5.08 10.16 15.24

DISTANCE ON DRAWING (cm)

Figure 1. The results of Experiment 1 : Classifica-tion times when subjects scanned different distancesover zero, one, or two intervening letters (I.L.).

tervening letters before reaching the target,F'(2, 27) = 22.65, p < .01. Interestingly, asis evident in Figure 1, the effects of distancewere the same regardless of how many in-tervening items were scanned over; therewas no interaction between the two variables(F' < 1). This lack of interaction also indi-cates, of course, that the effects of inter-vening items were the same for each of thethree distances—which is what one wouldexpect if this effect reflects time necessaryto "inspect" each of the intervening letters.Finally, there was no difference in time tocategorize letters of different case or to scanleft versus right, nor were any other effectsor interactions significant.

Errors tended to increase with increasingreaction times. For the 5.08-, 10.16-, and15.24-cm conditions, errors were .7%, 3.1%,and 1.4%, respectively. Although errors forthe 10.16-cm distances were relatively high,they were not significantly higher than the

errors for the 15.24-cm condition (p > .1).For the zero, one, and two intervening itemconditions, errors were .7%, 2.1%, and2.4%, respectively. Thus, it does not appearas if speed-accuracy trade-offs affected thedata.

Discussion

We found that more time is required toscan further distances across an image. Inaddition, more time also is required whenone scans over more items. Our findingsargue against the idea that people were notreally scanning a spatial image but rathersimply processing a serially ordered list ofletters. If so, we should only have foundan effect of number of intervening items (ifscanning the list were self-terminating).There is no reason to expect such a list tohave metric distance from each end to beassociated with each letter. Furthermore,we found effects of distance even when thetarget letter was not separated from thefocus point by any intervening letters. Fi-nally, we found that it took the same amountof time to scan right to left as it did to scanin the opposite direction. This last resultreplicates that of Kosslyn (1973) when hissubjects were asked to remember and thento scan visual images (left-to-right scan-ning was easier, however, when subjectsencoded and used verbal descriptions of thepictures instead of images). Thus, imagescanning would seem to involve processesor mechanisms different from those highlypracticed ones used during reading.

Given the existence of two independenteffects of distance and number of interven-ing letters, one might be tempted to askwhich factor is the more important. This isa nonsensical question: By increasing therange of distances, we surely could makedistance account for the lion's share of thevariance in scanning times—and by decreas-ing the range of distances, we could diminishthe importance of this variable. In addition,we could probably manipulate the impor-tance of number of intervening items bymaking the distractors more or less difficultto discriminate from the targets. Further-more, the present claim is not that distance


is more important than other variables, butonly that images do preserve metric distanceinformation—and that such information canbe used in real-time processing, affecting theoperating characteristics of cognitive pro-cesses.

One might argue that the effects of dis-tance on scanning time really reflect nothingmore than the enthusiastic cooperation ofour subjects, who somehow discerned thepurpose of the experiment and manipulatedtheir responses accordingly. Although 2 ofour subjects did hypothesize distance effects,they claimed to do so by introspecting upontheir performance during the task; no sub-ject confessed to consciously manipulatinghis or her responses. Nevertheless, we wouldbe more comfortable with a task that wasmore difficult to second-guess and manipulate.

Experiment 2

This experiment involves scanning be-tween the 21 possible pairs of seven loca-tions on an imaged map. Each of these dis-tances was different, and the task seemedsufficiently complex to thwart any attemptsto produce intentionally a linear relationshipbetween distance and reaction time. Sincethe critical question is whether images pre-serve metric information, it is importantthat scanning times be a function of someknown distance—otherwise, variations inscanning time cannot be taken to necessarilyreflect amount of distance traversed. Thus,we wished to ensure that subjects scannedonly the shortest distance between twopoints. In order to do so, we altered theinstructions slightly and asked these peopleto imagine a black speck moving along adirect path across the image. After memo-rizing the map, these subjects imaged it,focused on a location, and then decidedwhether a given named object was in facton the map. If so, the subjects were askedto scan to the named object on the imageand to push a button when they "arrived"there; if not, they pushed another button.The time necessary to scan between all pos-sible pairs of locations was measured. Asbefore, we expected times to increase with

distance (although not necessarily linearly,as rates may be variable).

Method

Materials. A map of a fictional island was con-structed containing a hut, tree, rock, well, lake,sand, and grass. Each distance between all 21pairs was at least .5 cm longer than the nextshortest distance. The precise location of each ob-ject was indicated by a red dot; these locations areindicated by a small x in Figure 2.

A tape recording was constructed containing 84pairs of words. Each location was named 12 timesand then followed 4 sec later by another word;on 6 of these trials, the second word did not namea location on the map. The "false" objects werethings that could have been sensibly included onthe map (e.g., "bench"). On the other 6 trials, thefirst word was followed by the name of each ofthe other locations. Thus, every pair of locationsoccurred twice, once with each member appearingfirst. The order of pairs was randomized, with theconstraint that the same location could not occurtwice within three entries, and no more than 4true or 4 false trials could occur in a row. Pre-sentation of the second word also started a clock.A new trial began 8 sec after the probe word waspresented. The test trials were preceded by 8practice trials naming pairs of cities in the UnitedStates for "true" items.

Procedure, The subjects first were asked tolearn the locations of the objects on the map bydrawing their relative positions. The subjects beganby tracing the locations on a blank sheet placed

Figure 2. The fictional map used in Experiment 2.


2.1

1.9

8 , . 7

tuz

1.5

LUo:

.96 8 10 12

DISTANCE (cm.)

14 16 18

Figure 3. The results of Experiment 2: Time to scan between all pairs of locations on theimaged map.

over the map, marking the locations of the reddots centered on the objects; this procedure al-lowed them to see the locations themselves in iso-lation. Next, they studied the map, closed theireyes and imaged it, and then compared their imageto the map until they thought their image wasaccurate.

The map then was removed, and the subjectsdrew the locations on a blank sheet of paper. Fol-lowing this, the subjects were allowed to comparetheir drawings with the original. This procedurewas repeated until all points were within .64 cmof the actual location. Between 2 and 5 drawingswere required for subjects to reach this criterion.

Next, subjects were told that they would hearthe name of an object on the map. They were topicture mentally the entire map and then to focuson the object named. Subjects were told that 5 secafter focusing on the named object, another wordwould be presented; if this word named an objectdepicted on the map, the subjects were to scan toit and depress one button when they arrived at thedot centered on it. The scanning was to be ac-complished by imaging a little black speck zippingin the shortest straight line from the first objectto the second. The speck was to move as quicklyas possible, while still remaining visible. If thesecond word of a pair did not name an object onthe map, the subjects were to depress the secondbutton placed before them. The clock was stoppedwhen either button was pushed, and response timeswere recorded. As before, we interviewed subjectsin the course of the practice trials, making surethat they were following the instructions aboutimagery use.

Subjects. Eleven new Johns Hopkins Univer-sity students served as paid volunteers in this ex-periment. Data from 2 additional people were notanalyzed because, when queried afterwards, theyreported having followed the imagery instructionsless than 75% of the time during the task.

Results

Only times from correct "true" decisions(where a distance was actually scanned)were analyzed. As before, wild scores wereeliminated prior to analysis. A wild scorewas now defined as one twice the size of themean of the other score for that distanceand the scores for the next shortest andlongest distances; only one score in any ad-jacent row of six could be so eliminated.Data were analyzed in two ways, over sub-jects and over items. We first analyzed eachsubject's times for the different distances inan analysis of variance. As expected, timesconsistently increased with increasing dis-tance, F(20, 200) = 13.69, p< .001. In ad-dition, we averaged over subjects and cal-culated the mean reaction time for each pair.The best fitting linear function was calcu-lated for these data by the method of leastsquares; not only did times increase linearly


with increasing distance but the correlationbetween distance and reaction time was .97.These data are illustrated in Figure 3.

Errors occurred on only 1.3% of the trialsand were distributed seemingly at random;more errors did not occur for the shorterdistances. Finally, subjects drew maps afterthe experiment. Not surprisingly, the cor-relation between the drawn and actual dis-tances between all possible pairs of pointswas quite high, r = .96.

Discussion

Time to scan across visual mental imagesagain increased linearly with the distance tobe scanned. This demonstration supportsthe claim that images are quasi-pictorial en-tities that can in fact be processed and arenot merely epiphenomenal. One of the de-nning properties of such a representation isthat metric distances are embodied in thesame way as in a percept of a picture, andthe present data suggest that this character-istic is true of visual mental images.

Interestingly, a number of subjects re-ported that they had to slow down whenscanning the shorter distances, because thefour objects at the lower left of the mapwere "cluttered together." The data show nosign of this, however, providing furthergrounds for taking with a grain of salt sub-jects' interpretations of their introspections.This experiment seems immune to the po-tential failings of Experiment 1; somewhatsurprisingly, no subjects reported suspectingthe hypothesis when it was explained tothem afterwards.

Experiment 3

Given the results of the first two experi-ments, how can we explain Lea's (1975)failure to find increases in reaction times asdistances increased? We earlier suggestedthat this failure was a consequence of hisinstructions: Subjects were not told to baseall judgments on consultation of their im-ages, but only to start off from an imagedlocation and to "scan" a certain number ofobjects from there. Although these peopleinitially began with an image, the actual

decisions could have been generated via pro-cessing of items in a list. If so, only ordinal—and not interval—relations among items(objects in the array, in Lea's case) shouldaffect time to sort through the list. Effectsof actual distance ought to occur only whenone scans the spatial image itself, whichseems to represent interval informationabout distance. If we find distance effectseven when people do not scan images, we arein trouble: We could not then infer that ef-fects of metric distance implicate scanningof quasi-pictorial images.

A second hypothesis for why Lea failedto obtain effects of distance on time to scanalso involves his instructions. Lea did notinsist that his subjects always construct theentire array ahead of time; instead, subjectswere told simply to image a starting placeand then to decide which object was somenumber of locations away. Perhaps distanceonly affects time to shift attention betweenlocations in an image when the locations areboth "in view" simultaneously. That is, ifan entire image is not kept in mind at once,the distance relations between visible andinvisible locations may not be represented;these relations could be an "emergent" prop-erty of constructing the whole image fromits component parts. One might shift to an"invisible" part by generating a sequence ofindividual images representing interveninglocations and not by actually scanning acrossan image. In this case, interval distancewould not be expected to affect time to shiftattention between parts.

The following experiment examines thehypotheses described above. In one group,subjects were asked to focus on a given lo-cation on an image of the map used in Ex-periment 2 and then to judge whether anamed object was on the map. Unlike thepeople in Experiment 2, however, these peo-ple were not required to consult their imageswhen making their judgments, but simplywere asked to reach decisions as quickly aspossible. In a second group, subjects alsoperformed the basic task of Experiment 2,but with one major modification: When fo-cusing on the initial location, these peoplewere asked to "zoom in" on it until that


object filled their entire image, causing theremainder of the island to "overflow." Thesepeople were told, however, that they must"see" an image of the second named objectbefore responding positively (if in fact itwas on the map). The two groups, then,were each instructed to perform in a waythat Lea's subjects may have acted spon-taneously.

Method

Materials. The same materials used in Experi-ment 2 were also used here.

Procedure. Subjects in both groups learned todraw the map as did subjects in the previous ex-periment. The procedure differed from that of Ex-periment 2 only in the following ways:

1. Rapid Verification Control Group. These sub-jects were given instructions like those of Experi-ment 2, except that no mention was made of scan-ning the image. After focusing on an initiallynamed object, these people were simply to decideas quickly as possible whether the second objectof a pair was in fact on the map. As before, sub-jects were urged to keep errors to a minimum.

2. Image Overflow Group. These subjects weregiven instructions that differed from those of Ex-periment 2 in two ways: First, these people wereasked to "zoom in" on the initially named objectuntil the rest of the map had "overflowed" (i.e.,

was no longer visible in) their image. Second, theywere instructed to be sure to "see" a second namedobject of a pair before responding positively. Thesesubjects were not told to scan to the second objectif it was on the map but only to be sure to "see"it prior to responding; no mention was made of aflying black speck or the like. As before, speedwith accuracy was stressed in both groups.

Subjects. Twenty-two new Johns Hopkins Uni-versity students volunteered as paid subjects inthis experiment. Half of these subjects were ran-domly assigned to one group, half to the other.An additional 3 people were assigned to the ImageOverflow Group but were not included, becauseafter the experiment they reported having fol-lowed the instructions less than 75% of the time.

Results

Data were analyzed as in Experiment 2.In the Rapid Verification Group, there weresignificant differences in time to evaluate dif-ferent pairs, F(20, 200) = 2.59, p < .01. Asis evident in Figure 4, however, times didnot increase systematically with distance. Infact, the relationship between distance andverification time was negligible, r = .09.

In the Image Overflow Group, in con-trast, times did increase systematically withdistance. Not only were times to evaluate

Ul(E

0.9

0.3

Ov«rflo*i-,89

Control

i «,09

6 8 10 12

DISTANCE (cm.)

16 IB

Figure 4. The results of Experiment 3: The effects of distance on response times for theImagery Overflow and the Rapid Verification Control Groups.


different pairs significantly different fromeach other, F(20, 200) = 4.59, p < .01, butthere was a respectable correlation betweendistance on the map and evaluation time,r = .89.

We also performed three additional anal-yses of variance, one comparing the resultsfrom each group with the data obtained inExperiment 2 and one comparing the twogroups with each other. Not surprisingly,there were less effects of distance in datafrom the Rapid Verification Control Groupthan in the Image Overflow Group or inExperiment 2 (/> < .01 for the interactionof distance and instructions in both cases).In addition, subjects in the Rapid Verifica-tion Control Group made decisions morequickly than those in either other condition(p < .01 for both comparisons). The com-parison between the results of the ImageOverflow Group and the findings of Experi-ment 2 produced a somewhat surprisingresult, or rather, lack thereof: In this case,the effects of distance were identical for bothinstructions (F < 1). Furthermore, therewas no significant difference overall inverification times (the mean for Experiment2 was 1.428 sec vs. 1.685 sec for the ImageOverflow Group), F(l, 20) = 1.04, p > .1.If "zooming in" increases the subjectivesize of an image, it should also increase the"distance" between portions of that image;hence, we would have expected that moretime should have been required by subjectsin the Image Overflow Group.

The error rate in the Rapid VerificationControl Group was 3.3%, whereas therewere only 1.47" errors in the Image Over-flow Group. As before, errors did not tendto increase with shorter distance for theImage Overflow Group, and seemed ran-domly distributed for the Rapid Verifica-tion Control Group. No subjects deducedthe purposes or predictions of this experi-ment.

Discussion

When people were not required to basedecisions upon consultation of their images,evaluation times did not increase with the

distance between a focus point and a probedobject that was in fact on the map. Thisresult allows us to argue against a non-imagery interpretation of the scanning re-sults obtained in the preceding experiments:If the effects of distance obtained previouslywere due to local activation and scanningthrough an abstract list structure (e.g.,perhaps a graph with "dummy nodes" in-terposed to mark off increasing distance),then we should have found effects of dis-tance here. Distance per se seems to affectresponse times only when people actuallyscan their images. Thus, Lea's (1975) re-sults may simply reflect the fact that hissubjects were not told to respond onlyafter seeing the probed object in their image.Clearly, before we draw inferences aboutimage processing from some data, we mustbe certain that such data were producedwhen people did in fact use their images.The instructions administered in the presentexperiments and elsewhere (Kosslyn, 1973,1975, 1976) seem capable of inducing sub-jects to use imagery, even if other means ofperforming a task are available.

Lea's results were probably not a con-sequence of subjects' not having the entirearray in their images prior to processing it,as witnessed by the results of the ImageOverflow Group. Although these peopleonly had the focus location in their images,times nevertheless increased with distanceto a probed object. We were surprised bythese results, which were not expected. Thisfinding seems to indicate that one may con-struct images such that portions are "waitingin the wings," ready to be processed if neces-sary. Thus, subjects seemed to have scannedto parts that were not visible initially intheir images but were available in a non-activated portion of the image.

There is one hitch in the above explana-tion of the data obtained from the ImageOverflow Group: If these people "zoomedin" closer to the imaged map than did thosein Experiment 2, the subjective distances be-tween parts should have been greater inthe Image Overflow condition. If so, thenmore time should have been required toscan these enlarged images, which was not


the case. One explanation of this disparityrests on a procedural difference between theImage Overflow condition and Experiment2: Subjects in Experiment 2 were instructedto image a small black speck flying betweenparts. This task may have required moreeffort than the simple shift-of-attention in-struction used in the present experiment,and thus slowed down scanning. In addition,it is possible that subjects in the two ex-periments simply scanned at different rates:If people in the Overflow condition scannedrelatively quickly, perhaps because distancestraversed were on the average relativelylarge, then we would not necessarily expectany differences in scanning times betweenthe two conditions. The following experi-ment eliminated the difference in instruc-tions and used a within-subjects design; wehoped that a given person would adopt aconstant scanning rate for different ma-terials.

Experiment 4

In this experiment we investigatedwhether more time is required to scan acrosssubjectively larger images. We worried thatif we used stimuli as complex as those in-cluded on the map, people might have to"zoom in" (if the image were small) or"pan back" (if it were large) in order to"see" parts clearly. Kosslyn (1975) demon-strated that parts of subjectively smaller im-

Figure 5. The schematic faces used in Experiment 4.

ages are more difficult to identify than partsof larger ones, and this may also be true ofparts of "overflowed" images. Not onlycould difficulty in identifying parts of rela-tively complex images obfuscate effects ofscanning images of different subjective sizes,but people may adjust their scanning ratesin accordance with the difficulty in identify-ing parts. Pilot data lent credence to thesefears, encouraging us to use more simplestimuli, where the parts were readily identi-fiable.

Thus, in this experiment people imagedone of three schematic faces at one of threesubjective sizes. The faces had either lightor dark eyes, and the eyes were one of threedistances from the mouth. These peoplefirst focused on the mouth of an imagedface and then shifted their attention to theeyes and decided whether a probe correctlydescribed them. As in Experiment 1, theseinstructions made no mention of a flyingspeck or the like. If distances determinescanning times, then subjectively smallerimages should be scanned more quickly thanlarger ones. In addition, the effects ofincreased distance should become more pro-nounced with larger images, since when sizeis multiplied, so are the distances.

Method

Materials. Six schematic faces were con-structed. The eyes were 7.62, 10.16, or 12.70 cmabove the mouth; for each distance, one face wasconstructed with light eyes and one was con-structed with dark eyes. The faces are illustratedin Figure 5.

Twelve copies of each face were made and usedin nine basic conditions, each of which was repre-sented by eight stimuli. These conditions weredefined by three subjective sizes—overflow, fullsize, and half size—and the three distances.Within a condition, half of the faces had lighteyes and half had dark eyes. Further, half of thefaces with each eye color were paired with theword dark and half with the word light on an ac-companying tape recording, producing an equal dis-tribution of true and false probes. The faces werethen randomized and placed in a booklet, withthe constraint that no given distance or sizecould occur twice within 3 trials.

A tape recording also was made. This tape con-tained stimuli consisting of three parts: First, thenumber of the trial was given. Second, 5 sec laterthe word cover was presented, followed 1 sec


later by one of three cues—overflow, full size, orhalf size. These stimuli indicated the size at whichthe subject should construct his or her image.Finally, 5 sec later the word light or dark waspresented, which also started a clock. Ten secafter this, a new number was presented and an-other trial began. For half of the trials in eachsize condition, the final word described the eyesof the imaged face, and for half it did not. The72 test trials were preceded by 8 practice trials.

Procedure. The subjects were told that theywere going to see schematic faces one at a time.As soon as a trial number occurred on a taperecording, they should turn to the correspondingpage of the book in front of them, exposing adrawing of a face. The subjects were asked tostudy the drawing well enough to form an ac-curate visual mental image of it with their eyesclosed. After 5 sec, the subjects would hear theword cover, at which point they would concealthe face with a small piece of cardboard; shortlythereafter they would hear a size specification,either overflow, full size, or half size. Upon hear-ing the word overflow, the subjects were to imagethe face so large that only the mouth was visible.Upon hearing full size, they were to image it aslarge as possible while still being able to "see"all of it at once in their image; as soon as thisimage was constructed, they were to mentallyfocus on the mouth and wait there until hearingthe next stimulus on the tape. Upon hearing halfsize, they were to image the face at half of thelength of the full-size version, again focusing onthe mouth. Following this, the subjects were toldthey would hear either the word light or dark. Atthis point, they were to "glance" up at the eyes intheir image and see if they were appropriatelydescribed. If so, they were to push one button;if not, they were to push the other. Hand ofresponse was counterbalanced over subjects; asbefore, the clock stopped as soon as either buttonwas pushed, and response times were recorded.Subjects were asked to respond as quickly aspossible, but always to base decisions on inspectionof the image (as in Experiment 1). During the8 practice trials preceding the test items (halftrue, half false, including all three size condi-tions and all three distances), the subjects wereasked repeatedly to describe their mental activity,and any misconceptions about the task were cor-rected.

Subjects. Sixteen new Johns Hopkins Uni-versity students volunteered to participate for pay;data from an additional subject were discardedbecause this person reported not following theinstructions at least 75% of the time.

Results

Only times from correct decisions wereincluded in an analysis of variance; errorsand occasional wild scores (denned as in

Experiment 1) were replaced by the meanof the remaining scores in that conditionfor that subject. As expected, times in-creased with further separation between themouth and eyes, F(2, 30) = 10.81, p < .01.In addition, times increased as subjectivesize of the image increased, F(2, 30) =17.33, p < .01. As is evident in Figure 6,increases in distance did have increasinglylarger effects as the subjective size in-creased; the interaction between size anddistance was in fact significant, F(4, 60)= 3.47, p < .025. Examination of Figure 6reveals, however, that the effects of distancewere not appreciably different in the full-size and half-size conditions. A marginallysignificant interaction between type of re-sponse (true or false) and distance, F(2,30) = 2.80, .05 < p < .10, led us to considerseparately data from true and false re-sponses. As in the main analysis, dis-tance and size both affected decision timesfor both types of responses (p < .01 in allcases in separate analyses of variance of thetrue and false responses). However, whereasthe effects of distance increased with sizefor true responses, F(4, 60) = 5.58, p < .01,they did not increase for false responses(F < 1). Furthermore, for true responsesthere was some difference between the ef-fects of distance in the full-size and half-sizeconditions. We observed that times in-creased an average of 109 msec for everyadditional 2.54 cm separating the eyes andmouth on the face in the half-size condition.On this basis, we predicted that time to scana face twice as long ought to be 2.050, 2.274,and 2.486 sec, respectively, for the three in-creasing distances. These predictions wereclearly off the mark; a chi-square test com-paring these expected results with the ob-served results was very significant, x2 =

23.8, p < .001. We then considered the pos-sibility that our subjects adjusted not thelength of their images, but the area. If so,then we expected that 1.858, 2.019, and2.167 sec, respectively, should be required toscan the three distances on a full-sizedimage; these estimates also failed to fit thedata, x2 = 33.18, p < .001. This failure wasmuch more severe for the middle distance


3.2

3.0

2.8

2.6

U)

-2.4oU0»

- 2.2wS

F 2.0zot-< 1.8LU

1.6

1.4

• • "TRUE" RESPONSES

0—0 "FALSE" RESPONSES

OVERFLOW

FULL SIZE

HALF SIZE

7.62 10.16 12.70

DISTANCE ON DRAWING ( c m )

Figure 6. The results of Experiment 4: The timerequired to classify eyes located three distancesfrom the mouth of faces imaged at three subjectivesizes (overflow, full size, and half size).

than the ends (the deviation from the ex-pected for the shortest distance was not sig-nificant [p > . 2 ] , whereas the deviation forthe longest was barely significant at the .05level). We then speculated that subjectsneither halved the lengths nor halved theareas but reduced size by performing somekind of a compromise between the two.Thus, we simply averaged our estimatesfrom the two procedures and discoveredthat these means did not deviate significantlyfrom the actual observed mean reactiontimes, x2 = 5.06, p > .05; again, the bestfit here was with the two extreme distances,

Finally, error rates again tended to bepositively correlated with reaction times.For true responses, error rates for the 7.62-,10.16-, and 12.70-cm stimuli were 6.25%,4.69%, and 6.25% for the overflow condi-tion; 1.56%, 1.56%, and 4.69% for the full-size condition; and 7.81%, 1.56%, and3.12% for the half-size condition. For thefalse responses, error rates for the 7.62-,10.16-, and 12.70-cm stimuli were 7.81%,7.81%, and 10.94% for the overflow con-

dition; 3.12%, 7.81%, and 6.25% for thefull-size condition; and 4.69%, 3.12%, and1.56% for the half-size condition. In twocases, the 7.62-em items incurred moreerrors than the 12.70-cm ones (true halfsize, false half size); * tests evaluating thesedifferences were not significant, but the"true" comparison was marginal, £(15) =1.86, .05 <• />< . ! . Thus, the faces incor-porating the shortest distance from themouth to the eyes may have been evaluatedfaster than they should have been, becauseof a lowered response criterion. If so, thenthe slope of the half-size condition (i.e.,effects of increased distance on scanningtime) may be steeper than is merited byscanning effects per se. In addition, in onecase the errors for corresponding distanceswere greater for the half size than the over-flow condition (true, 7.62 cm); this differ-ence was not significant, tf(15)= 1.00, p >.1, belying a speed-accuracy trade-off here.Finally, no subject deduced the purposes ormotivation of this experiment.

Discussion

As expected, people again required moretime to scan further distances across theirimages. This was reflected in three results:First, times increased with further separa-tion between the mouth and eyes of theimaged stimuli; second, more time was gen-erally required to scan across subjectivelylarger images; and third, there were in-creasingly large effects of increased distance(on the stimuli) for subjectively largerimages. This last result was observed onlywith "true" responses, however. Althoughthere was some difference in slope (i.e., theeffects of increases in distance on the face)between the full-size and half-size condi-tions, these differences were not as large aswould be expected if length were varied.This may have been because (a) peoplesometimes varied the area of their imagesand sometimes varied the length, or usuallyused a compromise of the two measureswhen determining how to scale the images,and/or (b) people may have performedsome other sort of processing when evaluat-ing short distances on subjectively small


images. That is, with the half-size images,the 7.62-cm separation may have seemedso slight that the eyes were visible even asone focused on the mouth. If so, scanningmay not have been necessary to evaluatethe imaged eyes, and these times thus mayhave been faster than predicted. This wouldresult in a larger difference between thetimes necessary to evaluate eyes of faceswith short and long distances than we ex-pected—and hence less of a difference in theeffects of increased distance in the half- andfull-size conditions. The error rates sug-gested that subjects may in fact have beendoing some more rapid, but less cautious,processing for the shortest distance in thehalf-size condition.

The failure to obtain slope differences fordifferent-sized images on the false trials isnot easily explained. There is some evidence,however (see Kosslyn, 1975), that peoplehave more difficulty in using images to arriveat a "false" decision; the present data maysimply reflect inconsistent use of imageryon the trials where the probed color wasnot in fact on the image.

Finally, it is worth noting that the resultsof this experiment allow us to eliminate onemore possible nonimagery interpretation ofthe scanning effects. That is, one could claimthat the closer two objects or parts are, themore likely it is that they will be groupedinto the same "chunk" during encoding.Presumably, parts encoded into the samechunk are retrieved in sequence morequickly than parts in different chunks. Inthis experiment, size of an image was notmanipulated until after the drawing was re-moved, precluding systematic differences inencoding among the three size conditions.Thus, the fact that subjectively larger im-ages generally required more time to scanthan did smaller ones seems to run counterto the notion that spatial extent affected scantimes only because of a confounding betweendistance and the probability of being en-coded into a single unit.

General Discussion

The present experiments converge indemonstrating that people can scan the dis-tances embodied in images. More time was

required to scan further distances, evenwhen the same number of items fell betweenthe focus and target locations. In addition,subjectively larger images required moretime to scan than did subjectively smallerones. Somewhat surprisingly, we found thatthe effects of distance persisted even when aperson "zoomed in" on one part, such thatthe remainder of the image seemed to over-flow. These results suggest that a part ofan image may exist "waiting in the wings,"ready to be activated into consciousness ifneeded. Finally, there were no effects of dis-tance on decision times when people did notactually use their images, even though animage had been generated and focused upon.These results taken together indicate thatimages are pictorial in at least one respect:Like pictures, images seem to embody in-formation about actual interval spatial ex-tents. The present experiments support theclaim that portions of images depict cor-responding portions of the represented ob-ject (s) and that the spatial relations be-tween portions of the imaged object (s) arepreserved by the spatial relations betweenthe corresponding portions of the image.These qualities are apparent in our intro-spections, and the present experiments sug-gest that people can operate on the repre-sentations we experience as quasi-pictorialmental images.

Given our results, how do we account forLea's (1975) failure to find systematic ef-fects of distance on evaluation times? First,the results of Experiment 3 suggest thatLea's results may simply reflect his failureto ensure that subjects responded only after"seeing" the target in their image. If leftto their own devices in making decisions,subjects would probably find a nonimagerystrategy to be faster, and such a nonimagerystrategy would not result in distance in-fluencing decision times. Second, Lea's taskwas so difficult (the mean reaction timesreach as high as 8 sec) that effects of dis-tance (which are measured in milliseconds,not seconds) may simply have been drownedout by the nonscanning components of thistask. Finally, even if imagery were used,Lea's ordered search task, which involvedcounting successive items, may have induced


subjects to generate a sequence of separateimages, each representing an object in thearray, rather than to attempt to hold andthen scan a complex image (see Kosslyn,1975, for evidence that more complex imagesare more difficult to maintain). Weber,Kelley, & Little (1972) report that peoplecan "verbally prompt" sequences of images,and something like this may have occurredin Lea's experiment. If so, then we have noreason to expect distance to affect responsetimes.

In closing, it seems worthwhile to con-sider briefly two possible conceptualizationsof how image scanning might operate. Themost obvious notion (two variants of whichwere suggested by Kosslyn, 1973) is thatscanning consists of moving an activatedregion over a spatial representation, some-what like moving a spotlight across an unlitbillboard. However, the spatial display usedin representing images presumably also isused in representing sensory input from theeyes (see Hebb, 1968; Segal & Fusella,1970) and hence only need represent in-formation from some limited visual arc thatcorresponds to the scope of the eyes. If so,then one ought to find that one "hits theedge of the billboard" if one scans too farin any given direction. Many people report,however, being able to scan to objects "be-hind" them in an image and even beingable to scan in a seemingly continuous circleacross all four walls of an imaged room.There is another way of conceptualizingimage scanning that deals with these sortsof observations naturally and easily.

In the Kosslyn & Shwartz (1977) com-puter simulation of visual mental imagery,it was most elegant to treat scanning as akind of image transformation, in the sameclass as mental rotation and size alteration.Here, scanning consists of moving an imageacross the image display structure, the cen-ter of which is posited to be most highlyactivated (and hence, portions of the imagefalling under the center are most sharply infocus). In this case, the analogy would beto moving a billboard or sequence of bill-boards under a fixed spotlight. Accordingto this notion, then, scanning 360° around

one in an image would be accomplished bycontinuously constructing new material atthe edge and shifting it across the imagedisplay. If nothing else, this approach mayhave heuristic value by leading us to lookfor similarities among scanning and otherimage transformations.

In conclusion, the present results convergein supporting the claim that the experienced,quasi-pictorial surface image is functionaland is not simply an epiphenomenal con-comitant of more abstract "deep" proc-esses. Comprehensive models of memorywill probably have to include more than thesort of prepositional list structures cur-rently in vogue (e.g., Anderson, 1976;Anderson & Bower, 1973).

References

Anderson, J. R. Language, memory, and thought.Hillsdale, N.J..: Erlbaum, 1976.

Anderson, J. R., & Bower, G. H. Human associa-tive memory. New York: Wiley, 1973.

Clark, H. H. The language-as-fixed-effect fallacy:A critique of language statistics in psychologicalresearch. Journal of Verbal Learning and VerbalBehavior 1973; 12, 33S-3S9.

Hebb, D. 0. Concerning imagery. PsychologicalReview, 1968, 75, 466-477.

Kosslyn, S. M. Scanning visual images: Somestructural implications. Perception & Psycho-physics, 1973, 14, 90-94.

Kosslyn, S. M. Information representation invisual images. Cognitive Psychology, 197S, 7,341-370.

Kosslyn, S. M. Can imagery be distinguishedfrom other forms of internal representation?Evidence from studies of information retrievaltime. Memory & Cognition, 1976, 4, 291-297.

Kosslyn, S M., & Pomerantz, J. R. Imagery,propositions, and the form of interval rep-resentations. Cognitive Psychology, 1977, 9, 52-76.

Kosslyn, S. M., & Shwartz, S. P. A simulationof visual imagery. Cognitive Science, 1977, 1,265-295.

Lea, G. Chronometric analysis of the method ofloci. Journal of Experimental Psychology: Hu-man Perception and Performance, 1975, 1, 95-104.

Segal, S. J., & Fusella, V. Influence of imagedpictures and sounds on detection of visual andauditory signals. Journal of Experimental Psy-chology, 1970, 83, 458-464.

Weber, R. J., Kelley, J., & Little, S. Is visualimagery sequencing under verbal control ? Jour-nal of Experimental Psychology, 1972, 96, 354-362.

Received May 25, 1977 •

Kosslyn (1978) Visual images preserve metric spatial ...wexler.free.fr/library/files/kosslyn (1978) visual images...Journal of Experimental Psychology: Human Perception and Performance

Documents