The Whole Is Equal to the Sum of Its Parts: A ...€¦ · result less than, equal to, or greater than the sum of its parts? Or perhaps the combination is just different as some Gestalt

The Whole Is Equal to the Sum of Its Parts: A Probabilistic Model ofGrouping by Proximity and Similarity in Regular Patterns

Michael Kubovy and Martin van den BergUniversity of Virginia

The authors investigated whether the gestalt grouping principles can be quantified and whether theconjoint effects of two grouping principles operating at the same time on the same stimuli differ from thesum of their individual effects. After reviewing earlier attempts to discover how grouping principlesinteract, they developed a probabilistic model of grouping by proximity, which allows measurement ofstrength on a ratio scale. Then, in 3 experiments using dot lattices, they showed that the strength of theconjoint effect of 2 grouping principles—grouping by proximity and grouping by similarity—is equal tothe sum of their separate effects. They propose a physiologically plausible model of this law.

Keywords: perceptual organization, vision, additivity, grouping by proximity, grouping by similarity

Gestalt phenomena are illustrated in all psychology textbooks be-cause they are striking examples of psychological emergent proper-ties, namely, properties that a whole does not share with its parts. Thefirst to study these phenomena empirically came to be known asGestalt psychologists. Unfortunately their thinking has the reputationof being vague and wooly, a view vividly voiced by the founder ofmodern visual science, Marr (1982), who wrote the following:

. . . the Gestalt school of psychology . . . was concerned with describ-ing the qualities of wholes by using terms like solidarity and distinct-ness, and with trying to formulate “laws” that governed the creation ofthese wholes. The attempt failed for various reasons, and the Gestaltschool dissolved into the fog of subjectivism. (p. 8)

In this article, we propose a theory of certain “qualities ofwholes” and formulate laws that “govern the creation of thesewholes”; we have tried to make it fog free and objective. Thequalities of wholes we explore are perceptual groupings formedconjointly by pairs of grouping principles; the laws we formulategovern the conjoint effects of such pairs.

Figure 1, an illustration in a seminal article by the father ofGestalt psychology (Wertheimer, 1923/1938, pp. 74–75), makesour problem concrete. In Figure 1A, grouping by proximity andgrouping by similarity join forces, giving the impression that they

both contribute to the strength of grouping into columns. But inFigure 1B, the two principles are opposed, creating an unstablepercept that fluctuates between grouping by columns and groupingby rows. These demonstrations raise the question that occupies ushere: If proximity and similarity join forces to produce a percep-tual organization, how much stronger is it than the organizationproduced by each of them alone? Likewise, if proximity andsimilarity are opposed, how much weaker is the resulting percep-tual organization than the organizations they produce separately?

In other words, when two grouping principles are conjoined, howdo the strengths of the individual grouping principles combine? Is theresult less than, equal to, or greater than the sum of its parts? Orperhaps the combination is just different—as some Gestalt psychol-ogists were fond of saying (Koffka, 1935/1963, p. 176)—and thealgebraic notion of sum is meaningless (for introductions to theseissues, see Corning, 2002; Harte, 2002; Nagel, 1961). In this articlewe show that the strength of a grouping produced by the conjoiningof two principles of grouping equals the sum of the strengths of thegroupings produced by each principle alone.

The first to broach this subject were Koffka (1915/1938), whoconcluded that gestalts are nonadditive emergent percepts (Ellis,1938, pp. 376–377), and Wertheimer, who at the time he formulatedthe laws of perceptual organization, realized that Figure 1 raisedinteresting questions. But it was Kohler (1920/1938) who first had aninkling that such questions could be addressed mathematically whenhe distinguished between weak and strong gestalts (Ellis, 1938, p. 29)and thought of them as forces (Ellis, 1938, p. 52; see also Lewin,1926, excerpted in Ellis, 1938, p. 287).

Along those lines, Koffka (1935/1963) proposed that we thinkof “group formation as due to actual forces of attraction betweenthe members of the group” (pp. 165–166). To investigate thisquestion, he created a demonstration (see Figure 2), which con-sisted of differently arranged rows of lines that could be blue or redand straight or wavy and could be either 1 or 1.6 units apart. Hereached three conclusions from the comparison of grouping inthese patterns. (a) The relative strength of grouping by proximityand grouping by similarity is independent of whether proximitybetween adjacent elements varies within a row (see Figures 2A and

Michael Kubovy and Martin van den Berg, Department of Psychology,University of Virginia.

Martin van den Berg is now at the Department of Psychology, CaliforniaState University, Chico.

This work was supported by National Eye Institute Grant R01 EY12926-06.We thank S. Grossberg and T. Lachman for their insightful advice; O. Da Pos,D. T. Devereux, W. Epstein, S. Gepshtein, J. Shatin, L. Strother, and C. vanLeeuwen for their advice and comments; and D. T. Gies and J. D. Wright for theirhospitality and support aboard the M. V. Explorer. We conducted statisticalanalyses and generated the data figures using the indispensable R language for dataanalysis and graphics (Ihaka & Gentleman, 1996).

Correspondence concerning this article should be addressed to MichaelKubovy, Department of Psychology, University of Virginia, Charlottes-ville, VA 22904-4400. E-mail: [email protected]

Psychological Review Copyright 2008 by the American Psychological Association2008, Vol. 115, No. 1, 131–154 0033-295X/08/$12.00 DOI: 10.1037/0033-295X.115.1.131

131

2B) or is uniform (see Figures 2D and 2E). (b) Grouping by shapesimilarity is stronger than grouping by color similarity because (1)when they are in competition with grouping by proximity, it iseasier to group elements by shape similarity (see Figure 2B) thanby color similarity (see Figure 2A), and (2) when they are not incompetition with grouping by proximity, it is also easier to groupelements by shape similarity (see Figure 2E) than by color simi-larity (see Figure 2D).

It would of course be a mistake to conclude from demonstra-tions such as this that one principle of grouping is always strongerthan the other. The strength of a grouping principle dependsentirely on the nature of the elements to be grouped. For example,in Koffka’s (1935/1963) demonstration we could always reducethe amplitude of the wavy lines enough so that grouping by colorsimilarity would be stronger than grouping by shape similarity.

Koffka (1935/1963) did not formulate the important principlesuggested by his observations: If, for all strengths of groupingprinciple A, the ordering of strengths of grouping principle B isinvariant, then the two principles operate independently. He did,however, come close to reaching one of our main conclusions: thatthese two grouping principles contribute independently to group-ing. This is apparent from his reformulation of the laws of group-ing as if they were laws of gravitation: “Two parts of the fieldattract each other according to their degree of proximity andequality [similarity]” (pp. 166–167). He doubted that one couldtest this claim empirically because he worried that “equality”would be hard to measure. We overcome this obstacle here.

Strategies for the Quantification of Grouping

To quantify the conjoined strength of two types of grouping, onemust (a) measure the strength of each grouping principle and (b)determine how these strengths combine when the principles areconjoined. After the earliest attempts to quantify grouping, re-searchers followed two strategies to accomplish these goals:

1. The trade-off strategy—to investigate the trade-off be-tween proximity and other grouping principles. In thisstrategy, the researcher tries to measure the strength ofgrouping principles while simultaneously trying to deter-mine how these strengths are combined

2. The proximity-first strategy—to first measure groupingby proximity and only then to study the relation betweenit and other grouping principles. Our experimental workfollows this strategy.

Figure 1. Wertheimer’s (1923/1938) demonstration in which he appliedgrouping by proximity and grouping by similarity to the same stimuli. �a�� distance between elements within columns; �b� � distance betweenelements within rows. A: Wertheimer’s Figure xii—both proximity andsimilarity favor columns (�b�/�a� � 1.083). B: Wertheimer’s Figure xiii—proximity favors columns (�b�/�a� � 1.104), but similarity favors rows.

Figure 2. Koffka’s (1935/1963, p. 165) exploration of the competitionbetween grouping by proximity and grouping by similarity (which he calledthe law of equality). In A–C, the larger distance between elements, �b�, is 1.594�a�, the shorter distance. In D and E, the distances are uniform. (We adaptedKoffka’s Figures 43A–43E, on the basis of instructions in his text. We took theratios of distances from his figure, rather than the implausible 3:1 ratio heclaimed them to be.) A: Pairs defined by proximity are easier to see than pairsdefined by color similarity. Hence, proximity is stronger than color similarity.B: Pairs defined by proximity are as easy to see as pairs formed by shapesimilarity. Thus, the strengths of grouping by proximity and by shape simi-larity are close to being equal. From this and A, Koffka inferred that groupingby form similarity is stronger than grouping by color similarity. C: Pairsdefined by shape similarity and color similarity are easier to see than pairsdefined by proximity. When shape similarity and color similarity are con-joined, they are stronger than proximity. D: Pairs are seen grouped by colorsimilarity. When proximity is uniform, color similarity determines grouping.E: Pairs are seen grouped by shape similarity. When proximity is uniform,shape similarity determines grouping. The grouping is stronger than in D,suggesting that the relative strength of these two factors is independent ofwhether proximity varies, as in A and B, or is held constant, as in D and E.

a

bc

d

basic parallelogram�

�

Figure 3. Defining features of a dot-lattice stimulus. �b�/�a� � 1.216; � �79°; � � 15°. A dot lattice seen through a circular aperture. This two-dimensional (infinite) regular periodic pattern is defined by two transla-tions, a and b, along which one can move the lattice without changing itsappearance. a and b define the sides of a basic parallelogram, which is thebuilding block of the dot lattice. The angle between a and b is �, which isthe third parameter needed to specify the lattice. Translations c and d alsoleave the lattice unchanged; they define the short and long diagonals of thebasic parallelogram. In experiments, the lattice is presented at a randomorientation �.

132 KUBOVY AND VAN DEN BERG

Dot Lattices

Before we review these strategies, we must introduce a widelyused tool in this type of research: dot lattices (used by manyresearchers, starting with Wertheimer, 1923/1938; see our Figure1). Their taxonomy (Kubovy, 1994) is summarized in Figures 3and 4. A dot lattice is formally defined as an infinite collection ofdots in the plane that remains unchanged if it is translated by avector a (i.e., a quantity that has a direction, as well as a magnitude�a�) or b (whose magnitude is �b�, where �b� � �a�). (A dot latticewill remain invariant under other translations as well, but it issufficient that it be invariant under two nonparallel translations.)These two vectors, and the angle between them, � (constrained by60° � � � 90°, for reasons explained by Kubovy, 1994), definethe basic parallelogram of the lattice and thus the lattice itself. Thediagonals of the basic parallelogram are c and d (where �c� � �d�).In its canonical orientation, a is horizontal; the angle � (measuredcounterclockwise) is the measure of the orientation of a dot lattice;�a� is called its scale. If we are not interested in the scale of a dotlattice, it can be located in a two-dimensional space with dimen-

sions �b�/�a� and � (see Figure 4), in which we can identify sixdifferent types of lattices, characterized by their symmetry prop-erties.

To study the interaction of grouping by proximity and groupingby similarity, we use at least two kinds of elements (called motifs)in the lattice, as did Wertheimer (1923/1938; see our Figure 1);they are called dimotif lattices (see Figure 5; see also Grunbaum &Shepard, 1987, pp. 215, 247–248). The two motifs are not assignedto dots at random, however. In Figure 5A, each string of dots in thea direction contains only one of the motifs, and the parallel stringadjacent to it contains the other. We say that this lattice ishomogeneous with respect to a. On the other hand, each stringof dots in the b direction consists of alternating instances of thetwo motifs; all the strings in this direction have an identicalarrangement of motifs. Thus, the lattice in Figure 5A is heter-ogeneous with respect to b. In contrast, the lattice in Figure 5Bis homogeneous with respect to b and heterogeneous withrespect to a.

What the Two Strategies Can Do

Researchers who pursued the trade-off strategy hoped to mea-sure the strengths of grouping by proximity and similarity in termsof each other. There are two ways to do this (see Table 1): (a) Foreach level of proximity, seek a level of similarity such that thepropensity to see a pattern grouped by proximity is equal to thepropensity to see it grouped by similarity. (b) For each level ofsimilarity, seek a level of proximity such that the propensity to seea pattern grouped by similarity is equal to the propensity to see itgrouped by proximity. Thus, in Figure 5A, we are most likely tosee the lattice as strings of dots parallel to a. One can reverse thistendency by (a) reducing the dissimilarity between the two motifsor by (b) increasing within-b distances (�b�) relative to within-adistances (�a�) by manipulating either. These approaches can pro-duce two types of grouping operating characteristics (GOCs; seeFigure 6), which show how one grouping principle trades offagainst another.

Figure 4. Two-dimensional space and nomenclature of dot lattices. Thesolid dot represents the oblique dot lattice shown in Figure 3.

b

a ab

A B

Figure 5. Dimotif dot lattices in which the tendency to group by proximity and the tendency to group bysimilarity can be manipulated. The differences between the two kinds of dots do not represent a condition of theexperiments reported later in the article. The distance between dots along a (�a�) is never greater than the distancebetween dots along b (�b�) (i.e., �a� � �b�). For both dot lattices shown in this figure, �b�/�a� � 1.25. A: Dots arehomogeneous along a and heterogeneous along b. Grouping is stronger along the a orientation than along theb orientation. Grouping by proximity is reinforced by similarity. � � 105°. B: Dots are homogeneous along band heterogeneous along a. There is approximate equilibrium (i.e., competition) between grouping along a andalong b. Grouping by proximity is opposed by similarity. � � 15°.

133A PROBABILISTIC MODEL OF GROUPING

To understand the nature of GOCs, we compare them to indif-ference curves and attention operating characteristics (AOCs).What microeconomists call indifference curves (Krantz, Luce,Suppes, & Tversky, 1971) represent different baskets of goods thatcan be acquired on a given budget. Imagine a consumer who wouldbe equally satisfied with a market basket consisting of 1 lb of meatand 4 lb of potatoes and another consisting of 2 lb of meat and 1lb of potatoes. In such a case, the meat, potato pairs 1, 4 and 2, 1lie on an indifference curve.

The AOC shows how observers can allocate attention. In one ofthe conditions of a typical experiment (Sperling & Melchner,1978), observers saw a stimulus (see Figure 7) and reported thetwo digits. In different blocks, they were told to allocate theirattention exclusively to the center, mostly to the center, equally toboth, mostly to the periphery, or exclusively to the periphery.Figure 8 shows the data and a fitted AOC for one observer (MJM),who could voluntarily trade off the amount of attention allocated tothe two parts of the stimulus. Had this observer not been able to doso, all these data points would be clustered.

There is an important difference between these trade-off func-tions and GOCs. The variable allocation of attentional or financialresources that produce an AOC or an indifference curve is con-trolled by the person’s choice. This is not the case with the GOCswe report here, which are controlled by the stimulus. We do notknow—and do not investigate here—whether observers can con-trol their location on a GOC.

We can now explain why the proximity-first strategy is prefer-able to the trade-off strategy. Although observers can judgewhether one grouping principle is stronger than another, theycannot say by how much. So the best that researchers can hope to

achieve with the trade-off strategy is to find the points of transitionbetween the conditions under which one grouping principle isstronger than another. Thus, they can only produce a single GOCfor a pair of grouping principles: the GOC for which the twogrouping principles are equal in strength (i.e., in equilibrium). Wecall these equilibrium GOCs.

The proximity-first strategy gives us more information thandoes the trade-off strategy. To claim that we have quantified thestrength of grouping, we need to produce a GOC map, an analogof an indifference map (see Figure 9). In microeconomics, oneassumes that a given pair of goods lies on only one indifferencecurve and that indifference curves do not intersect. So if the meat,potato pairs 1, 4 and 2, 1 lie on one indifference curve, the pairs 2,4 and 4, 1 might lie on a second indifference curve, which has ahigher utility because it contains more meat and the same quantityof potatoes. The proximity-first strategy allows us to produce GOCmaps for pairs of grouping principles.

The Trade-Off Strategy: A Brief History

The first to systematically study grouping by proximity and theinteraction of the two grouping principles was Rush (1937). In oneof her experiments, she showed observers a sequence of dimotifdot lattices in which she varied �b�/�a�. For each lattice, she askedthem to indicate how it was organized by giving them five alter-natives: (a) horizontal —, (b) vertical �, (c) oblique (1) �, (d)oblique (2) /, and (e) other ways (draw to indicate). Unfortunately,her meager summary of her data prevents us from presenting them

Figure 6. Two types of equilibrium grouping operating characteristics that can be produced using the trade-offstrategy. A: The experimenter manipulates the difference between proximity within rows and proximity withincolumns and asks observers to set the difference between similarity within columns and similarity within rowsso that the strength of organization by rows is equal to the strength of organization by columns. B: Theexperimenter manipulates the difference between similarity within rows and similarity within columns and asksobservers to set the difference between proximity within columns and proximity within rows so the strength oforganization by rows is equal to the strength of organization by columns.

Table 1Two Types of Trade-Off Strategy and the Corresponding GOCs

Manipulate. . . . . .to find level of. . . . . . such that GOC

proximity similarity strength(similarity) � strength(proximity) Figure 6Asimilarity proximity Figure 6B

Note. GOC � grouping operating characteristic.


in detail.1 Nevertheless, it seems that her data implied a competi-tion between grouping by proximity and grouping by similarityand that she thought she had measured the relative strengths of thetwo principles by finding their point of equilibrium: “ . . . it may besaid that Similarity equals about 1.5 cms. of Proximity” (p. 90).

Roughly 2 decades passed before Hochberg and Silverstein(1956; independent of Rush’s work, as a footnote in Hochberg &Hardy, 1960, attests) also pursued the trade-off strategy. Theirstimuli (see Figure 10) were 6 � 8 rectangular arrays of squares(later called split lattices by Kubovy, Holcombe, & Wagemans,1998). They asked observers to adjust proximity (for the type ofGOC, see Figure 6A) within rows �v2� � 42 mm � �v1� (whileholding the distance within columns constant at �w� � 18 mm) sothat the strength of grouping into rows or into columns would bein equilibrium (i.e., to find points at which attraction by proximitywas equal to attraction by similarity). They started the adjustmentprocess either from �v2� � �v1�, which was always seen organizedby rows (see Figure 10A), or from �v2 v1�, which was always seenorganized by columns (see Figure 10B).

We reanalyzed their data by computing log(v2/v1) for the ob-served transition between perceiving the lattice organized by prox-imity or by similarity. (Note that we distinguish here—and else-where in this article— between bold italic symbols, whichrepresent vectors, and their counterparts in the set of observerresponses. Thus, if observers indicate that the dominant organiza-tion of a stimulus is along vector v, then we denote this choice withv.) When they reduced the luminance difference between rows(thus reducing grouping by similarity and allowing grouping byproximity to manifest itself), the distance between rows for whichobservers reported an equilibrium between rows and columnsincreased. As Figure 11 shows, they produced two equilibriumGOCs.

Whereas Hochberg and Silverstein (1956) had observers adjustdistances in split lattices, Hochberg and Hardy (1960) asked ob-servers to adjust luminance differences (for the type of GOC, seeFigure 6B). Sixteen observers adjusted the luminance differencebetween columns in four 4 � 4 rectangular lattices of dots (seeFigure 12). After allowing for an error in Hochberg and Hardy’sreporting, we reconstructed their data by simulation (see the detailsin the Appendix) to produce a GOC that is probably close to theirs(see Figure 13).

Quinlan and Wilton (1998), in an innovative and elegant exper-iment, studied the relations between grouping by proximity andtwo forms of grouping by similarity (by color and by shape). Theirstimuli consisted of strips of seven elements; the center elementwas the target (see Figure 14). The width of each element sub-tended 1°; the gap between elements was 0.25°. They manipulatedproximity by shifting the left or right set of three elements by 0.5°.They also manipulated color and shape similarity. The observersused a scale � with range (�4, . . . , 4) to rate the degree to whichthe middle element grouped with the elements on the right or onthe left. They were assigned to one of three groups: (a) colorobservers, who saw Figures 14A–14E; (b) shape observers, whosaw Figures 14A, 14B, and 14F–14H; and (c) conflict observers,who saw Figures 14A, 14B, and 14I–14K.

The key to the inference of additivity of grouping principles inthis experiment is the effect of two ways to conjoin groupingprinciples: (a) �-conjoining, in which two grouping principles areconjoined so that they tend to strengthen grouping on the sameside. To be compatible with additivity, the strength of �-conjoinedprinciples should be greater than the strength of either principlealone. (b) �-conjoining, in which two grouping principles areconjoined so that they conflict. To be compatible with additivity,the strength of �-conjoined principles should be less than thestrength of either principle alone.

In our analysis of Quinlan and Wilton’s (1998) results (shown inFigure 14), we used the effect sizes � � �/(standard error of �)

1 The raw data G. P. Rush left on deposit at the Columbia UniversityPsychology Library were destroyed (V. Sukenik, Psychology Librarian,personal communication).

Figure 7. Stimulus for one condition of the Sperling and Melchner (1978)experiment (reconstructed).

Figure 8. Data for observer MJM in the Sperling and Melchner (1978)experiment. The open data points represent joint digit-detection accuracyfor the outside and inside digits during sessions in which MJM reportedboth the inside and outside digits. The area of the circles is proportional tothe percentage of attention MJM was told to give to the outside characters.The filled data points represent control sessions in which MJM reportedeither the inside or the outside digits. The dashed lines represent the meansof these sessions. The solid line is the best fitting linear attention operatingcharacteristic.


rather than their mean � ratings. We adjusted the � values to fallwithin the [0, 1] interval. Most of their data fall into an impres-sively regular pattern (see Figure 15). For example, in theproximity–color pair (in light red), the �-values of the individualfactors—�(color) � 0.65 and �(proximity) � 0.55—fall betweenthe two types of conjoining—�(proximity � color) � 1.0 and �(proximity � color) � 0.0. The same ordering applies to theproximity–shape pair (in light blue). The color–shape pair (inviolet) does not conform to this pattern.

Table 2 summarizes the inferences one can draw from thesedata. These inferences allow us to conclude the following: (a) Thestrength of grouping by proximity conjoined with grouping bycolor is compatible with their additivity, and (b) grouping by shapebehaves anomalously. However suggestive, these data are notdefinitive with respect to the question of additivity. Had Quinlanand Wilton (1998) used a design in which each type of groupingwas a multilevel factor, they might have been able to settle this

Good X (meat)

Goo

d Y

(po

tato

es)

$10

$20

$30

Figure 9. An indifference map with three of the indifference curvesshown. Each curve corresponds to a different budget. As the horizontal (orvertical) dashed line shows, unless one’s budget grows, one cannot in-crease the quantity of meat (or potatoes) without decreasing the amount ofpotatoes (or meat).

v1| v2

42 mm

w = 18 mm

v1 v2

42 mm

w = 18 mm

A B

Figure 10. Hochberg and Silverstein’s (1956) split lattices. A: If �v1� � �v2�, observers are most likely to seerows (grouping by similarity). B: If �v1��v2�, observers are most likely to see columns (grouping by proximity).

?

?

?

?

0.2

0.4

0.6

0.8

similarity

?

?

?

?

low high low high

proximity first similarity first

logv2

v1((proximity))

Figure 11. Hochberg and Silverstein’s (1956) two-point grouping oper-ating characteristics (11 observers); they are of the type shown in Figure6A. The difference in the observed distance between rows and columns(log[v2/v1]) at which the strengths of grouping by proximity and groupingby similarity are equal trades off with similarity (reflectance difference).Left—initially seen as grouped by proximity (�v1� �� v2�); right—initiallyseen as grouped by similarity (�v1� � �v2�). Error bars indicate 95% confi-dence intervals.


question. Such a study would be a valuable test of the generality ofthe theory we present here.

Another step toward answering the question of the additivity ofgrouping principles was taken by Oyama, Simizu, and Tozawa(1999), who created rectangular dimotif lattices (see Figure 16),which they presented to six or seven observers for 3 s. During thistime, the observers tilted a joystick to the right when they sawhorizontal grouping and pulled it when they saw vertical grouping.The horizontal separation was increased by 15� after a horizontalresponse and decreased by that amount after a vertical response. Adouble-staircase method determined the ratio of vertical to hori-zontal distances, �v�/�h�, that was in equilibrium with a particulardissimilarity.

We extracted four sets of results from Oyama et al.’s (1999)figures. Our Figure 17 gives four GOCs: The left panel shows thevalues of �v�/�h� that matched four levels of luminance dissimilarity(none, low, middle, and high), which we rescaled to the [0, 1]

range to allow us to compare conditions. The second panel showsthe values of �v�/�h� that matched four levels of size dissimilarity(none, low, middle, and high). The third panel shows four levels ofscaled color dissimilarity. Finally, the panel on the right shows thevalues of �v�/�h� that matched different numbers of features bywhich the rows differed (from none to four).

The research that has come the closest to producing proper GOCmaps is that of Claessens and Wagemans (2005), who �-conjoinedtwo grouping principles using a new type of lattice, the Gaborlattice (see Figure 18), in which they replaced dots with Gaborpatches or with unoriented Gaussian patches. They used the pro-cedure developed by Kubovy and Wagemans (1995), who pre-sented a dot lattice for 300 ms and asked the observers to indicatealong which of the four directions (a, b, c, or d; defined in Figure3) they saw the lattice organized. We denote these responses witha, b, c, and d. In Figure 19, we plot their data for rectangularlattices (defined in Figures 3 and 4) only (because of subtleties inthe data they obtained for other types of lattice). FollowingKubovy et al. (1998), they plotted the log odds of responding brather than a, log[p(b)/p(a)], as a function of the distance ratio|b| / |a|. The resulting function is called an attraction function. Theattraction functions are, as Kubovy et al. (1998) found, close tolinear. (We describe the Kubovy & Wagemans, 1995, experiment,and its analysis by Kubovy et al., 1998, extensively in the nextsection, The Proximity-First Strategy: A Brief History.) The at-traction functions were, as expected, close to linear. The effect ofaligning the Gabor patches with a or b addressed our question ofadditivity of conjoined grouping principles. The effects of prox-imity and motif alignment are additive: The intercept of the at-traction function (but not its slope) increases when the Gaborpatches are b aligned; it decreases by approximately the sameamount when they are a aligned.

Figure 12. The appearance of Hochberg and Hardy’s (1960) four stimuliafter observers adjusted the distances within columns so that grouping bysimilarity (columns) overcomes grouping by proximity (rows). A: �b�/�a� �1.0; D� R � 0.07 (s � 0.07). B: �b�/�a� � 1.3; D� R � 0.14 (s � 0.13). C: �b�/�a�� 1.7; D� R � 0.25 (s � 0.23). D: �b�/�a� D� R � 2.0; � 0.90 (s � 0.81). Thecontrast of the dots is an approximate representation of the means (D� R, infeetlambert) of the adjusted luminance differences. In A, rows and columnsshould be in approximate equilibrium. In B–D, organization by columnsshould be slightly stronger than organization by rows. �a� and �b� are thedistances between columns and rows, respectively. The standard deviationof these adjusted differences is s. We used the values of D� R and s togenerate Figure 13 (see Appendix).

Figure 13. Hochberg and Hardy’s (1960) grouping operating character-istics (of the type shown in Figure 6B). The abscissa is the manipulatedaspect ratio of the dot lattice. The bars represent 95% confidence intervals(see Appendix), and the dashed line represents the best fitting linearfunction.


Figure 14. A representative subset of Quinlan and Wilton’s (1998) stimuli and their mean ratings (SE) of thedegree to which the center element (the target) grouped with the elements on the left or on the right (ratings 0 mean groups with the left; ratings � 0 mean groups with the right).

Figure 15. Quinlan and Wilton’s (1998) data, arranged in three groups by pairs of grouping principles (thethree groups are color coded). Within each group, the heights of the labels reflects their value on the responsevariable �, which represents a normalized measure of the effect size they obtained. (The axis on the left appliesto the proximity–color and proximity–shape pairs; the axis on the right applies to the color–shape pair.) X � Y� the grouping principles X and Y are conjoined so as to strengthen grouping; X � Y � the grouping principlesare conjoined so as to operate in opposition.


For these data, we can go beyond plotting a GOC betweendistance ratio �b�/�a� and Gabor orientation because we have morethan equilibrium data. To see why, note the three horizontal dashedlines in Figure 19: The one labeled “equilibrium” represents thelevel at which log[p(b)/p(a)] � 1, and therefore p(a) � p(b). Thenext two lines represent nonequilibrium conditions, for which p(a)is either 4 or 16 times as high as p(b). Because p(a) � p(b) � 1,we cannot say without a model what these values of p(a) and p(b)are.

For the first time, we can construct a GOC map. To do so withthese data, we must make the questionable assumption that thealignment variable is not discrete with three ordered levels butrather a unidimensional continuum that ranges from b aligned to a

aligned. We show the resulting map in Figure 20, in which thecontour labeled “0” is the equilibrium GOC.

This survey leaves little doubt: Pairs of grouping principlestrade off against each other. The six studies we reviewed show thatGOCs can be obtained. As we pointed out earlier, the first five ofthese trade-off patterns do not give us enough data to claim that wehave measured the strength of either grouping principle becausethey are equilibrium GOCs. The sixth trade-off pattern is morepromising: Because Claessens and Wagemans (2005) obtainednonequilibrium data, we were able (with some good will) to extracta GOC map from their data. We now turn to the proximity-firststrategy, which allows us to construct GOC maps, which in turnenable us to measure the strength of the two grouping principles.

The Proximity-First Strategy: A Brief History

Oyama (1961) pioneered the proximity-first strategy. Heshowed that one could measure the strength of grouping by prox-imity on its own. Using rectangular dots lattices at a fixed orien-tation (see Figure 21), he recorded the amount of time observersreported seeing the competing vertical and the horizontal group-ings. His data show that the ratio of the time participants saw thevertical organization and the time they saw the horizontal organi-zation is a power function of the ratio of the vertical and horizontaldistances (because the logarithms of these ratios are linearly re-lated; see Figure 22).

Kubovy and Wagemans (1995) followed in Oyama’s (1961)footsteps, except for two important differences:

1. They used brief exposure durations to obtain probabilitiesof initial percepts.

2. They used dot lattices (see Figure 3), in which (a) the twoprincipal directions of grouping were not always perpen-dicular and (b) neither principal orientation of the latticewas necessarily vertical or horizontal. As a result, insteadof having two alternative responses, they gave their ob-servers four.

2.0

2.75

0.70

Figure 16. A reconstruction of a rectangular dimotif lattice used byOyama (1961), with �v�/�h� � 0.73. The dimensions are in degrees of visualangle. The rows differ by two out of the four features he manipulated: colorand shape (the others were luminance and size).

Table 2Inferences From Quinlan and Wilton’s (1998) Data (Figure 15)

Data Inference

Relation to additivity

Compatible Not incompatible Incompatible

�( p � x) �(x) The �-conjoining of color (or shape) with proximityis stronger than color (or shape).

✓

�( p � x) � �(x) The �-conjoining of color (or shape) withproximity is weaker than color (or shape).

✓

�([p � y] � z) � �( p � y)a The �-conjoining of shape (or color) with the�-conjoining of proximity with color (or shape)is weaker than the �-conjoining of proximity withcolor (or shape).

✓

�( p � s) �( p) The �-conjoining of proximity and shape is notstronger than proximity.

✓

�(c) �(c � s) �(s) Shape is anomalous: The �-conjoining of color andshape is weaker than color and stronger thanshape.

✓

Note. � � effect size from Quinlan and Wilton’s (1998) data; p � proximity; x � color or shape; s � shape; c � color.a y � color and z � shape, or vice versa.


Their data consist of two predictor variables (�b�/�a� and �) andfour response variables: p(a), p(b), p(c), and p(d). Because p(a) �p(b) � p(c) � p(d) � 1, these probabilities are not independent. Toovercome this problem, Kubovy et al. (1998) defined three re-

sponse variables: p(b)/p(a), p(c)/p(a), and p(d)/p(a). They reportedthe natural logarithms of these response variables.

Before we proceed, we review some notation we have already usedand introduce some notation that is new. We use v to refer to a genericdirection in the lattice (other than a), �v� to refer to the distancebetween dots along v, and v to refer to observers’ responses indicatingthat the lattice appeared to be organized along the direction v. Thus,the notation �v�/�a� means �b�/�a�, �c�/�a�, or �d�/�a� for a stimulus, andsimilarly, p(v)/p(a) means p(b)/p(a) for some data points, p(c)/p(a) forothers, and p(d)/p(a) for the remaining ones.

Figure 23 shows Kubovy and Wagemans’s (1995) data as re-analyzed by Kubovy et al. (1998). The linear function, whichaccounts for more than 95% of the variance, is called the proximityattraction function. Notice the three different data symbols in thefigure: They represent the data for the log odds of choosing, b, c,or d relative to a. Kubovy et al. found that

logp(v)

p(a)� s��v�

�a� � 1�,

or

p(v)

p(a)� es��v�

�a� � 1�,

where s � 0. This is a quantitative law of proximity attraction.Proximity attraction follows a decaying exponential function ofrelative interdot distances. This function is a law because it isinvariant over lattices of all shapes. Because proximity attractionin these lattices is based solely on interdot distances, we say thatit is a pure distance law.

Measuring the Strength of Grouping by Proximity

The pure distance law allows us to measure proximity attractionon a ratio scale. To support this claim, we need to specify the

Figure 17. Four grouping operating characteristics extracted from Oyama, Simizu, and Tozawa’s (1999)figures. Luminance: trade-off between distance ratio, �v�/�h�, and luminance dissimilarity between rows. Size:trade-off between distance ratio and size dissimilarity between rows. Color: trade-off between distance ratio andscaled color differences. Features: trade-off between distance ratio and number of features by which rowsdiffered. Because we normalized the x-axis, the similarity of slopes is not meaningful.

Figure 18. A rectangular a-aligned Gabor lattice with �b�/�a� � 1.25 and� � 112.5°, used by Claessens and Wagemans (2005). The inset shows amagnified Gabor patch.


b

a

logp((b))p((a))

-6

-4

-2

0

1 1.125 1.25 1.375 1.5

equilibrium

p((a)) = 4*p((b))

p((a)) = 16*p((b))

b −− alignedGaussa −− aligned

Figure 19. The Claessens and Wagemans (2005) data for rectangular Gabor lattices for Gaussian dots and twoalignments of Gabor patches.

b

a

Gab

or o

rient

atio

n

b −− aligned

Gauss

a −− aligned

1 1.125 1.25 1.375 1.5

-6

-5

-4

-3

-2

-1

0

Figure 20. Grouping operating characteristics map derived from the Claessens and Wagemans (2005) data.The numbers along the contours are values of log[p(b)/p(a)] from Figure 19.


meaning of the ratio p(v)/p(a). Denote the set of responses � {a,b, c, d} with generic element � . Because �� p(�) � p(a) � p(b) �p(c) � p(d) � 1,

p�v� �p�v�

��

p��. (1)

Dividing numerator and denominator by p(a), and writing �(v) �p(v)/p(a), and �(�) � p(�)/p(a),

p�v� ��v�

��

��, (2)

which is known as the Shepard–Luce choice rule (Logan, 2004; Luce,1959, 1963; Shepard, 1957). Equation 2 has the form of a strict-utilitymodel, defined by Roberts (1979/1984, pp. 280–283) as follows:

A closed structure of choice probabilities satisfies the strict-utilitymodel just in case there exists a real-valued function on A such that forall v � B � A,

p�v,B� ��v��

w�B

��w�,

where A is the set of all responses to all lattices, and B are theresponses to one lattice.

As Luce (1959) showed, � is measured on a ratio scale. So ourdependent variable, p(v)/p(a), is a proximity attraction strength,such that

��v� � es� �v��a��1� , where s � 0. (3)

Using our Equations 1 and 3, Kubovy et al. (1998) predicted thechoice probabilities with considerable accuracy.

This result implies that the visual system could perform this taskusing the competitive algorithm diagrammed in Figure 24. One canthink of a dot lattice as consisting of four stimuli competing foremergence into awareness, each of which is a set of dots lying inthe same straight line (i.e., they are collinear) oriented as a, b, c, ord, in which the dots are separated by distances �a�, �b�, �c�, or �d�,respectively. In the diagram, we represent them with four boxeslabeled �a�,�b�, �c�, and �d�. Each of these stimuli elicits a responsefrom an orientation-tuned interpolator, whose function is to detectpotential contours in a noisy environment and to generate a smoothperceived curve through a set of points that may lie exactly orapproximately on that curve. We assume that these interpolatorsare most sensitive to collinear dots in a particular orientation;hence, we say that they are orientation tuned. We also assume thatthere are many such interpolators; in the diagram, we show onlythose tuned to the orientations of the vectors in the dot lattice(because we assume that the others are quiescent). Each respondsto its input by firing at a rate determined by its strength �v�) and acoefficient k. (Note that we have two parallel notations: �(v)denotes a proximity attraction strength, and it is inferred frombehavior; and �(�v�)—the � is bold here—is a theoretical strengthof activation of an interpolator. We use parallel notations becausein our model, they play analogous roles but are conceptuallydifferent.)

If the model included only the four orientation-tuned interpola-tors and the visual system chose the strongest organization, wewould have a deterministic winner-take-all model (analogous tocompetitive learning algorithms first proposed by Grossberg,1972, and von der Malsburg, 1973). To account for the probabi-

Figure 21. One of the rectangular dot lattices used by Oyama (1961),with �v�/�h� � 2.0.

Figure 22. Oyama’s (1961) results showing the relative strength ofgrouping by proximity in the vertical versus the horizontal direction. v,vert � vertical; h, horiz � horizontal.


listic nature of our data, we made our model probabilistic: Wedivide the four firing rates by their sum (i.e., normalize) to obtainfour probabilities. (The role of normalization in such networks wassuggested by Grossberg, 1976a.)

Conjoining Two Grouping Principles

Having modeled grouping by proximity, we can ask what wewould expect to find if such grouping were additive with groupingby similarity when they are conjoined. According to Shepard(1987), the dissimilarity between two objects that differ in H waysis an exponential function of the distance between them in a spaceof H dimensions:

��x, y� � es dxy,

where the sensitivity parameter s (�0) captures the steepness ofthe generalization gradient and dxy is the distance between theobjects:

dxy � � �h�H

�uxh � uyh�r�1r

,

where �uxh � uyh� is the distance between x and y along dimensionh H. In the Kubovy et al. (1998) studies, dxy is unidimensional; inour Equation 3, dxy � (�v�/�a�) � 1.

The purpose of this article is therefore to determine the metricaccording to which two conjoined grouping principles combine. Ifthey combine with r � 1, then the metric is called a city-blockmetric: The two grouping principles combine additively, and theyare separable in Garner and Felfoldy’s (1970) sense (see alsoGarner, 1974).

If r � 1, we can predict the results of the experiment. Considertwo dots, p1 and p2, with a relative distance in space of (�v�/�a�),which differ in lightness by �. If proximity attraction and similarityattraction are additive, we can expand Equation 3 to give theattraction ( p1, p2) between p1 and p2 as follows:

�� p1, p2� � esdist� �v��a��1��ssim �,

where sdist is a spatial sensitivity parameter and ssim is a dissim-ilarity sensitivity parameter. This implies that in an experiment likethat by Kubovy et al. (1998), in which we �-conjoined or

Figure 23. Kubovy, Holcombe, and Wagemans’s (1998) proximity attraction function obeys a pure distancelaw for seven observers (ty, jbs, jw, ng, ech, jah, jbr) . The different symbols (square, circle, and triangle)represent the log odds of choosing b, c, or d relative to a, respectively. The solid lines represent the best fittinglinear functions. v � the three directions other than a (v � {b,c,d}).


�-conjoined grouping by proximity and grouping by similarity,we would observe

logp�v�

p�a�� sdist� �v�

�a� � 1� � ssim �.

Grouping by Proximity and Similarity: Three Experiments

As our review of the literature on grouping shows, none of theexperiments that conjoin two or more grouping principles allowsus to determine the relation between the principles’ conjoinedstrength and the strength of each principle alone. To obtain suchdata, we performed three experiments in which we manipulatedrelative proximity (�b�/�a�) and relative luminance (�). We describethese experiments together because they differ little.

Method

Observers. Observers were undergraduates at the Universityof Virginia, whom we paid $7.50 per session. No observer partic-ipated in more than one experiment. The numbers of participantsin each of the three experiments were 13, 15, and 6, respectively.

Stimuli. Observers saw rectangular dot lattices shown on agray circular background (500 pixels in diameter, 9.7° of visualangle, 19 cd/m2), which itself appeared on a black background.The dots (10 pixels in diameter, 0.20° of visual angle) were visibleonly in the circular gray area of the screen.

There are three ways to distribute two colors in dot lattices; inour experiments, we used two of these, shown in Figure 5. The dotlattices varied along two dimensions: the ratio �b�/�a� and thecontrast difference (�) between alternate columns. The shortestcenter-to-center distance between dots, a, was fixed at �a� � 50pixels (0.97° visual angle). Luminances, L, are given in terms ofthe Apple 8-bit RGB (red-green-blue) color system (where RGB �0 is black and RGB � 255 is white, with 254 levels in between).Dots in even columns had RGB � 172 (24 cd/m2). If we createdimotif lattices in which some dots have contrast r1 and othershave contrast r2, there is a dot-contrast difference, � � r2 � r1. Forthe purposes of these experiments, we consider these contrastdifferences to form an interval scale: � � {0, 1, 2, 3, 4}. On eachtrial, � (the orientation of the lattice; see Figure 3) was random.

In Experiments 1 and 2, dots were homogeneous along b (seeFigure 9 for the meaning of homogeneity and heterogeneity),which �-conjoins (creates a competition between) similarity andproximity. In Experiment 3, we had two conditions: (a) dotshomogeneous along b (�-conjoined) and (b) dots homogeneousalong a (�-conjoined), where similarity and proximity worked inconcert. We summarize the experimental designs in Table 3. InExperiments 1 and 2, we coupled low levels of � with small �b�/�a�ratios and high levels of � with large �b�/�a� ratios. In Experiment3, we crossed � with �b�/�a�.

Procedure. Observers sat in a dark room approximately 0.6 mfrom the monitor. Each trial (Figure 25 gives the duration of each

Figure 24. Model of proximity attraction for the dot lattice shown in Figure 3, with k � 150 and s � �1.9(Equation 3).


phase of a trial) started with a fixation cross followed by a dotlattice. The lattice was followed by dynamic mask (three randomdot patterns containing the same number and types of dots as thestimulus), which was followed by a response screen consisting offour response icons—gray circles traversed by a white line to formone of four plausible organizations of the lattice. The observers’task was to report the dominant orientation along which the dotswere grouped. After the observer clicked on one of the responseicons, the screen went black until the next trial began. At thebeginning of each session, participants did 20 preliminary trials.

Results

The raw data consisted of the number of times each observerresponded a or b for each combination of �b�/�a� and �. For eachcondition, we transformed the response frequencies #a and #b tolog odds:

logp(b)

p(a)� log

#b �1

6

#a �1

6

,

where we added 1/6 to the numerator and denominator to avoidzeroes and infinities, following a recommendation of Mostellerand Tukey (1977, pp. 112–114). We disregarded the relatively rarec and d responses.

For each experiment, we modeled the response variable,log[p(b)/p(a)], using mixed linear models in which our manip-ulated variables (�b�/�a� and �) were fixed effects within partic-ipants (using the lme4 package for model fitting and the gmod-els package for obtaining confidence intervals in the Rstatistical environment; Bates & Sarkar, 2007; Ihaka & Gentle-man, 1996; Warnes, 2007). The most parsimonious model ineach analysis was the model with the lowest Akaike informa-tion criterion (Gelman & Hill, 2007, pp. 525–527; Wagenmak-ers & Farrell, 2004). We obtained confidence intervals by a104-step Markov chain Monte Carlo (Gelman & Hill, 2007, pp.408 – 409).

We summarize the model for each of the three experiments intwo columns (see Table 4): The first gives the model’s estimate ofthe effect; the second gives an estimate of the corresponding effectsize (Cohen’s d � estimate/SE[estimate]). The first line gives theintercept of the model (i.e., the estimate of the baseline condition

fixation410 ms

dot lattice300 ms

mask 170 ms

mask 270 ms

mask 370 ms

responsead libitum

inter-trial580 ms

Figure 25. A typical trial in the three experiments reported in the article.

Table 3Experimental Conditions and Number of Observations Per Session for Each Condition

Experiment

L

�

�b�/�a�

RGB cd/m2 1.00 1.05 1.10 1.15 1.20 1.25 1.30 1.35 1.40 1.50 1.60

1 172 24 0 60 60 60176 25 1 60 60 60180 26 2 60 60 60184 27 3 60 60 60188 28 4 60 60 60

2 172 24 0 60 60 60176 25 1 60 60 60180 26 2 60 60 60184 27 3 60 60 60188 28 4 60 60 60

3 184 27 �3a 33 33 33 33180 26 �2a 33 33 33 33176 25 �1a 33 33 33 33172 24 0 33 33 33 33176 25 1 33 33 33 33180 26 2 33 33 33 33184 27 3 33 33 33 33

Note. Except where noted, similarity favors grouping along b. L � luminance; RGB � red-green-blue.a Similarity favors grouping along a.


with respect to which the effects are defined—the effect size refersto the estimate’s difference from 0), and each successive line givesthe amount one should add or subtract from this intercept to obtainan estimate of the effect (the effect size refers to the estimate’sdifference from the intercept). We also summarize the models inFigures 26, 27, and 28 (in which the error bars span a 68%confidence interval 1SE).

The two important findings of these analyses are that (a) all theattraction functions were more parsimoniously fit by linear func-tions than by nonlinear ones and (b) the data were most parsimo-niously fit by an additive model of the effects of the two groupingprinciples (i.e., there was no evidence of an interaction betweenthem).

The estimates of the slopes of the attraction functions for thethree experiments are �4.95 (SE � 1.03), �5.04 (SE � 1.09), and�4.59 (SE � 1.80), respectively. Given the size of the correspond-ing standard errors, these estimates are not accurate enough to bedeclared different.

To assess the applicability of these models to individual partic-ipants, we summarize the participant-by-participant distribution ofthe adjusted coefficient of determination, R2

adj (which is R2 de-flated by taking degrees of freedom into account) in Table 5.Although the participant-by-participant fit is by no means alwayshigh, the median R2

adj values for the three experiments are arespectable 0.63, 0.34, and 0.88, respectively.

Figure 26. Experiment 1: log[p(b)/p(a)] as a function of �b�/�a� and �. � � the magnitude of the differencebetween the contrasts of the two dots in the dimotif dot lattices. When � 0, grouping by similarity andproximity work in concert (�-conjoined); when � � 0, the types of grouping are in competition (they are�-conjoined); when � � 0, the dots are uniform in contrast (i.e., only grouping by proximity is possible).

Table 4Estimates of Fixed Effects and Values of Corresponding Cohen’s d Values in the Models for Experiments 1–3

Effect

Experiment

1 2 3

Estimate d Estimate d Estimate d

At �b�/�a� � 1.0, � � 0 �0.26 3.15 �0.11 0.75 0.02 0.21Relative distance (�b�/�a�) �4.96 4.79 �5.04 4.59 �4.59 2.55Dissimilarity (�) 0.41 4.49 0.16 5.12 0.28 16.28


Figure 27. Experiment 2: log[p(b)/p(a)] as a function of �b�/�a� and �. � � the magnitude of the differencebetween the contrasts of the two dots in the dimotif dot lattices. When � 0, grouping by similarity andproximity work in concert (�-conjoined); when � � 0, the types of grouping are in competition (they are�-conjoined); when � � 0, the dots are uniform in contrast (i.e., only group by proximity is possible).

Figure 28. Experiment 3: log[p(b)/p(a)] as a function of �b�/�a� and �. � � the magnitude of difference betweenthe contrasts of the two dots in the dimotif dot lattices. When � 0, grouping by similarity and proximity workedin concert (they were �-conjoined); when � � 0, the types of grouping were in competition (they were�-conjoined); when � � 0, the dots were of uniform contrast (i.e., only grouping by proximity was possible).

Discussion

The experiments replicated those of Kubovy et al. (1998): Theattraction functions were linear. This allows us to ask how theconjoined strength of similarity and proximity relates to thestrength of each individual principle. The answer is that they areadditive when the dependent variable is logarithmic:

logp�b�

p�a�� sdist � �b�

�a� � 1� � ssim �,

where sdist � 0, and ssim 0 (as long as � 0 if b is homoge-neous and � � 0 if a is homogeneous). In terms of response odds,

p�b�

p�a�� esdist� �b�

�a��1� essim �, (4)

and letting

��b� � esdist � �b��a��1� ,

and

�� essim,

we see that similarity and proximity are multiplicative:

p�b�

p�a�� b��.

This result allows us to draw GOC maps, as shown in Figures 29,30, and 31, which means that we have achieved our goal. We plotthem all on the same scale to allow us to compare the slopes of theGOC maps.

Theory. Figure 32 shows a model consistent with these results.Built on our model for proximity attraction (see Figure 24), itprovides an account of how a visual system might additively

Figure 29. The data in Figure 26 represented as a grouping operating characteristics map. � � the magnitudeof the difference between the contrasts of the two dots in the dimotif dot lattices. When � 0, grouping bysimilarity and proximity work in concert (�-conjoined); when � � 0, the types of grouping are in competition(they are �-conjoined); when � � 0, the dots are uniform in contrast (i.e., only grouping by proximity ispossible).

Table 5Applicability of Models to Individual Observers: Five-NumberSummaries of the Distribution of Radj

2 Values

Summary

Experiment

1 2 3

Minimum 0.08 0.00 0.651st quartile 0.54 0.11 0.76Median 0.63 0.34 0.883rd quartile 0.68 0.46 0.93Maximum 0.77 0.71 0.95n 13 15 6


combine the strengths of grouping by proximity and grouping bysimilarity when they are conjoined. It assumes two types of mech-anisms: orientation-tuned interpolators, which appeared in ourmodel for proximity attraction, and dissimilarity attenuators. Aswe did earlier, we consider a dot lattice to be four stimuli whosecollinear sets of dots are represented by the four boxes labeled �a�,�b�, �c�, and �d�. Each interpolator responds to its input by firing ata rate k�(v). However, if adjacent dots differ by �, a dissimilarityattenuator multiplies the firing rate by factor �(�) � 1.0. Thus, as� increases, �(�) decreases. If b is uniform and the dots comprisingthe other vectors are heterogeneous, �(�) � �(0) � 1.0 for b, but�(�) � 1.0 for a, c, and d because � 0. In Figure 32, we outlinein red the attenuator that corresponds to the uniform vector b. Theresult is a slightly more complex probabilistic winner-takes-some-but-not-all model.

We have laid out our model of similarity attraction as if weknew that (a) the two grouping principles are served by differentneural modules and (b) the effects of dissimilarity operate inparallel with the effects of interdot distance on the output of theorientation-tuned interpolators. The first assumption is not incon-sistent with current imaging evidence (Han, 2004; Han, Jiang,Mao, Humphreys, & Gu, 2005). We know of no support for thesecond assumption, and indeed, we are not committed to it. (We donot accept Han’s, 2004, claim that grouping by proximity producesshorter latency event-related potentials than does grouping by simi-

larity because he did not insure that the two grouping strengths wereequated.) For the time being, the parallel operation of the two mech-anisms in Figure 32 should be treated as a mere graphic convenience.

Two questions remain:

What do we know about �(�)? It would have been nice tohave found that the effect of � is invariant. However, theeffect of � varied by a factor of 2.5 (as can be seen in Table 4),even though the range of � was the same in the three experiments(although in Experiment 3, it was either applied to a or b).

Are the GOC maps invariant? From Figures 29 –31, whichwe plotted on the same scale, it is obvious that they are not:The slopes are 1.10, 1.02, and 1.97 units of � per 10%change of �b�/�a�. We do not know if these differences aresignificant because their accuracy depends on the accuracyof our estimates of the slopes of the attraction functions,which, as we saw earlier, are not accurate enough to bedeclared different.

These experiments differed in three ways: (a) the participants,(b) the context provided by the different ranges of values of �b�/�a�,and (c) whether similarity ever favored grouping along a (as it didin Experiment 3). Only further exploration of the role played bythese factors will allow us to quantify the effect of dissimilarity

b

a

δδ

–2

–1

0

1

2

1.1 1.2 1.3 1.4 1.5

-1.4

-1.2

-1.0-0.8-0.6

-0.4-0.2

0.0

0.2Figure 30. The data in Figure 27 represented as a grouping operating characteristics map. � � the magnitudeof the difference between the contrasts of the two dots in the dimotif dot lattices. When � 0, grouping bysimilarity and proximity work in concert (�-conjoined); when � � 0, the types of grouping are in competition(they are �-conjoined); when � � 0, the dots are uniform in contrast (i.e., only grouping by proximity ispossible).


and its trade-off with proximity beyond what we now know,namely, that their conjoint strengths are additive.

General Discussion

Additivity

What then happens when grouping by proximity and groupingby similarity are conjoined? Their conjoint effect is the sum oftheir individual effects. When the output of a system is the sum ofits inputs, the system is linear. There are two ways to think aboutour result: (a) that we have undermined the widely held assump-tion (e.g., Kruse & Stadler, 1990, 1995; Medawar & Shelley, 1980;Read, Vanman, & Miller, 1997) that no gestalt phenomenon,indeed, no emergent property (e.g., Bar-Yam, 1997), can be ac-counted for by a linear system, or (b) that the additivity of gestaltprinciples demonstrated here shows that a whole whose parts aregestalts need not itself be a gestalt with respect to those parts.Because the latter is the more conservative view, it is the onetoward which we lean.

Our results are consistent with an optimal Bayesian model forcontour grouping. Elder and his colleagues (Elder, 2002; Elder &Goldberg, 2002) investigated the statistical utility of proximity,good continuation, and luminance similarity for natural images.They had observers trace contours in natural grayscale images,

from which they extracted sequences of discrete tangent elements.From the relations between pairs of these tangents, they thenestimated the likelihood distributions required to construct anoptimal Bayesian model for contour grouping. They found thatgrouping by proximity and grouping by luminance similarity areapproximately uncorrelated, suggesting a simple factorial model forstatistical inference. This result could explain why the visual systemmanifests the additivity of proximity and similarity we have found.

The sort of decomposability we report here is not inevitable.Indeed, apparent motion—a prototypical gestalt concern (Werthei-mer, 1912)—is not analogously decomposable (Gepshtein &Kubovy, 2000). It is natural to think that vision takes a series ofsnapshots of dynamic scenes; motion perception ensues when thesnapshots are different. This metaphor suggests two questions: (a)How does the visual system form objects from elements withineach snapshot? This is the spatial grouping problem, which wehave addressed in this article. (b) When the snapshots are different,how does the visual system know which element in one snapshotgoes with each element in the next? This is the temporal groupingproblem. Some researchers have held that spatial grouping occursindependently of—and feeds into—temporal grouping, thus mak-ing spatiotemporal grouping decomposable. Using spatiotemporaldot lattices, Gepshtein and Kubovy (2000) independently manip-ulated the strength of spatial and temporal groupings and showed

b

a

δδ

-2

-1

0

1

2

1.1 1.2 1.3 1.4 1.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

Figure 31. The data in Figure 28 represented as a grouping operating characteristics map. � � the magnitudeof the difference between the contrasts of the two dots in the dimotif dot lattices. When � 0, grouping bysimilarity and proximity worked in concert (they were �-conjoined); when � � 0, the types of grouping werein competition (they were �-conjoined); when � � 0, the dots were of uniform contrast (i.e., only grouping byproximity was possible).


that the temporal configuration of stimuli could affect spatialgrouping, thus refuting decomposability.

Why Probabilistic Models of Grouping?

Our probabilistic model of grouping ensures that the visualsystem will not settle on the strongest percept to the exclusion ofothers. In this sense, our model has some affinity with neuralnetworks that exhibit “conscience”: If a processing element in acompetitive learning network wins too often, it “feels guilty” andprevents itself from winning excessively (a mechanism first pro-posed by Grossberg, 1976a, 1976b, and later developed by Rumel-hart & Zipser, 1985/1986; DeSieno, 1988; and Ahalt, Krishnamur-thy, Chen, & Melton, 1990).

More generally, randomness is often considered an adaptivefeature of sensory systems (Carpenter, 1999; Leopold & Logo-thetis, 1999). For example, Gepshtein and Kubovy (2005) foundthat perceptual multistability can be described by a probabilisticmodel in which perception depends on an orientation-tunedintrinsic bias that slowly (and stochastically) changes its orien-tation tuning over time. To explain why such a mechanismmight be adaptive, they argued as follows: When animals andhumans are given the choice between two alternatives, one ofwhich is rewarded more often than the other, they choose eachalternative roughly in proportion to the likelihood of reward(this is probability matching or probability learning; e.g., Estes& John, 1958; Gallistel, 1993, pp. 351–383). Although this

Figure 32. Model of proximity and similarity attraction for a dimotif dot lattice, the structure of which is shownin Figure 3, with k � 150, sdist � �1.9, and �(�) � 0.7 (Equation 4), where b is homogeneous and a isheterogeneous (see Figure 5B). Compare with the model in Figure 24.


strategy does not maximize payoff, it is suboptimal only in aworld in which probabilities are stable; if the most generoussource is likely to be exhausted, then the maximization strategy mayfail in the long run. A parallel argument would explain why it wouldbe adaptive for perceptual systems to see beyond the dominant inter-pretation. Indeed, probability matching occurs in visual behavior(Kowler & Anton, 1987; Triesch, Ballard, & Jacobs, 2002).

Implications for the Boolean Map Theory of VisualAttention

Our results suggest that some aspects of Huang and Pashler’s(2007) Boolean map theory of visual attention may need to bereexamined. In this article, we have not questioned the assump-tion that if grouping by proximity prevailed in Figure 33, wewould see a nine-stripe organization along a, whereas if group-ing by similarity prevailed, we would see a 10-stripe organiza-tion parallel to b. However, according to Huang and Pashler,when grouping by similarity prevails, one can select eitherstrips of circles or strips of squares, and an attempt to selectboth will force grouping by proximity (Huang and Pashler,2007, p. 626).

In our experiments, we used brief presentations to minimize theinvolvement of attention, and we rotated our lattices randomlyfrom trial to trial to minimize the effects of expectation. Thus, evenif Huang and Pashler (2007) were right, their predictions wouldhold only for much longer exposures. But even if one displayed alattice such as the one in Figure 33 indefinitely, the strength ofgrouping by similarity would depend on the similarity between thetwo motifs. For example, if we had rounded the corners of thesquares in Figure 33, grouping by similarity would have beenreduced, thus demonstrating that grouping by similarity does notmerely involve the selection of single features to segregate. This iswhy our theory does not countenance the possibility that groupingby similarity is affected by one shape feature at a time. It assumesthat grouping by similarity involves a comparison of features.

A reconciliation is possible, however, if one distinguishes be-tween grouping and object formation. According to Kubovy andVan Valkenburg (2001), grouping produces putative objects,

whereas attention selects some of them to undergo figure–groundsegregation, which is the essence of object formation. Thus, per-haps Huang and Pashler’s (2007) theory is actually a theory ofpostgrouping object formation, a view that is not contradicted byany of their or Huang, Treisman, and Pashler’s (2007) empiricalfindings.

Final Thoughts

Several issues remain open: (a) Is the separability of group-ing by proximity and grouping by similarity true of other pairsof grouping principles? This question is particularly interestingin the light of two kinds of evidence: (a) neurophysiologicalevidence (Gerlach, Marstrand, Habekost, & Gade, 2005) thatgrouping by shape similarity may be impaired, whereas group-ing by color and proximity are spared, and (b) the anomalousbehavior of grouping by shape similarity in the experiment ofQuinlan and Wilton (1998), which we discussed earlier in ourreview of the trade-off strategy. Does our model hold forirregular patterns? van den Berg (2006) has explored this issue,with complex results that are beyond the scope of this article.

We have shown that the questions raised by Gestalt psycholo-gists are tractable and meaningful, contrary to the criticism byMarr’s (1982) characterization quoted at the beginning of thisarticle. They are tractable because we have the experimental toolsto study them; they are meaningful because we have shown thatgestalts can be decomposed and their combination can be de-scribed using simple neural mechanisms.

References

Ahalt, S. C., Krishnamurthy, A. K., Chen, P., & Melton, D. E. (1990).Competitive learning algorithms for vector quantization. Neural Net-works, 3(3), 277–290.

Bar-Yam, Y. (1997). Dynamics of complex systems. Reading, MA: PerseusPublishing.

Bates, D., & Sarkar, D. (2007). The lme4 package: Linear mixed-effectsmodels using S4 classes (R package Version 0.99875–0) [Computersoftware]. Available at http://cran.r-project.org/doc/packages/lme4.pdf

Carpenter, R. H. S. (1999). A neural mechanism that randomises behav-iour. Journal of Consciousness Studies, 6(1), 13–22.

Claessens, P. M., & Wagemans, J. (2005). Perceptual grouping in Gaborlattices: Proximity and alignment. Perception & Psychophysics, 67(8),1446–1459.

Cleveland, W. S., & Devlin, S. J. (1988). Locally weighted regression: Anapproach to regression analysis by local fitting. Journal of the AmericanStatistical Association, 83(403), 596–610.

Corning, P. A. (2002). The re-emergence of “emergence”: A venerableconcept in search of a theory. Complexity, 7(6), 18–30.

DeSieno, D. (1988). Adding a conscience to competitive learning. InProceedings of the International Conference on Neural Networks (Vol.1, pp. 117–124). New York: Institute of Electrical and ElectronicsEngineers.

Elder, J. H. (2002). Ecological statistics of contour grouping. BiologicallyMotivated Computer Vision, Proceedings, 2525, 230–238.

Elder, J. H., & Goldberg, R. M. (2002). Ecological statistics of gestalt lawsfor the perceptual organization of contours. Journal of Vision, 2, 324–353.

Ellis, W. D. (Ed.). (1938). A source book of Gestalt psychology. New York:Harcourt, Brace.

Estes, W. K., & John, M. D. (1958). Probability-learning with ambiguity in

a

b

Figure 33. Dimotif dot lattice similar to the one used by Huang andPashler (2007) to suggest that when grouping by similarity prevails, onesees only a collection of strips of squares or a collection of strips ofcircles.


the reinforcing stimulus. American Journal of Psychology, 71(1), 219–228.

Gallistel, C. R. (1993). The organization of learning. Cambridge, MA:MIT Press.

Garner, W. R. (1974). The processing of information and structure. Po-tomac, MD: Erlbaum.

Garner, W. R., & Felfoldy, G. L. (1970). Integrality of stimulus dimensionsin various types of information processing. Cognitive Psychology, 1(3),225–241.

Gelman, A., & Hill, J. (2007). Data analysis using regression and multi-level/hierarchical models. New York: Cambridge University Press.

Gepshtein, S., & Kubovy, M. (2000). The emergence of visual objects inspace-time. Proceedings of the National Academy of Sciences of theUnited States of America, 97(14), 8186–8191.

Gepshtein, S., & Kubovy, M. (2005). Stability and change in perception:Spatial organization in temporal context. Experimental Brain Research,160(4), 487–495.

Gerlach, C., Marstrand, L., Habekost, T., & Gade, A. (2005). A case ofimpaired shape integration: Implications for models of visual objectprocessing. Visual Cognition, 12(8), 1409–1443.

Grossberg, S. (1972). Neural expectation: Cerebellar and retinal analogs ofcells fired by learnable or unlearned pattern classes. Biological Cyber-netics, 10(1), 49–57.

Grossberg, S. (1976a). Adaptive pattern classification and universal recod-ing: I. Parallel development and coding of neural feature detectors.Biological Cybernetics, 23(3), 121–134.

Grossberg, S. (1976b). Adaptive pattern classification and universal recod-ing: II. Feedback, expectation, olfaction, illusions. Biological Cybernet-ics, 23(4), 187–202.

Grunbaum, B., & Shepard, G. C. (1987). Tilings and patterns. New York:Freeman.

Han, S. (2004). Interactions between proximity and similarity grouping: Anevent-related brain potential study in humans. Neuroscience Letters,367(1), 40–43.

Han, S., Jiang, Y., Mao, L., Humphreys, G. W., & Gu, H. (2005). Atten-tional modulation of perceptual grouping in human visual cortex: Func-tional MRI studies. Human Brain Mapping, 25(4), 424–432.

Harrell, F. E., Jr. (2006). The Hmisc package (R package Version 3.0–12)[Computer software]. Available at http://cran.r-project.org/doc/packages/Hmisc.pdf

Harte, V. (2002). Plato on parts and wholes: The metaphysics of structure.Oxford, England: Clarendon Press.

Hochberg, J., & Hardy, D. (1960). Brightness and proximity factors ingrouping. Perceptual & Motor Skills, 10, 22.

Hochberg, J., & Silverstein, A. (1956). A quantitative index of stimulus-similarity: Proximity vs. differences in brightness. American Journal ofPsychology, 69(3), 456–458.

Holton, G. A. (2003). Value-at-risk: Theory and practice. San Diego, CA:Academic Press.

Huang, L., & Pashler, H. (2007). A Boolean map theory of visual attention.Psychological Review, 114(3), 599–631.

Huang, L., Treisman, A., & Pashler, H. (2007, August 10). Characterizingthe limits of human visual awareness. Science, 317, 823–825.

Ihaka, R., & Gentleman, R. (1996). R: A language for data analysis andgraphics. Journal of Computational and Graphical Statistics, 5, 299–314.

Koffka, K. (1938). Beitrage zur Psychologie der Gestalt. III. Zur Grundlegungder Wahrnehmungspsychologie. Eine Auseinandersetzung mit V. Benussi[Contributions to Gestalt psychology: III. Toward a foundation of thepsychology of perception—A debate with V. Benussi]. In W. D. Ellis (Ed.),A source book of Gestalt psychology (pp. 371–378). New York: Harcourt,Brace. (Reprinted from Zeitschrift fur Psychologie, 73, 11–90, 1915)

Koffka, K. (1963). Principles of Gestalt psychology. New York: Harcourt,Brace & World. (Original work published 1935)

Kohler, W. (1938). Die physischen Gestalten in Ruhe und im station-aren Zustand. Eine naturphilosophische Untersuchung [Physical ge-stalten at rest and in steady state: A natural-philosophical investiga-tion.]. In W. D. Ellis (Ed.), A source book of Gestalt psychology(pp.17–54). New York: Harcourt, Brace. (Reprinted from Die phy-sischen Gestalten in Ruhe und im stationaren Zustand. Eine natur-philosophische Untersuchung, by W. Kohler, 1920, Braunschweig,Germany: Friedr, Vieweg und Sohn)

Kowler, E., & Anton, S. (1987). Reading twisted text: Implications for therole of saccades. Vision Research, 27(1), 45–60.

Krantz, D. H., Luce, R. D., Suppes, P., & Tversky, A. (1971). Foundationsof measurement: Vol. 1. Additive and polynomial representations. NewYork: Academic Press.

Kruse, P., & Stadler, M. (1990). Stability and instability in cognitivesystems: Multistability, suggestion, and psychosomatic interaction. In H.Haken & M. Stadler (Eds.), Synergetics of cognition (pp. 201–215).Berlin, Germany: Springer.

Kruse, P., & Stadler, M. (Eds.). (1995). Ambiguity in mind and nature:Multistable cognitive phenomena. Berlin, Germany: Springer.

Kubovy, M. (1994). The perceptual organization of dot lattices. Psy-chonomic Bulletin & Review, 1(2), 182–190.

Kubovy, M., Holcombe, A. O., & Wagemans, J. (1998). On the lawfulnessof grouping by proximity. Cognitive Psychology, 35(1), 71–98.

Kubovy, M., & Van Valkenburg, D. (2001). Auditory and visual objects.Cognition, 80(1–2), 97–126.

Kubovy, M., & Wagemans, J. (1995). Grouping by proximity and multi-stability in dot lattices: A quantitative gestalt theory. PsychologicalScience, 6(4), 225–234.

Leopold, D. A., & Logothetis, N. K. (1999). Multistable phenomena:Changing views in perception. Trends in Cognitive Sciences, 3(7),254–264.

Logan, G. D. (2004). Cumulative progress in formal theories of attention.Annual Review of Psychology, 55(1), 207–234.

Luce, R. D. (1959). Individual choice behavior. New York: Wiley.Luce, R. D. (1963). Detection and recognition. In R. D. Luce, R. R. Bush,

& E. Galanter (Eds.), Handbook of mathematical psychology (Vol. 1, pp.103–189). New York: Wiley.

Marr, D. (1982). Vision: A computational investigation into the human rep-resentation and processing of visual information. San Francisco: Freeman.

Medawar, P., & Shelley, J. H. (Eds.). (1980). Structure in science and art(Proceedings of the Third C. H. Boehringer Sohn Symposium held atKronberg, Taurus, 2nd–5th May 1979). Amsterdam: Excerpta Medica.

Mosteller, F., & Tukey, J. W. (1977). Data analysis and regression: Asecond course in statistics. Reading, MA: Addison-Wesley.

Nagel, E. (1961). The structure of science: Problems in the logic ofscientific explanation. London: Routledge & Kegan Paul.

Oyama, T. (1961). Perceptual grouping as a function of proximity. Per-ceptual & Motor Skills, 13, 305–306.

Oyama, T., Simizu, M., & Tozawa, J. (1999). Effects of similarity onapparent motion and perceptual grouping. Perception, 28, 739–748.

Quinlan, P. T., & Wilton, R. N. (1998). Grouping by proximity or simi-larity? Competition between the gestalt principles in vision. Perception,24, 417–430.

Read, S. J., Vanman, E. J., & Miller, L. C. (1997). Connectionism, parallelconstraint satisfaction processes, and gestalt principles: (Re)introducingcognitive dynamics to social psychology. Personality and Social Psy-chology Review, 1(1), 26–53.

Roberts, F. S. (1984). Measurement theory. Cambridge, England: Cam-bridge University Press. (Original work published 1979)

Rumelhart, D. E., & Zipser, D. (1986). Feature discovery by competitivelearning. In D. E. Rumelhart, J. L. McClelland, & the PDP ResearchGroup (Eds.), Parallel distributed processing (Vol. 1, pp. 151–193).Cambridge, MA: MIT Press. (Reprinted from Cognitive Science, 9,75–112, 1985)


Rush, G. P. (1937). Visual grouping in relation to age. Archives of Psy-chology, 31(217), 1–95.

Shepard, R. N. (1957). Stimulus and response generalization: A stochasticmodel relating generalization to distance in psychological space. Psy-chometrika, 22, 325–345.

Shepard, R. N. (1987, September 11). Toward a universal law of general-ization for psychological science. Science, 237, 1317–1323.

Sperling, G., & Melchner, M. J. (1978, October 20). The attention operatingcharacteristic: Examples from visual search. Science, 202, 315–318.

Triesch, J., Ballard, D. H., & Jacobs, R. A. (2002). Fast temporal dynamicsof visual cue integration. Perception, 31(4), 421–434.

van den Berg, M. (2006). Grouping by proximity and grouping by goodcontinuation in the perceptual organization of random dot patterns.Unpublished doctoral dissertation, University of Virginia, Charlottes-ville.

von der Malsburg, C. (1973). Self-organization of orientation sensitivecells in the striate cortex. Biological Cybernetics, 14(2), 85–100.

Wagenmakers, E.-J., & Farrell, S. (2004). AIC model selection usingAkaike weights. Psychonomic Bulletin & Review, 11(1), 192–196.

Warnes, G. R. (2007). gmodels: Various R programming tools for modelfitting (R package Version 2.13.2). Available from http://cran.r-project.org/src/contrib/PACKAGES.html; http://www.sf.net/projects/r-gregmisc

Wertheimer, M. (1912). Experimentelle Studien uber das Sehen von Be-wegung [Experimental studies on seeing motion]. Zeitschrift fur Psy-chologie, 61, 161–265.

Wertheimer, M. (1938). Untersuchungen zur Lehre von der Gestalt, II[Investigations of the principles of gestalt II]. In W. D. Ellis (Ed.), Asource book of Gestalt psychology (pp.71– 88). New York: Harcourt,Brace. (Reprinted from Psychologische Forschung, 4, 301–350,1923)

Appendix

Reconstructing Hochberg and Hardy’s (1960) Grouping Operating Characteristics

Because Hochberg and Hardy (1960) reported implausibly largestandard errors of the mean (on the order of 90% of the means), weinferred—and J. Hochberg (personal communication, September25, 2005) thought it credible—that these were standard deviations(the values appear in the caption of Figure 12 in the main text). Toobtain confidence intervals, we inferred that the underlying distri-butions were lognormal (because of the high correlation betweenD� R and s). We used the following conversions (Holton, 2003, p.135, Equations 3.91 and 3.92):

D� sL � logD� R

2

�s2 � D� R2.

and

sL � �logR

D� R2 � 1

We randomly sampled 16 observations from each of the four lognor-mal distributions (the parameters of which we computed from Hoch-berg & Hardy’s, 1960, data using the above equations), from whichwe derived confidence intervals and performed a regression to obtainFigure 13. (We computed confidence intervals with a nonparametricbootstrap that does not assume normality by using the summarizefunction in the Hmisc R package; Harrell, 2006.)

Received May 8, 2006Revision received July 16, 2007

Accepted August 1, 2007 �


The Whole Is Equal to the Sum of Its Parts: A ...€¦ · result less than, equal to, or greater than the sum of its parts? Or perhaps the combination is just different as some Gestalt

Documents