-
A p RI L I 3 - I 8' I 9 9 6 CHI 96 PAPERS
Structuring Information With Mental Models: A Tour of Boston
Ishantha Lokuge
MIT Media Lab
20 Ames Street
Cambridge, MA 02139
[email protected]
Stephen A. Gilbert
Brain & Cognitive Sciences MIT, E 10-120
Cambridge, MA 02139
[email protected]
Whitman Richards MIT Media Lab
20 Ames Street
Cambridge, MA 02139
[email protected]
ABSTRACT We present a new systematic method of structuring
information using mental models. This method can be used both to
evaluate the efficiency of an information structure and to build
user-centered information structures. In this paper we present the
method using Boston tourist attractions as an example domain. We
describe several interfaces that take advantage of our mental
models with an activation spreading network. Multidimensional
Scaling and Trajectory Mapping are used to build our mental models.
Because of the robustness of the technique, it is easy to compare
individual difference in mental models and to customize interfaces
for individual models.
Keywords Cognitive models, multidimensional scaling,
visualization, interaction design, evaluation.
INTRODUCTION We all know that a curious person can more
efficiently absorb infonnation when it is well structured than when
it is arbitrarily scattered. The question that every information
architect then asks is, "How might I best organize the information
for that person?" This question contains three issues: what
structures are useful for organizing information in general, what
structures are useful for organizing that information, and what
structures are useful for organizing information for that
person.
To answer these above questions, we propose (i) collecting
experimental data from a number of subjects, (ii) analyzing the
mental models of those subjects with Multidimensional Scaling (MDS)
[7, 15, 16) and Trajectory Mapping (TM) [4, 9, 13], and then (iii)
using those models to design information structures. Such models
should lead more quickly to suitable interfaces rather than
beginning with trial and error explorations.
Permission to make digital/hard copies of all or part �f this
material fo_rpersonal or classroom use is granted without fee
�rov1ded that the copiesare not made or distributed for profit or
commercial advantage, the_ copyright notice, the title of the
publication and its date appear, and nollce is_ given that
copyright is by pennission of �e. ACM, Ii:ic, To
copy othe�1se,to republish, to post on servers or to red1stnbute
to hsts, requires specific
permission and/or fee. CHI 96 Vancouver, BC Canada @ 1996 ACM
0-89791-777-4/96/04 .. $3.50
Before continuing on to details, we offer definitions of
"information structure" and "mental model." By "information
structure" we mean an arrangement of pieces of information. The
arrangement might be a 2-D array on a table or a screen, as in a
card game or a spreadsheet. It might also be a 1-D ordering of
items, like a shopping list. It could also consist of a set of
information nodes with connecting association links, such as a
hypertext. All the pieces of information should belong to the same
conceptual type category, such as numbers to add, tasks to do, or
words to remember. By "mental model" we mean not the explanatory
model offered by Johnson-Laird [5], but rather a more general
definition: the cognitive layout that a person uses to organize
information in his or her memory.
THE EXAMPLE DOMAIN Consider the map of Boston shown in Figure
la. We have chosen fifteen different sites of activity taken from a
tour guide book (listed in Figure lb). This set of information
pieces (or "stimuli," as we will call them) is a very
highdimensional in feature space, and it has a relatively large
variance across people. That is, there are many features
Figure l a: Geographic layout of activities
413
-
Stimulus Set: Boston Tourist Attractions
I. Sports Museum
2. Children's Museum
3. Science Museum
4. Aquarium5. Swan Boats (boat tour of garden)
6. Newbury Street (elegant shopping)7. Quincy Market (outdoor
mall)
8. Trinity Church (historic site)
9. Magic Show
10. Salem (nearby historic town)
1 l. Harvard University
12. Museum of Fine Arts
13. Zoo 14. Fenway Park (baseball stadium)
15. Arboretum (nature preserve)
Note: Activities 9-15 are outside the scale of the map.
Figure 1 b: The stimuli for the MDS and TM experiments.
that can be used to describe the stimuli, and people do so in
significantly different ways. As a contrasting example, a deck of
playing cards contains very few features, namely the number of a
card, its suit, and perhaps its color, and different people
describe playing cards similarly.
MDS AND TM METHODS
A person who lives in Boston and knows the sites will have at
least two mental models of them, one based on their geographic
locations, and one based on their content. In order
• ooe 0 • • • 1 2 4 4 6 5
0 6 5 4 3 1 0 2 3 5 1
• 2 5 4 0 1 2
• 4
•
Figure 2a: Typical input data for the MDS algorithm; sim
ilarity data for black and white circles of different sizes. 7 =
very similar; 1 = very dissimilar.
414
CHI 96 A p R ! L I 3 - I 8. I 9 9 6
to illustrate the capabilities of Multidimensional Scaling
(MDS), first used by Torgerson [16], Shepard [15], and Kruskal [7],
we collected data from two subjects for each of these mental models
(Figure 3). Note that the judgments based on geographic similarity
(3a) are completely different from judgments based on content (3b).
The input data for MDS takes the form of pairwise similarity
judgments, and the output is an arrangement of the stimuli in a
metric space.
To clarify the procedure for obtaining these MDS plots, we offer
a simple example; consider a collection of seven black and white
circles of different sizes. In Figure 2a, these circles are
arranged randomly along the edges of a similarity matrix. The
numbers show a subject's similarity rating on a scale of I to 7.
From these numbers the MDS algorithm arranges the circles as shown
in Figure 4, with neighboring circles being the most similar.
The distances between the points in the output is usually a
non-linear transformation of the values in the similarity matrix.
For Figure 3a, we asked the subjects for every pair of stimuli, "On
a scale from l to 7, how similar are stimuli X and Y in terms of
their distance from each other?" For Figure 3b, we asked, "How
similar are stimuli X and Y in terms of their content or theme?" We
then used KYST2 [6] to run the MDS and generate the graphs
shown.
As one might expect, the distance-based MDS plot is similar to
the actual map of Boston, though somewhat warped; the warping could
stem from differing familiarity with the sites [11] or from
thinking of distance as travel time instead of geographic distance.
In the content-based MDS plot,
Figure 2b: Typical input data for the TM algorithm;
extrapolations and interpolations for black and white cir
cles of different sizes. Here are 12 of 21 possible pairs.
-
A p RI L I 3 - Is' I 9 9 6 CHI 96
Arboretum +Zoo+-
•
Salem Magic Show • ··---- 1'_ __________ t -------------
Sports • • Science Museum Museum
Harvard
Swan
Aquarium •
• Quincy Market
Boals;; ------
Newbury St.., • Trinity Church
Children's" Museum •
Figure 3a: MDS plot based on geographic similarity
similar activity sites like the Aquarium and the Zoo appear near
each other, as do shopping areas Newbury Street and Quincy Market.
One can use the groupings of points in an MDS plot to assign
features to clusters of points, as we have done in Figure 3b with
the features, "historical," etc. Likewise, many researchers who use
MDS would attempt to assign meaning to the axes of the plot,
suggesting perhaps that the X-axis runs from "playful to serious"
and Y-axis runs from "outdoors to indoors". Such labels or category
assignments must be done carefully and should be verified with
separate experiments, however, because they are often biased by the
experimenter's a priori knowledge of the data.
The reader might wonder what would be the outcome if we had
asked subjects to give general similarity ratings without
specifying what type of similarity. Would the resulting MDS plot
have been an unfortunate mixture of the two plots in Figure 3? This
issue raises the question of how we could
,,✓,20s+'"fefic;,','
•.¾f;;t;;:
-
C ~ " ~ @ ~ APRIL t 3 - 1 8 , 1 9 9 6
in the two "ex" columns, and the interpolant in the "int" col-
umn. As well as using a stimulus for each slot, the subject may
also enter an "X" or a "...". An "X" indicates that the subject did
not reel comfortable choosing a stimulus for that spot, and the
"..." indicates that the subject could imagine a stimulus that
would fit there, but that such a stimulus was not present in the
given data set to choose from.
From this set of quintuples, we can now extract a connected
graph of the stimuli in which the maximum number of quin- tuples
fit. For example, in the top row of Figure 2b, we have an ordered
set of white circles ending in a small dot in one column, and a set
of black circles ending in the same small dot in the other column.
If these quintuples are equally con- sidered as constraints on the
graph, the ending trajectory map will contain a path from the large
white circle to the large black circle, going through the small
dot, as shown in Figure 4.
BOSTON DATA Figure 5 shows a trajectory map for tourist site
data gathered from the authors. Note here that the positions of the
nodes are not important; the mental model lies in the connections
between the nodes, i.e. the topology of the graph. The weights on
the links are based not on similarity, but rather on the robustness
of that link across a gamut of parameters within the TM algorithm.
The weight on a link can thus be thought of as the strength of the
connection in the mental model. (A TM algorithm is being fine-tuned
for release by Gilbert.)
By pruning the trajectory map to only its strongest links, one
can see rough feature clusters emerge (see the heavier links
Mag c Show .8~;4;~ ~4 ~;~, . . . . . . . . . . . . . . . . .
Salem
; 4 : ~&
( " i
lildren's mourn
Science , M i i~..11 i m i
Museum ~iiiiii ~
O 0
Newbury St.
Arboretum Quincy Market
,:. ~,C? -0! ( ) ~ , ,
Figure 6: The mental models combined: a rough TM path
superimposed over the MDS content plot.
in Figure 5). In our example, the clusters are roughly simi- lar
to the estimated clusters in the MDS plot (Figure 2b). In a
trajectory map, however, the s~imuli are ordered within each
cluster, e.g. Arboretum, Swan Boats, Aquarium/Zoo. Figure 6 shows
the combined mental models, a rough TM path drawn over the MDS
content-based plot.
Because the tourist sites have several possible features that
could be used by subjects to report their TM extrapolations and
interpolations, one can discover the most salient fea- tures of the
data set by running the algorithm across many
m Fenway Park
Aquarium
O
Z q
Figure 7a: Children's activities energized by the region- al
activation spreading network.
m
F ¢nw,~tf Pw~
i~ ::~L ̧
I I tmi
Fine Arts Museum Trinity Church
Z ~
m
Figure 7b: Adults' activities energized by the regional
activation spreading network.
416
-
A p RI L I 3- I 8' I 9 9 6 CHI 96
II II
•-
II ..
FenwayPw�
II
•-
PAPERS
Figure 8: The network activates the path segments when the user
examines the sites Sports Museum, Fenway Park and Aquarium. The
activation levels are mapped to image and typographic size,
enabling the system to suggest to the user an appropriate next step
based on the MDS and TM results.
subjects. It is likely with this tourist site information, for
example, that some subjects would do the TM based on geographic
distance, while others would do it based on the content of the
sites. This feature difference would eventually manifest itself
from obviously disparate trajectory maps, and lead to the phrasing
of the MOS questions described earlier: similarity in terms of
distance and similarity in terms of content. Examples of different
trajectory maps for the same domain can be seen in [12].
EXPERIMENTATION
The mental models from TM and MDS provide an excellent basis for
measuring the efficiency of an information structure that has been
built from the models. One might arrange the stimuli serially
according to the TM paths or distribute them according to the MDS
plot. After collecting data from a new pool of subjects for
measures of readability, ease of remembering, etc. (see [2, 3] for
a typical array of memory measures), the information structure can
be systematically varied by changing the parameters that produce
the MOS and TM models.
EXAMPLE INTERFACES
If one considers the set of tourist attractions as an
information space to be explored, we can also use the mental models
to give the user well-founded suggestions as to his or her next
step of exploration. As a demonstration of feasibility, we have
designed a visualization system that allows the exploration of the
Boston tourist attractions. Since we are no longer restrained to
the geography of Boston, as an actual tourist would be, we must now
answer the question, "How should we order the various sites?" To
explore the different possible answers, we have built three
different interfaces for the system, each of which defines an
"attentional window" as the current region of interest in the
information space. An activation spreading network based on the
mental models defines the size of the attentional window as the
user explores. Thus, if the aquarium node of the network is
currently activated, and the models suggest that the swan
boats node is closely related, then the activation will spread
to swan boats, leading the attentional window to include swan boats
as a potential next focus.
In the first two interfaces, the 15 sites are arranged
twodimensionally according to the MDS plot in Figure 3b. Whichever
sites fall within the attentional window are rendered larger than
the others. In one interface, the attentional window envelops MDS
regions (Figure 7), and in the other, the window spreads across TM
paths (Figure 8). Figure 7a shows the display of regional
activation with emphasis on children's activities. Figure 7b shows
the activities in an adult context. In contrast, Figure 8 shows
several successive frames of path activation: when the user
investigates the Sports Museum, the network suggests that Fenway
Park or the Children's Museum might be a good next choice (those
two sites are slightly enlarged in Figure 8a). As the user shifts
attention to the next event, past activities fade away and
activities further along the path become more conspicuous.
The difference between these two styles of exploration can be
characterized by the width of the attentional window: narrow in the
case of paths, and wide in the case of regions. By smoothly varying
the width, one might change smoothly between modes of
exploration.
The third interface attempts to combine the path and region
following ideas of the first two. This interface depicts the sites
as semitransparent multimedia cubes in a 3-D space (Figure 9). The
cubes are arranged along the TM paths, but at each cube, other
cubes within the same region can be seen orbiting the current cube
nearby. Thus, the user can "fly" smoothly through the space along
the TM routes, or branch off to a similar site within the current
region. Also, the floor of each cube displays a geographic map with
the site's location high-lighted, thus incorporating all of the
features that we have discussed thus far.
417
-
CHI 96 A p RI L I 3- I 8, I 9 9 6
aquarium W11h vivid banners !lying overhead and a Susumu Shingu
sculpture dancing in the wind. the New England Aqu�r�'?Piil,16hed
at !he edge of Bos Ion harbor. is easy 10-spo1. In an ouldoor lank
near the enlrance. harbour seals f1olicalmost close enough 10
louch. The inside scene is even more colorful and act,ve. The
centorp,ece of the aquarium. a massive 187.000 gallon ocean tank.
is one of the world s larges! cyllnd11cal sail
Figure 9: An interface that combines path and region following;
the sites are represented by cubes in a 3-D space. The cubes lie
along the TM paths, and the other sites from the MDS region
encircle the cube within the attentional window.
The activation spreading network was designed by Lokuge and
Ishizaki [10], and the visualization system is implemented on a
Silicon Graphics Onyx workstation. This network links nodes in a
space (Figure 10) and uses the ordering in the space to control the
activation level of a node [ 1, 12]. When attention "jumps" to a
new node, the network partially reduces the activation levels in
the original set of nodes, leaving a faint trace as a history of
the attentional sequence.
DISCUSSION
We have designed this visualization system with its various
interfaces both as an existence proof that such a system could be
devised and as an illustration of our proposed methodology for
structuring information. We mentioned above the fact that a variety
of mental maps exist for each data set, both within and across
individuals. A further development would be an intelligent
activation spreading network, i.e. one which contained the full
repertoire of different mental models for a given data set and
chose the best model based on the user's behavior. Such a
repertoire could be described as a "hyper-mental-map" and could
likely formed by gathering MDS and TM data on the mental mod-
Goal: Kids' Tour
of Boston
' 1
Sports Museum
� / \enway Park
�/ Chnd='• •• :.rn
l
s�, """=
Magic Show l
� Salem
Figure 10: Schematic diagram of a regional activation spreading
network. Activation energy spreads in the direction of the arrows
initiated by the Goal: Kids' Tour of Boston.
418
els as stimuli themselves.
We have proposed and implemented a method of organizing
information space around cognitive maps. We plan next to explore
the degree to which this method can be used with more generalized
domains of knowledge, but examples from previous MDS and TM papers
lends us hope that the method could become generally applicable. By
accommodating an individual's search context (e.g. through paths or
regions) and his or her particular model of a domain (e.g. a
particular MDS or TM model), one can offer both a more personalized
tour of the information space and a more easily absorbable mass of
information.
ACKNOWLEDGMENTS
This work was in part sponsored by ARPA, JNIDS, NYNEX and
Alenia. However, the views and conclusions expressed here are those
of the authors and do not necessarily represent that of the
sponsors.
REFERENCES
l . Anderson, J. A. Spreading activation theory of memory.
Journal of Learning and Verbal Behavior 22, 261-295,1983.
2. Atkinson, R.C. & Shiffrin, R.M. The control of shortterm
memory. Scientific American, 225, 82-90, 1971.
3. Craik, F.I.M. & Lockhart, R.S. Levels of processing:
Aframework for memory research. Journal of VerbalLearning and
Verbal Behavior, 11, 671-684, 1972.
4. Gilbert, S. A. & Richards, W. Using trajectory mappingto
analyze musical intervals. Proceedings of the SixteenthAnnual
Conference of the Cognitive Science Society.1994.
5. Johnson-Laird, P.N. Mental models. Cambridge, MA:Harvard
University Press, 1983.
6. Kruskal, J.B. kyst2a. AT&T Bell Laboratories,Computing
Science Research Center, Netlib repository of software.
http://plan9.att.com/netlib/mds/index.html.
-
i
APRIL 13-18, 1996 Cl" l | 9 6
7. Kruskal, J.B. Nonmetric multidimensional scaling: A numerical
approach. Psychometrika, 29, 28-42, 1964.
8. Lokuge, I. Dynamic magical environments: Engaging interaction
based on the art of illusion. Masters Thesis, MIT Media Lab, 1995.
To receive a copy email docsOmit.edu or call MITDocument Services
at 6171253-5650.
12.
13.
Maes, P. Situated agents can have goals in Designing Autonomous
Agents: Theory and Practice from Biology to Engineering and Back,
ed. P. Maes, MIT Press/Bradford Books, 1992.
Richards, W. & Koenderink, J.J. Trajectory mapping ("TM"): A
new non-metric scaling technique. Proceedings of European
Conference on Visual Perception, Edinburgh (Abstract), 1993. In
press Perception.
9. Lokuge, I., Gilbert, S.A. & Richards, W. Visualizing
information spaces: A tour through Boston. Presented at European
Conference on Visual Perception '95, Ttibingen. Abstract in
Perception, 24 Supplement, p. 76.
14. Saint-Arnaud, N. Class~cafion of Sound Textures. Masters
Thesis, MIT Media Lab, 1995. To receive a copy email d o c s @ m i
t . e d u or call MITDocument Services at 617/253-5650.
10. Lokuge, I. & Ishizaki, S. GeoSpace: An interactive visu-
alization system for exploring complex information spaces.
Proceedings of ACM SIGCHI'95, Denver, Colorado, 1994.
15. Shepard, R.N. The analysis of proximities: Multi-
dimensional scaling with an unknown distance func- tion. I. &
II. Psychometrika, 27, 125-140, 219-246, 1962.
11. Lynch, K. Image of the City. Cambridge, MA: MIT Press,
1960.
16. Torgerson, W.S. Multidimensional scaling: I. Theory and
method. Psychometrika, 4, 401-419, 1952.
419