1 Size and Structure Matter to Mobile Users: An Empirical Study of the Effects of Screen Size, Information Structure, and Task Complexity on User Activities with Standard Web Phones Citation: Chae, M . and Kim, J. (2004), Size and Structure Matter to Mobile Users: An Empirical Study of the Effects of Screen Size, Information Structure, and Task Complexity on User Activities with Standard Web Phones, Behaviour & Information Technology, forthcoming Minhee Chae and Jinwoo Kim Human Computer Interaction Lab Yonsei University Seoul Korea [email protected], [email protected]Primary contact person: Jinwoo Kim, Professor School of Business Yonsei University Seoul, 120-749, Korea Tel) +822 2123 2528 Fax) +822 313 5331
57
Embed
Size and Structure Matter to Mobile Users: An Empirical Study of the Effects of Screen Size
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Size and Structure Matter to Mobile Users: An Empirical Study of the Effects of Screen Size, Information Structure, and Task Complexity on User
Activities with Standard Web Phones
Citation: Chae, M . and Kim, J. (2004), Size and Structure Matter to Mobile Users: An Empirical Study of the Effects of Screen Size, Information Structure, and Task
Complexity on User Activities with Standard Web Phones, Behaviour & Information Technology, forthcoming
Minhee Chae and Jinwoo Kim Human Computer Interaction Lab
The small screens of mobile Internet devices, combined with the increasing complexity of
mobile tasks, create a serious obstacle to usability in the mobile Internet. One way to
circumvent the obstacle is to organize an information structure with efficient depth/ breadth
tradeoffs. A controlled lab experiment was conducted to investigate how screen size and
information structure affect user behaviors and perceptions. The moderating effects of task
complexity on the relationship between screen size/information structure and user
navigation/perceptions were also investigated. Study results indicate that both information
structure and screen size significantly affect the navigation behavior and perceptions of mobile
Internet users. Task complexity was also found to heighten the influence of information
structure on user behavior and perceptions. The paper ends with a discussion of theoretical and
practical implications, among them a key implication for mobile Internet businesses: for
corporate intranet systems as well as m-commerce transaction systems, the horizontal depth of
information structures should be adapted to task complexity and anticipated screen size.
Keywords: Mobile Internet, task complexity, screen size, breadth/depth tradeoffs,
menu/information structure
3
Size and Structure Matter to Mobile Users:
An Empirical Study of the Effects of Screen Size, Information Structure, and Task
Complexity on User Activities with Standard Web Phones
1. Introduction
The mobile Internet offers wireless access via handheld devices to the digitized contents of the
Internet (Francis 1997). A survey study has projected that the number of mobile Internet users
in the world will grow eighteen-fold between 2000 and 2005, to about 729 million (Intermarket
Group 2002). The number of people in Japan using the mobile Internet already exceeds the
number using the traditional stationary Internet. Many forecasters, basing their predictions on
the increasing prevalence of standard mobile Internet phones, suggest that in the near future most
Internet access will take place by means of small wireless devices, equipped with a browser and
a wireless connection, that provide “anywhere and anytime” access (Buyukkoken and Garcia-
Molina 2000, Buyukkoken et al. 2000). The popularity of the mobile Internet is not surprising
when one considers the considerable benefits it offers Internet users, for it enables them to access
Internet information at the moment of need, whether or not a desktop PC is available
(Buyukkoken and Garcia-Molina 2000). The growth of various mobile technologies also holds
the promise of increasingly effective tools for wireless access, such as new protocols and
4
browsers adapted to the mobile interface (Intermarket Group 2002).
However, despite the industry’s conviction that the mobile Internet is the next “killer
application,” the reactions of actual users are quite negative in terms of usability (Nielsen and
Ramsay 2000). Their disappointing experiences with the mobile Internet result from the
limitations that distinguish mobile devices from conventional desktop PCs (Chae and Kim 2003).
Mobile Internet devices, especially Internet-enabled phones, have resources vastly inferior to
those of the desktop computers that access the traditional stationary Internet. Most current
mobile Internet devices suffer from small screens, low bandwidths, limited storage, a short
battery life, slow CPU speed, and cumbersome input facilities (Kamba et al. 1996, Albers and
Kim 2000, Buchanan et al. 2000). In short, usability is the greatest barrier between what the
mobile Internet could be and what it currently is (Venkatesh et al. 2003).
Though mobile devices will, in future generations, gradually redress many of their
present limitations, the display is not likely to become much larger, for the need for portability
will continue to constrain the size of the screen. Because text size cannot be reduced below a
threshold of legibility, only a small amount of information can be shown on the screen at a time;
on the typical mobile Internet phone, there are fewer than 15 lines vertically and fewer than 12
characters per line. Therefore, most of the time users cannot be shown complete lists of
available options within the screen’s display area. Instead they have to scroll through the menu
5
list, select an option, scroll through a sub-menu, select an option, and so on, repeatedly. They
are required to perform multiple key presses and may commit numerous navigation errors
(Albers and Kim 2000).
The usability problems faced by mobile Internet devices are exacerbated by the nature
of the tasks that users characteristically perform with them. Users frequently face situations in
which they must access complex information at the point of need, as, for instance, when a user
wishes to find the gas station nearest his or her current location. In such a situation, unless
users can retrieve the exact information they want immediately, they cannot reap the benefits of
the mobile Internet, namely, portability and instant accessibility. Moreover, mobile users
involved in complex tasks are more likely to be under time pressure than traditional Internet
users are, and thus may be more prone to errors when they try to accomplish a task.
One way to address these usability problems would be to develop an efficient menu
structure, one that took into consideration the small screen and the complex nature of the tasks
users can perform on the mobile Internet. Currently, information on a mobile screen is
presented to users in the form of a strict hierarchy. Compared to conventional menus in
stationary Internet systems, which provide users with multiple paths to a given target page, most
mobile content menus only allow one-path navigation. Moreover, the limited screen size forces
most mobile browsers to support only a line-based navigation, with a few soft keys such as OK,
6
Clear, Previous, and Next. The keys, when pressed, display a page on the next level down, the
page directly above, the next page on the same level, or the previous page on the same level,
respectively. Moreover, mobile users are unable to jump directly from page to page, and are
required instead to follow paths or links sequentially. Such a stepwise, line-based navigation
has been known to lead users to unexpected, undesired outcomes (Wallace et al. 1995). Several
studies have indicated that mobile Internet users suffer more severely from the problem of
undesired outcomes than stationary Internet users do (Kim et al. 2002, Nielsen and Ramsay
2000).
In sum, the portability of the mobile Internet poses a formidable design challenge:
how can the information necessary for complex tasks be presented effectively on small screens
that offer only limited navigation facilities? The main goal of this study is to examine how
information structure and screen size affect a mobile user’s navigation activities and perceptions.
A secondary goal is to identify the moderating effect of task complexity on the relation between
information structure/screen size and behavior/ activities. Thus the main research question is:
how may mobile Internet sites be structured to facilitate the complex tasks users will perform on
them, given that they will be displayed on small screens with limited navigation facilities? The
results may provide a starting point for the design of optimal menu structures for the mobile
Internet.
7
This paper consists of five parts. The following section presents the study’s research
background. The next section explains the research model and hypotheses, as well as the
research method – a controlled laboratory experiment. The results of the experiment follow,
and the paper closes with a discussion of the study’s implications and limitations.
2. Research Background
This section reviews three research areas closely related to our study: hierarchical menu
structure, the usability of small screens, and task complexity.
2.1 Hierarchical Menu Structure
In most Internet systems, menus use a complex mixture of various structures, but the overall
architecture of a website is generally hierarchical. In a purely hierarchal structure, each node
(that is, menu panel) in the hierarchy can be reached only from a single superordinate node that
lies directly above it in the hierarchy. The two key characteristics to be considered in the design
of a hierarchical structure are the depth and the breadth of the menu (Henneman and Rouse
1984). Depth (d) is usually defined as the number of levels in the hierarchy, breadth (b) as the
number of options per menu panel (Paap and Cooke 1997). When there are equal numbers of
options on each panel, the number of terminal nodes is a function of breadth raised to the power
8
of depth: N= bd.
Navigation problems (e.g., getting lost, or choosing an incorrect pathway to a goal)
become more severe as the hierarchy grows deeper. A hierarchical structure with several levels
requires a user either to recall or to discover a pathway from the present location to the target
location. In fact, a prior study (Snowberry et al. 1983) showed that error rates increased from
4.0% to 34.0% as depth increased from one to six levels. In addition, as the depth increases, so
does the number of page transactions, that is, the number of movements from one page to
another (Paap and Cooke 1997). Each page transaction requires an action from the user (e.g., a
keystroke or a mouse selection) and a response from the computer (e.g., a change of display).
Obviously, each transaction adds to the cumulative response time (Paap and Cooke 1997). In
sum, depth in an information structure increases the likelihood of navigational errors, and also
decreases execution speed.
Nonetheless, there are good reasons to consider a system with greater depth. Certainly,
when the amount of information exceeds the available space, at least some depth must be
introduced – and, in fact, a structure that favors depth can avoid the crowding brought about by
excessive breadth. Crowding – the presence on a single menu of more options than a user can
process quickly – increases the time it takes a user to make his or her selection. Thus, though
computer execution time is reduced, there is no net gain in the time it takes to accomplish a task
9
– and user frustration is likely greater. Papp and Cooke (1997) have found that a structure that
favors depth over breadth can avoid crowding by allowing funneling – that is a reduction in the
total number of options a user must choose among. Funneling can generate efficiency gains,
particularly in situations where more cognitive processing is required of users (Kiger 1984).
Thus it is generally agreed that developers should use depth to avoid crowding and to encourage
funneling.
In sum, the advantage of depth is that it encourages funneling, the disadvantage that it
induces errors and increases the number of page transactions. The advantage of breadth is that
it reduces navigation errors and the number of page transactions, the disadvantage that it leads to
crowding.
D. Miller (1981) examined the tradeoff between depth and breadth in menu structure
design. His study tested four different structures, all with 64 nodes at the bottom level: 26, 43,
82, and 641. The results indicated that increased breadth decreased the number of page
transaction but came at the expense of display crowding. The study results suggested that the
82 structure allowed the best ratio of performance speed to navigation errors among the four
structures. This level of breadth fits comfortably within the range of G. Miller’s (1956) finding
that short-term memory can typically retain 7 +/- 2 items. Kiger (1984) extended D. Miller’s
research, investigating which of his several menu structures users preferred. It turned out that
10
preference was consistent with performance: users were found to prefer the 82 structure.
Clearly, the balance of depth and breadth in hierarchical menu systems affects both user
navigation behaviors and user preferences – but whether the optimal balance is affected by
screen size remains to be seen. This is the topic of the next sub-section.
2.2 Usability of Small Screens
Considerable research has been done on the usability of small screens (Duchnicky and Kolers
1983, Dillon et al. 1990, Han and Kwahk 1994, Jones et al. 1999, Kim and Albers 2001). In
particular, researchers have been interested in the question of information presentation: how
does one display information effectively on screens far smaller than conventional computer
screens? Studies investigating the effects of small displays have indicated that reduced screen
size is closely related to various user behaviors, including navigation, searching, and browsing
(Duchnicky and Kolers 1983, Dillon et al. 1990).
However, the results of these studies are somewhat inconsistent. Some found that while
user performance, measured in terms of the time taken to select an option, worsened as the
display size decreased, the effect was not dramatic (Jones et al. 1999). However, a study of the
effect of display size on web interaction found that small screen size reduced user effectiveness
by up to 50% for the tasks being observed (Han and Kwahk 1994, Telstra 2001).
11
There are at least three possible causes for the inconsistency. First, typical displays
explored in studies such as Jones et al. (1999) ranged from one-quarter to one-half the size of
typical VGA displays (1024x768), whereas mobile displays are generally much smaller, ranging
from 128x128 (mobile phone) to 320x240 (PDA). It may be that real problems only occur
when a display is so small that only a few options can be displayed at one time, and that results
from prior studies like Jones et al. (1999) are not directly applicable to the much smaller displays
of mobile Internet devices. Second, many previous studies were interested in displays on
devices like typewriters and photocopiers (Buchanan 2001). Such early office-automation
devices allowed users to choose functions from a brief list presented on a small LCD screen.
Information structures encased in these devices were much simpler than mobile Internet
information structures are; the latter generally provide a great deal of information and arrange it
in complex structures. It is likely that mobile Internet users would not struggle if only a simple
list of choices were presented to them. Thus it may be that the findings of Han and Kwahk
(1994) are no more applicable to the mobile Internet than those of Jones et al (1999). Third,
though several studies, acknowledging that menu structure exerts an influence on user behavior
and cognitive process, have attempted to design effective mobile menu systems, they have done
little to consider the relation between the screen size of mobile devices and the hierarchical menu
structure of mobile Internet services.
12
In sum, prior studies have not directly treated the present research questions. Those
that considered a complex information structure considered too large a screen; those that
considered an appropriately small screen considered too simple an information structure; those
that have specifically taken up mobile Internet menus have failed to consider the relation of
menu structure to screen size. Thus the very small screens of mobile devices should be studied
anew, in light of the complex information structures of mobile Internet services.
Further, the effect of reduced screen size on user behavior may depend on task
complexity – the topic of the next sub-section.
2.3 Task Complexity and Menu Structure
The relation of task complexity to menu depth has been discussed extensively by Jacko and
Salvendy (1996). Their study demonstrated that the perceived complexity of a computerized
task increased as the depth of the hierarchical menu increased. The study takes as its theoretical
basis Campbell’s framework of the complexity of tasks (Campbell 1988) and Frese’s definition
of task complexity in a hypertext menu structure (Frese 1987). Campbell identified four
characteristics of a complex task: multiple paths, multiple outcomes, conflicting
interdependence among paths, and uncertain linkages (Campbell 1988). Frese proposed that
complexity is determined by the number of decisions that have to be made and by the relations
13
among those decisions (Frese 1987). In Jacko and Salvendy (1996), the central questions were,
first, which menu depths would yield the fastest responses and the fewest errors, and, second,
how these optimal designs would be perceived by users in terms of complexity. They found
that perceived complexity increased as menu depth increased, and also that perceived complexity
lengthened response time and reduced response accuracy. Thus minimizing perceived
complexity by creating a shallow menu structure might improve user response time.
3. Research Model and Hypotheses: A Theory of Information Structure for the Mobile
Internet
This section discusses three concepts critical to the present study: horizontal depth, screen size,
and task complexity. The study takes horizontal depth and screen size as independent variables
and task complexity as a moderating variable. The section ends with a discussion of dependent
variables, sub-hypotheses, and the research model, which is summarized in Figure 3 later.
3.1 Horizontal Depth
In terms of information structure, there is a subtle but important difference between the mobile
Internet and the conventional desktop-based Internet. Consider, for instance, a user finding
local traffic information (target, in Figure 1) through a mobile Internet–enabled phone. The
14
user starts searching at the top level (x in Figure 1) and moves down to the sub-level (y), where
he or she encounters a list of street names. If there are several street names on the list, there
may be several ways to present the whole list on the screen, such as providing the list on one
page (y - y’: Figure 1-A), or presenting the list on several pages (y1 - y5’: five pages in Figure
1-B). In the former case, the user searches for the street (list k) by scrolling down the long list,
while in the latter case, the user must move from the first page to the third page, which contains
list k (y1 y2 y3). After choosing list k, he or she continues, scrolling down to z and to the
target information.
In both cases, then, the user moves through three levels of vertical depth
(x y z target). But the cases are different in terms of what may be called horizontal depth.
To reach the page where list k is, the user in Figure 1-A only has to scroll down to level y, while
the user in Figure 1-B must move through two more levels of horizontal depth (y1 y2 y3).
We refer to the depth that exists between pages within a single level of the menu hierarchy as
“horizontal depth” (y - y1 - y2 - y3 - y4 - y’ in Figure 1-B), distinguishing it from the “vertical
depth” that exists between levels of the hierarchy (x y z target in both 1-A and 1-B).
Horizontal depth divides a unit of content into multiple sequential links, rather than
leaving a single larger unit on one page. “Horizontal depth” is horizontal because the content is
divided into multiple pages that sit on the same level of the information hierarchy (e.g., Main
15
Street and Sunset Boulevard in downtown LA), but it has depth because users must take more
steps in their search (e.g., from the page of “M” street names to the page of “S” street names).
It should be noted that horizontal depth differs from the “paging” process in traditional stationary
Internet systems. Users with mobile Internet phones must scroll to the last entry of the current
page (y1) in order to proceed to the first entry of the next page (y2), whereas traditional Internet
users can jump from one page to another directly (y - y1 - y2 - y3 - y4 - y5’). This difference is
caused by the limitation of most mobile Internet browsers to a line-based interface (rather than a
graphic user interface), as will be explained in more detail later. Therefore, mobile Internet
users may perceive horizontal depth as a form of depth rather than as a form of breadth.
[INSERT FIGURE 1 ABOUT HERE]
Because less information per page (breadth) leads to more pages, which leads to more
horizontal depth, there may be a tradeoff in a menu system between breadth and horizontal depth
similar to the well-known tradeoff between breadth and vertical depth (Geiser and Schumacher
1976, Henneman and Rouse 1984).
Previous studies on paging have shown that paging results in more errors (Geiser and
Schumacher 1976) and that error rates may rise as horizontal depth increases (Henneman and
Rouse 1984). Moreover, increasing horizontal depth may also increase perceived complexity
for users: encountering horizontal depth within one level of a hierarchy, they may mistake it for
16
vertical depth, and believe they are exploring a deeper structure than they actually are. As
perceived complexity increases, users may have more difficulty forming clear mental models or
structural frameworks (Schwarz 1983) – and, unless users have a correct mental model, they will
have difficulty sorting and structuring the information they receive as they progress through a
site (Albers and Kim 2000). The problem is especially pertinent to the mobile Internet, where
relatively deep menu structures are to some degree unavoidable, given the large amount of
information involved and the limited screen space available.
However, the negative effects of greater breadth in mobile Internet systems should also
be considered. The method the typical mobile device has for presenting options on a page
differs from that of the conventional Internet system. In the latter system, users just need to
scan a list of options in order to encode those options and decide whether to terminate their
search or continue (Miller 1981, Redish 1994). But in a mobile menu system, though many
options can reside on a single page, only a few are visible at a time, because only a few text lines
are available to the display screens. To view the options beyond the visible display, the user
must scroll line by line. When he or she presses the scroll-down button, for instance, all the
information on the screen moves up one line. On a four-line screen, three lines of the displayed
information will move up one line, the top line will disappear, and a new line will appear at the
bottom. After moving down the list by scrolling, the user must refocus on the correct part of
17
the content (Mechior 2001), the changes to which may increase task complexity: refocusing
becomes more difficult as more information changes. Thus processing time for mobile users
includes not just the time it takes to scan options displayed in the visible area, but also the time it
takes to scroll line by line to view as much of the list as necessary and to refocus on it. In a
mobile Internet system, reducing breadth by creating more horizontal depth might reduce
perceived complexity in serial information searches (Jacko and Salvendy 1996).
What is clear is that there are tradeoffs between horizontal depth and breadth, just as
there are tradeoffs between vertical depth and breadth. The study’s first main hypothesis is as
follows:
H1: Horizontal depth affects users’ navigation activities and perceptions.
3.2 Screen Size and Horizontal Depth
One aspect of screen size, as it relates to horizontal depth, is the rate at which information
changes as users scroll line by line through the menu structure. Consider two mobile devices
with different screen sizes, as shown in Figure 2.
[INSERT FIGURE 2 ABOUT HERE]
One screen has six lines, and the other has nine. Each screen devotes two lines to icons and
pictures (the top line showing a phone icon, the bottom line “UP” and “OK”) and one line to the
18
heading (“Best Melodies”). None of these lines can be used to display content. In other
words, the nine-line screen has only six lines of content area (Figure 2-A), the six-line screen
only three (Figure 2-B). The difference between the two screen sizes lies in the rate at which
information changes when users scroll up or down. If, for example, a user scrolls down line by
line on the six-line mobile screen (with three lines of content), 1/3 of the information changes per
scroll. However, if he or she scrolls down on the nine-line screen (with six lines of content),
only 1/6 of the information changes per scroll.
The smaller the screen, the more radical the information change users experience – and
the higher their cognitive load when they attempt to understand their current location relative to a
reference point. According to prior research (Kahneman et al. 1982), people tend to anchor a
reference point when they start to solve a problem, and to keep adjusting their current point on
the basis of their reference point. This result can be applied to the present case, namely,
navigation through an information space. If users scroll down one line on a standard PC
monitor, they can easily identify their reference point, because most of the screen content has not
changed. However, if they scroll down one line on a six-line screen, they may not be able to
relocate their reference point easily, because a large portion of the screen content has changed.
The higher the relative change, the more drastically users must adjust their anchoring points –
which will likely require greater cognitive effort of them. Thus our second main hypothesis is
19
as follows:
H2: Screen size affects users’ navigation activities and perceptions.
3.3 Task Complexity and Horizontal Depth
The mobile Internet has come to play an important role in numerous everyday tasks (Albers and
Kim 2000). Some tasks are relatively simple to perform; for example, mobile Internet users
access the Internet to learn the current local temperature (Kim et al. 2002). Others are
considerably more complex, as when users download popular animations or melodies through
their cellular phones, or bid on items in an on-line auction. In such cases, users have to
compare several alternative items and then decide on one, a procedure more complex than a
simple search. In fact, according to Frese’s (1987) theoretical background, this kind of task is
far more complex, because of the greater number of decisions to be made and the more
numerous increased inter-relations among these decisions (Frese 1987).
Users employ different navigation strategies for different sorts of task. In other words, a
user may have conceptual models for complex tasks different from his or her models for simple
tasks. It may be that performing complex tasks through the mobile Internet is made more
difficult by the smaller screen size and the greater horizontal depth. Since there is not enough
space on the screen to provide usable navigation cues or aids, such as a shopping cart or wish list,
20
the comparison process involved in selecting animations or melodies, for instance, can burden
the user with a heavier cognitive load (Albers and Kim 2000, Jacko et al. 1995). Therefore, the
effect of horizontal depth and screen size on user navigation activities and perceptions may vary
with the level of task complexity. Our third main hypothesis is as follows:
H3: The relation between horizontal depth/screen size and navigation
activities/perceptions will be affected by task complexity.
3.4 Dependent Variables and Sub-Hypotheses
Dependent variables for objectively measurable navigation activities are Between-Page
Navigation (BPN) and Within-Page Navigation (WPN). BPN represents the frequency of
“paging backwards and forwards.” BPN in the mobile Internet is different from traditional
paging (i.e., moving from one web page to another by clicking a button) because it includes
movement not just between vertical-depth pages, but also between horizontal-depth pages. In
addition to the page transactions necessary to complete a task, users may perform BPN in an
attempt to orient themselves, or to provide context as they progress through the text (Dillon
1990). BPN is expected to be closely related to screen size, because a smaller screen reduces
contextual information, leading users to rely on BPN as a source of contextual information.
BPN is also expected to be closely related to horizontal depth, because with greater horizontal
21
depth users become lost more easily, leading them to perform more BPN in an effort to get their
bearings. Therefore, two sub-hypotheses regarding BPN are as follows:
H1a: Greater horizontal depth will increase BPN.
H2a: Reduced screen size will increase BPN.
Second, WPN refers to scrolling activities within a single page. It is different from
traditional scrolling (scrolling up or down by moving the vertical scroll bar) in that most mobile
Internet users need to scroll line by line to the last item in order to proceed to the next page.
According to Jones et al. (1999), additional scrolling compromised users’ ability to accomplish a
task. Increased scrolling might result in a more pronounced “lost-in-space” effect or in greater
user frustration and fatigue (Nielsen and Ramsay 2000). The amount of scrolling reflects the
cognitive load a user experiences, and therefore WPN is also expected to be closely related to
screen size and horizontal depth. Since there is not enough space to provide contextual cues or
navigational aids, on a smaller screen users will scroll up and down more to make sense of the
page (Dillon 1990). However, greater horizontal depth is expected to decrease WPN, because
relatively few items will be presented in a single page. Therefore, two sub-hypotheses
regarding WPN are as follows:
H1b: Greater horizontal depth will decrease WPN.
H2b: Reduced screen size will increase WPN.
22
Dependent variables for the subjective features of navigation are perceived depth and
user satisfaction. Perceived depth was introduced to measure the user’s understanding of the
information structure (Jacko and Salvendy 1996). A perceived depth shallower than actual
depth indicates that a user has a relatively simple mental model for a given Internet system. A
perceived depth greater than actual depth suggests that a user is mistaking horizontal depth for
vertical depth. Indeed, it is proposed here that horizontal depth may be perceived as “real,” i.e.
vertical depth, increasing the level of perceived complexity. Reduced screen sizes will also
likely increase perceived depth: getting lost more easily, as they do with a smaller screen, users
will experience a heavier cognitive load, and therefore perceive more depth. Two sub-
hypotheses regarding perceived depth are as follows:
H1c: Greater horizontal depth will increase perceived depth.
H2c: Reduced screen size will increase perceived depth.
User satisfaction measures how satisfied users are with a given Internet system in
terms of navigation and structure. Kiger (1984), who extended D. Miller’s research by
investigating user preferences among various menu structures, showed that users’ subjective
preferences were consistent with their satisfaction. In the present context, it is expected that
user satisfaction with site navigation and structure will correspond with perceived depth.
Therefore, two sub-hypotheses regarding user satisfaction are as follows:
23
H1d: Greater horizontal depth will decrease user satisfaction.
H2d: Reduced screen size will decrease user satisfaction.
The overall research model for the study is summarized in Figure 3, with accompanying
research hypotheses. The research model consists of two independent variables (horizontal
depth and screen size), one moderating variable (task type), and two groups of dependent
variables, one objective (navigation activities) and one subjective (user perceptions).
[INSERT FIGURE 3 ABOUT HERE]
4. Experiment
4.1 Experimental Mobile Internet Site
How menus are categorized and labeled has been found to exert significant influence on user
behavior and perceptions (McDonald and Schvaneveldt 1988). To control for the confounding
effect of categorization and labeling, a pilot study was conducted with 60 mobile Internet users.
Each participant was asked to sort 100 index cards, each of which bore the name of a mobile
Internet menu (Nimwegen 1999). Participants were then asked to sort the cards into piles so
that cards representing similar concepts wound up in the same pile. After a subject had
completed this task, he or she was asked to arrange the piles in larger groups that seemed to
belong together, and then to invent a name for each group. The name, which offered further
24
insight into a user’s mental model of the information structure, was written on a Post-It note and
placed on the table next to the group.
On the basis of the pilot study results, the site for the main study was organized in a
3 3 3 3 structure (four vertical levels and three degrees of breadth). Each of the 81 (i.e. 34)
menu items at the bottom level was linked to a list of 60 selectable items. For example, if a
subject were given the task of determining when a Britney Spears fan club meeting was to be
held, he or she would navigate along this path: For Fun (level 1) Communities (level 2)
Fan Clubs (level 3) Movie Stars (level 4). Upon selecting “Movie Stars,” he or she would
encounter a list of 60 movie stars, each linked to specific information.
A standard navigation system, based on industry guidelines for mobile Internet phones
(Nokia 2001, Openwave 2001, Telstra 2001), was developed for the experimental site, as
follows:
The following functions were assigned to four hard keys: UP, DOWN, OK, and
CLEAR.
UP was used to scroll up one line, DOWN to scroll down one line. Movement was
only possible one line at a time.
OK was used to submit requests, CLEAR to go back to the previous page.
Scrolling could only go up and down, not left or right.
25
A cursor was placed (to start) on the top line. NEXT and PREVIOUS were offered
on the last line of the menu, in case the information was divided into multiple pages.
To move to the next page, users had to scroll down to the bottom line and select the
NEXT or PREVIOUS button.
There was no short-cut button (such as a “Home” button), or direct selection through
a number key or search engine.
The experimental site was built on a Game Virtual Machine (GVM)1 programmed in
Mobile C, a language based on ANSI C and optimized to the mobile environment. Next, the
developed content structures were uploaded to the GVM server group. Finally, the
experimental site was downloaded and executed on a GVM-installed mobile Internet phone, as
shown in Figure 4. The mobile Internet phone was an LG Cyon CX-300V featuring WAP and
256-color, 120x160 resolution LCD.2
[INSERT FIGURE 4 ABOUT HERE]
4.2 Experiment Participants
Participants were solicited for the main experiment through advertisements posted on several
1 The GVM is composed of a mobile application download platform that uses an OS independent black box. The GVM system consists of three parts: The GVM Server Solution, the GVM Module, and the GVM SDK. For more details, visit ShinjiSoft Corp. at http://www.shinjisoft.com 2 Detailed information about the LG Cyon CX-300V can be found at http://www.cyon.co.kr/job/product/feature.jsp?svci=10&pmd= CX-300DV
26
popular mobile Internet sites. To qualify, candidates had to be over the age of 12 and to use
mobile Internet services at least 30 minutes a month. Of the more than 300 people who applied,
90 were selected on the basis of demographic profile, average usage, and familiarity with the
mobile Internet. Their self-reported demographic and usage information was verified with the
mobile Internet carriers to which they subscribed. Out of a concern that behavior might differ
with gender, equal numbers of males and females were selected, although in fact gender was
found not to play a significant role in this study. Participant ages ranged from teens to thirties,
matching the customer profile for the mobile Internet. The average time they spent using
mobile Internet services was about 67 minutes a month. Subjects were thus well-balanced in
their experience and demographic profiles. They were compensated $40.00US for their
participation.
4.3 Experimental Design
A 2 x 3 x 2 factorial design was selected, as shown in Table 1. There were two between-
subject independent variables (screen size and horizontal depth) and one within-subject
moderating variable (task type). Screen size had two possible values: small (six-line screen)
and large (nine-line screen). Horizontal depth had three possible values: deep (six levels),
27
medium (four levels), and shallow (one level). Finally, task type had two possible values:
simple (single search) and complex (comparison search). Each of these variables is explained
in detail below.
[INSERT TABLE 1 ABOUT HERE]
First, horizontal depth was operationalized by dividing a content list of 60 items into
one-, four-, and six-page versions. As shown in Figure 5, if 60 options (e.g., 60 movie stars or
60 titles of incoming emails) are divided into four pages, 15 items (lines of content) are
presented per page, and there are four levels of horizontal depth. Thus the horizontal depth
(HD) is 4 and the breadth (B) per display is 15 (4HD/15B). Alternatively, if 60 options are
divided into six pages, each page has 10 items (6HD/10B). The study fixed vertical depth at 4
and the total number of items at 60 across all treatments.
[INSERT FIGURE 5 ABOUT HERE]
Second, two screen sizes were selected: six lines (the size most typical of mobile
Internet phones) and nine lines (the largest mobile phone display available at the time of
research). As seen already in Figure 2, a screen must devote two lines to icons and one line to
the heading. Thus the large screen had six content lines, the small screen three. Each line
could display eight Korean characters at most. There was no difference between the two
screens in terms of resolution or color.
28
Finally, four tasks were devised: two single-search tasks and two comparison-search
tasks. The single-search task was equivalent to a typical information retrieval on the Web.
Participants were asked to find the answer to a given question, such as “Where will the musical
The Lion King be performed in New York?” Comparison-search tasks involved comparing
several alternatives and then choosing only one. Subjects were given a task such as “Find the
five most popular ring melodies and then decide which is most cheerful.” A comparison-search
task is more complex than a single-search task, because a greater number of decisions have to be
made (Frese 1987), and because there are more paths and possible outcomes, with a higher
chance of conflicting interdependence among paths (Campbell 1988).
4.4 Experimental Procedure
To check the experimental system and operational validity, another pilot test was conducted with
18 subjects, after which some revisions were made to the experimental procedure. The final
experiment included two sessions: a practice session and the main session.
The practice session began with a brief explanation of the study’s purpose, and a general
introduction to the main features of the experimental mobile Internet site. Basic instructions
were given on use of the key buttons, as well as guidelines about navigation (for example, how
to go forward and backward). At the end of the practice session, participants were required to
29
perform two practice tasks.
In the main session, each participant was assigned the four study tasks in a random
order. Participants were asked to perform the tasks, one by one, taking up to 10 minutes per
task – a time period drawn from the pilot test.3 All the subjects succeeded in finding correct
answers within 10 minutes. The navigation behaviors of each participant were recorded in
system log files, which were transferred through a serial cable, in real time, from the mobile
Internet phone to a desktop PC, as shown in Figure 6(a). In order to maximize data integrity, a
small camera attached to the mobile phone recorded all user activity on a videotape in a remote
observation room, as shown in Figure 6(b). BPN and WPN were measured using the data from
the system log file and videotapes. BPN was calculated as the total number of movements from
one page to another, WPN as the number of up/down scrolling movements.
[INSERT FIGURE 6 ABOUT HERE ]
After completing each task, subjects were asked to answer a question about perceived
depth: “Please check ‘V’ where you think the information you have just found is located in
terms of depth.” The question was followed by a vertical line divided into equal sections. The
topmost node (i.e. the starting-point) was marked “Home,” and subjects were asked to indicate
the node where, relative to “Home,” the retrieved information had been found.
3 In the pilot test, all subjects found correct answers within 10 minutes. Most subjects in the main test also spent nearly 10 minutes per task,
30
In order to assess participants’ satisfaction in terms of navigational usability, a
questionnaire of seven items was administered at the end of each task. As shown in Table 2,
two sets of questions widely used in HCI studies were adapted to measure user satisfaction:
“Perceived Usefulness and Ease of Use” (PUEU) (Davis 1989), and “Questionnaire for User
Interface Satisfaction” (QUIS) (Chin et al. 1988). Participants answered each question on a
seven-point scale that ranged from “strongly disagree” (1) to “strongly agree” (7).
[ INSERT TABLE 2 ABOUT HERE]
5. Study Results
This section describes the results of the study, treating, first, objectively measurable navigation
activities, and, second, user perceptions about navigation and structure. The former results are
drawn from the system log and the video log; the latter based on the questionnaire responses.
5.1 Objectively Measurable Navigation Activities
Between-Page Navigation (BPN)
Three-way repeated-measures ANOVA results revealed a significant interaction effect between
task complexity and horizontal depth on BPN (F(2, 82) = 10.414, p < 0.01), but no significant
making comparison between the groups in terms of time taken less meaningful.
31
interaction effect between screen size and task complexity on BPN (F(2, 82) = 0.044, ns).
Therefore, H3 holds for BPN only in terms of horizontal depth, not in terms of screen size.
For simple search tasks, neither of the two independent variables showed statistically
significant main effects on BPN, nor any significant interaction effects on BPN (screen size (F(1,