1
Recommendations for Multi-Touch
Screen Design
Written by
Holly Peters, Rachel Gastrich, and Lyford Rome
Department of Educational Technology
San Diego State University
March 25, 2011
Prepared for the Museum of Photographic Arts
Balboa Park, San Diego, CA
2
Table of Contents Executive Summary ...................................................................................................................... 4
1. Introduction ............................................................................................................................... 5
1.1 Context ............................................................................................................................................... 5
1.2 Description of Task ............................................................................................................................ 5
1.3 Organizational Overview .................................................................................................................... 6
2. Analysis Process ........................................................................................................................ 7
2.1 Contextual Factors, Issues, and Concerns .......................................................................................... 7
2.2 Background Research ......................................................................................................................... 8
2.3 Survey ................................................................................................................................................. 8
2.4 Site Visits ............................................................................................................................................ 9
2.5 Literature Review ............................................................................................................................... 9
3. What is Multi-Touch Technology? ........................................................................................ 10
3.1 Definition .......................................................................................................................................... 10
3.2 History .............................................................................................................................................. 10
3.3 Variations.......................................................................................................................................... 11
3.4 Methods of Interaction ...................................................................................................................... 13
3.5 Advantages ....................................................................................................................................... 15
4. Findings .................................................................................................................................... 16
4.1 Survey Results .................................................................................................................................. 16
4.2 Site Visits .......................................................................................................................................... 16
4.3 Literature Review ............................................................................................................................. 19
5. Implications and Recommendations for Design .................................................................. 21
5.1 Physical Installation – Environmental Factors ................................................................................. 21
5.2 Interaction Mechanics ....................................................................................................................... 22
5.3 Designing for Groups and Multiple Users ........................................................................................ 25
5.4 Designing for Various Age Groups .................................................................................................. 27
5.5 Spatial Organization ......................................................................................................................... 27
5.6 Programming .................................................................................................................................... 28
6. Conclusions .............................................................................................................................. 29
3
Appendixes................................................................................................................................... 30
Appendix A – References ....................................................................................................................... 30
Appendix B – Survey Questions and Responses .................................................................................... 33
Appendix C – Annotated Bibliography .................................................................................................. 39
Appendix D – List of Figures ................................................................................................................. 46
4
Executive Summary This report describes best practices in multi-touch screen interactions. The Museum of
Photographic Arts (MoPA) in San Diego is installing a multi-touch screen in their gallery to help
visitors think critically about photography, improve the experience of patrons visiting the
museum in groups, create a crowd-sourced exhibit, and increase website traffic by linking multi-
touch screen interaction to MoPA’s website. A consulting team of three graduate students in
Educational Technology at San Diego State University were tasked with researching best
practices in multi-touch screen design that could be applied to a museum setting. The consultants
researched the capabilities of multi-touch screens, conducted site visits at institutions with multi-
touch screens, surveyed several museums with multi-touch screens, and reviewed previously
conducted research and case studies. This analysis effort revealed several best practices for
designing interactions using multi-touch screens, especially in the areas of physical installation,
interaction mechanics, target audience, spatial organization, and programming.
5
1. Introduction
1.1 Context
This report was prepared by Holly Peters, Lyford Rome, and Rachel Gastrich, graduate
students in the Educational Technology department at San Diego State University in San
Diego, California. It is the final deliverable for the ED 795A/B capstone course series
that students take to complete the Educational Technology Master’s degree. The team
partnered with Amber Lucero-Criswell, Director of Education and Public Programs, at
the Museum of Photographic Arts (MoPA) in San Diego, California to complete the
project, which culminated with this report. MoPA was founded in 1983 and is devoted to
collecting, conserving, and exhibiting the entire spectrum of the photographic medium.
The museum’s endeavors consistently address cultural, historical, and social issues
through its exhibitions and public programs. Approximately 100,000 people visit
MoPA’s museum and website annually.
1.2 Description of Task
As part of an Institute of Museum and Library Services (IMLS) grant, MoPA is installing
a multi-touch screen in their gallery entrance. The multi-touch screen will be vertically
mounted in the gallery entry, and will be visible when visitors enter the museum. By
installing the multi-touch screen, MoPA hopes to:
Help visitors understand and think critically about photography
Improve the experience of patrons visiting the museum in groups
Create a crowd-sourced exhibit through visitor selection of photos
Increase website traffic by linking the multi-touch screen interaction to
MoPA’s website
To this effort, the consultants were tasked with analyzing best practices in visitor
interactions with multi-touch screens at museums. Rather than identifying potential
content areas, the consultants researched general guidelines for multi-touch screen use in
museums that are applicable to all content areas.
6
1.3 Organizational Overview
This report is organized into five sections:
Introduction – An overview of the context, description of the task, and
an organizational overview
Analysis Process – An overview of the analysis process used by the
consultants
What is Multi-Touch Technology? – Definition, advantages, variations,
software, and methods of interactions specific to multi-touch technology
Findings – Results of research by the consultants
Implications and Recommendations for Design – Recommendations
based on best practices for MoPA to implement in their multi-touch
screen activities
7
2. Analysis Process
2.1 Contextual Factors, Issues, and Concerns
The recommendations in this report are based on the best research available to the
consultants during the time allotted to complete the appropriate analysis. The reader
should consider the following contextual factors, issues, and concerns that may have
influenced the research results.
New Technology
One factor that influenced our research is the sheer novelty of the topic. Not many
institutions have successfully incorporated multi-touch technology into their operations.
There are several possible factors contributing to the slow adoption of this technology
including cost, available space, unfamiliarity, lack of resources, and not recognizing a
need within their particular audience. The limited amount of multi-touch implementations
resulted in a narrow pool of reference material and led the consultant team to draw from a
wider variety of resources. In addition to museums, we also considered information from
other sources including retail stores and public spaces.
Different Orientation
Among the institutions that have implemented multi-touch technology, we have found
that the majority are in a horizontal orientation (table-style format). This format is
significantly different from the format that MoPA will use in several ways. With a
vertical format, users will physically interact with the screen in a standing position, view
the content at eye-level, and be able to see what other users are doing simultaneously. We
are confident, however, that examining the use of horizontal multi-touch screens yields
valuable information about best practices that can be applied to vertical orientations as
well.
Low Survey Response Rate
Of the thirteen institutions with multi-touch technology that we contacted and distributed
surveys to, only six returned a survey. Of these six, only three were complete. However,
the responses provided by the three respondents supported other research findings we
incorporated into our recommendations.
8
Single Observer for Site Visits
One person of the consulting team conducted on site observations at locations with multi-
touch technology. It is possible that a single observer may unconsciously distort data in
such a way as to make certain outcomes more likely, observing from their individual
perspective (gender, age, ethnicity, or other characteristics). The effect of this threat to
validity was reduced because the research team did not have a pre-determined hypothesis
that they were trying to prove. Also, the same team member observed each of the three
sites so that the findings would be as consistent as possible.
Financial and Temporal Limitations
While we would have liked to travel to other states and countries where multi-touch
technology is successfully used, we did not have the time or budget to do so. The team
also acknowledges that it would have been more desirable to visit each of the museums
we observed on more than one occasion, but our schedule could not accommodate this
luxury. In spite of any limitations, we believe that the time and effort invested into this
project was sufficient to produce a strong basis for our recommendations.
2.2 Background Research
Our analysis process began with background research to familiarize ourselves with multi-
touch technology, including its technical aspects and uses. We researched vendor sites,
including MultiTouch (www.multitouch.fi) and Ideum (www.ideum.com), to understand
the basic concepts behind multi-touch technology. Case studies linked on MultiTouch
and Ideum’s websites provided insight into the use of multi-touch screens in both
institutional and commercial settings. Although none of the case studies featured
museums, it was helpful to have an overview of topic.
2.3 Survey
At the beginning of the project, MoPA provided the consultants with a list of museums
using multi-touch technology. The team contacted each museum on the list to identify the
correct contact for further communication. Some museums were eliminated from that list
as they did not have a multi-touch screen, only had regular touch screens, or were using
their screens for research rather than with the general public. Thirteen organizations
confirmed that they had multi-touch screens and offered specific contact information. The
consultants designed a brief survey hosted on SurveyMonkey to poll the institutions
about their experience with their multi-touch screen(s). Six of the thirteen museums that
9
received the survey responded, including museums located in Sweden; Nashville, TN;
Anchorage, AL; Greenbelt, MD; and Vancouver, BC. Only three of the six, however,
returned a completed survey, with all the questions answered. Although the lack of
response was disappointing, some of the insights and experiences shared on the survey
proved to be helpful in our research. See Appendix II for a complete list of survey
questions and responses.
2.4 Site Visits
A member of the consulting team visited the following three locations that currently use
multi-touch technology:
The Microsoft Store (San Diego, CA) – This is a retail store owned and
operated by Microsoft, dealing in computers, computer software, and
consumer electronics.
The J. Paul Getty Museum (Los Angeles, CA) – The Getty Museum
houses European paintings, drawings, sculpture, manuscripts, decorative
arts, and European and American photographs. Their collection attracts a
wide variety of audiences spanning all age groups, education levels, and
nationalities.
The Annenberg Space for Photography (Los Angeles, CA) – The
home office need to provide support and direction during the rollout.
2.5 Literature Review
Finally, we conducted a literature review of best practices in multi-touch technology. We
reviewed articles, theses, and other research projects about multi-touch screens found
using Google, Google Scholar, and the Educational Resources Information Center
(ERIC). Though not all of the literature we reviewed focused on multi-touch screens in a
museum context, many of the findings were easily applicable to a broad range of multi-
touch screen uses. Our literature review provided us with a significant amount of useful
information.
10
3. What is Multi-Touch Technology?
3.1 Definition
The popularity of devices such as Apple’s iPad and iPhone, as well as the many new
tablets and touch screen computers, reflect the growing presence of touch technology in
day to day life. The phrase “multi-touch technology”, however, refers to any one of
several different interaction techniques and technologies that allow users to control a
computer using several fingers, or multiple inputs. These distinct approaches all utilize
different hardware and software, each with distinct advantages and disadvantages.
For the purposes of this report, we will define multi-touch as a system’s ability to register
more than three touch points directly on a display. Each input is detected and resolved
individually, even with multiple users. This greatly increases the speed, efficiency, and
intuitiveness of the technology. Our report will focus on best practices at user interaction
level, independent of a particular platform or vendor.
Figure 3.1: A vertical multi-user, multi-touch screen being used as an interactive directory at the Pori Jazz Festival. Image courtesy of Multitouch, Ltd.
3.2 History
Touch screen technology was first developed in the late 1960’s. Previous systems
allowed interaction with the computer screen using a light pen, but by 1972 technology
11
had advanced enough to enable a human finger to register input directly. This system was
developed by the Computer-based Education Research Laboratory at the University of
Illinois, Urbana-Champaign, and was called the PLATO IV Touch Screen Terminal
(Buxton, 2007). PLATO IV and other technology of this period recognized one single
touch at a time in separate zones on the screen.
By the mid 1980’s, systems had been developed to recognize multiple touches
simultaneously. Bob Boie at Bell Labs created the first multi-touch screen in 1984 that
used a transparent array of touch sensors overlaid on a cathode ray tube (CRT) screen
(Buxton, 2007).
In 1992, SUN Microsystems produced the Starfire demo, which imagined a day in the life
of a knowledge worker in the “far-off distant year of 2004.” This demonstration
introduced several multi-touch gestures and techniques (Tognazzini, 1994).
These ideas did not gain widespread adoption and remained in research labs until 2006
when NYU’s Jeffery Han presented a now-famous demonstration at the Technology,
Entertainment and Design (TED) Talks (Han, 2006). His system was relatively simple to
create as well as inexpensive, and it opened the door for much of the innovation we see
today.
3.3 Variations
With the advent of lower cost and more capable systems in the early 21st century, multi-
touch screens are now installed in many different settings and for as many different
purposes.
Modern multi-touch screens are used effectively in diverse environments such as the
military for command and control (Szymanski, 2008), science for data visualization and
collaboration (Wigdor, 2009), educational outreach for museums (Johnson, 2010), and
for business and data management applications (Morris, 2008).
Multi-touch systems are available in many form factors and sizes. Most of the literature
surveyed in our review described standalone table top systems, such as the Microsoft
Surface. A horizontal orientation permits more ergonomic virtual keyboard input and also
allows for the interaction with physical objects placed upon the table surface (Experience,
2011). A variation of the horizontal system is an angled, “drafting table” style surface.
This was the type first demonstrated by Jeffery Han in his 2006 TED Talk and was
described in an office setting by Morris (2009).
12
Figure 3.2: A multi-touch screen being used in a table format and with a stand for an angled horizontal orientation. Images courtesy of Multitouch, Ltd.
Multi-touch screens in a vertical orientation can offer their own unique advantages.
Large, wall-mounted screens are visible from far away, while horizontal screens require
the user to stand close by. Vertical screens are easier to read when displaying information
(Wimmer, 2009) and can be built to large sizes. Using modular units, a 4-meter-long
(13’) by 1.2-meter-high (4’) fully multi-touch video wall was constructed for the Chilean
Pavilion Wall of Chile display at Expo Shanghai 2010 (Multitouch.fi, 2010), for example.
Figure 3.3: A modular wall mounted cell-based design allows scalability. Image courtesy of Multitouch, Ltd.
MoPA has chosen to install a vertically oriented multi-touch screen in their front gallery.
The hardware technologies used to recognize multi-touch input vary by vendor and form
factor (Schöning, 2008). It is beyond the scope of this report to cover all the technical
details of the many different methods currently on the market. Rather, we will focus on
13
the software layer and the forms of interaction enabled by most modern multi-touch
displays.
3.4 Methods of Interaction
Despite multi-touch being a relatively newer method of interacting with a computer
display, there is already an emerging vocabulary of gestures that are standard on many
devices. A true multi-touch system does not just replace the mouse and keyboard with on
screen equivalents; on many such systems there is no cursor to move with a mouse at all.
Instead, a gestural “sign language” is used to interact with the display. Some of these
gestures are very intuitive as they mimic how one would naturally move, while some are
completely new to the experience of multi-touch.
Single Finger Tap or Touch
The simplest interaction with the screen is a single finger tap at a single point. This
gesture would be appropriate to activate button elements on screen, or any other status
controls such as “on/off” or “mute/unmute.” This is one of the most intuitive of all
gestures (Jacucci, 2010).
Single Finger Hold and Drag
The user holds down a single finger touch and then drags the finger across the screen.
This action can be interpreted different ways depending upon the screen element being
controlled (Malik, 2007). The movement can either be linear or rotational. This gesture is
appropriate for slider type controls such as volume or brightness, scrolling through lists
or mimicking rotating knobs. It is also used for moving onscreen objects, such as photos.
Single Finger Flick, Fling, or Swipe
This gesture resembles the single finger hold and drag but with a much faster motion.
This gesture can be interpreted by the system to mean the same as hold and drag but with
greater velocity, such as scrolling all the way to end of a list very quickly or moving a
photo off the screen.
Depending upon the physics of the software, this gesture could also be used to overcome
the “inertia” of some interface elements. Slower movements mean that the user is
intending to interact with what is displayed on screen, a fast swipe could tell the software
to switch to another view or part of the program.
14
Multiple Finger Touch, Drag, or Swipe
Variations of these gestures include using multiple fingers instead of a single touch.
Though, technically, these would be multi-touch gestures, all the fingers are performing
the same motion relative to each other. With a two finger swipe, both fingers would be
“swiping” in the same direction.
Figure 3.4: Examples of simple gestures: Single Touch, Multiple Touch & Drag, and Single Touch & Flick.
Multi-Touch Rotate/Spin
Rotating a two finger hold and drag around a common center will rotate or spin the
selected screen element. This could be a one or two handed gesture depending upon the
size of the element.
Multi-Touch Pinch/Shrink/Zoom Out
Holding two fingers and dragging them towards one another is interpreted by many
systems as a zoom out, shrinking whatever on screen element has been selected. This
may be performed with one or two hands, or with one stationary finger; what is important
is the motion relative to each point. If the motion is rotational as well, then the element
would be spun as well as shrunk (Malik, 2007).
Multi-Touch Unpinch/Stretch/Zoom In
Two fingers holding and dragging away from each other is the opposite of a pinch. This
gesture will zoom in or enlarge whatever element is selected. Again, the relative motion
between the fingers is what defines this gesture.
15
Figure 3.5: Examples of multi-touch gestures: Pinch/Zoom Out, Rotate/Spin, and Unpinch/Zoom In/Rotate combination.
3.5 Advantages
Multi-touch technology offers several advantages to museums over traditional kiosk style
installations (Multitouch.fi, 2011), such as:
The popularity of devices such as the iPad and touch screen phones has
raised the expectations of museum visitors who are drawn to the new
technology.
Modular multi-touch screens can be configured to make wall displays of
very large size, which become an integral part of exhibit experiences.
The size of wall mounted screens changes the relationship of the user to
the content. The visitor’s field of view is surrounded by the presentation.
Multi-touch displays by their nature allow for multiple users to interact at
the same time.
Multi-touch screens allow the visitor to interact more naturally with
onscreen content by allowing users to experience the presentation more
directly.
Visitors have a more memorable experience because they use more
senses than just sight.
16
4. Findings
4.1 Survey Results
Despite the low number of survey respondents, the information from our survey provided
several useful multi-touch screen practices. To summarize the findings:
Five out of six respondents used Ideum multi-touch screens.
Several of the respondents had multiple multi-touch screens; most were
oriented in the table-top format.
Most multi-touch screens were configured separately, rather than as one
large screen.
Visitors enjoyed interacting with screens that caught their eye, and liked
the relationship between exhibits and the screen.
Visitors preferred simple rather than complex interactions.
Interactions should be tested thoroughly before they are implemented.
Children tend to dive right into screen interactions, young adults
approach screen interactions with purpose, and older adults are more
cautious when interacting with multi-touch screens.
A complete list of survey questions and responses can be found in Appendix II.
4.2 Site Visits
The Microsoft Store
Our team first conducted a site visit at the Microsoft Store, where there are two horizontal
multi-touch screens built into sturdy table units. The units use Microsoft Surface
technology, a software and hardware combination product developed by Microsoft that
responds to natural hand gestures and real world objects placed on the screen by multiple
users. The platform features a 360-degree user interface on a 30 inch (76 cm.) reflective
surface. A projector underneath the surface projects an image onto its underside, while
five cameras in the machine's housing record reflections of infrared light from objects
and human fingertips on the surface.
17
A store employee demonstrated the basic functions of the multi-touch unit. He showed
how the technology allows for multiple points of contact to be recognized simultaneously
by asking two other people to place their fingertips on the screen with him all at the same
time. Each individual point of contact lit up on the screen. In addition, the screen was
able to detect and respond to physical objects such as a cup or identification card held
against the screen. The unit held games as well as applications for drawing and viewing
images. One of the two units in the store was not used as much as the other because,
according to the store employee, it was located by a window and the glare from the sun
caused problems.
The Getty Museum
Upon arriving at the Getty Museum, our team member realized that the Getty only had
single touch screens, not multi-touch screens. This means that each screen would
accommodate only one user and one input at a time instead of multiple users and inputs.
We did, however, learn valuable information as a result of the visit. Some of the screens
were in a horizontal table-top format, with one or two headphones attached, and located
in several locations throughout the museum, such as hallways and a computer room.
Additionally, units in the computer room were isolated from the rest of the museum
exhibits, and some visitors appeared to be reluctant to even enter the dimly lit room. In
fact, our team member observed that upon entering the computer room, several visitors
seemed as if they were attracted more to the comfortable seating as a place to rest rather
than the computers, as evidenced by their audible sighs and lack of computer usage.
A few computers were set up with a choice of three children’s games, and quite a few
families (adults with children) used these computers. Without exception, the children
began to interact with the touch screens immediately, while some of the adults just
watched their children or looked at the screen for several minutes before touching it. An
interview with a docent confirmed that, in his experience, children run right up to the
screens and adults are more hesitant.
Other computers throughout the room, either at counter-top height or at tables, were set
up to view pieces from the museum’s collection. The interactions available on most of
the touch screen computers consisted of viewing works of art in the museum collection,
detailed information about the piece, or a map of where the item is located in the
museum. The initial screen for these computers contained the text “Touch screen to
begin,” to prompt users. There were also a few computers that showed a library of videos
18
the user could choose from. The only observable gesture the screens recognized was
tapping, just as one would use a mouse to click in various areas on the screen.
The Annenberg Space for Photography
The last site visit conducted by the team was at the Annenberg Space for Photography.
The space housed two identical multi-touch screen units located in a corner of the
museum halfway between the front entrance and the beginning of the main exhibition.
The two units were arranged side by side in front of comfortable sofa-like seats with
shelves between them for users to place their belongings on. One of the units was not
working at the time of the site visit. The units were built into low horizontal tables in
front of each seating unit, which required the user to sit at the edge of the seat and lean
forward at a 45% angle in order to have a clear view of the screen, looking straight down
on it.
Once activated by touch, the screen displayed the following message along with an image
of a hand touching objects on the screen: “Have fun with photos and videos as you move,
resize, and rotate them. Enjoying photos and videos is a hands-on activity!” Our team
member observed that initially, users were unsure how to manipulate the onscreen images
until a museum employee came by to show them. Our team later learned that museum
employees had very limited, mostly on-the-job training on the screens due to the intuitive
design of the interactions. Employees did need, however, to know that they should not
tell visitors that the screens are “the same as” an iPhone because the technology is
different. Instead, they are instructed that they can say that the screens are “a little like”
an iPhone.
Users could drag images across the screen, and enlarge or reduce them by dragging the
corners. This could be done by multiple users simultaneously. One aspect of the design,
however, proved to be challenging for multiple users. Unless images were sized and
spaced far enough away from each other, only one image was visible at a time. The
interaction was designed so an image only appeared on the top of the “pile” of other
images while the user maintained physical contact. Once the user removes their hand
from an image, that image recedes behind other images, specifically behind images that
another user may be touching. As a result, we observed that when a group used the
screen, just one person from the group would touch the screen while the others looked on.
A museum employee also explained some of the features of the units that were removed
and why. For example, visitors were originally able to download and view photos from
19
their personal cameras, and also send photos from the current exhibit to a printer at the
front desk of the museum so they could take them home. These features were removed
due to usage time concerns and copyright issues.
4.3 Literature Review
Our literature review resulted in a broad spectrum of uses for multi-touch technology.
These various examples can be categorized on a continuum made up of the following
three categories: activation, manipulation, and creation. (See Figure 4.1)
Figure 4.1: Continuum of three types of multi-touch technology uses: activation, manipulation, and creation.
Activation
The activation level of interaction occurs when the user touches an area of the screen in
order to initiate a simple function. The outcome is the same for every user. Activation
includes accessing information from the organization or from outside the organization,
i.e. the Internet.
Activating information from the organization includes tapping a screen to see a list of
choices, filtering a selection, viewing more information, showing a map, or playing a
video. An example of this can be found in a study done at the Vancouver Aquarium
(Hinrichs and Carpendale, 2010), where visitors activated two types of multi-touch
learning experiences. The first experience was an interaction that launched photos,
videos, and graphic animations related to the museum’s Arctic exhibit. The second was a
timeline activity where users could overlay and compare different data sets on a timeline
by activating different layers. We also found examples of users accessing exclusive
content by bringing objects such as membership cards or bar codes into contact with the
multi-touch screen.
Information from outside the organization included items such as images from Flickr, a
video from YouTube, a Twitter feed, or a Facebook discussion. Flickr integration was
20
also used in the CityWall project (Peltonen, 2008). Photos tagged with "Helsinki" were
displayed in the top portion of the screen and mapped to a scrollable timeline with
thumbnails. Jacucci (2010) organized similar content into rotating 3D widget "worlds"
that unfolded into 2D planes when users interacted with them.
Manipulation
Manipulation occurs when the user’s action results in a corresponding change of the
information on the screen. The change is in direct relation to the degree of the user’s
interaction. The outcome is one of several possible outcomes available. Examples of the
manipulation level of interaction are the user generating special effects (like blowing
virtual bubbles, blowing virtual leaves in the air, or making ripples in a virtual pond of
water), resizing and rotating images, or playing games.
Again, the Vancouver Aquarium study shows that both types of their multi-touch
learning experiences feature fairly low levels of manipulation; users played videos,
resized and rotated images, and activated layers on a timeline, as opposed to higher levels
of manipulation found in games and similar interactions. The CityWall project also
included similar levels of manipulation, with users scrolling the timeline, selecting and
enlarging photographs. Jacucci's 3D worlds provided a greater degree of interaction,
with different widgets opening up to different activities. Kristensson's InfoTouch system
(2008) allowed direct interaction and manipulation of photo tag cloud. Users could drag
specific tags and keywords from the cloud to an onscreen "query area." The results of that
search would be displayed from the photo collection. The interactive screen at the Pori
Jazz Festival (MultiTouch Case Studies, 2011) allowed patrons to access schedule
information and track their favorites musicians by dragging spheres representing venues,
genres and artists.
Creation
Creation is the highest level of interaction and is when the user develops new content.
The result is something unique that has never existed before. Instances of creation include
editing images, adding comments to a discussion, or drawing “graffiti” on a virtual wall.
21
5. Implications and Recommendations for Design Based on our research, we have compiled the following best practices for multi-touch
screen interaction design. These practices are not only instructionally sound, but can also
be implemented across a wide range of content areas for multi-touch screen interactions.
5.1 Physical Installation – Environmental Factors
The physical location of multi-touch screens can greatly affect users’ experiences.
Careful attention should be paid to the physical location of screens to avoid any
detrimental effects to visitors’ enjoyment of the interaction resulting from environmental
factors.
Avoid Sunlight Glare
Feedback and observations from the California Science Center and the Microsoft Store
indicated that some of their screens are not used due the glare from sunlight. Multi-touch
screens should be installed in areas that are protected from sunlight glare or conversely,
from shadows that may make the screen difficult to view. Before a multi-touch screen is
installed, its intended location should be reviewed thoroughly to ensure the screen will
not be in an area with a lot of sunlight or other adverse environmental factors.
Install Screen in Well-Traveled Area
A site visit to the Getty Museum indicated that multi-touch screens should also be
installed in areas well-travelled by visitors. Several of the screens at the Getty Museum
are located in a side room off the main gallery, and many visitors would peek into the
room, but not actually enter the room to use the multi-touch screens. Hinrichs and
Carpendale (2010) reported a “honey pot effect” when they observed visitor interaction
with multi-touch screens at the Vancouver Aquarium; many visitors were drawn to the
multi-touch screens through other visitors’ interest in the screen. Thus, multi-touch
screens should be installed in an area where the honey-pot effect can be maximized.
Provide Storage Space
Hinrichs and Carpendale (2010) also observed that many users placed bags or backpacks
on the table-top multi-touch screen surface because there was no other place for them to
store their belongings when they used the screen. This caused an obstruction between the
user and the interaction, and caused several users to leave the interaction before it was
complete. Whether multi-touch screens are table-top or wall-mounted, users should be
22
provided with space to store belongings, such as a small shelf below the screen, in order
to free their hands for the on-screen interaction.
5.2 Interaction Mechanics
The haptic interaction users have with multi-touch screens, regardless of the content of
the interaction, can significantly affect the user’s experience. Poorly designed interaction
mechanics can frustrate users and render the intent of the interaction less effective. There
are several best practices in interaction mechanics to keep in mind when designing multi-
touch screen content.
Figure 5.1: Even with multi-touch screens, user interaction usually begins with a single touch (Jacucci 2010). Image courtesy of Multitouch, Ltd.
Mimic Natural Movements
Gestures and controls should mimic natural hand movements as much as possible
(Moscovich, 2007). For example, if users are required to grasp an object, the
programmed hand motion should mimic a natural grasping motion, with the thumb as one
input and the rest of the fingers acting as another input. Similarly, since many people are
already familiar with smartphone technology; gestures and controls such as expanding,
collapsing, or zooming in on objects should use similar motions on multi-touch screens as
are used with smartphones (Hinrichs and Carpendale, 2010). For example, on many
23
smartphones, expanding, collapsing, or zooming in on objects is controlled with a user’s
thumb and pointer finger. Any interactions on a multi-touch screen using this control
should mirror this action as much as possible to leverage users’ existing knowledge of the
motion.
Use Gradual Engagement
Brignull (2008) recommends providing an initial set of display-based interactions that are
intuitive and can be easily and comfortably followed. Jacucci (2010) found that users
were most likely to attempt to initially manipulate onscreen objects with one finger.
Initial interactions should require a single touch only, with more complex multi-touch
gestures allowed as the interaction unfolds. (See Figure 5.1)
Provide Large Controls
All onscreen controls should be large enough for users to easily manipulate. For example,
Hinrichs and Carpendale (2010) found that many guests at the Vancouver Aquarium
complained of a “fat finger” effect with controls that were too small or too close resulting
in unwanted actions. One guest said that when controls were too close they “would do
some unpredictable things” and launch unintended screens or interactions. This lack of
response or unintended action quickly caused frustration among users. Therefore, careful
consideration should be made when designing controls to ensure they are large enough
for users to easily interact with and control.
Avoid Easy Deletion
If users have the ability to delete or hide objects in an interaction, it should not be too
easy for users to delete or hide those objects. The accidental deletion of objects can
frustrate users (Hinrich and Carpendale, 2010) and cause them to leave the interaction.
The specific mechanics for deleting objects should be designed carefully; for example,
frequently used controls should not be placed near the delete control. If a gesture or
action is used to delete objects, those actions should not mirror other commonly used
gestures or actions.
Avoid Sustained Contact
Gestures should not require sustained contact to be effective. A vertically-oriented touch
screen can cause discomfort if users are required to hold their hands out at arms' length
for an extended period of time (Szymanski et al., 2008). This condition, coined "gorilla
arm," was described in the 1980's after light pen users reported fatigue while working
24
with monitors (Avons, 1983). Any interactions should require short periods of contact if
possible and activities should be designed to allow users to rest their arms periodically.
Limit Activities Requiring Text Entry
Similarly, Wimmer (2009) found text entry to be best handled with a keyboard in a
horizontal position. Requiring users to do text entry using a virtual QWERTY style
keyboard in a vertical orientation quickly leads to fatigue.
Ensure Onscreen Changes Result from User Interaction
All changes on the screen should be a direct result of user interaction (Moscovich, 2007).
Interactions should not change in ways that are not intuitive from direct user interaction.
An animation that automatically starts or an object that moves without input from the
user can confuse users, especially if they are focused on another object or interaction.
Provide Feedback for User Input
Animations should be used to acknowledge user input when a control is selected or
activated (Szymanski et al., 2008). For example, the Annenberg Space for Photography
featured a water ripple effect on one of their multi-touch screens that began when users
touched it, rather than on its own. This feedback confirmed when users had selected an
area of the screen, assuring that the user’s touch had been received by the screen.
Use Animation to Indicate Interface Controls
Since controls are not physical objects, animations are helpful to draw attention to screen
elements which are controls. Pulsing, color cycling and fading are animations that do not
change a control element’s location on the screen, but instead invite interaction from the
user (Szymanski et al., 2008).
Demonstrate Possibilities of the Interface
Vanacken (2008) proposed an avatar-based help system to assist users with onscreen
interactions. These “TouchGhosts” would be displayed when the system senses repeated
failed attempts at a gesture. Alternately, the multi-touch system would use the
TouchGhosts when the system has been inactive for some time, showing the possibilities
of the interface to bystanders. This guidance helps distinguish multi-touch screens from
other static video displays in the museum, and invites bystanders to interact. Another way
to demonstrate the possibilities of the interface is to have a museum representative
stationed near the interaction and show visitors how to use the screen, as observed during
the site visit to the Annenberg Space for Photography.
25
Move Objects Along the Shortest Path Possible
Users should also be able to move objects using the shortest path possible (Moscovich,
2007). Users will naturally move objects using the short path from the starting point to
ending point. Interaction designs should reflect this concept and allow users to move
objects efficiently, especially if the interaction requires users to move objects to a fixed
position or static target. The only exception to this practice is pursuit-type tasks often
found in games where users move objects in a more varied path will help you determine
whether the training was successful in the long term.
5.3 Designing for Groups and Multiple Users
The nature of multi-touch screens makes them ideal for group interactions. However,
special considerations must be made when designing interactions for multiple users.
Provide Workspace for Multiple Users
It is important to consider the space each user will control on the multi-touch screen if
designing onscreen workspaces for multiple users. A common problem with multi-
screens with simultaneous interactions for multiple users is interference with another
user’s workspace (Hinrich and Carpendale, 2010). With undefined workspaces, users can
“steal” other user’s objects unintentionally if they accidentally touch or move that object.
Users can also block another user’s work if they move objects onto another workspace.
Good design can mitigate this problem by defining clear workspaces for users,
prohibiting objects from being placed on top of each other, or preventing objects from
reacting to other touches when they are in use.
Use Audio Feedback Carefully
Audio feedback can help make users aware of the presence of other users, and indicate
“collisions” when actions run into a workspace border. Positive audio feedback (reward
sound) that is shared and synchronized between users can also help foster social
interaction across workspaces. However, if negative audio feedback (error sound) meant
for and individual is heard by all users it can cause confusion as to which actions caused
the error (Hancock et al, 2005).
Use Localized or Coded Audio
If sound must be used in an interaction, it should be limited and directed at the individual
user, rather than the entire screen. If the screen is divided into multiple workspaces, then
audio should be channeled separately to those zones with separate speakers. Coded audio
26
(where the waveforms are varied so that different sounds are played for each user) can be
useful if activities are individualized but a single audio speaker must be shared with the
display (Hancock et al, 2005).
Provide Challenges to Increase Cooperation
Several articles reported that although strangers completed on-screen interactions
separately, they interacted with each other when faced with a conflict while using the
multi-touch screen (Peltonen et al, 2008; Hinrichs and Carpendale, 2010). Visitors often
helped each other troubleshoot interactions or shared media items, which promoted brief,
conversational experiences. While interactions should not be designed with any controls
meant to intentionally confuse users, if the goal of an interaction is to foster cooperation
between users, then elements of that interaction should feature challenges that users must
work together to overcome. For example, Peltonen et. al. (2008) suggests designing
performative interactions where each user must take a different role to increase
interactions between visitors. However, Hinrichs and Carpendale (2010) reported that
some users preferred not to have a social experience while using the multi-touch screen.
Therefore, if possible, visitors should have the option of participating in an individual
interaction or a group interaction on the multi-touch screen. By designing individual and
group interactions, solitary visitors can complete interactions by themselves, and those
users visiting in groups or who wish to engage other patrons can participate in more
social interactions.
Plan for Varying Interaction Times
Several studies indicate that visitors use multi-touch assets for varying amounts of time
depending upon the complexity of the content and the age of the user. For a simple photo
collection application, the average interaction time was less than three minutes, with
children staying slightly longer than adults (Hinrichs, 2010 and Jacucci, 2010). For
material of more complexity and depth, adults spent longer with the display than
children. Hinrichs (2010) noted some adult visitor groups interacted with the Vancouver
Aquarium Arctic Choices table for as long as 20 minutes. Depending upon location, this
could affect foot traffic patterns if crowds gather and stay at the screen for long periods of
time. Alternatively, experiences could be designed to time out instead of being open
ended and unlimited.
27
5.4 Designing for Various Age Groups
As with all types of instruction, the age and maturity of the audience is a key
consideration in design. Interactions designed for children will differ from interactions
designed for adults.
Level of Prompting Differs for Age Groups
Based on interviews at sites and the survey results, interactions should feature guidance
appropriate to the age of the intended audience. Children tend to begin interactions with
little guidance, young adults approach interactions with purpose, and adults are more
cautious. Therefore, interactions designed for children do not need to feature lengthy
instructions or introduction screens as children tend to begin interactions with little
external prompting and tend to disregard onscreen text. Interactions designed for young
adults should feature at minimum a brief description of the interaction; although many
young adults are familiar with various types of technology, both site visits and survey
responses stated that young adults approach multi-touch screens with the intent of
figuring out the interaction. Conversely, designs for adults should provide more guidance
or step by step instructions, such as prompts, arrows, examples, or brief introduction text.
Data from site visits and survey responses indicates that adults are more cautious than
children or young adults when interacting with multi-touch screens, perhaps due to a lack
of familiarity with that type of technology. Many adults began interactions only after
watching other visitors interact with the screen for an extended period of time. Regardless
of the reason for their uncertainty, multi-touch screens for adults should feature the most
built-in guidance and prompting to encourage adults to use the screen.
5.5 Spatial Organization
The spatial organization of a multi-touch screen is also a key consideration in design.
Provide Central Work Area
Many examples of multi-touch screens feature objects that are manipulated by users.
Since users can only manipulate one or two objects at a time, this means that some
objects will be “active” and other objects will be “inactive.” Users typically organize
digital objects by storing inactive items around the outside of the screen, and using the
middle of the screen as a working area to manipulate active items (Steimle et.al. 2010).
Therefore, interactions where users manipulate multiple digital objects should feature a
working area in the center of the screen for users to manipulate active objects, and a
28
storage area around the outside of the screen for users to store inactive objects. For
example, if users are manipulating set of photographs in an interaction, users should be
able to manipulate chosen photographs in the center of the screen while storing inactive
photos around the outside of the screen, and easily move photographs between the
working and storage areas.
5.6 Programming
As with all types of technology, reliable programming is essential to a positive user
experience.
Test Interaction Thoroughly
Feedback from the survey indicated that interactions should be planned and tested
thoroughly before they are implemented. One respondent recommended building a paper
prototype, to scale, before the multi-touch screen is installed to help visualize the space
the screen will occupy. Another respondent recommended rigorously testing interactions
before they are shared with the public, and ensuring all firmware and software is up to
date. Outdated software or hardware upgrades could potentially cause interactions to slow
or malfunction, and frustrate users to the point where they leave the multi-touch screen
interaction.
29
6. Conclusions Multi-touch technology is a rapidly developing field, with new products and possibilities
constantly emerging. MoPA’s decision to embrace such a current, innovative tool places
them as a leader in visitor engagement at museums. While being ahead of the curve has
its advantages, implementing such a new technology does present challenges, especially
for those tasked with designing meaningful learning experiences. In order to create a
successful visitor experience, MoPA should consider.
The physical installation and environmental factors that may affect
multi-touch screens
The mechanics of an interaction
The intended audience, including patrons visiting in groups and their age
range
The spatial organization of onscreen interactions
The type of programming
While this report does not address specific content of the experience, these fundamental
design guidelines can be used in a wide variety of content areas. Sound instructional
design is critical in creating a lasting, memorable museum experience.
30
Appendixes
Appendix A – References
Avons, S., Beveridge, M., Hickman, A., & Hitch, G. (1983). Considerations on using a
lightpen-interactive system with young children. Behavior Research Methods &
Instrumentation, 15(1), 75-78.
Benko, H., Morris, M., Brush, A., & Wilson, A. (2009). Insights on interactive tabletops:
a survey of researchers and developers (Microsoft Research Technical Report
MSR-TR-2009-22). Redmond, Washington: Microsoft Research
Brignull, H., Izadi, S., Fitzpatrick, G., Rogers, Y., & Rodden, T. (2004). The introduction
of a shared interactive surface into a communal space. Proceedings of the 2004
acm conference on computer supported cooperative work (pp. 49-58). New York,
New York: ACM Press.
Buxton, B. (2007, January 12). Multi-touch systems that I have known and loved.
Retrieved from http://www.billbuxton.com/multitouchOverview.html Experience
things in a whole new way. (n.d.). Retrieved March 6, 2011 from
http://www.microsoft.com/surface/whatissurface.aspx
Han, J. (Writer). (2006). Jeff Han demos his breakthrough touchscreen [Web]. Available
from
http://www.ted.com/talks/jeff_han_demos_his_breakthrough_touchscreen.html
Hancock, M., Shen, C., Forlines, C., & Ryall, K. (2005). Exploring non-speech auditory
feedback at an interactive multi-user tabletop. In Proceedings of Graphics
Interface 2005 (GI '05). Canadian Human-Computer Communications Society,
School of Computer Science, University of Waterloo, Waterloo, Ontario,
Canada, 41-50.
Hinrichs, U., & Carpendale, S. (2010). Interactive tables in the wild: visitor experiences
with multi-touch tables in the arctic exhibit at the Vancouver aquarium.
Informally published manuscript, Computer Science, University of Calgary,
Calgary, Canada. Retrieved from http://hdl.handle.net/1880/48199
Jacucci, G., Morrison, A., Richard, G., Kleimola, J., Peltonen, P., Parisi, L., & Laitinen,
T. (2010). Worlds of information: designing for engagement at a public multi-
31
touch display. Proceedings of the 28th international conference on Human factors
in computing systems, 10, 2267-2276.
Johnson, L., Witchey, H., Smith, R., Levine, A., and Haywood, K., (2010). The 2010
Horizon Report: Museum Edition. Austin, Texas: The New Media Consortium.
Kristensson, P., Arnell, O., Björk, A., Dahlbäck, N., Pennerup, D., Prytz, E., Åström, N.
(2008). Infotouch: an explorative multi-touch visualization interface for tagged
photo collections. Proceedings of the 5th Nordic conference on Human-computer
interaction: building bridges, 2008, 491-494.
Malik, S., Brush, A., & Meyers, B. (2007). An exploration of multi-finger interaction on
multi-touch surfaces. Toronto, Ont., Canada: University of Toronto.
Morris, M. (2008). A field study of knowledge workers’ use of interactive horizontal
displays. Proceedings of the Ieee tabletops and interactive surfaces 2008 (pp.
105-112). Amsterdam: IEEE.
Moscovich, T. (2007). Principles and applications of multi-touch interaction. Providence,
Rhode Island: Brown University.
Multi-touch case studies. (n.d.). Retrieved March 6, 2011 from http://multitouch.fi/case-
studies/
Peltonen, P, Kurvinen, E, Salovaara, A, Jacucci, G, et al. (2008). It's mine, don't touch!:
interactions at a large multi-touch display in a city centre. Proceeding of the
twenty-sixth annual SIGCHI conference on Human factors in computing
systems, 08, 1285-1294.
Schöning, J., Brandl, P., Daiber, F., Echtler, F., Hilliges, O., Hook, J., Löchtefeld, M.,.
(2008). Multi-Touch Surfaces : A Technical Guide Technical Report TUM-
I0833, Technical Reports of the Technical University of Munich, October 2008,
1-19.
Steimle, J, Khalilbeigi, M, Mühlhäuser, M, & Hollan, J. (2010). Physical and digital
media usage patterns on interactive tabletop surfaces. Proceedings of the ACM
International Conference on Interactive Tabletops and Surfaces 2010.
Szymanski, R, Goldin, M, Palmer, N, Beckinger, R, Gilday, J. & Chase, T. (2008).
Command and control in a multitouch environment. Proceedings of the Army
science conference Orlando, Florida: http://handle.dtic.mil/100.2/ADA503423.
32
Tognazzini, B. (1994). The "starfire" video prototype project: a case history. Retrieved
March 6, 2011 from http://www.asktog.com/papers/videoPrototypePaper.html
Vanacken, D, Demeure, A, Luyten, K, & Coninx, K. (2008). Ghosts in the interface:
meta-user interface visualizations as guides for multi-touch interaction.
Horizontal Interactive Human Computer Systems, 2008, 81 - 84.
Wigdor, D., Jian, H., Forlines, C., Borkin, M., & Shen, C. (2009). The wespace: the
design, development, and deployment of a walk-up and share multi-surface
visual collaboration system. Proceedings of the ACM Conference on Human
Factors in Computing Systems (CHI) Boston, Massachusetts, 1237-1246
Wimmer, R., Schulz, F., Hennecke, F., Boring, S., & Hussman, H. (2009). Curve:
blending horizontal and vertical interactive surfaces. Proceedings of the 4th ieee
workshop on tabletops and interactive surfaces (pp. 49-58). Banff, Canada: IEEE.
.
33
Appendix B – Survey Questions and Responses
What type of multi-touch screen does your organization have? Answer Options Response Percent Response Count Ideum 83.3% 5 MultiTouch US 16.7% 1 HP 0.0% 0 Microsoft 0.0% 0 Perceptive Pixel 0.0% 0 Evoluce 0.0% 0 KeyTec 0.0% 0 24-7mediatechnology 0.0% 0 Other (please specify)
AV&C New York
AV&C, Design PT
33.3% 2
answered question 6 skipped question 0
What is the positioning of your multi-touch screen(s)? Answer Options Response Percent Response Count Wall-mounted 33.3% 2 Table-top 100.0% 6 Kiosk 33.3% 2 Other (please specify) 0.0% 0 answered question 6 skipped question 0 How many multi-touch screens do you use in your organization? Answer Options Response Percent Response Count 1 33.3% 2 2 16.7% 1 3 0.0% 0 4 0.0% 0 5 0.0% 0 More than 5 50.0% 3 answered question 6 skipped question 0
34
If you have multiple screens, are they used separately or configured together as one screen? Answer Options Response Percent Response Count Used separately 75.0% 3 Configured together as one unit
0.0% 0
Both 25.0% 1 answered question 4 skipped question 2 Are multiple people able to touch the screen(s) at once? Answer Options Response Percent Response Count No 0.0% 0 Yes (please explain) 100.0% 6 There is room for about 8 children or 5 adults around each table. I believe it has up to 50 points of contact. There are 12 points of interactive tracking. Multiple people can touch and interact with the screens simultaneously. It's a multitouch 100 inch table that about 8 people can interact with at a time. Multiple users can interact with the content. answered question 6 skipped question 0 Please describe specifically how your visitors interact with the multi-touch screen(s). "One is a top down view of the Arctic with two control stations on either end that are made up of iPhone style switches, sliders and roller wheels. Visitors can enable and disable various overlays such as sea ice extent, or migration routes. The other is a picture viewer with attached metadata. Pictures spawn and visitors select the ones that interest them. They can also select video. They can scale both and there is attached metadata so they can flip the picture and read about it. Sometimes a connection will spawn and link two pictures that are different parts of the same story." Touch. Move images of objects through different wavelengths of energy. answered question 3 skipped question 3
35
What types of multi-touch interactions have you found that work best? Visitors enjoy the non-linear interactions with our media database. They like to choose whatever catches their eye. They like the fact that the touchtable content is connected in them and information to print content that is part of the exhibits. It's an additive experience from table to wall or wall to table. Pairing of screens and other exhibits. Simple and straightforward; can't utilize for layering of content. answered question 3 skipped question 3 What types of multi-touch interactions have you found that do not work well? If people need to look at controls, then look at changes on the screen, those changes need to be clearly seen as the effect of the interaction. There can be accidental stealing of focus so you have to make sure your interactions can survive that. None. See above. answered question 3 skipped question 3 What challenges regarding visitor usage have you encountered using multi-touch technology? Shake-out. It took us a long time to get the applications stable. But we put a thousand people a day past the exhibits, and they are always busy and now they are rock solid. Testing can only be truly done in a live gallery setting. We tested extensively and creatively in the lab, but people still surprised us once it went live. Not enough Money for more, larger and faster screens. Ours is a projection screen unit that has to be aligned. answered question 3 skipped question 3
36
Do you collect statistics for how many visitors use the multi-touch screen? Answer Options Response Percent Response Count Yes 33.3% 1 No 66.7% 2 answered question 3 skipped question 3 If so, how do you collect that data? Answer Options Response Percent Response Count Survey 0.0% 0 Exit interviews 0.0% 0 Program internal to the multi-touch screen
0.0% 0
Other (please specify) 100.0% 3 One time study. No. We do have an evaluation form people can fill out. answered question 3 skipped question 3 On an average day, approximately what percentage of museum visitors use the multi-touch screen(s)? Answer Options Response Percent Response Count Less than 10% 0.0% 0 10% 0.0% 0 20% 0.0% 0 30% 66.7% 2 40% 0.0% 0 50% 0.0% 0 60% 0.0% 0 70% 33.3% 1 80% 0.0% 0 90% 0.0% 0 100% 0.0% 0 answered question 3 skipped question 3
37
Have you noticed any particular usage patterns? (For example, males use screens more than females, children more than adults, etc.) Children run to the screens and dive in. Young adults approach and act with a sense of purpose. Older adults 50+ watch from a distance as others interact and then move in cautiously. Nope. Adults, teenagers, and middle schoolers. answered question 3 skipped question 3 Have you received any feedback from your staff about the multi-touch technology? Answer Options Response Percent Response Count Yes 100.0% 3 No 0.0% 0 answered question 3 skipped question 3 If so, please summarize the feedback most commonly received. They love it, though IS hated the shake out process. Like it. Easy to care for; enjoyed by visitors. answered question 3 skipped question 3 Have you received any feedback from visitors about the multi-touch technology? Answer Options Response Percent Response Count Yes 100.0% 3 No 0.0% 0 answered question 3 skipped question 3 If so, please summarize the feedback most commonly received. They love that we have something different and new. They also like the self-selecting, non-linear nature of the storytelling. Like it. They like it; find it engaging. answered question 3 skipped question 3
38
What criteria do you use to determine this technology’s success? Answer Options Response Percent Response Count Multi-touch screen usage statistics
33.3% 1
Visitor feedback 100.0% 3 Increased museum web site traffic
0.0% 0
Increased attendance at museum
33.3% 1
Increased interest in other museum exhibits
33.3% 1
Increase in membership
33.3% 1
Other (please specify) 0.0% 0 answered question 3 skipped question 3 What suggestions do you have for a museum that is looking to implement similar technology? Paper prototype, to scale, before building. Make sure you are clear on your desired outcome. Rigorously test the equipment to work exactly the way you want it to work, before you share it with the public. Always make sure that all firmware and software is up to date. answered question 2 skipped question 4 Please provide us with your contact information in case we have any follow-up questions. Name: Nick Lynch Company: Anchorage Museum at Rasmuson Center Email Address: [email protected] Phone Number: 907-929-9252 Name: Herschell Parker Company: Adventure Science Center Email Address: [email protected] Phone Number: 615-401-5070 answered question 2 skipped question 4
39
Appendix C – Annotated Bibliography
Avons, S., Beveridge, M., Hickman, A., & Hitch, G. (1983). Considerations on using a
lightpen-interactive system with young children. Behavior Research Methods &
Instrumentation, 15(1), 75-78.
Summary: Describes a study in the 1980's with children, computers and lightpens,
an early form of touch screen interaction. Notable for describing the arm fatigue
that sets in when using a screen in a vertical orientation.
Benko, H., Morris, M., Brush, A., & Wilson, A. (2009). Insights on interactive tabletops:
a survey of researchers and developers (Microsoft Research Technical Report MSR-TR-
2009-22). Redmond, Washington: Microsoft Research.
Summary: A survey of 58 tabletop researchers and developers with longer-term
tabletop use experience, to find out how they use their devices, what they use
them for, and what features they consider important for novice, single-user, and
collaborative scenarios. Identifies obstacles to mainstream adoption and input and
ergonomic challenges.
Brignull, H., Izadi, S., Fitzpatrick, G., Rogers, Y., & Rodden, T. (2004). The introduction
of a shared interactive surface into a communal space. Proceedings of the 2004 acm
conference on computer supported cooperative work (pp. 49-58). New York, New York:
ACM Press.
Summary: Describes a user study of a large multi-user interactive surface
deployed for an initial period within a real world setting. The surface was
designed to enable the sharing and exchange of a wide variety of digital media.
The setting for the study was the common room of a high school where students
come together to mix, socialize, and collaborate throughout the day. Reports on
how the students use the new technology within their own established communal
space. Findings show that the system was used extensively by the students in a
variety of ways, including sharing of photos, video clips, and websites, and for
facilitating social interaction. Discusses how the interactive shared surface was
40
appropriated by the students and introduced into their everyday lives in ways that
both mirrored and extended their existing practices within the communal space.
Buxton, B. (2007, January 12). Multi-touch systems that I have known and loved .
Retrieved from http://www.billbuxton.com/multitouchOverview.html
Summary: Overview of the history and application of multi-touch systems.
Provides a good reference regarding both technical details and design from one of
the pioneers in the field.
Experience things in a whole new way. (n.d.). Retrieved March 6, 2011 from
http://www.microsoft.com/surface/whatissurface.aspx
Summary: Microsoft's website describes the features and functionality of their
Surface product line. Includes videos, case studies and software demonstrations.
Han, J. (Writer). (2006). Jeff Han demos his breakthrough touchscreen [Web]. Available
from http://www.ted.com/talks/jeff_han_demos_his_breakthrough_touchscreen.html
Summary: Video of Jeff Han's revolutionary demonstration at the 2006
Technology Entertainment and Design conference.
Hancock, M., Shen, C., Forlines, C., & Ryall, K. (2005). Exploring non-speech auditory
feedback at an interactive multi-user tabletop. In Proceedings of Graphics Interface 2005
(GI '05). Canadian Human-Computer Communications Society, School of Computer
Science, University of Waterloo, Waterloo, Ontario, Canada, 41-50.
Summary: Discusses experiments on the use of non-speech audio at an interactive
multi-touch, multi-user tabletop display. Investigates both affirmative sounds that
confirm user actions and negative sounds that indicate errors. The results show
that affirmative auditory feedback may improve one's awareness of group activity
at the expense of one's awareness of his or her own activity. Also compares
localized sound, where each user has his or her own speaker, to coded sound,
where users share one speaker, but the waveform of the sounds are varied so that
a different sound is played for each user. Users can identify who caused a sound
when either localized or coded sound is used.
Hinrichs, U., & Carpendale, S. (2010). Interactive tables in the wild: visitor experiences
with multi-touch tables in the arctic exhibit at the Vancouver aquarium. Informally
41
published manuscript, Computer Science, University of Calgary, Calgary, Canada.
Retrieved from http://hdl.handle.net/1880/48199
Summary: Investigates visitors’ use of two different tabletop applications—the
Collection Viewer and the Arctic Choices table—that are part of the Canada’s
Arctic exhibition at the Vancouver Aquarium. Describes what factors attracted
visitors’ attention and triggered interaction with both tabletop exhibits, the
character and duration of information exploration, general exploration strategies,
and factors that triggered social and collaborative information exploration.
Usability issues are also presented with possible solutions.
Jacucci, G., Morrison, A., Richard, G., Kleimola, J., Peltonen, P., Parisi, L., & Laitinen,
T. (2010). Worlds of information: designing for engagement at a public multi-touch
display. Proceedings of the 28th international conference on Human factors in computing
systems, 10, 2267-2276.
Summary: Study of different 3D interface elements and how they affect social
interaction on a public multi-touch screen in Helsinki. Describe methods of
effectively partitioning the large surface to allow simultaneous usage, the need
and location of help assets and how best to initiate a multi-touch session.
Johnson, L., Witchey, H., Smith, R., Levine, A., and Haywood, K., (2010).
The 2010 Horizon Report: Museum Edition. Austin, Texas: The New Media Consortium.
Summary: The New Media Consortium’s Horizon Reports focus on emerging
technologies that they have identified as being instrumental for the future of
educational organizations. The Museum Edition focuses on 6 such technologies
and provides context on how they will affect the experience of museums in the
near future. The six identified are: mobile devices, social media, augmented
reality, location based services, gesture based computing and the semantic web.
Kristensson, P., Arnell, O., Björk, A., Dahlbäck, N., Pennerup, D., Prytz, E., . . . Åström,
N. (2008). Infotouch: an explorative multi-touch visualization interface for tagged photo
collections. Proceedings of the 5th Nordic conference on Human-computer interaction:
building bridges, 2008, 491-494.
Summary: Describes multi-touch interfaces for manipulation of tag clouds of a
tagged photo collection. Allows for creation of queries by physically moving
42
different attributes of photos into a query location. Video of the technique at:
http://www.youtube.com/watch?v=DHMJJwouq5I
Malik, S., Brush, A., & Meyers, B. (2007). An exploration of multi-finger interaction on
multi-touch surfaces. Toronto, Ont., Canada: University of Toronto.
Summary: Describes the development of the Visual Touchpad, a low-cost vision-
based input device that allows for detecting multiple hands and fingertips over a
constrained planar surface. Investigates unanswered questions as to how multiple
fingers from a single hand can best be utilized on these touch-sensitive surfaces.
Proposes a set of more advanced bi-digit widgets and gestures that facilitate
smooth transitioning from novice to expert usage.
Morris, M. (2008). A field study of knowledge workers’ use of interactive horizontal
displays. Proceedings of the Ieee tabletops and interactive surfaces 2008 (pp. 105-112).
Amsterdam: IEEE.
Summary: Field study to collect and analyze over a month of use data from eight
participants who used horizontal displays in conjunction with their existing office
computer setups. Analyzes the system logs, observations, and interview data to
reveal clear differences in preference and use patterns for horizontal and vertical
display configurations. Formulates hardware and software design guidelines that
would increase the utility of interactive horizontal displays for office scenarios
based upon the findings.
Moscovich, T. (2007). Principles and applications of multi-touch interaction.
Providence, Rhode Island: Brown University.
Summary: Studies the human factors governing multi-touch interaction, with
special emphasis on finger coordination. Presents a number of novel interaction
techniques that illustrate the benefits of multi-touch interaction. These techniques
let users work faster and more fluently than traditional single-point interaction
methods. Describes a family of techniques that use multiple finger contacts to
control digital instruments for manipulating graphical objects, allowing for
parallel control of multiple parameters (such as position and orientation) and
reduce the need for separate interaction modes by unifying operations (such as
grouping and moving objects).
43
Multi-touch case studies. (n.d.). Retrieved March 6, 2011 from http://multitouch.fi/case-
studies/
Summary: Describes real world uses of multi-touch screens. Includes case studies,
photographs and videos of such multi-touch installations as the Pori Jazz Festival,
a Virtual Autopsy Table, an Interactive Bar and the world’s largest public
multitouch display is installed for Siemens in Vienna.
Peltonen, P, Kurvinen, E, Salovaara, A, Jacucci, G, et al. (2008). It's mine, don't touch!:
interactions at a large multi-touch display in a city centre. Proceeding of the twenty-sixth
annual SIGCHI conference on Human factors in computing systems, 08, 1285-1294.
Summary: Studies social interaction around a public multi-touch display in
Helsinki. Describes different interface methods, thresholds of interaction with the
device and fellow users, behaviors of users, methods of inviting usage and
obstacles to usage.
Schöning, J., Brandl, P., Daiber, F., Echtler, F., Hilliges, O., Hook, J., Löchtefeld, M.,.
(2008). Multi-Touch Surfaces : A Technical Guide Technical Report TUM-I0833,
Technical Reports of the Technical University of Munich, October 2008, 1-19.
Summary: Describes the different methods that devices use to detect multiple
touch inputs. Investigates the different hardware designs and provides a good
technical brief of the current state of the art.
Steimle, J, Khalilbeigi, M, Mühlhäuser, M, & Hollan, J. (2010). Physical and digital
media usage patterns on interactive tabletop surfaces. Proceedings of the ACM
International Conference on Interactive Tabletops and Surfaces 2010.
Summary: Investigates the challenges of hybrid usage: users with physical and
digital objects interacting with a multi-touch table. Reports the results of a study
of spatial usage patterns when physical and digital items are grouped and sorted
on a tabletop work surface. Discusses issues with occlusion, involving both
inconvenient and desirable aspects. Describes design implications for hybrid
tabletop systems.
Szymanski, R, Goldin, M, Palmer, N, Beckinger, R, Gilday, J. & Chase, T. (2008).
Command and control in a multitouch environment. Proceedings of the Army science
conference Orlando, Florida: http://handle.dtic.mil/100.2/ADA503423.
44
Summary: Discusses the use of advanced user interface technology to create a
computer enhanced multi-touch command and control collaborative environment.
The COMET (Command and Control Multitouch Enabled Table) team out of
CERDEC C2D Communications Electronics Research Development and
Engineering Center Command and Control Directorate) is building and
researching the system discussed in this paper.
Tognazzini, B. (1994). The "starfire" video prototype project: a case history. Retrieved
March 6, 2011 from http://www.asktog.com/papers/videoPrototypePaper.html
Summary: Discusses the development of the Starfire video project by Sun
Microsystems in the 1990's. Described many revolutionary technologies and user
interface conventions such as touch screen, voice activation and the World Wide
Web.
Vanacken, D, Demeure, A, Luyten, K, & Coninx, K. (2008). Ghosts in the interface:
meta-user interface visualizations as guides for multi-touch interaction. Horizontal
Interactive Human Computer Systems, 2008, 81 - 84.
Summary: Discusses how most users are not familiar with multi-touch interfaces
beyond the traditional move-rotate-scale interactions and higher order interactions
can become exceedingly complex. The paper introduces the concept of
TouchGhosts: visual guides that are embedded in the multi-touch user interface
and that demonstrate the available interactions to the user. These provide
guidance in real time and within the context-of-use. Strategies are presented so
that the online help should not interrupt the flow of use.
Wigdor, D., Jian, H., Forlines, C., Borkin, M., & Shen, C. (2009). The wespace: the
design, development, and deployment of a walk-up and share multi-surface visual
collaboration system. Proceedings of the ACM Conference on Human Factors in
Computing Systems (CHI) Boston, Massachusetts, 1237-1246
Summary: Describes the "WeSpace" – a collaborative work space that integrates a
large data wall with a multi-user multi-touch table, developed for scientists who
frequently meet in small groups for data exploration and visualization. Presents a
yearlong effort including ethnographic studies, design, development and user
45
testing results of scientists carrying out their collaborative research using these
tools.
Wimmer, R., Schulz, F., Hennecke, F., Boring, S., & Hussman, H. (2009). Curve:
blending horizontal and vertical interactive surfaces. Proceedings of the 4th ieee
workshop on tabletops and interactive surfaces (pp. 49-58). Banff, Canada: IEEE.
Summary: Argues that horizontal and vertical surfaces serve different purposes
and offer different advantages. Proposes a combination of both surfaces by a soft
curve forming one large, L-shaped surface. This setup preserves the unique
properties of horizontal and vertical surfaces while allowing interaction across
boundaries without disruption. Describes findings from ergonomic studies
supporting this idea.
46
Appendix D – List of Figures
Figure 3.1: A vertical multi-user, multi-touch screen being used as an interactive directory at the Pori Jazz Festival. Image courtesy of Multitouch, Ltd. Figure 3.2: A multi-touch screen being used in a table format and with a stand for an angled horizontal orientation. Images courtesy of Multitouch, Ltd. Figure 3.3: A modular wall mounted cell-based design allows scalability. Image courtesy of Multitouch, Ltd. Figure 3.4: Examples of simple gestures: Single Touch, Multiple Touch & Drag, and Single Touch & Flick. Figure 3.5: Examples of multi-touch gestures: Pinch/Zoom Out, Rotate/Spin, and Unpinch/Zoom In/Rotate combination. Figure 4.1: Continuum of three types of multi-touch technology uses: activation, manipulation, and creation. Figure 5.1: Even with multi-touch screens, user interaction usually begins with a single touch (Jacucci 2010). Image courtesy of Multitouch, Ltd.
.