1 Fall 2006 6.831 UI Design and Implementation 1 Lecture 20: HCI Research Topics
1
Fall 2006 6.831 UI Design and Implementation 1
Lecture 20:
HCI Research Topics
2
Fall 2006 6.831 UI Design and Implementation 2
UI Hall of Fame or Shame?
Suggested by Chris Child
Our Hall of Fame/Shame candidate for today is the undo behavior of Outlook’s sticky notes.
You can try this yourself in Outlook: create a sticky note (File/New/Note). Type some text into
the note, and move the note to a different place on the screen. Then press Ctrl-Z to undo. It
undoes not only what you typed, but also the position of the note – and the note animates
through all the different positions you moved it to on the screen.
Recall the important dimensions of an undo model:
- what stream of actions is undone? Only the actions that affected this sticky note; other sticky
notes, and other Outlook windows, aren’t affected.
- how is the stream divided into units? It turns out that the entire stream of actions since the note
was created is a single unit – everything gets undone when you press Ctrl-Z once.
- what state is actually restored? everything about the note – its position, its size, even its color.
- how far back can you undo? As far as the creation of the note – unless you switch to another
window. Switching away from the note clears the note’s undo history, so further undo is
impossible.
What else is wrong here? As the screenshot shows, the animation wasn’t even done properly –
instead of animating using automatic redraw, Outlook paints the moving note directly on the
screen, leaving a smear behind it. Notice that the smear is visible in some parts of the Outlook
window, but not in others. Why do you think that is?
3
Fall 2006 6.831 UI Design and Implementation 3
Today’s Topics: HCI Research
• Input modalities
–Vision, speech, pen, tangible
• Output modalities
–Large, small, ambient
• Information visualization
• End-user programming
• Ubiquitous computing
This is only a sample. There’s a ton of HCI research out there.
4
Fall 2006 6.831 UI Design and Implementation 4
Input Modalities
• Vision
• Speech
• Pen gesture & sketching
• Tilting & bending
• Tangible objects
• Multimodal: several kinds of inputs at
once
5
Fall 2006 6.831 UI Design and Implementation 5
Input: Vision
• Camera watches the user to detect:
–Eye gaze
–Head pose
–Body gestures
–User presence
–Movement of physical objects
6
Fall 2006 6.831 UI Design and Implementation 6
PlayAnywhere
http://research.microsoft.com/~awilson/PlayAnywhere%20final%201Mbps.wmv
7
Fall 2006 6.831 UI Design and Implementation 7
Eye Tracking in User Testing
Source: eyetools.com
8
Fall 2006 6.831 UI Design and Implementation 8
Input: Pen & Touch
• Handwriting
• Sketching
• Touch
9
Fall 2006 6.831 UI Design and Implementation 9
The Perils of Handwriting Recognition
10
Fall 2006 6.831 UI Design and Implementation 10
Pen Sketching
http://www.ai.mit.edu/projects/rationale/video/oxygen-sketch%20design.mpg
11
Fall 2006 6.831 UI Design and Implementation 11
Touch & Play
1 Mbps human body network
Connecting a source of digital media (camera, DVD player, laptop) to a sink (display, printer,
overhead projector)
File or information transfer: file delivery, copy/paste, business card transfer
12
Fall 2006 6.831 UI Design and Implementation 12
Input: Speech Dialog Systems
Mercury: Welcome. How may I help you?
USER: I WOULD LIKE TO FLY FROM BOSTON TO LONDON
Mercury: Flights from Boston to London. What date will you be travelling?
USER: NEXT FRIDAY IN THE EVENING
Mercury: Okay. flights from Boston to London departing between 5:00 pm and midnight
on June 25. Can you provide a departure or arrival time? The earliest nonstop flight from
Boston to London departing between 5:00 pm and midnight on June 25 leaves at 5:30
p.m. and arrives at 4:55 a.m. on British Airways. You arrive one day later.
USER: IS THERE A UNITED FLIGHT
Mercury: Okay. United flights from Boston to London departing between 5:00 pm and
mid night on June 25. One moment please... I have a nonstop United flight leaving at
6:40 p.m. and arriving at 6:25 a.m.?
USER: BOOK THAT FLIGHT
13
Fall 2006 6.831 UI Design and Implementation 13
Input: Tangible Objects
http://tmg-video.media.mit.edu/senseboard/senseboard_352x240.mpg
14
Fall 2006 6.831 UI Design and Implementation 14
Output Modalities
• Large screens–Multiple monitors, desk surfaces, walls
• Small screens–Handhelds, cellphones, watches
• Ambient displays
• Other senses
–Audio: speech, “earcons”
–Touch: haptic feedback
–Olfactory
15
Fall 2006 6.831 UI Design and Implementation 15
Output: Large Screens
http://www.dgp.utoronto.ca/~ravin/videos/uist2005_distantpointing.avi
16
Fall 2006 6.831 UI Design and Implementation 16
Output: Small Screens
http://www.patrickbaudisch.com/publications/2005-Baudisch-CHI05-
SummaryThumbnails.wmv
17
Fall 2006 6.831 UI Design and Implementation 17
Output: Ambient Displays
Dangling string (Weiser & Brown, 1995)
8 foot plastic string hanging from ceiling
Mounted to motor
Motor is connected to Ethernet
Motor turns slightly with every Ethernet packet
Bursty traffic causes string to jiggle
Constant heavy traffic causes string to whirl: not just visual information, but also
auditory (the frequency of the hum)
Information Percolator, also called BubbleVis: 32 tubes filled with water, with aquarium air
pumps at the bottom of each tube, each separately controlled. Turning on the pump produces a
pixel of bubbles that rises up the tube. Can be used as a textual display (marquee-style). Also
hooked up to a camera to display awareness of someone moving down the corridor. Both
visual and auditory, since each pump makes a little noise.
Pinwheels: each pinwheel has a motor. Visual and tactile (feeling breeze generated by the
pinwheel).
Digital family portrait: provides awareness of a loved one’s life, even at a distance. Kinds of
information displayed: health (sleeping well? Eating? Feeling good?), activity, social
interaction, quality of environment (cold? Snowy?) Picture doesn’t change, but frame displays
icons (updated daily). Tested in Wizard of Oz study.
18
Fall 2006 6.831 UI Design and Implementation 18
Information Visualization
Goal of info viz: shift load from cognitive system to perceptual system, representing data in
such a way that relationships become evident to visual perception
From Tufte’s Visual Display of Quantitative Information
A famous example of an excellent static information visualizations
6 variables:
- direction of travel
- size of army
- x and y position of army
- temperature during return
- dates during return
Simple use of color for a nominal attribute
Appropriate use of size as scalar attribute
Lines linking two different displays that use position in two different ways (map location vs.
date & temp)
Note detachments that later rejoined the army
Note disastrous crossing of the Berezina river
Note abstraction: no heavy grid, only important landmarks (rivers and cities) identified
19
Fall 2006 6.831 UI Design and Implementation 19
Dynamic Queries
20
Fall 2006 6.831 UI Design and Implementation 20
Dynamic Queries
21
Fall 2006 6.831 UI Design and Implementation 21
Dynamic Queries
22
Fall 2006 6.831 UI Design and Implementation 22
Exhibit
http://simile.mit.edu/exhibit/examples/presidents/presidents.html
23
Fall 2006 6.831 UI Design and Implementation 23
End-user Programming
• Why do users need to program?– User analysis can target:
• Whole population: general-purpose applications like MS Excel
• Specific user class: bookkeeping for beekeepers
• One user: a particular beekeeper’s bookkeeping practices
– The tighter the user analysis, the more functional and usable the application
• But the less revenue will pay for its development
• Can a single user ask for more features, more iterations?
• Common kinds of EUP– Customization (e.g. toolbars)
– Automation of repetitive tasks (e.g. macros)
– Application development (e.g. spreadsheets)
24
Fall 2006 6.831 UI Design and Implementation 24
Programming by Example (PBE)
• User demonstrates how on concrete examples, using familiar UI operations
• System records user’s demonstration as a program
• Also called programming by
demonstration (PBD)
25
Fall 2006 6.831 UI Design and Implementation 25
Ubiquitous Computing
• Computing infrastructure constantly surrounding us
–Small, portable, wearable devices
–Large and ambient displays
–Sensors throughout the world for vision &
speech
–Location sensing & context awareness
• Also called pervasive computing
Computing technology will submerge in the environment around us, becoming invisible, just
making everything work better
26
Fall 2006 6.831 UI Design and Implementation 26
Prototyping for Ubiquitous Computing
http://hci.stanford.edu/dtools/dtools-apr06.mov
27
Fall 2006 6.831 UI Design and Implementation 27
What To Do Next
• Courses in spring 2007– 6.883 Pervasive, Human-centric Computing (Rudolph)– 6.870 Multimodal/Intelligent Interfaces (Darrell/Davis)– MAS.630 Affective Computing (Picard)– MAS.632 Speech Interfaces & Mobile Devices (Schmandt)– MAS.672 New Paradigms for HCI (Maes)
• Research at MIT– CSAIL
• Vision (Darrell)
• Pen (Davis)
• 3D (Durand, Popovic)
• Speech (Glass, Seneff, Zue)
• End-user programming & usable security (Miller)
• Ubiquitous (Rudolph)
– Media Lab• Tangible (Ishii)
• Agents, end-user programming (Lieberman)
• Ambient & ubiquitous (Maes)
• Wearable (Pentland)
• Emotion (Picard)
• Speech and audio (Schmandt)
• Context aware (Selker)