Evaluating Touch Gesture Usability Kevin Arthur Sr. User Experience Researcher [email protected], twitter @karthur 1 Copyright © 2010 Synaptics Inc. 9/22/2010
Jul 14, 2015
Evaluating Touch
Gesture Usability
Kevin Arthur
Sr. User Experience Researcher
[email protected], twitter @karthur
1Copyright © 2010 Synaptics Inc.9/22/2010
Overview
Gestures
• Technical properties
• Usability challenges
Methods for testing gesture usability• Test framework
• Examples from TouchPad gesture studies
• Considerations for mobile
2Copyright © 2010 Synaptics Inc.9/22/2010
Synaptics
3
TouchPad ClearPad
+ Portable music players, computer peripherals, large touchscreens+ Gesture suites
Capacitive touch technology
Copyright © 2010 Synaptics Inc.9/22/2010
Synaptics User Experience
• Usability
• Research
• Concepts
• Design (*hiring)
• Customer partnerships
• Integrated into product development and customer projects
4
Fuse Concept (with TAT, TheAlloy, Immersion, Texas Instruments)
Copyright © 2010 Synaptics Inc.9/22/2010
Gesture Properties and
Challenges
5Copyright © 2010 Synaptics Inc.9/22/2010
Defining Touch Gestures
• Typically a continuous sequence of touch events and a motion trajectory
• Performs an action or adjusts a parameter
• Multi-finger or single-finger• Touchscreen (direct touch)• Touchpad (indirect touch)
• Gestures don’t belong everywhere:“Everything is best for something and worst for something else.” – Bill Buxton*
Touchpad examples
6
Two-Finger Pinch Zoom
Two-Finger Rotate
Three-Finger Flick
Circular Scrolling
* http://www.billbuxton.com/multitouchOverview.html
Copyright © 2010 Synaptics Inc.9/22/2010
Gestures: Reactions & Challenges
• Useful, usable, enjoyable“Efficient,” “Super smooth,”
“Finger-flicking fun”
• Discoverability, learnability“Did not know how to do
the gesture”
• Gesture recognition“Tuned too tightly,” “Failed
more often than not”
• Misrecognition and unintended gestures“Accidentally resize”
• Responsiveness, satisfying results“Taking an eternity to
respond”
7Copyright © 2010 Synaptics Inc.9/22/2010
Important Properties of Gestures
8
• Gestures must be taught, discoverable
• Testing the documentation is importantLack of Affordances
• Results are not always the sameNondeterministic
• Recognition trade-offs exist between gestures in same parameter space
• Gestures should be evaluated as a setInterdependent
• Touchpad’s primary use is still for pointing and scrolling; gestures shouldn’t interfere
Interface Overloading
• Hand size, long fingernailsUser Variation
Copyright © 2010 Synaptics Inc.9/22/2010
Phases in Performing a Gesture
[Intention]
Registration
Continuation
Termination
9
“I want to enlarge this photo and I’ll use pinch zoom”
Two fingers touch down and move together or apart
Fingers continue moving
Fingers lift
Does user discover and do the gesture properly? (Q1)
Is the gesture recognized correctly and reliably? (Q2)
Is the result satisfactory? (Q3)
Has the user completed the task successfully? (Q4)
Phase Example Research Questions
Phases from Mike Wu et al., 2006.Copyright © 2010 Synaptics Inc.9/22/2010
Goals when Testing Gestures
10
Respond to stakeholder needs
• Developers want feedback – formative usability studies
• Sales and OEM partners want data – summative and competitive studies
Test in context
• Holistic evaluation; test with users in realistic scenarios with working system
• Different from offline gesture recognizer tests
Answer research questions
• Discoverability, learnability, accuracy, task success and satisfaction
Copyright © 2010 Synaptics Inc.9/22/2010
Usability Test Framework and
Methods
11Copyright © 2010 Synaptics Inc.9/22/2010
Test Framework
12
1. Gesture Introduction
• Scripted new-user experience
2. Familiarization and Practice Task
• Structured practice; assess discovery, understanding (Q1 - Intention)
3. Accuracy Task
• Measure gesture recognition rates (Q2 – Registration)
4. Satisfaction Questionnaire and Debrief
• Assess overall ease of use and preference (Q3, Q4 – Continuation, Termination)
Copyright © 2010 Synaptics Inc.9/22/2010
Example Study
• 2 devices, 5 gestures, 12 participants
• One-hour session plan
13Copyright © 2010 Synaptics Inc.9/22/2010
The examples on the following slides are drawn from studies like this one. Their purpose here is just to illustrate the methods and not to indicate real results.
1. Gesture Introduction Example
14
Simulate new user experience• “Out of box experience” material
• Help videos
• Try the gesture until success
• If no success, moderator assists
Copyright © 2010 Synaptics Inc.9/22/2010
2. Familiarization and Practice
Task: Pinch Zoom Example
Zoom in on the South Residences, near the top of the map, and find the building called The Knoll. Zoom all the way in on The Knoll and then zoom all the way back out.
15
2000 × 2000 pixels or higher.Approximately five pinch-zooms required.
Copyright © 2010 Synaptics Inc.9/22/2010
16Copyright © 2010 Synaptics Inc.9/22/2010
2. Familiarization and Practice:
Flick and Rotate Example
17
Scenario: sorting and tagging vacation photos
Copyright © 2010 Synaptics Inc.9/22/2010
2. Familiarization and Practice Task:
Flick and Rotate Example
18
1. Use the rotate gesture to make the image upright.2. Type the image’s title into the caption field and press Enter.3. Use the flick gesture to go to the next image.
Copyright © 2010 Synaptics Inc.9/22/2010
19
Users given a set of randomly oriented photos to rotate and tag.
Copyright © 2010 Synaptics Inc.9/22/2010
3. Accuracy Task Example
User performs a set number of gesture attempts.
Moderator records system responses (correct / no response / misrecognized).
Example with sample data:
20
Gesture Correct No ResponseMisrecognized Other/
NotesAs Pinch As Rotate
Pinch Zoom In
7 2 - 1
Pinch Zoom Out
10 0 - 0
Rotate Clockwise
6 3 1 -“requirestoo much motion”
Rotate Counter-clockwise
7 1 2 -
Copyright © 2010 Synaptics Inc.9/22/2010
3. Accuracy Task: Sample Result Chart
21
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Pinch Zoom In
Pinch Zoom Out
Rotate CW
Rotate CCW
Average
Avera
ge R
eco
gn
itio
n R
ate
Misrecognized
No Response
Correct
Unified Measure
Target~90% Correct
Copyright © 2010 Synaptics Inc.9/22/2010
3. Accuracy Task: Sample Results from
a Competitive Study
22
61
82
96
83
88
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2F Pinch Zoom
2F Rotate 3F Flick LR
3F Flick UD
2F Scroll
System AAverage Correct 82%
Correct No response Incorrect
94
73
85
94 96
90
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2F Pinch Zoom
2F Rotate 3F Flick LR
3F Flick UD
3F Press 2F Scroll
System BAverage Correct 89%
Correct No response Incorrect
Copyright © 2010 Synaptics Inc.9/22/2010
On rotate, the high rate of misrecognition as pinch here indicated a need to balance the gesture algorithms better.
4. Satisfaction Questionnaire: Pinch
Zoom Example
23Copyright © 2010 Synaptics Inc.9/22/2010
Note mix of generic questions (1-2, 5-8) and specific questions about a particular gesture’s performance (3,4).
4. Satisfaction Questionnaire: Pinch
Zoom Example
24Copyright © 2010 Synaptics Inc.9/22/2010
4. Satisfaction Questionnaire: Sample
Result
25
7.38
4.38
7.88
8.75
8.25
1 2 3 4 5 6 7 8 9
Pinch
Rotate
3F Flick
3F Press
2F Scroll
I would use this gesture if it were available(strongly disagree = 1, strongly agree = 9)
Strongly
Disagree
Strongly
Agree
Copyright © 2010 Synaptics Inc.9/22/2010
4. Satisfaction Questionnaire,
Alternative Rating Format
How well do the gestures work on each system?
Please rate from 1 (Poor) to 5 (Excellent)
Gesture System A System B System C
Pinch Zoom
Rotate N/A
Flick Left-Right
Flick Up-Down N/A
Three-Finger Press N/A N/A
Two-Finger Scrolling
26Copyright © 2010 Synaptics Inc.9/22/2010
A rating format like this can be more practical for a large study when there is less time to complete questionnaires.
1
2
3
4
5
Pinch Zoom 2F Rotate 1F Rotate (Chiral)
Flick LR Flick UD 1F Scroll (Linear)
1F Scroll (Circular)
2F Scrolling
User Ratings With 95% Confidence Intervals
System A System B System C
4. Satisfaction Questionnaire: Sample
Results, Alternative Rating Format
27
Excellent
Poor
Copyright © 2010 Synaptics Inc.9/22/2010
Related Tests
Measure gesture side-effects
• Unintended gestures
• Degradation on other tasks
Other usability tests for touch devices
• Pointing/selection
• Scrolling
• Text input
28Copyright © 2010 Synaptics Inc.9/22/2010
Mobile Considerations
Direct touch
• Occlusion
Different contexts
• One-handed, two-handed
• Thumb vs. finger
• On the move
Existing vs. new gestures
• Accuracy already determined or not
29Copyright © 2010 Synaptics Inc.9/22/2010
Mobile App Example
30
Steps in gesture test plan:
1. Introduction
• Game startup
2. Familiarize and practice
• Go through practice mode, play first level
• Assess success at learning gestures
3. Accuracy
• Repeat gestures using practice mode or similar scenario
• All are one-finger gestures in same parameter space
4. Assess satisfaction
MonsterKill by Origin8
Copyright © 2010 Synaptics Inc.9/22/2010
An outline of how you might do a usability test focused just on the gesture components of this game.
Conclusions
Elements of successful gestures• Discovery and understanding
• Robust recognition
• Satisfying results that mesh with the entire user experience
Design usability tests to address each element
Used at Synaptics to• Guide engineering improvements
• Help marketing team choose gestures
• Improve the end user’s experience
31Copyright © 2010 Synaptics Inc.9/22/2010
Resources
32
References
• Mark Billinghurst and Bill Buxton, “Gesture Based Interaction” in Human Input to Computer Systems (draft), www.billbuxton.com/inputManuscript.html
• Dan Saffer, Designing Gestural Interfaces, O’Reilly 2008.
• Craig Villamor et al., Touch Gesture Reference Guide, www.lukew.com/touch/
• Jacob Wobbrock et al., “User-Defined Gestures for Surface Computing,” CHI 2009.
• Mike Wu et al., “Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces,” IEEE Tabletop 2006.
• HumanCentric Gesture Research, 2010. blog.humancentric.com/gesture-research/
• Don Norman and Jakob Nielsen, “Gestural Interfaces: A Step Backwards in Usability.” Interactions Sept/Oct 2010. interactions.acm.org/content/?p=1401
Acknowledgments
• Usability colleagues at Dell, HP, Lenovo, Synaptics
Contact
• [email protected], twitter: @karthur , blog: touchusability.com
Copyright © 2010 Synaptics Inc.9/22/2010