Theory-Driven Design in Theory-Driven Design in HCIHCI
Mary CzerwinskiMary Czerwinski
Microsoft ResearchMicrosoft Research
UIST 2004UIST 2004 22
OverviewOverview
Why theory, especially now?Why theory, especially now? Brief history of HCI, psychology and theoryBrief history of HCI, psychology and theory
The importance/role of theory The importance/role of theory Examples of theory-driven researchExamples of theory-driven research How to do theoryHow to do theory Summary of benefits of theory to HCISummary of benefits of theory to HCI
UIST 2004UIST 2004 33
Why is it a good time to revisit Why is it a good time to revisit theory? Yesterday…theory? Yesterday…
UIST 2004UIST 2004 44
Dichotic Listening Task (Cherry, Dichotic Listening Task (Cherry, 1953)1953)
UIST 2004UIST 2004 55
Cocktail Party PhenomenonCocktail Party Phenomenon
Cocktail party Cocktail party problemproblem How is it that out of a sea of voices we How is it that out of a sea of voices we
can focus on a single conversation?can focus on a single conversation? Cocktail party Cocktail party effecteffect
Moray (1959) Moray (1959) • While you are usually unaware of identity of While you are usually unaware of identity of
words in a non-attended conversation…words in a non-attended conversation…• A notable exception is your A notable exception is your namename
UIST 2004UIST 2004 66
Attenuation Theory of Attention Attenuation Theory of Attention (Treisman, 1960)(Treisman, 1960)
Blocking out Blocking out the irrelevant the irrelevant content easy content easy until….until….
It’s It’s semantically semantically meaningful or meaningful or important to important to youyou
Hey, Mary!
UIST 2004UIST 2004 77
Guidelines for Speech Communication Guidelines for Speech Communication Applications, Cockpits, etc.Applications, Cockpits, etc.
Provide a mechanism to “pull” one voice Provide a mechanism to “pull” one voice into focus into focus Mostly focusing on physical attributes of the Mostly focusing on physical attributes of the
messagemessage Do not present too much information Do not present too much information
simultaneously simultaneously Provide enough time for the user to fully Provide enough time for the user to fully
fuse streams if necessaryfuse streams if necessary
UIST 2004UIST 2004 88
80s and 90s: Personal Computer80s and 90s: Personal Computer
UIST 2004UIST 2004 99
User-Centered Theories in HCIUser-Centered Theories in HCI
The Psychology of Human-Computer The Psychology of Human-Computer InteractionInteraction, by Card, Moran and Newell , by Card, Moran and Newell (1983)(1983)
Simon and Newell’s contributions to Simon and Newell’s contributions to psychology, AI and HCI psychology, AI and HCI
Cognitive and perceptual psychology Cognitive and perceptual psychology theories codified into guidelinestheories codified into guidelines A LOT of theory from this era is used in HCI A LOT of theory from this era is used in HCI
design practice todaydesign practice today
UIST 2004UIST 2004 1010
And Then, Time Stood Still…And Then, Time Stood Still…
Advances to GUI Advances to GUI desktops arguably desktops arguably stalledstalled
Lots of evolution Lots of evolution on designs but on designs but less new theoryless new theory
Do we need it?Do we need it? Where are the Where are the
breakthroughs?breakthroughs?
Evolution
UIST 2004UIST 2004 1111
Today: UbicompToday: Ubicomp
UIST 2004UIST 2004 1212
Theoretical Guidance Missing…Theoretical Guidance Missing…
Social proxemics and etiquette?Social proxemics and etiquette? Multi-cursor interaction?Multi-cursor interaction? New mental models?New mental models? New metrics for productivity and New metrics for productivity and
acceptance?acceptance? New artifacts?New artifacts? Privacy?Privacy? New input approaches?New input approaches?
UIST 2004UIST 2004 1313
Tomorrow TodayTomorrow Today
Brain-computer Brain-computer interaction devices interaction devices like BrainGatelike BrainGate
Controlling objects Controlling objects with thought is with thought is becoming a realitybecoming a reality
Good science—Good science—new theories?new theories?
UIST 2004UIST 2004 1414
UIST 2004UIST 2004 1515
VeriChipVeriChip
FDA approved FDA approved implantable chipsimplantable chips
RFID tagsRFID tags Rooted in the skin for Rooted in the skin for
accessing medical accessing medical recordsrecords
Privacy issues are Privacy issues are becoming pervasive becoming pervasive in our researchin our research
Philosophy->Theory-?Philosophy->Theory-?
UIST 2004UIST 2004 1616
A Tools Stage of HCIA Tools Stage of HCI (Painting by (Painting by ZdenekZdenek
BurianBurian)) Cro Magnons occurred Cro Magnons occurred
~40,000 years ago ~40,000 years ago Hunted mainly with Hunted mainly with
spears, (bow and spears, (bow and arrows were later arrows were later developed). developed).
Made tools from blades Made tools from blades of Flint stone, used for of Flint stone, used for preparing animal skins.preparing animal skins.
UIST 2004UIST 2004 1717
Shneiderman—Why Theories in Shneiderman—Why Theories in HCI?HCI?
DescriptiveDescriptive: clarify terms, key concepts: clarify terms, key concepts ExplanatoryExplanatory: reveal relationships and : reveal relationships and
processesprocesses PredictivePredictive: about performance and situations: about performance and situations PrescriptivePrescriptive: convey guidance for decision : convey guidance for decision
making in design by recording best practicemaking in design by recording best practice GenerativeGenerative: enable practitioners to create, : enable practitioners to create,
invent or discover something newinvent or discover something new
UIST 2004UIST 2004 1818
Examples of Theory-Driven Examples of Theory-Driven ResearchResearch
Just a few examplesJust a few examplesNot exhaustive!Not exhaustive!Still, a tough task to identify well-Still, a tough task to identify well-
known examples across all types known examples across all types of theoryof theory
UIST 2004UIST 2004 1919
Explanatory Theory ExampleExplanatory Theory Example
Norman’s seven stage model from POET (1988)Norman’s seven stage model from POET (1988) An approximate model with a continuous An approximate model with a continuous
feedback loopfeedback loop Forming the goalForming the goal Forming the intentionForming the intention Specifying the actionSpecifying the action Executing the actionExecuting the action Perceiving the state of the worldPerceiving the state of the world Interpreting the state of the worldInterpreting the state of the world Evaluating the outcomeEvaluating the outcome
UIST 2004UIST 2004 2020
Using the model predictively…Using the model predictively… Miyata & Norman (1986)Miyata & Norman (1986)
Predicted interruptions between task Predicted interruptions between task execution and evaluation as less harmful execution and evaluation as less harmful when multitaskingwhen multitasking
UntestedUntested In 2000, we decided to test this using IM In 2000, we decided to test this using IM
and multiple tasksand multiple tasks Examined planning, execution and evaluation Examined planning, execution and evaluation
phases of tasksphases of tasks
UIST 2004UIST 2004 2121
UIST 2004UIST 2004 2222
Attention-Based Principles of Attention-Based Principles of NotificationsNotifications
Early in a task was the worst time to Early in a task was the worst time to interrupt if you want user to rememberinterrupt if you want user to remember
Make notifications situation-awareMake notifications situation-aware Look for cognitive breakpoints in users’ Look for cognitive breakpoints in users’
interactions. interactions. When possible, use smart monitoringWhen possible, use smart monitoring
Monitor the user (what stage in task?)Monitor the user (what stage in task?) Content of interruption—similar is betterContent of interruption—similar is better
• Obvious privacy issues, etc.Obvious privacy issues, etc.
UIST 2004UIST 2004 2323
Predictive Theory ExamplesPredictive Theory Examples
Large display research (Tan, Large display research (Tan, Czerwinski & Robertson, 2001-Czerwinski & Robertson, 2001-2003)2003)• Most early research carried out Most early research carried out
around cockpit design around cockpit design • New hardware often necessitates the New hardware often necessitates the
need for new software/interactionneed for new software/interaction• Serendipitous gender and spatial Serendipitous gender and spatial
cognition findings based on theories cognition findings based on theories of perception and cognitionof perception and cognition
UIST 2004UIST 2004 2424
Dsharp DisplayDsharp Display
43"
11"
UIST 2004UIST 2004 2525
Prescriptive Theory ExamplesPrescriptive Theory Examples Example: Gestalt Theory of Perception Example: Gestalt Theory of Perception
(similarity, closure, good continuity, (similarity, closure, good continuity, proximity/figure-ground)proximity/figure-ground)
Example: Feature Integration Theory Example: Feature Integration Theory (Triesman et al., 80s), guided visual (Triesman et al., 80s), guided visual search and pop out effectssearch and pop out effects
Utilized well in design guidelines todayUtilized well in design guidelines today
UIST 2004UIST 2004 2626
Information Visualizer SystemInformation Visualizer System
Access CostsAccess Costs
Interaction CostsInteraction Costs
Assimilation CostsAssimilation Costs
Larger WorkspaceDenser Workspace
Larger WorkspaceDenser Workspace
Highly InteractiveHighly Interactive
Information Visualization
Information Visualization
COST STRUCTUREOF INFORMATION
INFORMATIONWORKSPACE
Analysis Goals UI Artifacts
CASE STUDIES EXPLOIT HUMANPERCEPTION
INFORMATIONVISUALIZATIONS
ANIMATED GUI
3D/RoomsInteractive Objects
Cognitive Coprocessor
UIST 2004UIST 2004 2727
•ACT-IF (Pirolli & Card ’99)
•Theory based on information foraging, sensemaking and the scatter/gather approach
•Published in Psychology Review
UIST 2004UIST 2004 2828
Scatter/Gather Document BrowserScatter/Gather Document Browser
UIST 2004UIST 2004 2929
UIST 2004UIST 2004 3030
Generative Theory ExamplesGenerative Theory Examples
Shneiderman: “The future of HCI Shneiderman: “The future of HCI must be tied to more effective must be tied to more effective generativegenerative theoriestheories that enable HCI to that enable HCI to become the bright shining source of become the bright shining source of innovation; a much stronger role than innovation; a much stronger role than the usability testers and refiners of the usability testers and refiners of ideas initiated by others.”ideas initiated by others.”
UIST 2004UIST 2004 3131
Buxton’s 3-State Model of Buxton’s 3-State Model of Graphical Input (1990)Graphical Input (1990)
Model inspired Model inspired Mackinlay, Card & Mackinlay, Card & Robertson (1991) to Robertson (1991) to write “A Semantic write “A Semantic Analysis of the Design Analysis of the Design Space of Input Devices”Space of Input Devices”
Hinckley et al. (1998) Hinckley et al. (1998) extended the ideas to extended the ideas to add notation for add notation for continuous properties continuous properties during state transitions during state transitions of devicesof devices
UIST 2004UIST 2004 3232
Pointing and Fitts’ law – The abc’s of user Pointing and Fitts’ law – The abc’s of user interfaces (Predictive and Generative)interfaces (Predictive and Generative)
Fitts’ lawFitts’ law )1(log2 WDbaT
UIST 2004UIST 2004 3333
Zhai et al.: Laws of actionsZhai et al.: Laws of actions
PointingPointingCrossingCrossingSteeringSteering
Thank you Shumin!Thank you Shumin!
UIST 2004UIST 2004 3434
Crossing – more than dotting the i’sCrossing – more than dotting the i’s
Why crossing?Why crossing? increasing interaction increasing interaction
“vocabulary”“vocabulary” Pen based computingPen based computing
How does crossing How does crossing compare with pointing?compare with pointing?
What is the theoretical What is the theoretical foundation of crossing?foundation of crossing?
Accot & Zhai 2002
UIST 2004UIST 2004 3535
Systematic comparison Systematic comparison
Continuity: Continuity: continuous vs discretecontinuous vs discrete
Target type:Target type:pointing vs crossingpointing vs crossing
Constraint direction:Constraint direction:collinear vs orthogonalcollinear vs orthogonal
Accot & Zhai, CHI 2002
UIST 2004UIST 2004 3636
Crossing-based interfacesCrossing-based interfacesAccot & Zhai 2003
UIST 2004UIST 2004 3737W
D
Law of steeringLaw of steering
Rashevsky 1959, Drury 1971; Accot & Zhai, 1997, 1999, 2001
Accot & Zhai 1997, 1999, 2001)
UIST 2004UIST 2004 3838
Law of Steering - General formLaw of Steering - General form
dsdsW(s)W(s)
IDIDCC = =
CC
TTCC = a + b ID = a + b IDCC
dsW(s)
CC
UIST 2004UIST 2004 3939
Computing off the Computing off the desktopdesktop
Desktop computing Desktop computing “workstation” interface “workstation” interface foundationfoundation Large and personal displayLarge and personal display Input device (mouse)Input device (mouse) Typewriter keyboardTypewriter keyboard
HCI Frontier – beyond the HCI Frontier – beyond the desktopdesktop Interfaces without display-Interfaces without display-
mouse-keyboard tripodmouse-keyboard tripod Numerous difficult challengesNumerous difficult challenges
UIST 2004UIST 2004 4040
“Fitts-digraph energy”
27
1
27
12 1
i j i
ijij
W
DLog
IP
Pt
Alphabetically Tuned and Optimized Mobile Interface KeyboardAlphabetically Tuned and Optimized Mobile Interface Keyboard
(ATOMIK)(ATOMIK)Human Movement Study: Fitts’ law
MT = a + b Log2(Dsi/Wi + 1)
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
sp E T A H O N S R I D L U W M C G Y F B P K V J X Q Z
English Letter Corpus(News, chat etc)
01
0
Eif
EifeBAW kTE
Metropolis “random walk” optimization
))(( i
z
aii xyite
Alphabetical tuning
N
i
icifCI1
)()(
Word connectivity
Zhai, Hunter, Smith 2000; Zhai, Sue, Accot 2002, Drews
UIST 2004UIST 2004 4141
Limitations and hints from Limitations and hints from ATOMIKATOMIK
Tapping one key at a time – tedious. The Tapping one key at a time – tedious. The stylus can be more expressive and dexterous.stylus can be more expressive and dexterous.
Does not utilize language Does not utilize language redundancy/statistical intelligence.redundancy/statistical intelligence.
People tend to remember the pattern of a People tend to remember the pattern of a whole word, not individual letters.whole word, not individual letters.
UIST 2004UIST 2004 4242
Zhai et al.--SharkZhai et al.--Shark
UIST 2004UIST 2004 4343
Metrics Development—”Metrics Development—”Subjective Subjective Duration Assessment”Duration Assessment”
HCI and iterative usability metricsHCI and iterative usability metrics Performance (task times, success rates)Performance (task times, success rates) Preference (user satisfaction Preference (user satisfaction
questionnaires)questionnaires) Usually correlated, but not alwaysUsually correlated, but not always Users notoriously “positive” w/ratingsUsers notoriously “positive” w/ratings Neilsen & Levy (1994): need an Neilsen & Levy (1994): need an
average of 5 on a 7 point scaleaverage of 5 on a 7 point scale
UIST 2004UIST 2004 4444
Ziegarnik Effect (1927)Ziegarnik Effect (1927)
People remember uncompleted tasks People remember uncompleted tasks better than completedbetter than completed
Weybrew (1984) used time estimationWeybrew (1984) used time estimation People overestimate time on unfinished tasksPeople overestimate time on unfinished tasks People underestimate time on completed People underestimate time on completed
taskstasks Van Bergen (1968) task has to be Van Bergen (1968) task has to be
engaging or more difficult to get the effectengaging or more difficult to get the effect
UIST 2004UIST 2004 4545
Spool (2001)Spool (2001)
Found a strong correlation between Found a strong correlation between perceived download time and whether perceived download time and whether users successfully completed their tasks users successfully completed their tasks on a siteon a site
When people accomplish task on a site, When people accomplish task on a site, they perceive that site to be fast, and vice they perceive that site to be fast, and vice versaversa
Goes against emphasis on fast download Goes against emphasis on fast download times only, need to support user’s task times only, need to support user’s task mostmost
UIST 2004UIST 2004 4646
67% 83% 100%-50
-40
-30
-20
-10
0
10
20
30
Task Success Rate
MSN Study—SDA Better Predictor of Difficulty than User Satisfaction
UIST 2004UIST 2004 4747
So Are We in Good Shape?So Are We in Good Shape?
Obviously, great work has come Obviously, great work has come out of theory in HCIout of theory in HCI
Do we need more?Do we need more?
UIST 2004UIST 2004 4848
Why I Think Tomorrow’s HCI Why I Think Tomorrow’s HCI Needs TheoryNeeds Theory
We’ve come full circleWe’ve come full circle Many new tools used both singly and by multiple Many new tools used both singly and by multiple
peoplepeople
Theory is one of the most important things we Theory is one of the most important things we
should be doing as a disciplineshould be doing as a discipline Not everyone needs to do it, but some of us have to!Not everyone needs to do it, but some of us have to!
A way to guarantee progressA way to guarantee progress Invite new disciplines to work with usInvite new disciplines to work with us
Cognitive neuroscientists, biologists, ethicists?Cognitive neuroscientists, biologists, ethicists?
UIST 2004UIST 2004 4949
How to Do Theory?How to Do Theory? Whittaker, Terveen & Nardi (2000)Whittaker, Terveen & Nardi (2000)
Adopt a reference task agendaAdopt a reference task agenda A set of well-defined “tasks” that candidate A set of well-defined “tasks” that candidate
systems could be evaluated against like systems could be evaluated against like TRECTREC
Even skeptics (Landauer, Carroll et al., Bannon) Even skeptics (Landauer, Carroll et al., Bannon) agree on the importance of having an adequate agree on the importance of having an adequate descriptive base for HCI as a prerequisite for descriptive base for HCI as a prerequisite for more theory development more theory development
UIST 2004UIST 2004 5050
Engestrom’s Activity System Model Engestrom’s Activity System Model (1987)(1987)
Most human activities are collective ones taking Most human activities are collective ones taking place in rich social environments place in rich social environments
Model used to explain collective activities and Model used to explain collective activities and cooperative work, including cultural conventions cooperative work, including cultural conventions and established rulesand established rules
UIST 2004UIST 2004 5151
Activity Theory and HCIActivity Theory and HCI Transformation of tools, rules and divisions of labor is Transformation of tools, rules and divisions of labor is
key key Utilized more in CSCW communityUtilized more in CSCW community Exacty how Activity Theory is operationalized in Exacty how Activity Theory is operationalized in
studies is a common problemstudies is a common problem Can provide a framework to reinterpret the concept of Can provide a framework to reinterpret the concept of
user needs and societyuser needs and society
UIST 2004UIST 2004 5252
Challenges to Theory BuildingChallenges to Theory Building Advancing technologyAdvancing technology Influence of individual differencesInfluence of individual differences Contextual effectsContextual effects Longitudinal effectsLongitudinal effects New metrics neededNew metrics needed
But paradigms exist that can be But paradigms exist that can be leveragedleveraged
UIST 2004UIST 2004 5353
It’s Good for Your Career!It’s Good for Your Career! Top 10 Most Frequently Cited CHI AuthorsTop 10 Most Frequently Cited CHI Authors
Stu Card (484 citations)Stu Card (484 citations) Bill Buxton (351)Bill Buxton (351) Thomas Moran (344)Thomas Moran (344) Ben Shneiderman (322)Ben Shneiderman (322) Hiroshi Ishii (298)Hiroshi Ishii (298) Brad Myers (287)Brad Myers (287) Jakob Nielsen (286)Jakob Nielsen (286) Allen Newell (222)Allen Newell (222) Jock Mackinlay (217)Jock Mackinlay (217) George Robertson (215)George Robertson (215)
UIST 2004UIST 2004 5454
ConclusionConclusion Benefits of a theoretical Benefits of a theoretical
approach to HCI:approach to HCI: Move our community toward a Move our community toward a
solid, scientific disciplinesolid, scientific discipline Identify gaps in the design spaceIdentify gaps in the design space Engage in a communal Engage in a communal
discussion about designdiscussion about design Career advancementCareer advancement InnovationInnovation Holy Grail for
HCI
UIST 2004UIST 2004 5555
Thank You!Thank You!
UIST 2004UIST 2004 5656
Tom Erickson’s PoemTom Erickson’s Poem Theory Theory: A Designer's View Theory Theory: A Designer's View
Theory weary, theory leery, why can't I be theory cheery? Theory weary, theory leery, why can't I be theory cheery? I often try out little bits, wheresoever they might fit. I often try out little bits, wheresoever they might fit. (Affordances are very pliable, though what they add is quite (Affordances are very pliable, though what they add is quite deniable.) deniable.) The sages call this bricolage, the promiscuous prefer menage... The sages call this bricolage, the promiscuous prefer menage...
A savage, I, my mind's pragmatic, I keep what's good, discard A savage, I, my mind's pragmatic, I keep what's good, discard dogmatic. dogmatic.
//// Oddly now, I'm theory cheery -- I find I have a theory theory! Oddly now, I'm theory cheery -- I find I have a theory theory!
Neither holy grail, nor deep disgrace, theory's useful in its Neither holy grail, nor deep disgrace, theory's useful in its place, place, (Framing, talking, predicting, bonding, evoking discourse--(Framing, talking, predicting, bonding, evoking discourse--Others responding) Others responding) Like goals and methods, plans and actions, theory's situated, Like goals and methods, plans and actions, theory's situated, not pure abstraction. not pure abstraction. So make your theory a public way, where passers by may So make your theory a public way, where passers by may pause and staypause and stay