Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April 5, 2012
Making Checklists Social
Scott Klemmer Jesse CirimeleNicolas Kokkalis
1
POMI research group
Thursday, April 5, 2012
How many of you...?
Milkman, K et. al. in PNAS 2011
Thursday, April 5, 2012
Checklists Everywhere
Pronovost et al, 2006
Thursday, April 5, 2012
Checklists Everywhere
Pronovost et al, 2006
Ten-day Trial: infection rates from 11% to 0.
Michigan hospital study:infection down 66%saved $175 millionsaved 1500 lives
Thursday, April 5, 2012
Checklists are Mobile
N Seyff et. al. in Software IEEE 2006
Thursday, April 5, 2012
Checklists can be Socially Created or Used
Our Work
Thursday, April 5, 2012
Creating Checklists Socially:Providing Action Plans helps people
complete Tasks
Nicolas Kokkalis, Johannes Huebner, Steve Diamond, Dominic Becker, Michael Chang, Thomas Koehn,
Florian Schulze, Moontae Lee, Scott Klemmer
Thursday, April 5, 2012
Can personal task lists benefit from social creation?
Thursday, April 5, 2012
Complete more and easier
No Action Plans
Provided Action Plans
Self-created Action Plans
Less
Ef
fort
Mor
e Ef
fort
More CompletedLess Completed
Thursday, April 5, 2012
Can the crowd provide actionability to people’s tasks?
Thursday, April 5, 2012
To enable our studies we created an online community around tasks: TaskGenies
People can submit their tasks, others can decompose them
Access is provided via WWW for desktop, App for mobile, and e-mail
Taskgenies Platform
Thursday, April 5, 2012
TaskGenies User Interfaces
Desktop: Submit Task & Create Plans Mobile: Submit Tasks
Thursday, April 5, 2012
Experiment: Method & Measures
280 people submitted 20 tasks they planed to complete
Control: no interventionCrowd: provided w/ crowd-created plansSelf: asked to create own plans
1 week later: measured # of tasks completed
Thursday, April 5, 2012
F(2,230)=6.10, p<.05
Crowd-created plans helped people complete more tasks
Thursday, April 5, 2012
Crowd Plans help diverse Tasks
High Level Tasks Lingering Tasks
!"#
$!"#
%!"#
&!"#
'!"#
(!"#
)!"#
*+,--#.#/0--#102301## 45647-080-##
Control Crowd
Task
s C
ompl
eted
(χ2=16.00, p<0.0001) (χ2=3.68, p=0.052) (χ2=17.72, p<0.0001) (χ2=6.24, p<0.05)
!"#
$!"#
%!"#
&!"#
'!"#
(!"#
)!"#
*+*,-.*/01.*/#23454## -.*/01.*/#23454##
Thursday, April 5, 2012
69% of task had at least one analog
Thursday, April 5, 2012
Scaling Up by reusing Action Plans
Using a corpus of 6000 action plans95% of tasks used an existing plan5% of tasks used crowd
NLP condition better than Control
Thursday, April 5, 2012
Email: de facto task list of people
Thursday, April 5, 2012
GmailValet.com
Thursday, April 5, 2012
Restricting Access: Only 1 label, Only the last 100 messages
Thursday, April 5, 2012
The Valet View
Thursday, April 5, 2012
Trust but verify: Record all actions
Thursday, April 5, 2012
Take aways
Action plans facilitate effective work, but we have them too rarely. Making checklists social helps us get things done.
Thursday, April 5, 2012
Using Checklists Socially:Supporting crisis response teams
with checklists and multiple displays
Jesse Cirimele, Leslie Wu, Kyle Barrett, Katherine Chen, Kyle Harrison, Larry Chu, Stu Card, Scott
Klemmer
Thursday, April 5, 2012
Surgery
Thursday, April 5, 2012
Surgery
• Pervasive and essential (234 million surgical procedures per year)
Thursday, April 5, 2012
Surgery
• Pervasive and essential (234 million surgical procedures per year)
• Complex178 tasks per day in ICU
Thursday, April 5, 2012
Surgery
• Pervasive and essential (234 million surgical procedures per year)
• Complex178 tasks per day in ICU
• High mortality1/300 die in surgery
Thursday, April 5, 2012
Types of errors
Thursday, April 5, 2012
Types of errors
• Social
• team coordination problems
• communication issues (miss something)
Thursday, April 5, 2012
Types of errors
• Social
• team coordination problems
• communication issues (miss something)
• Individual
• lose time (late)
• get treatment algorithms wrong
Thursday, April 5, 2012
Braincan be slow, error prone
Thursday, April 5, 2012
Braincan be slow, error prone
Cognitive AidFast and Accurate
Thursday, April 5, 2012
Braincan be slow, error prone
Cognitive AidFast and Accurate
... butThursday, April 5, 2012
Cognitive AidStatic, Visible to One
Interactive AidDynamic, Visible to Group
Thursday, April 5, 2012
insights
Thursday, April 5, 2012
insights
• Always ready
Thursday, April 5, 2012
insights
• Always ready
• Nurse as primary controller
Thursday, April 5, 2012
insights
• Always ready
• Nurse as primary controller
• Big display facilitates group coordination
Thursday, April 5, 2012
Thursday, April 5, 2012
Cog Aid => Resource Management
• cognitive aids are a resource to help the doctor during a crisis
• but so are
• other people in the room
• information about the patient
• people on-call in case of emergency
• communication
Thursday, April 5, 2012
OR layoutThursday, April 5, 2012
Tablet Controller
Large Screen Display
Thursday, April 5, 2012
Scenario walkthrough
Thursday, April 5, 2012
The Wizard of Oz and the OR
Thursday, April 5, 2012
Thursday, April 5, 2012
• Checklists (cognitive aids, task lists)
• can play an important role in many situations
• can help us more when we carry them with us
• can be more effective when support socially
Conclusion
Thursday, April 5, 2012
Thank you!
Questions?
37Thursday, April 5, 2012
http://hci.stanford.eduhttp://aim.stanford.edu
Jesse Cirimele, Leslie Wu, Kyle Barrett, Katherine Chen, Kyle Harrison, Larry Chu, Stu Card, Scott Klemmer
Thursday, April 5, 2012