UX LONDON Research Together! 1
UX LONDON
Research Together!
1
Hello!
2
I have a question…
3
Do you enjoy being right?
4
You are correct!
5
YESSS!
6
p0wned!
7
No.
8
?
9
>? !
10
Flickr/Chris Voll
11
Ego!
12
13
14
Where are you?
15
What We’re Doing Today Research & Collaboration
A Framework for Research Questions & Activities Biases & Objections
Understanding the Organization
Break
User Research Analysis
Models and Reports Getting Buy-In at Every Stage
Wrap Up
16
Research & Collaboration
17
18
19
20
21
22
23
People
24
25
Collaboration!
26
Collaboration!
27
28
True Collaboration
29
Chris Noessel
Collaboration requires behavior change
30
For Effective Collaboration
Have a plan Provide a rationale Define roles and responsibility Set expectations Communicate progress Reflect on performance
31
The 4 Virtues of Collaboration
Clarity & definition Accountability & ownership Awareness & respect Openness & honesty
32
Embrace Conflict
33
Why research?
34
Real World
Context
Organization Users
35
Things you
don’t control
Goals, assumptions &
resources
Needs & behaviors
36
Personal View
Personal View
Personal View
37
Shared Reality
38
Shared Understanding
39
A design project is a series of decisions.
40
What is
What ought to be
41
Research leads to evidence-based
decisions.
42
Data (alone) doesn’t change minds.
43
Objections!
44
6We don’t have the time.
45
$We don’t have the money.
46
$We don’t have the expertise.
47
We have a guy.
48
0
25
50
75
100
April May June July
We’ll just A/B test.
We have analytics.
49
!Let’s just make a prototype.
50
Steve Jobs
51
Any others?
52
Everyone wants better products, faster.
53
No one wants to read a report.
54
Research Advantages
55
¿Faster decisions
56
CLower risk
57
Saves money$58
JIncreased value
59
Continuing returnsC60
Everyone is more effective and
efficient
61
62
How to Do Research
63
Dogma
64
Design-Led
Research-Led
Expert Mindset
Participatory Mindset
Users seen as subjects Users seen as partners
Design-led with
expert mindset
Design-led with
participatory mindset
Research-led with expert
mindset
Research-led with participatory
mindset
Dubberly Design Office
65
Goal Driven
Skeptical Mindset
Increase chance of success Reduce risk
Willing to question the value of any approach
66
One Simple Process
67
Form Questions Analyze Data
Gather Data
68
Form Questions Analyze DataThink Critically
69
Form Questions Analyze DataObserve
70
Form Questions Analyze DataInterview
71
Form Questions Analyze DataRead
72
Form Questions Analyze Data
Read
Experiment
Interview
Observe
Think
73
Your Process Success!!!!Insights
74
You need a plan!
75
76
Form Questions Analyze Data
Gather Data
77
Questions determine results.
78
Questions give research meaning.
79
Research high-priority questions.
80
Good Questions
Specific Actionable Practical
81
A Bad Question
“How do we get Millennials to like us?”
82
Better Question
“How do recent college graduates living in cities decide what to have for dinner?”
83
A Bad Question
“What do people do around here all day?”
84
A Better Question
“How do editors and designers work together?”
85
The Best Question
The unknown that carries the most risk.
86
Project Risks
•Target customers don’t value product
•Customers need something else
•Business model doesn’t support it
•Organization can’t produce it
•Someone else is doing it better
87
Practice!!88
Practice Exercise: Writing questions Time:10 minutes
• I will give you a project scenario • Discuss potential research questions • Write down the 3–5 highest priority
89
What research questions might serve this project?
A e-commerce startup wants to create an app to
help people give gifts.
90
Time is up!691
Let’s talk about your questions.
92
How to answer those questions
93
Form Questions Analyze Data
Gather Data
94
Research activities are simply ways to answer questions.
95
Questions About
Users
ProductOrg
Competition
InterviewsInterviews
Usability Testing
A/B Testing
Contextual Inquiry
Literature Review
SWOT Analysis
Brand Audit
Usability Testing
Competitive Analysis
Heuristic Analysis
Descriptive
Evaluative
Evaluative
Evaluative
Analytic
Analytic
Generative
Descriptive
96
Research Activity
Topic Purpose
Time
Money
97
Phone Interviews
What do we need to know
about?What kind of
decision will it inform?
How long do we have?
What is our budget?
In-Person Interviews Contextual
Inquiry
Usability Testing
Competitive Analysis
98
Why not just make a prototype?
99
100
“If we only test bottle openers, we may never realize customers prefer screw-top bottles.”
– Victor Lombardi, Why We Fail
101
There is no one right method or activity.
102
How to get the most out of any method •Clarify goals •Enumerate assumptions •Identify questions •Prioritize questions •Work collaboratively •Present strategically
103
Don’t focus on the method
104
Tell the story
105
Critical Thinking
106
Critical Thinking
•Disciplined •Self-correcting •Clear •Logical
107
Uncritical Thinking
“I hate yellow, so a yellow website won’t succeed.”
108
Critical Thinking
“I hate yellow, but based on the evidence, it might work for our audience.”
109
Uncritical Thinking
“The information on this website is too dumbed-down for me.”
110
Critical Thinking
“Our target audience needs clear, simple information.”
111
Critical Thinking
“I don’t know.”
112
Bias
113
Bias: Something that causes an influence or
prejudice
114
Confirmation Bias: You selectively weight the information that confirms what you already believe.
115
Sampling Bias: Your sample of research subjects isn’t sufficiently
representative.
116
Interviewer Bias: You insert your opinion
into interviews.
117
Social Desirability Bias People don’t say the true things that they worry will
make them look bad.
118
Hawthorne Effect Observation changes the behavior being observed.
119
Ease
Clear Display
Related Experience
Primed Idea
Good Mood
Feels True
Feels Familiar
Feels Good
Feels Effortless
Daniel Kahneman
120
Feeling confident? It’s not a good sign.
121
You might have a bad case of Dunning-Kruger.
122
Research Topics
123
Organizational Research
124
Real World
Context
Organization Users
125
All organizations have baggage
126
Organizational research helps you understand
•Requirements •Politics •Workflow •Capabilities •Goodwill
127
Requirements
What are the top business priorities for this project/product?
128
Politics
What does success mean for our individual roles?
129
Workflow
(How) do we have to change how we work together to be successful?
130
Capabilities
What are the strengths and weaknesses of our team?
131
Capabilities
Where is the internal expertise?
132
Goodwill
How might product decisions make someone’s job easier (or harder)?
133
Stakeholders
134
How to extract knowledge from these people?
135
Get them alone.
136
Butter them up.
137
Basic Stakeholder Questions What is your title? How long have you been in this role?
What are your essential duties and responsibilities?
What does a typical day look like?
Who are the people you work most closely with?
How is that going?
138
Project Specific Questions What does success mean from your perspective?
What will have changed for the better once this project is complete?
Do you have any concerns about this project?
What do you think the greatest challenges to success are?
Internal and external?
139
Stakeholder Power Moves
“Why are you asking me this?”
“I don’t understand that question. It doesn’t make any sense.”
“I don’t feel comfortable talking to you about that.”
“No one pays attention to anything I have to say, so I don’t know why I should bother talking to you.”
“How much more time is this going to take?”
140
Practice!!141
Practice Exercise: Stakeholder interviews Time:10 minutes x 2
• Find a partner. This is a pair exercise. • We will show you a set of questions • Interview your partner • Really listen • No need to take notes
142
Stakeholder Questions
•What is your title? How long have you had this job?
•What are your essential duties and responsibilities?
•What is a typical day like?
•Who are the people you work most closely with?
•How is that going?
•What do you think the greatest challenges to your success are?
•Internal and external?
143
Switch!∞144
Stakeholder Questions
•What is your title? How long have you had this job?
•What are your essential duties and responsibilities?
•What is a typical day like?
•Who are the people you work most closely with?
•How is that going?
•What do you think the greatest challenges to your success are?
•Internal and external?
145
Time is up!6146
What did you learn?
147
Empathy
148
In summary
• A shared reality is as important as “new” facts. • Research is a simple process. • Be goal-oriented, not dogmatic. • Questions threaten authority. • Facts alone won’t convince those who feel threatened. • Use what you learn to fit your findings into the project story.
149
User Research
150
Real World
Context
UsersOrganization
151
Photo: Flickr/theloushe
Ethnography
152
How to do bad user research: Ask people what they want.
153
How to do bad user research: Ask people what they like.
154
Never ask users what they want or like.
155
The Four Ds of Design
Ethnography
156
Deep Dive Daily Life
Data Analysis Drama
157
“...true ethnography reveals not just what people say they do, but what they actually do.”
–PARC
158
159
Photo: Flickr/lintmachine
160
The Art of The Interview
161
Interviewing is not talking.
162
Interviewing is listening.
163
Good Interviewers 1.Know Your Question 2.Warm Up 3.Shut Up
164
Interview Structure
1 Intro
2 Body
3 Conclusion
165
Introduction:
Smile
Express gratitude
Describe the process
Ask to record
Warm up questions
Body:
Ask open-ended questions
Probe for more
Allow silence
Use questions as checklist
Conclusion:
Transition to wrap-up
Ask if there is anything else
Thank for time
Interview Structure
166
Introduction:
Smile
Express gratitude
Describe the process
Ask to record
Warm up questions
Body:
Ask open-ended questions
Probe for more
Allow silence
Use questions as checklist
Conclusion:
Transition to wrap-up
Ask if there is anything else
Thank for time
Interview Structure
167
Introduction:
Smile
Express gratitude
Describe the process
Ask to record
Warm up questions
Body:
Ask open-ended questions
Probe for more
Allow silence
Use questions as checklist
Conclusion:
Transition to wrap-up
Ask if there is anything else
Thank for time
Interview Structure
168
You are the host You are the student
169
Out of your comfort zone,
and into theirs.
170
Interview Checklist • Create a welcoming atmosphere.
• Always listen more than you speak.
• Take responsibility to accurately convey the thoughts and behaviors of the people you are studying.
• Start each interview with a general description of the goal, but be careful of focusing responses too narrowly.
• Avoid leading questions and closed yes/no questions. Ask follow-up questions.
• Prepare an outline of your interview questions in advance, but don’t be afraid to stray from it.
• Also note the exact phrases and vocabulary.
171
Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
Make a note of
172
RolesInterviewer Notetaker Observer Participant
173
Practice!!174
Exercise: User research interview Time: 15 min x 3
•We will give you a research scenario and an interview script
•In your groups, assign roles of participant, interviewer, note taker, and observer (optional)
•Conduct the interview in 15 min, then switch roles. •Hang on to your notes. We’ll need them.
175
Interview Scenario
You are working on a new service to help people give gifts.
The goal of the research is to identify unmet needs people might have with regard to giving gifts.
176
Listen for:
Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
177
!Ready? Go!
178
Switch!∞179
Remember: This is a research interview, not a friendly conversation.
Listen for:
Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
180
Switch!∞181
Clear your mind.
Listen for:
Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
182
Time is up!6183
How did that go?
184
How about a focus group?
185
186
“Even when the subjects are well selected, focus groups are supposed to be merely the source of ideas that need to be researched.” –Robert K. Merton, Sociologist, invented focus groups
187
Everybody Lies
188
Anything else about activities?
189
Analysis
190
Analysis turns data into useful insights
191
192
Analysis isn’t complicated
Compile data Look for patterns
Identify insights
Create models
…but it is challenging
193
You need to create meaning from data
194
Data > Meaning > Actionable Inputs
195
Observation Observation
Observation
Observation Observation
Observation
Observation Observation
Observation
196
Observations are
Verbatim quotes Reported behaviors
Observed behavior
197
Observation Observation
Observation
Observation Observation
Observation
Observation Observation
Observation
Collaborates on purchases
Uses several devices Needs affirmation
198
Make product information sharable
Save state in purchase process
Users collaborate on purchases
Insight Mandate/Action
199
Ground rules The goal of this exercise is to better understand the context and needs of the user.
Wait to identify larger patterns until you’ve gone through the data.
Clearly differentiate observations from interpretations (what happened versus what it means).
No specific solutions until after you’ve gone through insights and principles. Solutions come next.
200
Practice!!201
Practice Exercise: Analysis Part 1 Time: 15 minutes.
1.Gather all notes 2.Scan notes for interesting observations 3.Write observations on notes 4.Put the notes in a pile
202
Look for quotes and observations that indicate
Goals (what the
participant wants to
accomplish)
Priorities (what is most important to
the participant)
Tasks (actions the participant
takes to meet their goal)
Barriers (what prevents accomplishing
the goal)
Motivators (the situation or
event that starts down the
task path)
Habits (things the participant does on a
regular basis)
Relationships (who the
participant interacts with
doing the tasks)
203
You have 15 minutes.6
204
Get ready to finish!6
205
Time is up!6206
207
Practice Exercise: Analysis Part 2 Time: 15 minutes.
1.Put notes on the board 2.Group notes into patterns 3.Label the group 4.Negotiate and advocate for your perspective
208
1
23
209
Do this part last
210
You have 15 minutes.6
211
Get ready to finish!6
212
Time is up!6213
How did that go?
214
Creating a Model
215
Personal View
Personal View
Personal View
216
Shared Reality
217
Thinking is not useful until shared
218
A model is thinking made visible
219
Some examples
220
221
222
223
224
225
226
227
228
229
230
In summary •A model simplifies and clarifies complex ideas.
•Thinking isn’t useful until it’s out where people can see it.
• An effective model makes it easy to incorporate new information.
•A model diagram is a tool, not an end in itself.
•Avoid the temptation to make diagrams more pretty than useful.
•An effective model communicates without needing an audio guide.
231
Personas
232
A persona is “the user” made visible
233
Busy mom is the ur-persona.
234
I’ve never seen a persona called “Married woman, no kids, with pristine hardwood.” God, how I aspire to see that persona.
-Steve Portigal
235
They can still be useful
tools
236
Personas
• Distill ethnographic research • Document representative groups of needs and
behaviors as archetypes • Represent relationships among user types • Allow team to advocate for user needs • Act as a reference point for decision-making • Maintain empathy throughout design process
237
Personas must be based on actual research data
238
Fun, but not useful
239
A good model represents and simplifies knowledge
240
Practice!!241
MAKE A SIMPLE JOURNEY
Experience Journey Map | Task: Eating Lunch
242
Practice Exercise: User Journey Time: 15 minutes.
1.As a group, review your user interview notes to identify steps in gift purchase journey
2.Decide whether that step is positive or negative 3.Decide how much of a positive or negative 4.Fill in diagram
243
Time is up!6244
How did that go?
245
Reporting
246
2A useful report supports
Clear goals Shared values Access to information Confident decisions
247
2You decide the purpose
Informing? Inspiring? Focusing? Remembering? Recording? Deciding?
248
Research ReportStudy Title
Date Completed
Research Goal
Activities
Related Decisions
Key Insights
Supporting Observations
Recommended Actions
Questions for Further Study
Keep it as brief as you can.
249
Building a research-driven
culture250
Capture the value of research
251
Use your report to tell a story
252
How most people do itMethods
(what we did, usually in vast
detail)
Findings (what we found,
often disconnected
from biz goals)
Meaning (the
implications for our design
work)
Framing (How it
connects to the project story…
maybe)
zzzz
253
How you shouldMethods (a brief
summary up front, most as an appendix)
Findings (what we found, leading with the interesting bits)
Meaning (the
implications for your business)
Framing (Setting the stage with
context and a good story)
$$
$
254
Finish line!
255
In summary • Research creates a shared understanding of reality.
• Asking questions is uncomfortable. Embrace that feeling.
• A truly collaborative approach and environment is necessary for research to be effective, and it also makes it more fun.
• Clear goals and good questions are required.
• Choose only the research activities that answer real questions and inform your top priority design and development decisions.
• Practice! Observe and listen every day.
• Document! Report! Share! It’s easy to lose what you learn.
256
Any questions?
257
Thank you!
258