…if you know what you are doing Risto Sarvas Futurice & Aalto University INUSE seminar 6/12/12 User Studies Are a Waste of Money!
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
…if you know what you are doing J
Risto Sarvas Futurice & Aalto University
INUSE seminar 6/12/12
User Studies Are a Waste of Money!
› Head of UX & Service Design at Futurice.
› Docentship at Aalto University (Media Technology).
› A decade at HIIT.
HELSINKI
TAMPERE
BERLIN
LONDON
150 Founded in
2000
1
Futurice in brief
DÜSSELDORF
We offer tailored solutions done with
agile methods.
Overview
› How to spend the UX budget?
› What to focus on if not on user studies?
› Discussion points
20 000 € …to make a newsreader for iOS.
How should we spend it?
Budget breakdown
› Development
› Estimated 12k€
› Management
› Estimated 3k€
› UX design & research
› What is left is… 5k€.
Typical choices for user research. » Focus groups
» Observations
» User interviews
» PD workshops
» Stakeholder workshops
» etc.
» 1 FG, 2 desgnrs, 4 days of work = 6400€
» 5 usrs, 1 designer, 5 days of work = 4000€
» 5 usrs, 1 designer, 5days of work = 4000€
» 1 workshop, 2 designers, 3 days = 4800€
» 1 workshop, 2 designers, 3 days = 4800€
» Our UX budget is 5000€ and above we have allocated no design yet!
One person day = 800€ in budget
“Let’s cut the developmet
budget?”
› Which is more critical a working software or UX? Really.
› Interviewing 5 people would be 33% of the development budget.
“Let’s scope it down to make
it possible”
› Sure… but does that really save money here?
› User studies do not scale down: Running focus groups or workshops takes few days each anyway.
› We have to scope this down anyway to do it only with 20k€.
“The UX budget is too small!”
“We should not
accept this project!”
› Yes the UX budget is too small.
› Nevertheless, that is what the customer was willing to pay. This is their budget.
› Should we take the project, then?
› Can we make the newsreader with a 5k€ UX budget? With no user research!
11 6/12/12 Futurice
Should we do the UX for 5k€?
In ~6 days?
› Is this the first newsreader app in the world?
› Have we done newsreader apps before?
› Does the customer have this already done on some other platform?
› Are we users ourselves?
› Is this a familiar customer?
› What features does the customer want? How simple is it?
› Are we familiar with the platform UI guidelines?
World of a difference: ~5k€ ~50k€
Porting an existing app. Creating from scratch.
“You guys did a great job with the Android newsreader for our company. Could you do the same reader for iOS as well? It will be for public use to accompany our newspaper.”
“We want something previously unseen in the world of newsreaders. We do not know what it is. It will be a tool for stock market analysts and investors. This would be a good first project between our companies.”
Our 25 latest mobile cosnumer apps are rated 4.2 in average. Amount of user studies varies, but it is never what schoolbooks would expect.
How is it possible to make successful software without emphasizing user
studies?
Adapted from Hyysalo 2011.
Existing cultural genres
Eliciting user reqs
User participa
tion
Designer as user
Designer as
professional Tech &
components
What shapes the product, and its UX design?
• Design for myself • Common sense • Social distance
• Professional experience
• Existing trends • Shared visions among
designers
• Existing UI components • Existing platform APIs • Existing content
• UI genres • Metaphores and
associations • Use practices
• Lead users • Participatory design • User testing
• Observation • Literature • Existing studies • Focus groups
Adapted from Hyysalo 2011.
Existing cultural genres
Eliciting user reqs
User participa
tion
Designer as user
Designer as
professional Tech &
components
What shapes the product, and its UX design?
Do we have specific user domain knowledge?
Does the customer know their customers?
How experienced is our designer? Have we done these kinds of
products/services? Are we learning enough?
What is the platform? What is the back-‐end?
What are the UI guidelines? What are the customer’s brand
guidelines?
Is this rocket science? What is the “metaphor” of the
design? Are we using well-‐known
conventions?
How much does it cost? Is it worth it?
User testing is often worth it. Are we gathering user data
after launch?.
Bechmark competition and reverse engineer user reqs. Ask the customer to tell
about the users.
18
New ideas
Concept ready
Familiar domain
Unknown domain
Specific users
Consumers
No user data
Plenty of user data
Customer knows agile
Customer new to agile
Single functionality
Complex service
Small organization
Large organization
Strategy documents
Gfx for coding
0 1 2 3
4 5
5 4 3
2 1
1 2 3
4 5
5 4 3
2 1
0
19
New ideas
Concept ready
Familiar domain
Unknown domain
Specific users
Consumers
No user data
Plenty of user data
Customer knows agile
Customer new to agile
Single functionality
Complex service
Small organization
Large organization
Strategy documents
Gfx for coding
0 1 2 3
4 5
5 4 3
2 1
1 2 3
4 5
5 4 3
2 1
“ERP app”
0
“Mobile app”
20
New ideas
Concept ready
Familiar domain
Unknown domain
Specific users
Consumers
No user data
Plenty of user data
Customer knows agile
Customer new to agile
Single functionality
Complex service
Small organization
Large organization
Strategy documents
Gfx for coding
0 1 2 3
4 5
5 4 3
2 1
1 2 3
4 5
5 4 3
2 1
0
The 20k€ newsreader
› Is the designer also a user?
› Has the designer done newsreaders?
› Has the designer worked with this customer?
› Does iOS have clear guidelines?
› Does the customer have clear visual & brand guidelines?
› Is a newsreader app a common product?
› Can we re-‐use the design from existing products?
› Can the customer give us good feedback on behalf of the end-‐users?
› Can we do the UX design for 5000€?
› Sure we can!
Key points
› User studies are expensive. They are not always worth the cost.
› Take advantage of other factors shaping the UX:
› The designer as a user.
› The designer as a professional with experience.
› Existing UI patterns, guidelines & components.
› Obvious use patterns and practices.
› Consulting other people who know the users (e.g., product owner)
User Studies are a Waste of Money… …under specific circumstances.
Discussion But what next?
Discussion point 1: We need more understanding on what
shapes the design of technology and how to apply that into design.
Domain familiarity
Expe
rienc
e of designe
r
# of features
ERP for Z
Web page for G
Online service for X
Online service for WZ
ERP for X
Mobile app for XY
Mobile app for ZZ
Mobile app for ZZ
ERP for XXY
ERP for XXY
Enterprise system for XX
Enterprise system for XY
The Multi-‐Dimensional Space
Theory Practice • Social construction of technology • Ethnography, ethnomethods • Qualitative & quantitative approaches • Use practices, domestication of tech • Design research
• Empirical data on projects • Case studies • Real world context • Views of professionals • Reality check
This should be a rapid cycle!
Discussion point 2: Real user knowledge is created after launch.
From waterfall to agile/lean.
6/12/12 29
Development
Business objectives
Research & Design
Maintenance
x months
User studies as guesswork
Several user studies based on actual use
Related Documents
