‣Evgenia (Jenny) Grinblo ‣UX Specialist, Future Workshops
USABILITY TESTING: THE DIY APPROACH
!1
AGENDA !2
BY THE END OF THIS SESSION, YOU WILL BE ABLE TO...‣ Plan a usability test ‣ Facilitate a test and identify usability problems ‣ Decide which usability problems to tackle first
INTRODUCTION
‣ Trained in ethnographic research ‣ Work with 12 devs, multiple projects ‣ Introduced usability testing inside our process
EVGENIA (JENNY) GRINBLO UX SPECIALIST, FUTURE WORKSHOPS
!3
INTRODUCTION !4
http://bit.ly/1f9Pnzt
OUR STORY: BRIDGING THE GAP !
!
!CLIENTS USERS
!!Intuitive!!Simple
!Designed for
everyone
!Unique
!Easy to use
INTRODUCTION
AND YOU..?
!5
‣ Name ‣ Company / role ‣ What do you want to get out of this workshop?
AGENDA !6
!How to run a £3000 fool-proof test
✓ The easiest, quickest, cheapest methods ✓ Tips on facilitating, note-taking, & prioritising ✓ How to run a usability test tomorrow ✓ Resources & tools to keep learning !!
WHAT WE’LL COVER
AGENDA !7
HOUSEKEEPING‣ You don’t need laptops yet ‣ These slides will be on SlideShare.net/JennyEvgenia ‣ Questions always welcome ‣ Short break after this part ‣ If you need to leave early, please tell me now
USABILITY TESTING: THE DIY APPROACH
0 / USABILITY TESTING: WHAT & WHY 1 / PREPARING A TEST PLAN
2 / RUNNING A USABILITY TEST
3 / TURNING DATA INTO ACTIONS
!8
USABILITY TESTING: WHAT & WHY
USER EXPERIENCE BUILDING BLOCKS
http://www.flickr.com/photos/7855449@N02/2779597129 Diagram by David Armano
USABILITY TESTING: WHAT & WHY
http://www.flickr.com/photos/7855449@N02/2779597129 Diagram by David Armano
USABILITY IS A MAJOR BUILDING BLOCK OF A GOOD EXPERIENCE
USABILITY TESTING: WHAT & WHY
1. INTUITIVE (n.) Using what one feels to be true without conscious reasoning
www.youtube.com
USABILITY TESTING: WHAT & WHY !12
NEEDING INSTRUCTIONS MEANS SOMETHING COULD BE MORE SIMPLE.
USABILITY TESTING: WHAT & WHY !13
2. EASY TO USE Doesn’t make people feel stupid or inferior to the tool they are using. Helps them feel in control.
www.errorwallofshame.tumblr.com
USABILITY TESTING: WHAT & WHY !14
3. ACCESSIBLE People of different abilities can use it
USABILITY TESTING: WHAT & WHY !15
YOU CAN’T DIFFERENTIATE IF THE EXPERIENCE IS BROKEN.
USABILITY TESTING: WHAT & WHY !16
67.45% = AVERAGE E-COMMERCE CART ABANDONMENT RATE
http://baymard.com/lists/cart-abandonment-rate
July 28, 2013
USABILITY TESTING: WHAT & WHY !17
http://www.shopify.com/blog/8484093-why-online-retailers-are-losing-67-45-of-sales-and-what-to-do-about-it
USABILITY TESTING: WHAT & WHY !18
http://www.shopify.com/blog/8484093-why-online-retailers-are-losing-67-45-of-sales-and-what-to-do-about-it
USABILITY TESTING: WHAT & WHY !19
Analytics (Impersonal)
Survey (Tricky)
Usability test (Immediate, easy)
DIFFERENT WAYS TO FIND PROBLEMS
USABILITY TESTING: WHAT & WHY !20
USABILITY TESTING BENEFITS
http://bit.ly/1f9Pnzt
‣ Behavioral vs. Self-reported
‣ Opportunity to ask “why?”
‣ Motivates teammates
‣ Removes “designer bias”
‣ Can be done on prototypes
‣ Can be cheap & quick
USABILITY TESTING: WHAT & WHY !21
WHAT HAPPENS IN A USABILITY TEST?Video: http://www.youtube.com/watch?v=QckIzHC99Xc
USABILITY TESTING: WHAT & WHY !22
USABILITY TESTING SETUP
‣ Who is there
‣ Equipment and setup
‣ Space
‣ Context
USABILITY TESTING: WHAT & WHY !23
MOST IMPORTANT: TEST YOUR IDEAS REGULARLY WITH REAL PEOPLE.
USABILITY TESTING: WHAT & WHY !24
Recruit Participants
Prepare a test plan Test Analyse +
iterate
USABILITY TESTING: STEPS
USABILITY TESTING: WHAT & WHY !25
HOW TO CHOOSE PARTICIPANTS
‣ Anyone who hasn’t seen the app
‣ Similar demographics to real users (whenever possible)
USABILITY TESTING: WHAT & WHY !26
CRITERIA TO CONSIDER
‣ Demographics: age, gender, tech ability
‣ Behaviour: new/ experienced users, habits, loyalty
‣ Mindset: angry customers, feel strongly about social media, insecure about technology, philosophy about saving
USABILITY TESTING: WHAT & WHY !27
‣ Friends, family, colleagues
‣ Social media followers
‣ Existing customers
‣ Streets and coffee shops
‣ Recruitment agencies (£££)
WHERE DO WE FIND PARTICIPANTS?
USABILITY TESTING: WHAT & WHY !28WHEN DO WE TEST? AS EARLY AS POSSIBLE
Idea stage Definition stageExploring FocusingMany ideasHigh-level“Dream world”
Best ideasRefinedRealistic
USABILITY TESTING: WHAT & WHY !29
DO USABILITY TESTING REGULARLY, WITH YOUR WHOLE TEAM
USABILITY TESTING: WHAT & WHY !30
The number of team members exposed to users trumps the number of users you test with
http://www.uie.com/articles/user_exposure_hours/
USABILITY TESTING: WHAT & WHY !31
SOMETIMES IT’S HARD TO ACCEPT YOUR PRODUCT NEEDS HELP
https://twitter.com/adrianhungate/status/391954628184915970
USABILITY TESTING: WHAT & WHY !32FACE-TO-FACE “MAGIC:” WHEN YOU SEE SOMEONE IN PAIN, YOU WANT TO HELP THEM
http://www.flickr.com/photos/martin_mcdonald/371884786/
USABILITY TESTING: WHAT & WHY !33
— DANIEL Lead Developer, Future Workshops
“!
I was completely surprised at how cumbersome the users experience with the app was. Basic tasks were which was so obvious to me were suddenly a huge barrier.
USABILITY TESTING: WHAT & WHY !34
— DANIEL Lead Developer, Future Workshops
“!
... What the usability testing did for me was really prove how much tech people can lose sight of who they are building for. It about going back to basics in some cases.
USABILITY TESTING: WHAT & WHY !35
IN SUMMARY: USABILITY TESTING BENEFITS
1. Highlights where our product isn’t intuitive / easy to use
2. Puts team in the shoes of users
3. Helps sell product improvements to stakeholders/ clients
4. Helps prioritise feature backlog
!
USABILITY TESTING: THE DIY APPROACH !36
5 MINUTE BREAK
USABILITY TESTING: THE DIY APPROACH
0 / USABILITY TESTING: WHAT & WHY
1 / PREPARING A TEST PLAN 2 / RUNNING A USABILITY TEST
3 / TURNING DATA INTO ACTIONS
!37
PREPARING A TEST PLAN !38
To run a test, you need
‣ Some representation of your product
‣ Preferably interactive
‣ Doesn’t have to be finished
‣ Or: competitor product!
‣ Idea of what you want to test
PREPARING A TEST PLAN !39
What do we test?
‣ General impressions
‣ Ease of use of critical features
‣ Features we’re not sure about (e.g., is the help clear?)
‣ Response to design, iconography
Tip: start from high-level when testing prototypes, test the details with later builds
PREPARING A TEST PLAN !40
Wide (Generative)
Narrow (Validating)
WHY will people use this product?
SET THE SCOPE
HOW do these people use this product?
PREPARING A TEST PLAN !41
Wide (Generative)
Narrow (Validating)
SET THE SCOPE
Explore How is the value proposition of this tool fit in with people’s lives
Validate How do people perform specific task with this tool?
Confirm/iterate Observing what people do with this tool, in their own environment
PREPARING A TEST PLAN !42
Wide (Generative)
Narrow (Validating)
SET THE SCOPE
Explore How is the value proposition of this tool fit in with people’s lives
Validate How do people perform specific task with this tool?
Confirm/iterate Observing what people do with this tool, in their own environment
First...
PREPARING A TEST PLAN !43
FIRST IMPRESSIONS: THE 5 SECOND TEST
Ready?
PREPARING A TEST PLAN !44
PREPARING A TEST PLAN !45
• What is the purpose of the website/app?
• Name 3 things you can do with this
website
• Who does this site or app belong to?
• How do you feel about this organisation?
5 SECOND TEST QUESTIONS
PREPARING A TEST PLAN !46
Wide (Generative)
Narrow (Validating)
SET THE SCOPE
Explore How is the value proposition of this tool fit in with people’s lives
Validate How do people perform specific task with this tool?
Confirm/iterate Observing what people do with this tool, in their own environment
Now...
PREPARING A TEST PLAN !47
THE MORE DETAILED THE ARTIFACT, THE MORE DETAILED THE FEEDBACK.
PREPARING A TEST PLAN !48
PREPARING A DETAILED USABILITY TEST
Recruit Participants
Prepare a test plan Test Analyse +
iterate
KEY OBJECTIVE(S)
TIME
DELIVERABLE
STEPS
EXERCISE 1: CHOOSE A PRODUCT TO TEST
Choose an app or website to focus on today
10 mins !49
1. As a group, choose a website or app you will usability test
2. It can be your app/website but I must be able to interact with it
3. Avoid sites that require logging in (unless you have a login to share)
A group decision about an app/website
PREPARING A TEST PLAN !50
Test plan!
‣ Tasks: What are we testing? Why?
‣ Success Paths: What do we expect to happen?
‣ Scenarios: How can we find out if our expectation is true?
!
!
// SUCCESS PATH
// TEST PLAN FOR: // PART:// GROUP:
TASK
!
!
// SCENARIO
// GENERAL SCENARIO
// INFO (passwords, etc)
!// SUCCESS PATH
TASK
!
!
// SCENARIO
// INFO (passwords, etc)
@ T
empl
ate
by @
futu
rew
orks
hops
!
!
// SUCCESS PATH
// TEST PLAN FOR: // PART:// GROUP:
TASK
!
!
// SCENARIO
// GENERAL SCENARIO
// INFO (passwords, etc)
!// SUCCESS PATH
TASK
!
!
// SCENARIO
// INFO (passwords, etc)
Set up the motivation
What we are testing
What we want/expect the user to do
Motivation to do the task + instructions
Information necessary for the task
!
!
// SUCCESS PATH
// TEST PLAN FOR: // PART:// GROUP:
TASK
!
!
// SCENARIO
// GENERAL SCENARIO
// INFORMATION
!// SUCCESS PATH
TASK
!
!
// SCENARIO
// INFORMATION
You need to cook a new recipe for the company potluck. You found a recipe online and now you need to buy the ingredients and cook it.
You need to remember what to buy. Use the app to make a shopping list you can take to the supermarket.
Print-out of a recipe from the internet
Create a text note and make a bulleted list of ingredients
E-mail a note to a friend using swipe to share
Text note Share a note via e-mail
You really liked the new recipe and you think your friend Jenny will like it too.
Send the recipe to Jenny from within the app.
Jenny’s email: [email protected]
Don’t show this to the participant!
PREPARING A TEST PLAN !54
Scenario Structure !
‣ Your character
‣ Your motivation
‣ What you need to do
‣ Any details like login information, etc
!
// SCENARIO
PREPARING A TEST PLAN !55
Scenario Tips
‣ Avoid using terminology from the interface (e.g., search, tag, filter)
‣ Scenarios can have steps (e.g., find a recipe and add it to your collection)
‣ Escalate the difficulty as the test progresses
Tip: make the scenario specific enough so someone can relate to it, but not specific enough to exclude people
KEY OBJECTIVE(S)
TIME
DELIVERABLE
STEPS
EXERCISE 2: CREATE A TEST PLAN
Learn to write a non-biased test that covers the features you need tested
20 mins !56
1. Using the Test Plan Worksheet, define 3 key tasks that must be intuitive and easy to use in the application
2. Identify the success paths for each task (what we expect users to do)
3. Write specific tasks, then a general scenario
4. Run through the complete test once
Filled out test plan worksheet, ready to be used in a test with a participant
USABILITY TESTING: THE DIY APPROACH
0 / USABILITY TESTING: WHAT & WHY
1 / PREPARING A TEST PLAN
2 / RUNNING A USABILITY TEST 3 / TURNING DATA INTO ACTIONS
!57
RUNNING A USABILITY TEST !58
Recruit Participants
Prepare a test plan Test Analyse +
iterate
USABILITY TEST STEPS
RUNNING A USABILITY TEST !59
As a facilitator, your jobs are:
‣ “Tour guide”
‣ “Therapist”
‣ Burst the thought bubbles
http://books.google.co.uk/books?id=9Q3OQVyX_-QC
From: Rocket Surgery Made Easy by Steve Krug
RUNNING A USABILITY TEST !60
‣ Practice staying quiet
‣ Remind the participant to speak out loud
‣ Don’t give anything away
‣ Be reassuring, say thank you
‣ If they get stuck, wait (a little)
Facilitating
RUNNING A USABILITY TEST !61
Facilitator Cheat-SheetIf they forget to think aloud
‣ "What are you thinking right now?"
‣ "What are you finding confusing on this screen?"
‣ "I know it's an unusual thing to do but could I please ask you to keep thinking out loud and talk as you go along? It really helps me understand your experience better."
RUNNING A USABILITY TEST !62
Facilitator Cheat-Sheet
If they get upset
‣ "This is very helpful for us, we are uncovering a lot of problems in the design that will help us make the app better.”
‣ You’re struggling with the same areas that other participants found confusing - you’re totally normal
RUNNING A USABILITY TEST !63
Facilitator Cheat-Sheet
If they have a question
‣ "Please continue in the way that makes sense to you. We will learn a lot from seeing how you proceed with this without any help."
RUNNING A USABILITY TEST !64
Facilitator Cheat-Sheet
The golden go-to
‣ "We expected to uncover problems in the design. You are not doing anything wrong - we knew there would be issues and you are helping us find them."
RUNNING A USABILITY TEST !65
PEOPLE WILL FEEL STUPID. MAKE THEM FEEL AT EASE.
RUNNING A USABILITY TEST !66
Facilitator checklistIntroduce yourself and anyone else present
Thank the participant for coming
Explain why they’re here & what will happen
“We are not testing you”
Introduce thinking aloud
Explain questions, breaks, and ask if they have any questions
Optional: consent forms, recording, interview
RUNNING A USABILITY TEST !67
Ways to record our findings
From easy to complicated
‣ Pair up with a note-taker
‣ DIY recording equipment
‣ Professional recording equipment
‣ Live-stream to a group of stakeholders
RUNNING A USABILITY TEST !68
NOTE-TAKING BRINGS DOWN 2 BARRIERS TO TESTING: TIME & EQUIPMENT
(But I’ll give you some tips on recording later!)
RUNNING A USABILITY TEST !69
Note-taking tips
Only record what’s important
‣ If someone is really happy
‣ If someone is really frustrated
‣ If someone fails a task
‣ Write down short quotations
RUNNING A USABILITY TEST !70
Notes Example • Taps Map on merchant profile but this opens the system
Maps app — confused !• Goes to Search and types “coffee shops” but there are no
results — “why can’t I see what’s around me?”
!• "If I had to think about something around me, I’d start
the merchant profile and expect to see other things around me"
!• The icons aren’t telling me that Cafe Nero is a coffee shop
!!
RUNNING A USABILITY TEST !71
What to bring to the testPrinted scenarios & tasks for participant
Printed scenarios & tasks with success paths for facilitator
Facilitator script
Consent form (if needed)
Pen + paper / recording tools
Test setup checklist
Optional: List of pre-test and debrief questions
Tip: Prepare a printed copy of the scenarios & steps for the participant, they will use it
RUNNING A USABILITY TEST !72
Setting up the testReset the mobile app / site to the default settings
Clear the test area of any open documents, written notes, etc.
Prepare participant forms: scenarios, intro script, consent form
Optional: Verify the recording equipment
Optional: Make sure the brightness on the screen is reduced
KEY OBJECTIVE(S)
TIME
DELIVERABLE
STEPS
EXERCISE 3: USABILITY TEST
1. Gain experience facilitating
2. Collect notes containing usability problems
3. Identify weaknesses in your test plan
40 mins !73
1. Every round, swap one member of your group in exchange for a member from another group
2. The ‘foreign’ member is the participant
3. Start running through the test you’ve planned
4. Everyone in the group should try to be a facilitator at least once
A collection of notes detailing usability problems found in the product
DISCUSSION TIMEEXERCISE 3: USABILITY TEST !74
USABILITY TESTING: THE DIY APPROACH !75
5 MINUTE BREAK
USABILITY TESTING: THE DIY APPROACH
0 / USABILITY TESTING: WHAT & WHY
1 / PREPARING A TEST PLAN
2 / RUNNING A USABILITY TEST
3 / TURNING DATA INTO ACTIONS
!76
TURNING DATA INTO ACTIONS!77
Recruit Participants
Prepare a test plan Test Analyse +
iterate
USABILITY TEST STEPS
TURNING DATA INTO ACTIONS!78
What to do with all the data?
‣ Identify changes with the biggest impact & smallest effort as a team
‣ Find the “low-hanging fruit”
‣ What has impact on the bottom line?
‣ Put everything else in the ice-box
TURNING DATA INTO ACTIONS!79
Identifying key findings
High-priority: people fail, or abandon the task
Medium: people succeed but are frustrated and unhappy
Low: people would enjoy the app more if...
Tip: You can’t fix everything. Pair usability priority with business needs and product roadmap to find the fixes with the biggest impact.
TURNING DATA INTO ACTIONS!80
Presenting findings - an example
TURNING DATA INTO ACTIONS!81
Presenting findings
Interface screenshot here
Task
What we expected to happen
What really happened
What we can fix right now
KEY OBJECTIVE(S)
TIME
DELIVERABLE
STEPS
EXERCISE 4: IDENTIFYING KEY FINDINGS
1. Consider how to prioritise usability problems
2. Discuss and agree on priorities as a team
15 mins !82
1. Individually, write down 2 high-level and 2 medium-level usability problems on post-its
2. As a group, arrange your post-its from high to low priority.
3. Jot down a quick summary of key findings to present to the larger group
A group decision about an app/website
DISCUSSION TIMEEXERCISE 4: IDENTIFYING KEY FINDINGS !83
SUMMARY & CONCLUSION !84
Summary‣ Match your test artifact to your scope
‣ Plan realistic scenarios that don’t give away the test purpose
‣ Be encouraging and open to feedback
‣ Record only what’s needed
‣ Prioritise findings for maximum impact
SUMMARY & CONCLUSION !85
DIVE IN MORE DEEPLY
Remote Research Nate Bolt & Tony Tulathimutte !
DIY Usability Testing
Steve Krug
Lean UX Jeff Gothelf
See “Resources and Further Reading” printout for more
SUMMARY & CONCLUSION !86
REMEMBER THIS?
http://www.flickr.com/photos/7855449@N02/2779597129 Diagram by David Armano
SUMMARY & CONCLUSION !87
“
http://www.humanfactors.com/ROIposter.asp
Any intelligent fool can make things bigger and more complex. It takes courage, and a touch of genius, to move in the opposite direction.
– Albert Einstein
A FINAL THOUGHT
SUMMARY & CONCLUSION !88
Thank you! [email protected] www.futureworkshops.com Twitter: @grinblo / @futureworkshops !Questions + comments welcome!
!
GET IN TOUCH