Guerrilla Usability Testing
Nov 01, 2014
Guerrilla Usability Testing
Usability"Good design is problem solving"
Jeffrey Veen(Analytics, Typekit, author...)
Book: Psychology of everyday things
by Donald Norman
What’s Usability
• Usable, "easy to use". It's a very intuitive and simple concept, yet many companies don't do it (officially)
• Like CT, but ABE one of the simplest Booking Engines.
• User Centered design. It has to be self-explained.
• Technology shouldn’t make us feel stupid, neither force us to learn complicated stuff. “Don’t Make me Think”, book by Steve Krug
• Jakob Nielsen : Focus on monetizing
Example 1 : Useit.com
Example 2 : Craiglists, another dull yet successful (usable) site
Example 3 : Amazon
Example 6 : Apple
and more; Google, Youtube...
Example 7 : ABE step 1. 3 simple steps (not just good for marketing)
ABE Step 2 grid layout. Not the most common layout, but it works.
ABE Step 3
Why things are unusable?
• Development focus on the machine
• It's hard to achieve
• Teams don't work integrated
• Design vs implementation
• Developer’s lack of perspective.
• like...
Unusable: Irishrail.ie : Complex, uses its own language...
Why things are usable?
• Early focus on users and task. Interactive Design
• Prototyping and wireframing
• Evaluation and measurement
• Research, Surveys, Observation
• like...
Usable : Dropbox
Analysis
Design
Implementation
Evaluation
Development process stages
Usability TestingUsability testing is a technique used to evaluate a product by testing it on users...
Wikipedia
Usability testing generally involves setting a series of tasks for people to complete and noting any problems they encounter
Andy Budd
Usability Testing Lab
Benefits on usability testing
• Help inform the design process
• Uncover unknown or unexpected issues
• Fix problems early (easier)
• Test assumptions
• Highlight unnecessary features
• Provide Objectivity solve opinion battles
• Set baselines and measure improvement
For designers For site owners
Andy Budd
• Improve customer satisfaction and retention
• Increase conversion rates
• Reduce maintenance, trainning and support costs
• Reduce project risk
• Highlight unnecessary features
• It’s so cheap you’d be stupid not to
Effective ways of testing
• Eye tracking. Heat maps
• A/B Testing
• Formal testing (made by a company, lab, focus groups...)
• Guerrilla style
Eye tracking: Heat map
A/B Testing : Used by Argus to implement Lightbox
A/B Testing : Used by Argus to implement Lightbox
“The results were encouraging: after 3 weeks of testing, the average landing page conversion rate had gone from 2.70% to 3.33% (an average rise of 20% in the conversion rate)
Following the positive impact of the test, the light box functionality was rolled out across all PPC landing pages, leading to marked increase in conversions (+19%)
This was then rolled out on to the rest of the site leading to an improvement of 5.99% in the conversion rate for all traffic mediums”
(Thanks Dave)
Guerrilla Usability TestingGuerrilla warfare is the unconventional warfare and combat in which a small group of combatants use mobile tactics in the form of ambushes and raids to combat a larger and less mobile formal army.
Guerrilla usability testing
• Get somebody from the street, and ask them to do simple task, observe and record (using Camtasia or Silverback)
• Affordable (small rewards), informal. Everybody can help or bring somebody
• Small group of people (5 get 85% issues, 12 get 99%)
• Moderator and “Think Aloud” protocol
• Precise task definition
• Using tools for metrics like analytics, surveys and feedbacks...
Pros and Cons of Guerrilla style
• Help inform the design process
• Uncover unknown or unexpected issues
• Fix problems early (easier)
• Test assumptions
• Highlight unnecessary features
• Provide Objectivity solve opinion battles
• Set baselines and measure improvement
PROS CONS
• Improve customer satisfaction and retention
• Increase conversion rates
• Reduce maintenance, trainning and support costs
• Reduce project risk
• Highlight unnecessary features
• It’s so cheap you’d be stupid not to
Silverback
How does it work
Gerrilla Testing
• We did it for the first time in September 2010 before push live the new Step 3
• We found just a few issues; actually step 1: Magic locations
• For a moment we though, Is this really working?
Results
Gerrilla Testing
• Then we did it again, in the process of developing the map, this time we did it earlier stage.
• Very different results, tough but positive feedback. They liked the idea.
• Things to work on and improve. But at least we knew what to focus on.
2nd Testing: The Map
• Pick up locations, most didn’t know they could put the address
• Zooms and controls
• Some didn’t see “I’m here”, confused the best price icon...
• Missing elements like scales
• Ignoring funcionalities, (Legend, Sort by, Refres)
• Didn’t fully understand which car was which flag in the map
Issues we found
Gerrilla Testing
Video
Thanks
questions?
Work in progress...