3 Methods to Master Agile UX Testing | UserZoom Webinar

1!

Webinar: 3 Methods to Master Agile UX Testing

Presented by: Ann Rochanayon,

Senior Director of UX/CX Research, UserZoom

2!

Rapidly Test Usability & Measure UX

UserZoom is the All-in-One UX Research Platform for the Data-Driven Enterprise

3!

•  Chat box is available if you have any questions!

•  Look out for a quick poll question!

•  There will be time for Q&A at the end!

•  We will be recording the webinar for future viewing!

•  All attendees will receive a copy of the slides/recording!

•  Continue the discussion using #UZWebinar!

Quick Housekeeping

3 Methods to Master Agile UX Testing!

4!

Ann Rochanayon Sr. Director UX/CX Research

UserZoom

Meet Your Speaker


Continue the discussion using #UZWebinar

5!

Agenda •  What is Agile?!

•  UX Research Challenges in Agile!

•  3 Methods to Master Agile UX Testing!

•  Tips for UX Testing in Agile Development!

•  Questions!



6!

What is Agile?


•  Lightweight, iterative approach to software development that builds

software incrementally as opposed

to delivering it all at once at the end!

!

•  An alternative to Waterfall development or traditional,

sequential development!


7!

Agile vs. Waterfall


Source: http://www.business-software.com/blog/waterfall-vs-agile-development-differ-matters/


8!



9!

Benefits of Agile Development


Allows for change

Better risk management

Improved quality

Increased collaboration

Better visibility and transparency


10!

UX Challenges in Agile Development


•  Agile is a framework for software development

•  Does not necessarily take UX into account

•  Speed, not enough time for ‘formal usability testing’

•  May not have minimum viable product to test

•  Designers may focus on immediate needs within user stories, neglect bigger picture


11!

Addressing UX Challenges in Agile


•  Utilize remote testing techniques

•  Run quick, iterative tests

•  Copy prior studies, update and rerun

•  Use real-time automated reporting

•  Determine UX metrics to measure and benchmark the user experience

•  Observe how design changes impact UX Agile metrics


12!

Quick Poll



13!

3 Methods for Agile UX Testing


Screenshot Click Test Tree Test Task-based Testing


14!

Method 1: Screenshot Click Testing



15!

UX Research Needs in Agile

3 Methods to Master Agile UX Testing!!

Top Agile UX Pain-Points Top UX Testing MethodsDesign – Which design has the best conversion? A/B test with screenshot click testing or task-based

testing

Image / Ad testing – Which image / ad do users prefer? A/B test with screenshot click testing or task-based testing

Attention – What grabs users’ attention? Screenshot click test

Navigation optimization – Can users efficiently find content? Screenshot click test, tree test, task-based testing, card sorting

Usability – Can users complete key tasks on the site? Task-based testing

Nomenclature / Terminology: Do users understand the terminology / naming used on the site?

Tree testing

Information architecture – How do users expect to find information? How should content be organized on the site?

Tree test, card sorting

Brand impression – How do users feel about the brand? Timeout testing, screenshot click testing, task-based testing


16!



•  Determine findability of content

•  Find out where users click first

•  Evaluate first impressions

•  Understand what grabs their attention

•  Uncover UI flaws

•  Conduct A/B testing with two or more designs


17!

Method 1: Screenshot Click Testing Metrics


•  Success Rates

•  First click testing – Difficulty

o  How difficult or easy was it to find where to click on the site to find

information about _________?

•  First click testing – Confidence

o  How confident were you that you clicked the correct area to find

_______?

•  First impression – Emotional response, overall impression

o  What 3 words would you use to describe the page to a friend?

o  What is your impression of the page?


18!

Method 1: Screenshot Click Testing Metrics


•  Attention

§  Which area of the page first grabbed your attention?

•  A/B Testing – Comparison / Preference

§  Which homepage do you prefer?

§  Which version of the product details page best presented the product features?

§  Which version best presented clickable links?

•  Include images of the screenshots in the question to aid with recall•  Give the versions neutral name; avoid old vs. new or names that imply order•  Ask follow-up questions to understand why•  Include a “No Preference” or “None of the above” option

Tips for Preference Questions

19!




20!

Method 1: Screenshot Click Test Results


Click Effectiveness / Click Success Rates


21!



Heatmaps


22!



Areas Clicked


23!

Method 2: Tree Testing



24!




testing






Tree testing





25!



•  Test information architecture or navigation without visual cues, distractions, and

search

•  Measure the effectiveness of a site

structure

•  Evaluate the findability of content

•  Determine how to organize content


26!

Method 2: Tree Testing Metrics


•  Success rate - % of participants who successfully select the right item from

the navigation tree

•  Efficiency – Success after one attempt vs. multiple attempts

•  Time – Amount of time spent determining which item to select

•  Confidence

§  How confident were you that you selected the right item?

•  Ease of use

§  How difficult or easy was it to determine where you would click to find _____?


27!




28!

Method 2: Tree Test Results


% Clicked

% Selected

User Clicks


29!



Click Effectiveness / Success Rates, Time


30!



Click Summary


31!

Method 3: Task-Based Testing



32!




testing






Tree testing





33!



•  Test navigation, design

•  Test wireframe w/ hotspots, low fidelity prototypes

§  Enable links users are likely to click to track clickstreams

§  Must be hosted and web accessible

•  Test user stories or tasks


34!

Method 3: Task-Based Testing Metrics


•  Task completion / success – Defined by researcher

•  Task efficiency – Time on task, Unique page views, # of clicks

•  Task Difficulty / Ease of use

§  How difficult or easy was it to customize a _______?

•  Satisfaction

§  Based on your experience with the prototype, how satisfied were you with the process of

customizing _______?

•  Conversion

§  How likely are you to purchase from this site?


35!




36!

Method 3: Task-Based Testing Results


Quantitative: Task Effectiveness (Task Success)


37!



Quantitative: Task Efficiency (Number of Clicks, unique page views, time on task


38!



Quantitative: Rating scales, Net Promoter Score (NPS)


39!



Qualitative: Open-ended Questions


40!



Qualitative: Videos of participants’ screens with video and/or audio


41!



Behavioral: Heatmaps


42!



Behavioral: Clickstreams


43!



•  Shop Direct – UK’s leading multi-brand digital retailer with annual sales of £ 1.7 billion

•  Challenge: Lab-based research indicated that customers were annoyed with the checkout process

•  Goal: Make checkout easier and improve conversion

•  Solution: Conducted a series of remote task-based studies A/B testing two different HTML prototype designs

•  Results: Identified and resolved 54 usability issues prior to release. Increased conversion by 5.7%.

Method 3: Case Study – Task-Based Testing

44!

Other Methods


•  Card Sorting

•  Understand users’ mental models for grouping content

•  Use to inform site’s information architecture

•  Timeout Testing / 5 Second Test

•  Learn how users feel about your webpage design

•  Gain insight into their emotional response

•  Determine if they understand the value proposition


45!

Tips for Agile UX Testing



46!



•  Include a UX professional as part of the agile team

•  Research should be conducted 1 or 2 sprints ahead of development

•  Consider setting up a parallel track for UX


47!



Source: http://uxmag.com/articles/the-rite-way-to-prototype

•  Consider using RITE (Rapid Iterative Testing and Evaluation)

•  Don’t need high fidelity prototype or

working code

•  Use low fidelity prototype, “paper

prototypes”, PowerPoint mockups,

Simple black and white wireframes with HTML


48!



•  Have a recruitment plan

§  Consider utilizing customer list, customer council / online community, private

panel of users, or outside panel

•  Take advantage of online reporting / data visualization


49!

Additional Resources


•  Webinars•  How to Integrate UX into an Agile Environment

http://info.userzoom.com/integrate-agile-ux.html?source=Website&llr=resources&medium=&olr=resourcesDean T. Barker, Sr. Director of User Experience at Optum

•  How to Master UX Testing in an Agile Design Processhttp://info.userzoom.com/how-to-master-ux-testing-in-an-agile-design-process-webinar.html?source=Website&olr=resources&llr=resourcesJon Innes, ‘The Agile UX GURU’ at UX Innovation LLCJeff Sauro, ‘The UX Measurement MASTER’ at Measuring Usability

•  Formative Usability Testing in Agile: Piloting New Techniques at Autodeskhttp://info.userzoom.com/formative-usability-testing-webinar.html?source=Website&olr=resources&llr=resourcesEunice Chang, Sr. Principal User Researcher at AutodeskOlivia Williamson, Principal User Experience Designer at Autodesk


50!

Additional Resources


•  Articles•  Waterfall to Agile UX: How Autodesk Made the Switch

http://www.userzoom.com/agile-ux-research-and-testing/waterfall-to-agile-ux-how-autodesk-made-the-switch/•  How Shop Direct Improved their Conversion Rates with UserZoom•  http://www.userzoom.com/successstory/shop-direct/•  Dispelling 4 Common Agile UX Myths

http://www.userzoom.com/agile-ux-research-and-testing/dispelling-4-common-agile-ux-myths/Jen Kirschniok, Digital Experience Expert UserZoom

•  Agile & Remote Usability Testing: A Love Storyhttp://www.userzoom.com/agile-ux-research-and-testing/agile-remote-usability-testing-a-love-story/

•  12 Best Practices for UX in an Agile Environment, Part 1 and 2http://www.uie.com/articles/best_practices/, https://www.uie.com/articles/best_practices_part2/Jeff Patton, AgileProductDesign.com

•  eBook•  The Essential Guide to Integrating User Experience and Usability Testing in Agile

http://info.userzoom.com/agile-user-experience-guide.html?source=website&olr=resources&llr=resourcesDean T. Barker, Sr. Director of User Experience at OptumJon Innes, ‘The Agile UX GURU’ at UX Innovation LLCJeff Sauro, ‘The UX Measurement MASTER’ at Measuring UsabilityEunice Chang, Sr. Principal User Researcher at AutodeskOlivia Williamson, Principal User Experience Designer at Autodesk

51!

Questions


52!

Need more info?

Get answers about UserZoom. We are here to help. !Talk with one of our knowledgeable specialists.!

Contact us

UserZoom 1054 South De Anza Blvd, Suite 102

San Jose, CA 95129 Toll free number: (866) 599 1550 !

www.linkedin.com/company/userzoom @UserZoom!Follow us!

3 Methods to Master Agile UX Testing | UserZoom Webinar

Technology