HXR 2016: Don't Fear Small Numbers: Why a Few Design Research Participants Can Provide Big Insights - Dan Berlin & Susan Mercer, Mad*Pow

#HxR2016

Don’t Fear the Small Numbers

Why a Few Design Research Participants

Can Provide Big Insights

Dan BerlinVP, Experience Research, Mad*[email protected]@banderlin

Susan MercerExperience Research Director, Mad*[email protected]@susanamercer

#HxR2016

Topics

Prevalent requests for large numbers of participantsHow small numbers provide big insightsDetermining the “right” number of participants

#HxR2016

Large vs. Small Numbers

#HxR2016

Design Research isn’t…Market Research

#HxR2016

#HxR2016

But we’re not in the Mad Men days any more

#HxR2016

Good design follows an iterative process

#HxR2016

Design Research studies aren’t…Clinical Trials

#HxR2016

#HxR2016

Design Research

#HxR2016

In-home study

#HxR2016

Operating Room Study

#HxR2016

Five to eight users for usability

finds 80-90% of problems

Proportion of Usability Problems Found by Number of Participants

#HxR2016Source: https://www.nngroup.com/articles/which-ux-research-methods/

UX Research Methods

#HxR2016

Find 85% of the problems, update design, repeat

#HxR2016

Small Numbers

#HxR2016

UX Research is Typically Directional

#HxR2016

We conduct hundreds of interviews each year

#HxR2016

Trends emerge after a few participants

#HxR2016

Outliers provide incremental data during qualitative research

#HxR2016

Usability studies are qualitativePositive: Users were able to find the “Buy Now” button.

3 out of 8 participants mentioned that the picture had no relevance to them.Consider: replacing this with an infographic or another picture that may draw in users.

6 out of 8 participants mentioned that the text in the buttons was hard to read.Consider: increasing the contrast of the text to ease readability.

#HxR2016

Priority Finding RecommendationMedium Participants said that the text was hard to read Increase the contrast of the text

High Participants were unable to locate the product; they said they expected it under Products & Services

Move XYZ product to the Product & Services area

High Participants were unable to discern the top from the bottom of the vial

Include a visual cue as to which is the top and bottom of the vial

Medium Participants were able to figure out how to use the insulin pump, but said that the instructions were very hard to follow

Add detail throughout the instruction manual

Typical Usability Study Results

#HxR2016

The “Right” Number of Participants

#HxR2016

Keep Design RFPs Open-Ended

#HxR2016

http://bit.ly/IndiPersonas

Demographics don’t indicate reasoning

#HxR2016Source: http://howiechang.com/post/3045331253

Segment Users by Needs

Transactional Informational

#HxR2016

Project Scope Influences Number of Participants

#HxR2016

Segment Larger Projects

Application

Features Navigation Content

#HxR2016

• 12-16 participants per project segment & user group

Discovery Interviews

• 8-10 participants per project segment

Usability Studies

Very General Rules of Thumb

#HxR2016

Why? To Maximize ROI on UX Research

#HxR2016

Thank you!Dan Berlin@[email protected]

Susan Mercer@[email protected]