Gaining design insight through recruiting research participants

Gaining design insights from your research recruiting process

Dana Chisnell@danachis

Lesson learned

How long?

How often?

How bad?

Medications?

Employed?

Insurance?

TERMINATE (interview).

Recruiting as user research: bonus data

Recruiting as user research: conversational interviews

How do I get started?

Where do I find people?

What do I do with old customer data?

Why is recruiting so time-consuming?

How can I stop no-shows?

• Sourcing

• Screening

• Scheduling

• Compensating

Steps to recruiting happiness

keep it in house

bond closely with consistent resource

Do it yourself.

Include the recruiter in the study planning

Objective of the study

Format of the sessions

Behavior to be observed

This is part of the customer experience

First contact

Recap:

• Recruiting is part of the research.

• You should do it yourself.

Snowball recruiting can be more effective than your database

1. Sourcing

• Snowball

• Online social networks

• Churches, school groups, social clubs

• Professional associations

• User groups, conventions, and conferences

• Support groups

• Intercepts

• Craigslist

Sources

Panels

• Can take time to get a pull from a company database

• Customer data is out of date

• People have to opt in

• Some make a living responding to surveys and doing studies

Biases: Every sample has a bias. Mix them up.

The art of the open-ended interview.

2. Screening

The art of the interview

• This is not a test.

• It’s a conversation to learn about the person

• Frame the conversation based on the study objectives

• Get the respondent to volunteer information

Be open

• “tell me about...”

• “when was the last time you...”

• “say more about that...”

• “what’s the thing you like most about x...?”

Know who shouldn’t be there

• Create a question that will show a faker is a faker

• Be clear about who you don’t want in your study

• Demographics don’t predict behavior

• Focus on actions you want to observe

Behavior versus demographics

• Regulations may bound age ranges

• Beware targeted segments

Behavior versus demographics

plays first-person shooter games

downloads movies

applies for benefits

files claims

buys groceries

books hotels

shares photos

• Attributes that may indicate differences in behavior

Classifiers

Set expectations • Recording

• NDAs

• Homework

• Possible personally identifying information

• Using their device or yours

• Alone or with other participants

• Being observed

NameBooks own travel?

Business trips in 12 mo

Pleasure trips in 12 mo

Terry Y 25 2

Pat N 30 5

Tracy Y 10 1

Keith Y 9 2

Leslie Y 15 1

Ari Y 2 2

Kelly N 50 0

Erin Y 5 5

More hints

• Ask all the questions

• You’ve just started a relationship

• Respondents get invested

So far• Treat recruiting

like research

• Recruiting is first contact

• Every sample has bias

• Use networks to find sources of participants

• Focus on behaviors

• Screening establishes a relationship

Coming up• Scheduling

• Compensating participants

• Case studies

BreakQuestions & Answers

• Scheduling

• Compensating participants

• Case studies

It’s all about tradeoffs.

3. Scheduling

Where = timing

• Remote sessions offer the greatest flexibility

• Visiting participants can make them more available

• Going to you is actually the most time-consuming for participants

Be as generous as possible.

4. Compensation

Compensation, not incentive

• Compensate as soon as possible

• Cash is ideal

• Gift cards can work

• Licenses or subscriptions

• Donations

Pay or gift?

Remember to thank the people who helped you find participants, too.

Send a thank-you note.

Gaining design insights from the research recruiting process

• People with epilepsy who track symptoms and meds

Case 1: Usability test of a health monitoring app

QuestionsWhat’s it like to have this condition? Have you always had it? How do you cope? Tell me about meds.

SurpriseA lot of the people we wanted to meet were not patients. They were caregivers.

Case 2: Interviews about travel experiences

• People who travel between two major cities

• A mix of business travelers, pleasure travelers, and weekenders

QuestionsTell me about the last time you made the trip.[If needed] Why did you make the trip? [If needed] Who were you with? [If needed] How long did it take? [If needed] How often do you make this trip?

SurpriseIt’s not just the purpose of the trip, it’s the people you’re with.

Case 3: Usability test of a hotel booking site

• People who stay in hotels when they travel

QuestionsTell me about your last trip. Where’d you stay? How did you decide where to stay? How did you make a reservation?

SurpriseWe actually wanted people who book their own hotel rooms.

Summary• Do it yourself or include the recruiter when

you plan your study

• Continuous, snowball recruiting pre-qualifies & expands the sample

• Biased samples aren’t all bad

• Open-ended, voice-to-voice interviews are key to show rates

Summary• Open-ended screening reveals nuances

that can bust your assumptions

• You may lose real users if you aren’t

• open

• flexible

• attentive

Bonus tip!

• Confirm the appointment by email

• Remind the participant by email

• Remind the participant by phone

usabilitytesting.wordpress.com