Top Banner
Collaborative Research with Erika Hall
227

Collaborative Research

Aug 11, 2014

Download

Design

Erika Hall

Slides from my Collaborative Research workshop from Webstock 2014.

Buy the book: Just Enough Research
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Collaborative Research

Collaborative Research

with Erika Hall

Page 2: Collaborative Research

Hello!

Page 3: Collaborative Research

I have a question...

Page 4: Collaborative Research

Do you enjoy being right?

Page 5: Collaborative Research

You are correct!

Page 6: Collaborative Research

Yessss!

Page 7: Collaborative Research

?

Page 8: Collaborative Research

Flickr/Chris Voll

Page 9: Collaborative Research
Page 10: Collaborative Research

Ego!

Page 11: Collaborative Research

Assuming!=

Determining

Page 12: Collaborative Research

Research!

Page 13: Collaborative Research

Agenda

Page 14: Collaborative Research

Agenda

9–10:30

10:30–11

11–12:30

12:30–1:30

1:30–3

3–3:30

3:30–5

Introduction. Collaboration and

Research. Forming Your Questions.

Morning Tea

Activities. Interviewing.

Lunch

Analysis & Models

Afternoon Tea

Analysis & Models Contd. Reporting

& Sharing.Wrap-up

Page 15: Collaborative Research

Flickr/Jerome Collins

Page 16: Collaborative Research
Page 17: Collaborative Research

Design-Led

Research-Led

ExpertMindset

ParticipatoryMindset

Users seen as subjects

Users seen as partners

Design-ledwith

expert mindset

Design-ledwith

participatory mindset

Research-ledwith expert

mindset

Research-ledwith participatory

mindset

Dubberly Design Office

Page 18: Collaborative Research

Goal Driven

Skeptical Mindset

Increase chance of success

Reduce risk

Willing to question the value of any approach

Page 19: Collaborative Research

Collaboration

Page 20: Collaborative Research
Page 21: Collaborative Research
Page 22: Collaborative Research

Dogma!

Page 23: Collaborative Research

How?

Page 24: Collaborative Research
Page 25: Collaborative Research

Chris Noessel

Page 26: Collaborative Research
Page 27: Collaborative Research

Why?

Page 28: Collaborative Research

Goals

Page 29: Collaborative Research
Page 30: Collaborative Research
Page 31: Collaborative Research

LeanUX Principles:

Design thinking

Agile methods

Lean startup method

Page 32: Collaborative Research

Extreme Uncertainty

Page 33: Collaborative Research

Design Thinking

Page 34: Collaborative Research

“Design thinking is a human-centered approach to innovation that draws from the designer's toolkit to integrate the needs of people, the possibilities of technology, and the requirements for business success.”

– Tim Brown, president and CEO IDEO

Page 35: Collaborative Research
Page 36: Collaborative Research

Dogma

Page 37: Collaborative Research

Context

Page 38: Collaborative Research

Real World

Context

YourUsers

YourOrganization

Page 39: Collaborative Research

UX

Page 40: Collaborative Research

Business

UX

Business

UX

Business

UX UX

UX

Business

UX

Page 41: Collaborative Research

Locate your risk

Page 42: Collaborative Research

Flickr/Kris Krug

Page 43: Collaborative Research
Page 44: Collaborative Research

Where are you coming from?

Page 45: Collaborative Research

Commit to working

collaboratively

Page 46: Collaborative Research

Establish a process.

Page 47: Collaborative Research
Page 48: Collaborative Research

Overcome objections

Page 49: Collaborative Research

We don’t have the time.

Page 50: Collaborative Research

We don’t have the money.

Page 51: Collaborative Research

We don’t have the expertise.

Page 52: Collaborative Research

We’re already A/B testing

Page 53: Collaborative Research

Everyone wants better products.

Page 54: Collaborative Research

No one wants to read a report.

Page 55: Collaborative Research

Design is active.

Reading is passive.

Research is active.

Page 56: Collaborative Research

Embrace conflict

Page 57: Collaborative Research
Page 58: Collaborative Research

One Simple

Process

Page 59: Collaborative Research

Form Questions

AnalyzeData

GatherData

Page 60: Collaborative Research

Form Questions

AnalyzeData

Think Critically

Page 61: Collaborative Research

Form Questions

AnalyzeDataObserve

Page 62: Collaborative Research

Form Questions

AnalyzeDataInterview

Page 63: Collaborative Research

Form Questions

AnalyzeDataRead

Page 64: Collaborative Research

Form Questions

AnalyzeData

Think Critically

Observe

Interview

Experiment

Read

Page 65: Collaborative Research

Interpretation

Interpretation

Interpretation

Interpretation

Page 66: Collaborative Research

SharedReality

Page 67: Collaborative Research

Collaborating with

Strangers

Page 68: Collaborative Research

A design project is a series of

decisions.

Page 69: Collaborative Research

Researchis a craft.

Page 70: Collaborative Research

Bias

Page 71: Collaborative Research

Confirmation Bias:

You selectively weight the information that confirms what you already believe.

Page 72: Collaborative Research

Sampling Bias:

Your sample of research subjects isn’t sufficiently representative.

Page 73: Collaborative Research

Interviewer Bias:

You insert your opinion into interviews.

Page 74: Collaborative Research

Social Desirability Bias

People don’t say the true things that they worry will make them look bad.

Page 75: Collaborative Research

Dunning-Kruger Effect

Unskilled people feel overly confident. Competent people are less so.

Page 76: Collaborative Research

Forming Questions

Page 77: Collaborative Research

Good Questions

SpecificActionablePractical

Page 78: Collaborative Research

A Bad Question

“What do people think about pets?”

Page 79: Collaborative Research

A Better Question

“How do single urban adults choose and acquire a pet?”

Page 80: Collaborative Research

A Bad Question

“What do people do around here all day?”

Page 81: Collaborative Research

A Better Question

“How do we coordinate communication priorities across departments?”

Page 82: Collaborative Research

Critical Thinking

Page 83: Collaborative Research

Critical Thinking

DisciplinedSelf-correctingClearLogical

Page 84: Collaborative Research

Uncritical Thinking

“I hate yellow, so all yellow websites are total failures.”

Page 85: Collaborative Research

Critical Thinking

“I hate yellow, but based on the evidence, it might work for our audience.”

Page 86: Collaborative Research
Page 87: Collaborative Research

Research and Collaboration

Working together across disciplines and making decisions based on evidence shouldn’t be hard, but they can be.

Done right, research and working collaboratively reinforce each other through a shared understanding of reality.

Start with your goal in mind, not with any process or buzzword.

Asking questions and cutting across traditional roles can both be threatening to the established order.

Commit to clear communication and critical thinking.

Research questions follow from goals, assumptions, and risk.

Always have a framework and a plan.

Page 88: Collaborative Research

Break!

Page 89: Collaborative Research

Activities!

Page 90: Collaborative Research

Form Questions

AnalyzeData

GatherData

Page 91: Collaborative Research

Gather Data

Page 92: Collaborative Research
Page 93: Collaborative Research
Page 94: Collaborative Research

Questions About

Users

ProductOrg

Competition

InterviewsInterviews

UsabilityTesting

A/BTesting

ContextualInquiry

LiteratureReview

SWOTAnalysis

BrandAudit

UsabilityTesting

CompetitiveAnalysis

HeuristicAnalysis

Descriptive

Evaluative

Evaluative

Evaluative

Analytic

Analytic

Generative

Descriptive

Page 95: Collaborative Research

ResearchActivitiesTopic

Purpose

Time

Money

Page 96: Collaborative Research

Purpose (Decision Type):

What needs doing?

What are people doing?

How is this thing working?

Page 97: Collaborative Research

Purpose (Decision Type):

Find new product idea.

Better meet an identified need.

Iterate on existing product.

Page 98: Collaborative Research

Purpose (Decision Type):

Generative.

Descriptive

Evaluative

Page 99: Collaborative Research

Why Not Just Prototype?

Page 100: Collaborative Research
Page 101: Collaborative Research

If we only test bottle openers, we may never realize customers prefer screw-top bottles.

– Victor Lombardi, Why We Fail

Page 102: Collaborative Research

Organizational Research

Page 103: Collaborative Research
Page 104: Collaborative Research

Organizational research is good for:

Requirements

Politics

Workflow

Capabilities

Goodwill

Page 105: Collaborative Research

Requirements

What are the top business priorities for this project/product?

Page 106: Collaborative Research

Politics

What does success mean to the individual stakeholders?

Page 107: Collaborative Research

Workflow

Do we have to change how people are working together to be successful?

Page 108: Collaborative Research

Workflow

How do we have to change how people are working together to be successful?

Page 109: Collaborative Research

Workflow

How can we possibly change how people are working together?

Page 110: Collaborative Research

Capabilities

What are the strengths and weaknesses of our team?

Page 111: Collaborative Research

Capabilities

Where is the internal expertise?

Page 112: Collaborative Research

Goodwill

How can this project make your job easier (or harder)?

Page 113: Collaborative Research
Page 114: Collaborative Research
Page 115: Collaborative Research

Basic Stakeholder Questions

What is your title? How long have you been in this role?

What are your essential duties and responsibilities?

What does a typical day look like?

Who are the people you work most closely with? How is that going?

What does success mean from your perspective, what will have changed for the better once this project is complete?

Do you have any concerns about this project?

What do you think the greatest challenges to success are? Internal and external?

Page 116: Collaborative Research

For each stakeholder, note the following:

What’s their general attitude toward this project?

What’s the goal as they describe it?

To what extent are this person’s incentives aligned with the project’s success?

How much and what type of influence do they have?

Who else do they communicate with on a regular basis?

To what extent does this stakeholder need to participate throughout the project, and in which role?

Is what you heard in harmony or in conflict with what you’ve heard from others throughout the organization?

Page 117: Collaborative Research

Stakeholder power moves

“Why are you asking me this?”

“I don’t understand that question. It doesn’t make any sense.”

“I don’t feel comfortable talking to you about that.”

“No one pays attention to anything I have to say, so I don’t know why I should bother talking to you.”

“How much more time is this going to take?”

Page 118: Collaborative Research

10 minutes practice.

What is your title? How long have you been in this role?

What are your essential duties and responsibilities?

What does a typical day look like?

Who are the people you work most closely with? How is that going?

What do you think the greatest challenges to success are? Internal and external?

Page 119: Collaborative Research

UserResearch

Page 120: Collaborative Research

Photo: Flickr/theloushe

Ethnography

Page 121: Collaborative Research

The Four Ds of Design Ethnography

Page 122: Collaborative Research

Deep DiveDaily Life

Data AnalysisDrama

Page 123: Collaborative Research

“...true ethnography reveals not just what people say they do, but what they actually do.”

–PARC

Page 124: Collaborative Research
Page 125: Collaborative Research

Photo: Flickr/lintmachine

Page 126: Collaborative Research

The Art of The Interview

Page 127: Collaborative Research

How to do bad user research:

Ask what people want

Page 128: Collaborative Research

Everybody Lies

Page 129: Collaborative Research

Interviewing is not talking.

Page 130: Collaborative Research

Interviewing is listening.

Page 131: Collaborative Research

You SubjectThe

Comfort Zone

Page 132: Collaborative Research

You SubjectThe

Comfort Zone

Page 133: Collaborative Research

You SubjectThe

Comfort Zone

Page 134: Collaborative Research

You SubjectThe

Comfort Zone

Page 135: Collaborative Research

You SubjectThe

Comfort Zone

Page 136: Collaborative Research

Good Interviewers:

Know Your Question

Warm Up

Shut Up

Page 137: Collaborative Research

IntroductionBody

Conclusion

Page 138: Collaborative Research

Introduction:

Smile

Express gratitude

Describe the process

Ask to record

Warm up questions

Page 139: Collaborative Research

Body:

Ask open-ended questions

Probe for more

Allow silence

Use questions as checklist

Page 140: Collaborative Research

Conclusion:

Transition to wrap-up

Ask if there is anything else

Thank for time

Page 141: Collaborative Research

You are the hostYou are the student

Page 142: Collaborative Research

Interview Checklist

Create a welcoming atmosphere to make participants feel at ease.

Always listen more than you speak.

Take responsibility to accurately convey the thoughts and behaviors of the people you are studying.

Start each interview with a general description of the goal, but be careful of focusing responses too narrowly.

Avoid leading questions and closed yes/no questions. Ask follow-up questions.

Prepare an outline of your interview questions in advance, but don’t be afraid to stray from it.

Also note the exact phrases and vocabulary that participants use.

Page 143: Collaborative Research

Look for

Goals

Priorities

Tasks

Motivators

Barriers

Habits

Relationships

Tools

Environment

Page 144: Collaborative Research

Lunch!

Page 145: Collaborative Research

Interview Scenario

You work for an e-Commerce site that wants to develop a new service to help people give gifts. The goal of the research is to identify unmet needs people might have with regard to giving gifts.

Page 146: Collaborative Research

Interview Practice

Break into groups of 3-4 people

1 interviewee, interviewer , 1 notetaker, 1 observer (optional),

Switch in 15 minutes

3 rounds

Page 147: Collaborative Research

Listen for:

Goals

Priorities

Tasks

Motivators

Barriers

Habits

Relationships

Tools

Environment

Page 148: Collaborative Research
Page 149: Collaborative Research

How did that go?

Page 150: Collaborative Research

How about a focus group?

Page 151: Collaborative Research
Page 152: Collaborative Research

“Even when the subjects are well selected, focus groups are supposed to be merely the source of ideas that need to be researched.”

–Robert K. Merton, Sociologist, the guy who invented focus groups

Page 153: Collaborative Research

Competitive Research

Page 154: Collaborative Research

How else might your target

customer solve the same problem?

Page 155: Collaborative Research

Competitive Review

How do they explicitly position themselves? What do they say they offer?

Who do they appear to be targeting? How does this overlap or differ from your target audience or users?

What are the key differentiators? The factors that make them uniquely valuable to their target market, if any?

How do the user needs or wants they’re serving overlap or differ from those that you’re serving or desire to serve?

What do you notice that they’re doing particularly well or badly?

Based on this assessment, where do you see emerging or established conventions in how they do things, opportunities to offer something clearly superior, or good practices you’ll need to adopt or take into consideration to compete with them?

Page 156: Collaborative Research

Your target customershave to love youmore than theyhate change.

Page 157: Collaborative Research

(Usability) Testing

Page 158: Collaborative Research
Page 159: Collaborative Research

A good research activity:

• Answers a key question

• Addresses identified assumptions

• Informs specific decisions

• Involves your team

• Fits your level of expertise

• Fits your schedule and budget

Page 160: Collaborative Research

Collaborative Recruiting!

Page 161: Collaborative Research

How to find people:

• From your existing, high-traffic site

• Social networks

• Friends and family

• Mailing lists

• Flyers

Page 162: Collaborative Research

A good research activity:

• Answers a key question

• Addresses identified assumptions

• Informs specific decisions

• Involves your team

• Fits your level of expertise

• Fits your schedule and budget

Page 163: Collaborative Research

• Fundamentally research is a simple process

• There are many activities and definitions

• No pressure!

• Select the methods that inform decisions

• Begin by understanding your organization

• Never ask what people like

• People are lazy, forgetful creatures of habit

• Keep each other honest

• Practice and learn

Page 164: Collaborative Research

Analysisand Models

Page 165: Collaborative Research

Creating Meaning From Data

Page 166: Collaborative Research

1. Compile data2. Analyze

3. Identify Insights4. Create Model

Page 167: Collaborative Research

Analysis

Page 168: Collaborative Research

Basic Analysis

Closely review the notes.

Look for interesting behaviors, emotions, actions, and verbatim quotes.

Write what you observed on a sticky note (coded to the source, the actual user, so you can trace it back).

Group the notes.

Watch the patterns emerge.

Rearrange the notes as you continue to assess the patterns.

Page 169: Collaborative Research
Page 170: Collaborative Research

Observation

Observation

Observation

Observation

Observation

Observation

Page 171: Collaborative Research

Observation

Observation

Observation

Observation

Observation

Observation

Observation

Page 172: Collaborative Research

Collaborates on purchases

Observation

Observation

Observation

Observation

Observation

Observation

Observation

Page 173: Collaborative Research

Collaborates on purchases

Uses several devices

Observation

Observation

Observation

Observation

Observation

Observation

Observation

Page 174: Collaborative Research

Collaborates on purchases

Uses several devices

Needs affirmation

Observation

Observation

Observation

Observation

Observation

Observation

Observation

Page 175: Collaborative Research

Ground rules

Acknowledge that the goal of this exercise is to better understand the context and needs of the user. Focus solely on that goal.

Respect the structure of the session. Refrain from identifying larger patterns before you’ve gone through the data.

Clearly differentiate observations from interpretations (what happened versus what it means).

No specific solutions until after you’ve gone through insights and principles. Solutions come next.

Page 176: Collaborative Research
Page 177: Collaborative Research

Look for

Goals

Priorities

Tasks

Motivators

Barriers

Habits

Relationships

Tools

Environment

Page 178: Collaborative Research

25 minutes analysis.

Break into groups of 6-8 people

Each group work together to fill out one diagram with the strongest patterns.

Negotiate and advocate for your perspective.

Page 179: Collaborative Research

Models & Pictures

Page 180: Collaborative Research

Extract information

Page 181: Collaborative Research

Extract information

Page 182: Collaborative Research

Get thoughts out of your

head

Page 183: Collaborative Research

Personas

Page 184: Collaborative Research
Page 185: Collaborative Research

I’ve never seen a persona called “Married woman, no kids, with pristine hardwood.”

God, how I aspire to see that persona.

-Steve Portigal

Page 186: Collaborative Research
Page 187: Collaborative Research

Make a persona based on your interviews

Back into the analysis groups

One person will describe the personas to everyone and we’ll decide whether they can be collapsed.

Page 188: Collaborative Research
Page 189: Collaborative Research

Other Models

Page 190: Collaborative Research

ThoughtWorks

Page 191: Collaborative Research
Page 192: Collaborative Research
Page 193: Collaborative Research
Page 194: Collaborative Research
Page 195: Collaborative Research
Page 196: Collaborative Research
Page 197: Collaborative Research
Page 198: Collaborative Research
Page 199: Collaborative Research

A concept map is a picture of our understanding of something.

–Dubberly Design Office

Page 200: Collaborative Research

Generate lists of words related to the main concept.

The list can come from research, reading, experts, brainstorming, or any other source.

The second step is to edit the list. Some terms may be related to the subject, but not in a way that meets the project goals.

The third step is to define the terms on the edited list. This is particularly important with unfamiliar or technical terms. But it also helps with familiar terms, too.

Create a matrix listing all the terms down one side and repeating the list across the top. Note the relationship in the boxes where a row and column intersect. The resulting matrix of relationships provides a checklist for building the concept map.

Page 201: Collaborative Research

• List terms

• Edit the list

• Define the remaining terms

• Create a matrix showing the relations of terms

• Rank the terms

• Decide on main branches or write framing sentences

• Fill in the rest of the structure

• Revise

• Apply typography to reinforce structure

• Revise

Page 202: Collaborative Research
Page 203: Collaborative Research
Page 204: Collaborative Research

Analysis and Models

Everyone on the team should be involved in turning data into insights. A productive session requires rules.

Once you and your team have extracted insights from data, document those insights in models.

A model distills and documents thinking so everyone on the team can see it and make decsions based upon it.

Remember than models are still an interim document. They are tools. Think “useful” not precious. Update as needed.

The affinity diagram comes straight out of analysis sessions.

Personas are one of the most intelligible research outputs for people throughout the organization.

Page 205: Collaborative Research

Break!

Page 206: Collaborative Research

Reporting and Sharing

Page 207: Collaborative Research

How to make research meaningful to your organization

Page 208: Collaborative Research

Flickr/Kris Krug

Page 209: Collaborative Research

Flickr/Jerome Collins

Page 210: Collaborative Research

You are collaborating with your future selves.

Page 211: Collaborative Research

Design synthesis is the most critical part of the design process. Yet in our popular discussions of design and innovation, we've largely ignored this fundamental role.

–John Kolko

Page 212: Collaborative Research

Building a Culture of Research

Page 213: Collaborative Research

How to make research meaningful to your organization

Page 214: Collaborative Research

It is your job to make it easy for everyone

else.

Page 215: Collaborative Research

Research ReportStudy Title

Date Completed

Research Goal

Activities

Related Decisions

Key Insights

Supporting Observations

Recommended Actions

Questions for Further Study

Page 216: Collaborative Research

Clear goals

Shared values

Access to information

Clear decision-making

Page 217: Collaborative Research

The goal determines the form

Page 218: Collaborative Research

How to apply research

Page 219: Collaborative Research

Reasons to Share

Page 220: Collaborative Research

Flickr/loozrboy

Page 221: Collaborative Research

The report is not the research.

Page 222: Collaborative Research

Why report at all?

Page 223: Collaborative Research

Informing?

Inspiring?

Focusing?

Remembering?

Recording?

Deciding?

Page 224: Collaborative Research

Wrap Up

Page 225: Collaborative Research

In summary

Research creates a shared understanding of reality.

Asking questions is uncomfortable. Embrace that feeling.

A truly collaborative approach and environment is necessary for research to be effective, and it also makes it more fun.

Clear goals and good questions are required.

Choose only the research activities that answer real questions and inform your top priority design and development decisions.

Practice! Observe and listen every day.

Document! Report! Share! It’s easy to lose what you learn.

Page 226: Collaborative Research

Any questions?

Page 227: Collaborative Research

Brief books for people who make websites No.

9

JUST ENOUGHRESEARCH

Erika Hall

www.abookapart.com

You might enjoy the book.