C-RITE: How to run impactful iterative studies in a fast paced environment V2

C(ollab) RITE How to run impactful iterative studies in a fast paced environment

Jhilmil JainAndroid

#crite

@jhilmiljain

#androidux

Wednesday, July 30, 14

C-RITE• Iterative UX research method suitable for agile

environments

• From research to new design solution in 1 day

• 7 research sessions in 2 days

• Collaborative design workshop each day

• Prototype changes between day 1 and 2

• Can be repeated weekly

• Stakeholders collaborate in all UX phases

#crite #androiduxWednesday, July 30, 14

ThePlayers

The Players and Their Roles

Wednesday, July 30, 14

Research AnalysisReporting

&Prioritization

Design PrototypingPlanning

& Recruiting

Traditional Roles

Reporting &

Prioritization

Planning & Recruiting Research Analysis Design Prototyping

Reporting &

Prioritization

Planning & Recruiting

Designer Designer

Researcher

PM PM

EngineerWednesday, July 30, 14

Collaboration in C-RITE

Research Analysis Design PrototypingReporting

&Prioritization

Planning & Recruiting

Designer

PM

Engineer

Researcher

Wednesday, July 30, 14

Time Line - Traditional Usability

Research AnalysisReporting

&Prioritization

Design PrototypingPlanning

& Recruiting

1- 2 weeks 1- 2 weeks 1/2 - 1week 1/2 -1week

4 - 8 weeks

#crite #androidux

1/2 - 1 week1/2 -1 week

Wednesday, July 30, 14

Time Line - RITE

Research AnalysisReporting

&Prioritization

Design PrototypingPlanning

& Recruiting

1 - 2 weeks 1 - 2 weeks

2 - 4 weeks

#crite #androiduxWednesday, July 30, 14

Time Line - C-RITE

Research AnalysisReporting

&Prioritization

Design PrototypingPlanning

& Recruiting

1-2 weeks 2.5 days

1.5 - 2.5 weeks

#crite #androiduxWednesday, July 30, 14

Time Line - Cyclic C-RITE

Research AnalysisReporting

&Prioritization

Design PrototypingPlanning

& Recruiting

1 week 2.5 days

1.5 weeks

#crite #androiduxWednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

Example: TV Voice Search

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

1st TV Voice Search Design

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

C-RITE Research Timing

10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm

Day 1 Day 2

#crite #androidux

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

C-RITE Collaborative Observation

10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm

Day 1 Day 2

#crite #androidux

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

Observation Board Example

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

User Goals Example

Observation ExampleGoal example

P1P1 P2P2Bob Lin Ali Nur Bob Lin Ali Nur

User knows what to say 2 1 2 2

The system understood user’s intent 5 3 1 1

The system doesn’t slow down user w unnecessary confirmations 4 4 2 2

User can easily tell where speech will work 2 2 2 2

User comfortable using their voice 4 4 4 4

Rating on a scale from 1- 5 1 = poor 5 = good

Wednesday, July 30, 14

Mocks Example

Wednesday, July 30, 14

Mocks Example

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

Collaborative Observation

#crite #androidux

• Re-call to count observations, analysis step reduced

• Disagreements about observations are resolved immediately

• Research protocol can be adjusted based on stakeholder feedback

• Researcher can explain what observations mean

• Small sample size discussion is obsolete

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

C-RITE Analysis

10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm

Day 1 Day 2

2:00 pm 3:00 pm

#crite #androidux

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

From Observation to Insight

Observation ExampleGoal example

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

Prioritizing Insights

Goal example

Insights

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

C-RITE Design Workshop

3:00 pm

Day 1

10:00 am 11:00 am 1:00 pm 2:00 pm 10:00 am 11:00 am 1:00 pm 2:00 pm

Day 2

6:30 pm5:30 pm

#crite #androidux

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

Design Workshop

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

Design Workshop Actions

Observation ExampleGoal example

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

Action Item Owners & Due Dates

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

Before & After

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

C-RITE Design Workshop

#crite #androidux

• Turn panic into confidence

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

C-RITE Design Workshop

#crite #androidux

• Turn panic into confidence• Designers demonstrate problem solving skills

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

C-RITE Design Workshop

#crite #androidux

• Turn panic into confidence• Designers demonstrate problem solving skills• Engineers solve technical constraints

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

C-RITE Design Workshop

#crite #androidux

• Turn panic into confidence• Designers demonstrate problem solving skills• Engineers solve technical constraints• Researchers guide insight interpretation

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

C-RITE Design Workshop

#crite #androidux

• Turn panic into confidence• Designers demonstrate problem solving skills• Engineers solve technical constraints• Researchers guide insight interpretation• PM gives business and scheduling input

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

C-RITE Design Workshop

#crite #androidux

• Turn panic into confidence• Designers demonstrate problem solving skills• Engineers solve technical constraints• Researchers guide insight interpretation• PM gives business and scheduling input• Everyone participates in creative process

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

C-RITE Reporting

Day 1

Send action items to team

Day 2

Send action items to team Short report

Day 3

#crite #androidux

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

C-RITE Prototyping

3:00 pm10:00 11:00 1:00 2:00

Day 2

6:30 pm

Day 1

10:00 11:00 1:00 2:00 5:30 pm

#crite #androidux

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

C-RITE Cycles

Week1 Week 2 Week 3

#crite #androidux

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

C-RITE Recruiting

• Create a panel of users that meet general recruitment criteria

• Panel members commit to availability at short notice on research days

• Each week, schedule panel members• Recruit for specific criteria when needed

#crite #androidux

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

C-RITE Preparations

• Set expectations with all stakeholders beforehand about time commitment, collaboration and workshop format

• Decision makers need to participate in C-RITE studies and workshops

#crite #androidux

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

C-RITE Summary

#crite #androidux

• Iterative UX research method suitable for agile environments

• From research to new design solution in 1 day

• 7 research sessions in 2 days

• Collaborative design workshop each day

• Prototype changes between day 1 and 2

• Can be applied weekly

• Stakeholders collaborate in all UX phases

Wednesday, July 30, 14

Thank you

#crite #androiduxWednesday, July 30, 14

Appendix

Wednesday, July 30, 14

Agile Methods: RITE

• Changes are made as soon as a problem is identified, sometimes after 1 session

• 1-2 sessions a day. Breaks between sessions long enough to make changes

• 1-2 weeks of sessions

• Requires stakeholder attendance and participation

Medlock, Wixon, Fulton, Terrano and Romero, UPA, 2002

#crite #androiduxWednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

Agile Methods: KRUG + RITE

• 4 participants over 2 days, at least 1 hour between sessions to make changes

• Researcher sends out observations and recommendations 30 min after last session

• Team debrief of observations and action items on day 2, 1 hour after last session

• No collaborative design workshop

McGinn, Chen, Journal of Usability Studies, 2013

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Prototyping

Planning & Recruiting

Agile Methods: PULSE

• 3 participants, 1 day every week

• Analysis and write-up of results by researcher within 2 days

• No collaborative analysis and design workshop

Konno, Agile 2012

Wednesday, July 30, 14

Research Analysis Reporting &Prioritization Design Implementation

Planning & Recruiting

Variations We’ve Tried

• C-RITE Remote• If time zones allow, stakeholders participate through ‘Hangouts’

• Sessions are recorded and watched with time delay

• Next day debriefs and design workshops for incompatible time zones

• C-RITE Diary Studies• In-lab diary kick-offs and debriefs are run in C-RITE format

• Researcher analyzes and summarizes diary data for debrief and design workshop

Wednesday, July 30, 14