TAKING A HYPOTHESIS DRIVEN APPROACH · 2019-05-20 · AARRR “PIRATE” FUNNEL METRICS Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”

TAKING A HYPOTHESIS DRIVEN APPROACH Nicola Boyle & Rob Harrison

1

2

Assumptions

Assumptions

Assumptions

Assumptions

Assumptions

Assumptions

Assumptions

Assumptions

Assumptions

AssumptionsAssumptions

Assumptions

AssumptionsAssumptions

Assumptions

Assumptions

Assumptions

3

Us.

4

You.

RETWEETI’ll try a

#HypothesisDrivenApproach #AgileAus

@nicolaboyle27 @remoterobbo

CREATING A HYPOTHESIS

6

We believe that <this change>

Will result in <this outcome>.

We will know we have succeeded when <we see change in X metric>.

Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”

SKYPE FOR STAFF

7

We believe that enabling Skype functionality to staff

Will result in less staff travel and quicker turnaround

times on decision making.

We will know we have succeeded when we see that the flights taken by staff, for face to face meetings, have been reduced by 5%.


MVP - THE HYPOTHESIS’S BEST FRIEND

8Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”

METRICS THAT MATTER

9

Acquisition

Activation

Retention

Revenue

Referral

AARRR “PIRATE”FUNNEL METRICS


- Dave McClure

Acquisition - Number of people who visit your service

Activation - Number of people who have a good initial response

Retention - Number of people who come back for more

Revenue - Number of people from the cohort who engage in revenue-creating activity Referral - Number of people from the cohort who refers other users

ITERATE TO VALIDATE

10

THE VALIDATED LEARNING LOOP

• REDUCED RISK & UNCERTAINTY

• MORE VALUE SOONER

• LAYER-IN COMPLEXITY

Create hypothesis

Design and run

experiment

Study results

Evolve model and

implement changes


RETWEETI’ll try a



CREATE AN INNOVATION CULTURE


DISRUPTS


ACCEPT OR CANCELPHONE, TEXT, EMAIL OR NOTIFICATION

MANAGE BOOKING ONLINE

FROM REQUIREMENTS TO HYPOTHESES

14

We believe allowing passengers to change their flight within a seven day period

Will result in a better customer experience as passengers will not have to call to manage their booking.

We will know we have succeeded when 10% of customers select a new flight within the seven day period.

Create hypothesis

Design and run

experiment

Study results

Evolve model and

implement changes

SEVEN DAY CHANGE


BUILD AN MVP


THREE DAY CHANGE

Create hypothesis

Design and run

experiment

Study results

Evolve model and

implement changes

CHECK THE RESULTS

16

• Most passengers accepted new flight.

• Some passengers changed on day or cancelled.

• Very few passengers changed to next day and fewer changed on the third day.


Create hypothesis

Design and run

experiment

Study results

Evolve model and

implement changes

CONTINUE, PIVOT, STOP

17

• Cost to build seven day change deemed not worth the ROI.

• Seven day change feature stopped.

• Team able to pick up more valuable feature.


Create hypothesis

Design and run

experiment

Study results

Evolve model and

implement changes

FROM DONE TO MONITORING

18

Team walls and portfolio wall now have Monitoring instead of Done.

Definition of done to move out of monitoring column is ‘Measure outcome’


19

BECOME A DATA-LED BUSINESS


BUILD MEASUREMENT IN

20

“Without data you’re just another person with an opinion” - W. Edwards Deming

-

• Agreed before development.

• An acceptance criteria for every story.

• Part of the way of working - it’s not done until you’ve studied the results and acted on them.


ISOLATE CHANGE

21

We believe providing the ability to add an extra bag prior to payment only

Will result in increased customer satisfaction and speed to complete bookings.

We will know we have succeeded when:

• There is reduction in complaints from customers related to overloading of information;

• Decrease in time to complete a booking (from search to payment);

• There is no statistically significant drop-off in customers adding extra bags.


ADD EXTRA BAGS

RETWEETI’ll try a



TRANSFORM YOUR ORGANISATION


GUILD WORKSHOP


TRAINING BUSINESS ANALYSTS ON HYPOTHESIS DRIVEN DEVELOPMENT

We believe running a one hour session where we introduce the concept, core theory and have participants

build a hypothesis based on their existing project

Will result in an increase in attendees applying a hypothesis driven approach in their teams.

We will know we have succeeded when the average score for attendees is at least a three on a self-assessment scale.

MEASURING SUCCESS


1. I know nothing about hypothesis driven development.

2. I know a little, but not how to apply it.

3. I could work in a team using it.

4. I could apply the concepts and explain it to others.

5. Move over, I will be presenting this session.

SELF-ASSESSMENT

SECOND TIME RUNNING THE SESSION

PIVOT


We believe running a ninety minute session where we introduce the concept, core theory and have participants

build a hypothesis based on a fictional project

Will result in an increase in attendees applying a hypothesis driven approach in their teams.

We will know we have succeeded when the average score for attendees is at least a three on a self-assessment scale.

27

I have not failed. I’ve just found 10,000 ways that

don’t work. — Thomas A. Edison

RESOURCES

28

The Lean Startup - Eric Ries

Entrepreneur/Author - Barry O’Reilly

The Lean Enterprise - Jez Humble, Joanne

Molesky and Barry O’Reilly


THANK YOU

[email protected] [email protected]

“I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @Remoterobbo”

mailto:[email protected]?subject=

mailto:[email protected]

TAKING A HYPOTHESIS DRIVEN APPROACH · 2019-05-20 · AARRR “PIRATE” FUNNEL METRICS Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”

Documents