Smoke Tests - the hustle'rs guide to experimentation

SMOKE TESTS HUSTLE YOUR EXPERIMENTS w/o DEVS

@DistroDom - #WMD2015

WHO IS THIS GUY DOMINIC?● Co-Founded GarmentValet● Co-Founded Curebit / Talkable

Now I’m @DistroDom @500 Startups investing time and $ in promising startups #500STRONG

@DistroDom - #WMD2015

PRODUCT GROWTH+

@DistroDom - #WMD2015

A little story...

@DistroDom - #WMD2015

MY OWN F*CKUP - one of the many ;)

@DistroDom - #WMD2015

BAD LINES FROM THE “OPS” APP

Sales

Cash

@DistroDom - #WMD2015

WAITING ON

ONE MODULE

1 YEAR WASTED

@DistroDom - #WMD2015

WE’VE ALL BEEN

HERE

@DistroDom - #WMD2015

Growth Is SLOWER than we want

@DistroDom - #WMD2015

Product Backlog is LONGER than we want

@DistroDom - #WMD2015

YOU EXPECT….

@DistroDom - #WMD2015

But you get….

@DistroDom - #WMD2015

#ProductGROWTHDilemma

@DistroDom - #WMD2015

KILL THAT SHIT

Before it KILLS YOU@DistroDom - #WMD2015

SMOKE TEST

@DistroDom - #WMD2015

WTF is this Smoke Test?

WAY TO PROVE THAT YOUR PRODUCT BACKLOG IS WORTH IT

task: validate do

@DistroDom - #WMD2015

@DistroDom - #WMD2015

- Alberto Savoia

LET’S RUN A PROCESS

1. What is the MINIMUM PROOF You Need?2. How can you get that proof with NO ENGINEERS?3. Drive the RIGHT TRAFFIC to your experiment4. MEASURE Conversion5. Analyze and OPTIMIZE6. Make a DECISION

@DistroDom - #WMD2015

WHAT IS THE PROOF YOU NEED?

@DistroDom - #WMD2015

CAN I DO IT WITHOUT CODE?

@DistroDom - #WMD2015

AUDIENCE MATCH

@DistroDom - #WMD2015

KEEP SCORE

@DistroDom - #WMD2015

NEXT STEPS

@DistroDom - #WMD2015

DECIDE

@DistroDom - #WMD2015

ITERATE CONSTANTLY

@DistroDom - #WMD2015

Vs. This Design

- Asked for email and zip before live

- Had a <3% Email Capture Rate

@DistroDom - #WMD2015

RESULTS

<= 30% of visitors entered their

zipcode&

25 % of people entered their

email =>&

10% shared

@DistroDom - #WMD2015

AN EXAMPLE FROM LAST WEEK

@DistroDom - #WMD2015

ADD TO BOX

@DistroDom - #WMD2015

“TYPICAL” PRODUCT WAY

BUILD a New Feature Called “Add To Box”(~2 Weeks)

PLAN a New Feature Called “Add To Box”

(~1 Week)

TEST & RELEASE

a New Feature Called “Add To

Box”(~1 Week)

ONE MONTH POTENTIALLY WASTED

@DistroDom - #WMD2015

“TYPICAL” PRODUCT WAY

LAUNCH(THAT DAY)

PLAN IDEA => HACK

(~1 hour)

MAKE $$$$ BOoooOM

:)

And the RESULTS….

@DistroDom - #WMD2015

FIRE

● 9.55% of Active Subs converted

● Overall monthly gross profit grew by 45% because of this feature

● Marginal gross profit per PijonBox, on those boxes, grew over 357%.

●

@DistroDom - #WMD2015

BUT EVEN IF WE LOST….

● Saved a month of dev time● Learned about our customers● FELT GOOD AS A TEAM●

WE WOULD HAVE STILL WON!!

@DistroDom - #WMD2015

[email protected]

@DistroDomwww.facebook.com/dominic.coryell

HUSTLE WITH ME

@DistroDom - #WMD2015

GROWTH => PRODUCT

@DistroDom - #WMD2015