Top Banner
Experiment Driven Product How do we evolve our product at Redbubble
13

Experiment driven product - LAST 2014

Feb 22, 2017

Download

Technology

Ivan Vysotskiy
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Experiment driven product - LAST 2014

Experiment Driven ProductHow do we evolve our product at Redbubble

Page 2: Experiment driven product - LAST 2014

Who is Redbubble

Artwork Marketplace

~10M visits a month

Melbourne based

San Francisco office

2 http://www.redbubble.com/people/luckyvegetable

Page 3: Experiment driven product - LAST 2014

Engineering at Redbubble

4 teams

Very-very Lean

Even more Agile

up to 12 deploys / day

We test assumptions3

http://www.redbubble.com/people/luckyvegetable

Page 4: Experiment driven product - LAST 2014

Purchase Experience

4

Page 5: Experiment driven product - LAST 2014

A/B test

5

FAILSUCCESS

Page 6: Experiment driven product - LAST 2014

Why A/B test

Isolate changes !

Establish causal relationship

6http://www.redbubble.com/people/joust/works/567864-e-mc-bananas

Page 7: Experiment driven product - LAST 2014

Funnel

1. Identify major

steps

2. Measure each

3. Profit

7http://www.redbubble.com/people/anfa/works/8412967-i-put-the-fun-in-funnel

Page 8: Experiment driven product - LAST 2014

Free Shipping Experiment

8

Page 9: Experiment driven product - LAST 2014

Free Shipping Experiment

1 week

No significant change

~$3500 loss

~$50000+ saved

9

Page 10: Experiment driven product - LAST 2014

Amazon Payment Fake Door

10

FAIL

SUCCESS

Page 11: Experiment driven product - LAST 2014

Amazon Payment Fake Door

Minimal cost to run

People who use only Amazon Payments exist!

Looks like ~8% increase in conversion

11

Page 12: Experiment driven product - LAST 2014

Amazon Payments Experiment

12

Page 13: Experiment driven product - LAST 2014

Stop assuming - start testing

13Duncan Hull - https://flic.kr/p/iVLZt