This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Optimizing Your Optimization Strategy
Benn Stancil Co-‐Founder and Chief Analyst
Mode @bennstancil
Kyle Rush Head of
Optimization Optimizely @kylerush
Focus on high value flows
What to test: focus
!
HIGH VALUE FLOWS vs.
ONE OFFS
What to test: focus
Define success before you start
Track many goals
This is what 16 goals looks like
Define success before you start
But choose one primary goal.
The right primary goal.
What to test: geJng ideas
Comb through your data.
17% error rate
What to test: geJng ideas
Submit Successa
b
a
b
3.5% absolute difference.
26.3% relaNve difference.
What to test: geJng ideas
Go through the flow yourself.
100 Nmes.
What to test: geJng ideas
CompeNtor research
What to test: geJng ideas
Doesn’t have to be a compeNtor
What to test: geJng ideas
User tesNng
• Talk to friends
*She misses you.
• Call your mom*
• Talk to peers
Your strategy should revolve around a variable called MDE.
(minimum detectable effect)
Sample size
Everyone wants a low MDE.
But most of us don’t have the luxury.
Sample size
Myth: Small startups can’t a/b testYour small website gets 2,000 visitors a month