Meetup Turin (UX Torino and Optimizely)

Lessons learned from over 686,000 A/B tests

Marco Sgrignuoli (Software Engineer) [email protected]

Pedro B. Meyer (Country Manager – Italy, Spain & Portugal) [email protected]

Lesson #1: Define Success

Lesson #1: Define Quantifiable

Success Metrics

$ Per Recipient Visitors Per Day

Visit

Sign-Up Rate

Email $$

Media

Button

Button

● Sign Up

● Learn More

● Join Us Now

● Sign Up Now

Splash Page Experiment’s Variations Media

• “Get Involved” Image

• “Family” Image

• “Change” Image

• Barack’s Video

• Springfield Video

• Sam’s Video

Button

● Sign Up

● Learn More

● Join Us Now

● Sign Up Now

Splash Page Experiment’s Variations Media

• “Get Involved” Image

• “Family” Image

• “Change” Image

• Barack’s Video

• Springfield Video

• Sam’s Video

Button

● Sign Up

● Learn More

● Join Us Now

● Sign Up Now

Splash Page Experiment Media

• “Get Involved” Image

• “Family” Image

• “Change” Image

• Barack’s Video

• Springfield Video

• Sam’s Video

Splash Page Experiment

Splash Page Experiment

Splash Page Experiment

Email Subscriptions

Volunteers Amount Raised

Original 7,120,000 712,000 $143,000,000

+40.6% +2,880,000 +280,000 +$57,000,000

New 10,000,000 1,000,000 $200,000,000

Lesson #1: Define Quantifiable

Success Metrics

Lesson #2: Less Is More.

Reduce Choices.

Lesson #2: Less Is More.

Reduce Choices.

Lesson #3: Words Matter!

Focus on your Call to Action

Donation Button Experiment

Never Signed-Up

Signed-Up No Donation

Previously Donated

0.0% 0.0% 0.0%

2.3% 27.8% 16.3%

-27.8 N/A N/A

15.2% -24.6% 11.9%

8.5% 2.9% 18.4%

Lesson #3: Words Matter!

Focus on your Call to Action

Lesson #4: Seek the Global Maximum

Lesson #4: Seek the Global Maximum

Lesson #5: Fail Fast

Lesson #5: Fail Fast

Lesson #6: Nothing is Sacred.

Challenge Assumptions.

Lesson #6: Nothing is Sacred.

Challenge Assumptions.

Lesson #7: Get Started Today!

10,000+ Happy Customers!

alesport.com

Live Demo!