MBO15 Presentation: Jeff Carl, Angie's List

mboindy.com

Introduction to Multi-Variate Testing

Most marketing professionals are using some form of testing to determine which is the best message to send to their customers.

• Angie’s List has had success using multi-variate testing

• When used correctly, this method allows you to test the effectiveness of an unlimited number of variations.

• Used in conjunction with A/B testing to eliminate false positives or other analytical pitfalls.

mboindy.com

History of MVT

1747 – First origins Of a MVT When James Lind

Discovered a cure for scurvy

1980 – Genichi Taguchi Develops MVT method to Improve manufacturing

1876 – Charles S. Peirce writes the First English Language Publication on optimal design

2000 – Daniel McFadden Wins Nobel Prize in Economics

For Choice Modeling MVT technique

2015 – Angie’s List Embraces a technique first developed in

1747 to sell more stuff

mboindy.com

Testing 101

At Angie’s List we A/B test every new idea we have.

This method is tried and true, easy to analyze, and generally produced significant results.

New Idea ≠ Best Idea

To determine the best, we would need rounds of A/B testing.

mboindy.com

Benefits of MVT Optimal design lets you test 1000's of variations in the time it takes you to run a simple A/B test.

Isolates specific elements to determine which exact part of the new design performed so well. Eliminates the need to run several sequential A/B tests on the same page with the same goal. Work smarter, not harder.

mboindy.com

Thanks!

Start getting meaningful results faster by switching from A/B tests to MVT tests!

Questions?