A WordStream & KISSmetrics Webinar: A/B Testing; You’re Most Likely Doing It Wrong

FROM THE CLICK AND…

BEY ND

A/B Testing; You’re Most Likely Doing It WrongJess Armstrong & Tomasz Borys 06.11.2015

Copyright 2015 WordStream, Inc. & KISSmetrics All rights reserved.

LIVE WEBINAR#BeyondTheClick

• Customer Onboarding Specialist

• UMass Amherst- Economics

• Foodie- Traveler- Gymnast

@Jess_Armstrong_

About

Jess

What Makes a Worthwhile Test?

Creating a Fail-Safe A/B Test

3 A/B Test to Consider

A/B Testing in PPC

Why PPC?

• No Cost Set Up

• The Data You need is already there!

• High Commercial Intent


What to Test?

Ignore the Trivial Things!

Tiny Optimizations are a waste of time!


Ignore the Trivial Things!

Gains from small changes tend to flatten out over time.


Fix Obvious Problems!


The Mountain Analogy

Goal to get here

You are

here


Aim for results

that are 2-3xHigher than your

baseline, not 2-3%

Team

Effort!

Creating a Fail-Safe Test

Define Success

There are so many

metrics in AdWords, You

need to prioritize ahead

of time!


Which One Wins?

0

5

10

15

20

25

30

35

40

45

50

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6

Total conversions by Week

Control Test

0

10

20

30

40

50

60

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6

Average CPA by Week

Control Test

Conflicting Winners!?


Quantity vs. Quality

Example


Quantity vs. Quality

Example


Save Yourself the Frustration


Set a Timeline and

Stick to It

Choosing in a Tie

Set Up a Safety Net

• Remarketing Campaigns

• Catch lost conversions missed

during your test

• Users are 2x more likely to

convert after seeing your ad 7+

times

3 A/B Tests to Consider

1. Change Your Call to Action

• This is a Business Decision

• Ads and Landing Pages

• What makes them click and

convert?

• Collaborate with other teams


2. Test Mobile Strategies

Some Business see huge increases in

conversions on Mobile!



….Others don’t.



Things to consider:

• Mobile Bid Modifier

• Mobile Text-Ads

• Call Extensions

• Mobile Friendly Landing Pages

• Set up an experiment in AdWords

• 50/50 impression share


3. Change Your Flow

• Think about where on the funnel people are

falling off and change it


Key Take-Aways

Think Big! Aim for 2-3x conversion rate

Set Clear Goals and Priorities in advance

Capture lost conversions with remarketing

Tests to try:

o Change Call-to-Action

o Test Mobile Strategies

o Change the Conversion Flow


Tomasz BorysDirector of Marketing

KISSmetrics

@tbcali

How We A/B Test at KISSmetrics

• Set a Goal

• Baby Steps

• Aim for Statistical Significance

• Never Lose Sight

• Blind Eye for 1 Week


Set a Goal

Baby Steps

People want to move the

needle and see results fast, by

implementing multiple changes

at once.

But how can you be sure what

element had an impact?

Aim for Statistical Significance SET A GOAL

Aimfor Statistical

Significance

getdatadriven.com

http://getdatadriven.com

LIVE WEBINAR

Blind Eye

for 1 Week

Data can be very erratic the

first several days, so it’s

easy to hit the panic button

#BeyondTheClick

Never Lose

Sight

We can caught up on

what we think is best

to optimize

conversions or

traffic…but data

doesn’t lie

A/B Testing Beyond Click Conversions


Original


Variant


Signups


Signups


Signups


Signup Install JSCustom

DataOpps


Signups

Installed JS


Signups

Install JS


Signups

Install JS


Custom Data


Custom Data


Custom Data


Custom Data

Opportunities Created


Custom Data



Custom Data



Social Ad Funnel


Keeping a Pulse on Your Funnel



DataOpps



DataOpps

LIVE WEBINAR


DataOpps



DataOpps


Ways of Improving Funnel

• Find the Bottleneck

• Learn why there is a Bottleneck


Learn Why There is a Bottleneck

• A/B Testing

• Surveys and Reach

• Usability Testing


@tbcali

@ Jess_Armstrong_Questions?

A WordStream & KISSmetrics Webinar: A/B Testing; You’re Most Likely Doing It Wrong

Marketing

week live webinar

frustration live webinar

teams live webinar

conversion flow live

quality example live

failsafe test

failsafe ab test

test users