Avalon's DM 101 - Testing Best Practices

1Cover Page

Client Training DayDirect Marketing 101: Testing

September 7, 2017

Avalon Consulting Group

www.avalonconsulting.net

©2017 Avalon Consulting Group.

All Rights Reserved.

2

AUDIENCE POLL

Is this you?

3

I have made major changes to a campaign

without testing them first.

4

Each organization is different and

not all best practices are universal.

When in doubt about what’s best for

your organization, test to find the

data-driven answers you need.

Ongoing testing is essential to a

healthy, productive direct marketing

program.

Testing Overview

5

Testing Goals

Get people to open your solicitation

Increase response/revenue

Determine what messaging resonates

Settle an argument

Save costs

6

QUIZ

Which Test Won?

7

8

9

Join a Nation of Gardeners as a Member of the American Horticultural Society r Yes, I

want to enhance my skills as a gardener and share my love of gardening with people nationwide. I have

completed the attached 2015 National Gardening Survey and will join the American Horticultural Society with the

enclosed donationof:

r $35$25* r $50 r $100 r $250 r $500 r Other $

*Special introductory offer! Send $25 for FULL membership & receive our exclusive Gardener’s Tote – FREE!

Join a Nation of Gardeners as a Member of the American Horticultural Society r Yes, I want

to enhance my skills as a gardener and share my love of gardening with people nationwide. I have completed the

attached 2015 National Gardening Survey and will join the American Horticultural Society with the enclosed

donationof:

r $40$30* r $50 r $100 r $250 r $500 r Other $

*Special introductory offer! Send $30 for FULL membership & receive our exclusive Gardener’s Tote – FREE!

10

11

And the winners are…

$25 (lower amount)

12

Test to get people to open your

solicitation

13

Image vs. No Image

The red/gold carrier had a higher response rate, average gift, and 16% higher rev/m.

14

Teasers and Photos

21% increase in revenue/m –response rate and avg. gift stronger

15

Deadlines

Deadline won with a 26%

higher rev/m due to a

higher average gift and

flat response rate

16

Control

Test

Plain vs. Image Carrier

Back

Plain carrier had a

26% higher

response rate and

higher average gift

Back

17

Email Tests

Send timing (day of week and time of day)

From line (e.g. organization vs. email signer)

Subject line (e.g. length, direct vs. vague, etc.)

Emoji

18

Test to increase

response/revenue

19

Ask Strategy

Test ask strings:

Different ask amounts (e.g. odd vs. round numbers)

Shorter or longer ask string lengths (e.g. adding $1,000)

Order of asks (e.g. ascending vs. descending)

Asterisk or circle driving attention to target ask vs. none

Different asks for different segments (e.g. outside lists vs. lapsed donors

in acquisition)

Monthly option

TIP: Make sure asks in the letter/email match reply or donation form.

20

ROI after 1 year

Ask String Testing

21

Discounts and Premiums

Discount brought

in 10% more

members,

discount offer

won at 90%

confidence.

22

Engagement Techniques

Surveys Member Cards

23

Mission-Relevant Engagement

Placemat

Place Cards

24

Petitions

Offline Online

25

Testing to determine what

messaging resonates

26

Aggressive vs. Cooperative

27

Part I: Pilot Acquisition Conversion

Campaign

Target Launch: October 2013

Story vs. Traditional RSVP Invitation

Concept #1: Letter from the Field Concept #2: RSVP/Invitation

28

Concept: Member Card

RSVP vs. Member Card

29

Testing across channels

30

Testing to settle an argument

31

Long-Standing Control

32

Four-Color Images

Black and white carrier generated

28% more revenue per thousand

mailed!

Performance metrics tied but

significantly lower cost for white

carrier.

33

2 Page vs. 4 Page

34

Testing to reduce costs

35

Cost Saving Tests

1st vs. 3rd class postage or an RAE vs. BRE

Test eliminating an insert or upfront premium

Bare-bones, standard package vs. more expensive and/or non-standard package

Test an E-R0 to invite people to renew online and not receive a DM renewal

Machine vs. autopen vs. actual handwriting

Use less color for more impact

Reduce personalization and resulting matching costs

Use a drop-cut format rather than a separate reply form

Use a window carrier instead of a closed face envelope

36

Measuring your test results

37

Test Set-up

To ensure valid testing results, it is important to

follow the guidelines below:

Ensure the statistical

significance of your test – use a

sample size calculator to

accurately project your panels.

Ensure the goal of the test is

clear – what will be the measure

of success?

• 20% lift in response rate? 10% lift in average gift? 15% reduction in cost?

Test one element at a

time to isolate the impact of the variable

change.

• Carrier? Copy? Ask string?

Choose tests where there is a “reason to believe” the change will improve the campaign.

• What information informs the testing decision?

38

Check Test Panels

39

Measuring Your Test

It is important to accurately read the results of your

test, beyond just “eye-balling”:

A difference in results may be insignificant and within the margin of error.

Use a calculator designed to identify significant differences before determining a winning test.

Use roll-out costs to extrapolate the true impact of the test at full quantities.

40

Have Fun!

41

Thank you!Anne SenftVice President

Avalon Consulting Group

805 15th Street, NW Suite 700

Washington, DC 20005

202-627-6529

annes@avalonconsulting.net

Connect with Avalon Today!