1 Cover Page Client Training Day Direct Marketing 101: Testing September 7, 2017 Avalon Consulting Group www.avalonconsulting.net ©2017 Avalon Consulting Group. All Rights Reserved.
1Cover Page
Client Training DayDirect Marketing 101: Testing
September 7, 2017
Avalon Consulting Group
www.avalonconsulting.net
©2017 Avalon Consulting Group.
All Rights Reserved.
2
AUDIENCE POLL
Is this you?
3
I have made major changes to a campaign
without testing them first.
4
Each organization is different and
not all best practices are universal.
When in doubt about what’s best for
your organization, test to find the
data-driven answers you need.
Ongoing testing is essential to a
healthy, productive direct marketing
program.
Testing Overview
5
Testing Goals
Get people to open your solicitation
Increase response/revenue
Determine what messaging resonates
Settle an argument
Save costs
6
QUIZ
Which Test Won?
7
8
9
Join a Nation of Gardeners as a Member of the American Horticultural Society r Yes, I
want to enhance my skills as a gardener and share my love of gardening with people nationwide. I have
completed the attached 2015 National Gardening Survey and will join the American Horticultural Society with the
enclosed donationof:
r $35$25* r $50 r $100 r $250 r $500 r Other $
*Special introductory offer! Send $25 for FULL membership & receive our exclusive Gardener’s Tote – FREE!
Join a Nation of Gardeners as a Member of the American Horticultural Society r Yes, I want
to enhance my skills as a gardener and share my love of gardening with people nationwide. I have completed the
attached 2015 National Gardening Survey and will join the American Horticultural Society with the enclosed
donationof:
r $40$30* r $50 r $100 r $250 r $500 r Other $
*Special introductory offer! Send $30 for FULL membership & receive our exclusive Gardener’s Tote – FREE!
10
11
And the winners are…
$25 (lower amount)
12
Test to get people to open your
solicitation
13
Image vs. No Image
The red/gold carrier had a higher response rate, average gift, and 16% higher rev/m.
14
Teasers and Photos
21% increase in revenue/m –response rate and avg. gift stronger
15
Deadlines
Deadline won with a 26%
higher rev/m due to a
higher average gift and
flat response rate
16
Control
Test
Plain vs. Image Carrier
Back
Plain carrier had a
26% higher
response rate and
higher average gift
Back
17
Email Tests
Send timing (day of week and time of day)
From line (e.g. organization vs. email signer)
Subject line (e.g. length, direct vs. vague, etc.)
Emoji
18
Test to increase
response/revenue
19
Ask Strategy
Test ask strings:
Different ask amounts (e.g. odd vs. round numbers)
Shorter or longer ask string lengths (e.g. adding $1,000)
Order of asks (e.g. ascending vs. descending)
Asterisk or circle driving attention to target ask vs. none
Different asks for different segments (e.g. outside lists vs. lapsed donors
in acquisition)
Monthly option
TIP: Make sure asks in the letter/email match reply or donation form.
20
ROI after 1 year
Ask String Testing
21
Discounts and Premiums
Discount brought
in 10% more
members,
discount offer
won at 90%
confidence.
22
Engagement Techniques
Surveys Member Cards
23
Mission-Relevant Engagement
Placemat
Place Cards
24
Petitions
Offline Online
25
Testing to determine what
messaging resonates
26
Aggressive vs. Cooperative
27
Part I: Pilot Acquisition Conversion
Campaign
Target Launch: October 2013
Story vs. Traditional RSVP Invitation
Concept #1: Letter from the Field Concept #2: RSVP/Invitation
28
Concept: Member Card
RSVP vs. Member Card
29
Testing across channels
30
Testing to settle an argument
31
Long-Standing Control
32
Four-Color Images
Black and white carrier generated
28% more revenue per thousand
mailed!
Performance metrics tied but
significantly lower cost for white
carrier.
33
2 Page vs. 4 Page
34
Testing to reduce costs
35
Cost Saving Tests
1st vs. 3rd class postage or an RAE vs. BRE
Test eliminating an insert or upfront premium
Bare-bones, standard package vs. more expensive and/or non-standard package
Test an E-R0 to invite people to renew online and not receive a DM renewal
Machine vs. autopen vs. actual handwriting
Use less color for more impact
Reduce personalization and resulting matching costs
Use a drop-cut format rather than a separate reply form
Use a window carrier instead of a closed face envelope
36
Measuring your test results
37
Test Set-up
To ensure valid testing results, it is important to
follow the guidelines below:
Ensure the statistical
significance of your test – use a
sample size calculator to
accurately project your panels.
Ensure the goal of the test is
clear – what will be the measure
of success?
• 20% lift in response rate? 10% lift in average gift? 15% reduction in cost?
Test one element at a
time to isolate the impact of the variable
change.
• Carrier? Copy? Ask string?
Choose tests where there is a “reason to believe” the change will improve the campaign.
• What information informs the testing decision?
38
Check Test Panels
39
Measuring Your Test
It is important to accurately read the results of your
test, beyond just “eye-balling”:
A difference in results may be insignificant and within the margin of error.
Use a calculator designed to identify significant differences before determining a winning test.
Use roll-out costs to extrapolate the true impact of the test at full quantities.
40
Have Fun!
41
Thank you!Anne SenftVice President
Avalon Consulting Group
805 15th Street, NW Suite 700
Washington, DC 20005
202-627-6529
Connect with Avalon Today!