Attribution Management Forum 3.0: How To Build Accurate Models To Solve Attribution

©2009 Third Door Media, Inc.

Attribution Management Forum 3.0: How To Build Accurate Models

To Solve Attribution

Tuesday May 5, 2009 1 PM EDT

Speakers: Adam Goldberg, Dr. Purush Papatla


Viewing Tips

Turn Off Pop-Up BlockersTechnical difficulties?

Click on “Help?” link Use Q+A box

Submitting questions to speaker Q+A session at end of webcast Use “Ask a Question” box to submit

questions Send questions at any time


Adam Goldberg, Chief Innovation Officer

Co-Founded ClearSaleing Inc. in 2006Columbus, OH

Started Google’s inside sales organization in 2003-2006New York City, NY

Started Actuate’s inside sales organization in 2000-2003San Francisco, CA

Worked for Oracle Corp. in 1998-2000 in Major Account SalesRedwood Shores, CA

Speaker and Trainer at events such as: (SMX), (SES), (DMA)


Dr. Purush Papatla

Ph.D. from Kellogg School of Management at Northwestern University

Associate Professor, MarketingSheldon B. Lubar School of Business

President and Founder; Vetra AnalyticsPublished in top-tier marketing journals

» Marketing Science» Journal of Marketing Research» Journal of Business Research» Journal of Retailing» Journal of Interactive Marketing


EVOLUTION OF ONLINE ADVERTISING

Number of Clicks

Online Conversions

OfflineConversions

AttributionManagement

PortfolioManagement


Last Click

Pur

chas

e P

ath™

Even Attribution

Exclusions

Rules Based Attribution

Mathematical Attribution Models

ATTRIBUTION MANAGEMENT HIERARCHY


THE CONSUMER BUYING CYCLE






Poll Question 1

• Currently, how are you attributing conversion credit to your various ad sources?

1. Last click2. Other attribution method


Recap of Modeling Framework From “Measuring the Immeasurable”

www.AttributionManagement.com


CONSUMER DECISIONS


DECISION INFLUENCER

What we know

Our Communications┼Paid Search┼Banner Ads┼e-mail┼Onsite Promotions┼Comparison Shopping┼Affiliate ad

Consumer Search┼Organic search┼Site visits to us

What we don’t know yet

┼Competitor Communications┼Consumer search

┼ Site visits to competitors┼ Product trials┼ …….

┼Other sources┼ Social Media┼ Word of mouth┼ Opinion sites┼ Expert opinions┼ Traditional Mass Media

Uncertainty


MODELING CONSUMER DECISIONS

┼ Build a mathematical model to predict consumer decisions┼ Using data on influencers that we are able to track and measure

┼ Representing data on influencers that we can’t yet track and measure - our

uncertainty - through a statistical distribution

┼ Calibrate the model on observed consumer decisions┼ Purchase - yes/no

┼ Purchase size - dollar volume, # of units

┼ Repeat purchases

┼ Word of mouth

┼ Etc.

┼ Test the model’s quality by comparing predicted and actual behavior


Consumer’s Decision = f (Our Communications, Consumer Search, Competitor Communications, Other Sources)

= f ([Paid Search, Banner Ads, e-mail, Onsite Promotions, Comparison Shopping, Affiliate ads], [Organic search, Site visits to us], [uncertainty])

CONSUMER DECISION MODEL


MEASURING THE EFFECTS OF KNOWN FACTORS?

We assume that each of the known influencers

has an influence potential


MATHEMATICAL MODEL FOR CONSUMER’S DECISION

* The β’s are the attribution weights


We calibrate the model on data from the ClearSaleing platform

The data includes but is not limited to:

Purchase Path™ data

Record of consumer’s decisions• Purchase/non-purchase

• Product(s) purchased• Amount spent• Repeat visits and purchases

GETTING THE ATTRIBUTIONS


Calibrate the model on the ClearSaleing data

• Find the values of β’s which will help us predict consumer decisions as accurately as possible

Model is calibrated using:

• Maximum Likelihood • Bayesian methods

The β’s are the attribution weights!

GETTING THE ATTRIBUTIONS


MODELING THE INFLUENCE POTENTIAL

Influence potential of an influencer = f (# of exposures,

when each of the exposures occurred,

decay rate of the effect of exposures)


Poll Question 2

• What challenges have you run into when trying to build an attribution model?

1. Our technology cannot track beyond the last ad clicked2. We cannot build a sound mathematical model3. We cannot incorporate offline, social media, and word of mouth advertising4. All of the above5. We haven’t tried to build an attribution model


PROGRESS SINCE LAST WEBINAR

1. Selection of businesses for the first

round of model testing

2. Identification of unique influencers

3. Set up the data for calibrating and

testing the model

4. Calibrate and test multiple versions of

the model


SELECTING BUSINESSES

Selecting businesses that:

• Have a high level of ad spend

• Wide array of advertising sources (paid search, email, banner, etc)

• We have a least 6 month of dataWe have 2+ years of data in some cases

• Seasonal variations


SELECTED BUSINESS VERTICALS

We will be developing and testing the model on nine businesses in the following verticals

• Retail – web only

• Retail – multi-channel

• Insurance

• Financial Services


Identification of unique influencers



INFLUENCER CATEGORIES

We organized the influencers into the following categories:

• Direct

• Organic Referrers (e.g., Google)

• Paid Search

• Comparison Shopping

• e-mail

• Display advertising

• Affiliate

• Social Media

• Video


UNIQUE INFLUENCERS

Each category was further sub-categorized into a number of unique influencers:

• Direct - 1

• Organic Referrers – 11 (Google, Yahoo, MSN, etc)

• Paid Search Engine – 11 (Ex: Brand vs. Non-Brand)

• Comparison Shopping – 3 (Ex: Model Number vs. Product Name)

• e-mail - 3 (Ex: Direct Response vs. Brand)

• Display advertising - 4

• Affiliate - 2

• Social Media - 1

• Video - 1


OVERALL

We have:

• 9 categories of influencers• 37 types of unique influencers across the nine

categories

Our model develops attributions for these 37 unique influencers across the nine businesses.



Set up the data for calibrating and testing the model



135 predictors

64,653 Purchase Paths™• 11,353 paths resulting in a purchase• 53,300 abandoned paths that did not end in a purchase

o A path was defined as abandoned based on some proprietary criteria

Model can explain abandonment too

Another frontier: Attributions for abandonment


To date, we have calibrated over 70 versions of the model

We plan to calibrate and test the model at least 500 more times in various forms before firming up our

conclusions• 45,000 models run across the nine data sets

Testing

• Do the estimated attributions make intuitive sense?• Is the model able to predict consumer behavior?

o Can it predict purchases?o Can it predict non-purchases?

RESULTS TO DATE


Intuitive assessment of attributions

• Findings: not yet firmed up since we have 37 unique influencers to assess across hundreds of model runs

Predictive Testing

• 85% or more of the purchases being predicted correctly• 95% or more of the non-purchases predicted correctly• Lift charts for calibration and prediction samples

o Top decile indices average between 450 and 500

RESULTS TO DATE


Consumer ratings and reviews

Social networks

Blogging

Social commerce

Instant messaging

Twitter

You Tube

RSS and multiple feeds

TYPES OF NON-CLICK/ PASSIVE INFLUENCERS


Why do we need to?

• Benefits of including non-click influencers in attribution models

• Risks of not-including non-click influencers in attribution models

Challenges

• How do we include them, if they don’t click and we don’t track??

ATTRIBUTIONS FOR NON-CLICK INFLUENCERS


Use survey data on the use of non-click influencers

• Statistically infer the likelihood of the use of each type of passive influencers by different demographic, lifestyle and psychographic segments

• Vetra is currently working on this approacho Vetra Passive Survey™

• Vetra Passive Survey™ can be used to include passive influencers in attribution models

VETRA PASSIVE SURVEY™


Monitor sources of passive influence • YouTube• Facebook• Myspace• epinions.com

Statistically infer the proportions of buyers who engage in discussions and exchanges regarding products

• Vetra has completed preliminary work on a model for this inference

Vetra Passive Proportions™ can also be used to include passive influencers in attribution models

VETRA PASSIVE PROPORTIONS™


Vetra and ClearSaleing will:

• Continue to test a number of attribution models and influencers

• Analyze the performance of models across different verticals

• Identify the best attribution models for different verticals

• Expand attribution models to include passive influencers usingo Vetra Passive Survey™o Vetra Passive Proportions™

ONGOING RESEARCH ON ATTRIBUTION AND NEXT STEPS


Poll Question 3

• What is your timeframe in switching from a last click model to an advanced attribution model?

1. Less than 6 months2. Within a year3. Within 2 years4. More than 2 years5. No timeframe



Adam Goldberg- www.attributionmanagement.com [email protected]

www.ClearSaleing.com

QUESTIONS?

[email protected]

Upcoming SMN webcast:May 12: Ask iProspect: Strategies &Tactics for World-Class SEO

May 14: Search Marketing for Small Business: The Basics for Online Success

Attribution Management Forum 3.0: How To Build Accurate Models To Solve Attribution

Business

door media

attribution model

modeling consumer decisions

clearsaleing data

testingthe model

consumer buying cycle

years of data

month of data