Webinar: Survival Analysis for Marketing Attribution - July 17, 2013

Revolution Confidential

Survival Analysis for

Marketing Attribution

Webinar

July 2013

Andrie de Vries

Business Services Director – Europe

@RevoAndrie

Revolution Analytics

@RevolutionR

Revolution ConfidentialWho am I?

CRAN package ggdendro

StackOverflow

Revolution Analytics Webinar, July 2013 2

Revolution ConfidentialFrom Toledo to Albacete

Revolution ConfidentialRent a car?

Revolution Confidential… or take a bus?

Revolution Confidential… but what’s happening here?

Revolution ConfidentialMarketing attribution: the Question

How to attribute conversion success to marketing spend?

Where to spend the next marketing dollar?

Revolution ConfidentialAgenda

Digital marketing attribution

Using Survival models

At scale, on big data

Revolution ConfidentialTwo-click conversion journey

Click1:

Open landing page

Click 2:

Sign up to offer

Revolution ConfidentialTypical conversion journey…

Impressions

• Banner ad

• Page post ad

• Sponsored tweet

• Search ad

• Landing page

• Special offer

• Application form

Conversion

• Sign up

• Ask for more detail

11Revolution Analytics Webinar, July 2013 11

Revolution Confidential…but no two journeys are the same…

Impressions

• Banner ad

• Page post ad

• Sponsored tweet

• Search ad

• Landing page

• Special offer

• Application form

Conversion

• Sign up

• Ask for more detail

12Revolution Analytics Webinar, July 2013

Person 2

Person 3

Person 1

Attribution models

Last click only

All events even

Rule based

Statistical modelling

…so how to attribute the value?

13Revolution Analytics Webinar, July 2013

Person 2

Person 3

Person 1

Revolution ConfidentialAttribution with statistical modelling

Regression

In many cases, log data is available only

for conversions

And when non-conversion data is available,

these people may convert in near future

Revolution ConfidentialAttribution with statistical modelling

Regression

In many cases, log data is available only

for conversions

And when non-conversion data is available,

these people may convert in near future

Survival analysis

Use time to conversion as dependent

variable

Can use each interaction (view or click)

as an observation

Can include censored (incomplete) data

No need to flatten the data

Revolution ConfidentialSurvival models

Kaplan-Meier survivor function

Cox proportional hazards model

𝑆𝑘𝑚 =

𝑡𝑖<𝑡

𝑟 𝑡𝑖 − 𝑑(𝑡𝑖)

𝑟(𝑡𝑖)

𝐿(𝛽) = 𝐿𝑖(𝛽)

𝐿𝑖(𝛽) =𝑟𝑖 𝑡∗

𝑗 𝑌𝑗 𝑡∗ 𝑟𝑗 𝑡∗

𝜆 𝑡; 𝑍𝑖 = 𝜆0(𝑡)𝑟𝑖(𝑡) Hazard function

𝑟𝑖 𝑡 = 𝑒𝛽𝑍𝑖(𝑡) Risk score

Likelihood that

individual i dies

Partial likelihood

> library(survival)

> Surv(…)

> library(survival)

> coxph( Surv(…) ~ …)

Revolution ConfidentialWhat is death?

Revolution Analytics Webinar, July 2013

Medicine: actual death of patient

Engineering: failure of component

Revolution ConfidentialWhat is death?

For attribution: cookie conversion

Worked example

Attribution of digital media for

telecoms client

Revolution ConfidentialRead the data

> rdsFile <- "survival_data.rds"

> xd <- readRDS(rdsFile)

> class(xd)

[1] "data.table" "data.frame"

> nrow(xd)

[1] 775782

> ncol(xd)

[1] 31

Revolution ConfidentialWhat does the data look like?> xd[1:25, 1:6, with=FALSE]

id Conversion.Time Event.Number Event.Time Event.Type Campaign

1: 10101:49721794 01/10/2012 00:05 1 01/10/2012 00:02 Click Free Sims

2: 10101:49721801 01/10/2012 00:05 1 29/09/2012 16:25 View BAU High Media

Revolution ConfidentialHistogram of cookie lifetime

Impressions and clicks in customer journey

Cookie duration (days)

clicks)

0 5 10 15 20 25 30

Revolution ConfidentialFitting the model

> library(survival)

> fitp <- coxph(

Surv(times, event=Converted) ~ Type +

Event.Type +

Supplier +

PrevClicks +

AdFormat,

data=xd)

Revolution ConfidentialWhat does the data say?AdFormat Event.Type PrevClicks Supplier Type

60x600

Leaderb

- 728x90

00x250

Specific

Google

Facebook A

AdFormat Event.Type PrevClicks Supplier Type

60x600

Leaderb

- 728x90

00x250

Specific

Google

Facebook A

What does the data say?

• Advertise the right product!

• Some suppliers are better at generating

conversion

• But note the data wasn’t an unbiased

experiment!

Revolution ConfidentialEstimated hazard function

> x <- survfit(fitp)

> xx <- with(x, data.frame(time, surv, upper, lower))

> ggplot(xx, aes(time, surv)) + geom_step() …

Revolution ConfidentialWhere Revolution helps

Import

• Text formats

• SAS

• High-speed database

• Hadoop

Pre-process

• DataStep

• Clean

• Refactor

• Sort

• Merge

Analyse

• Cube

• Summarise

• Parallelise (rxExec)

• Regression

• GLM

• Tweedie

• Clustering

• Decision trees

• Predict

Deploy

• Web services

Confidential to Revolution Analytics 29

Revolution R Enterprise

Parallel external memory algorithms (PEMAs)

Revolution ConfidentialCase study: Datasong

Profile:

Multi-channel marketing analytics

Software developer and service provider

Growing, innovative, cost-conscious

Technology:

Revolution ConfidentialModeling the Baseline Hazard

Capture nonlinear trends in

baseline, while overlaying

marketing treatment variables

as well as other customer

attributes

Revolution R package used:

• RevoScaleR

Revolution R functions used:

• rxImport()

• rxSummary()

• rxCube()

• rxLogit()

• rxPredict()

• rxRoc()

Revolution ConfidentialTransformations

Catalog Email

Revolution ConfidentialOutcome

Massively scalable infrastructure

Attribution and optimization at individual customer level for clients

such as Williams-Sonoma

Client saved $250K in one campaign

Rapid deployment of customer-specific models

Innovative techniques, e.g. GAM Survival models

Performance improvement

Experienced 4x performance improvement on 50 million records

www.revolutionanalytics.com Twitter: @RevolutionR

The leading commercial provider of software and support for the popular open source R statistics language.

Webinar: Survival Analysis for Marketing Attribution - July 17, 2013

Technology

Ultimate Survival Kit to share - · PDF fileBuilding...

Attribution from a Research Library Perspective, on NISO...

Getting Away from Widgets: Design Thinking as a Survival...

ClearSaleing-Forrester Webinar Excerpt - Interactive...

Stay Tuned! At the end of this Webinar, Bliss will announce....

Webinar 7: Attribution Patient Roster Layout...

eMarketer Webinar: Making Attribution Work in the Age of Big...

DIGITAL MEDIA ATTRIBUTION - Smart Insights · Media...

Webinar Slides: The Mobilegeddon Survival Kit for Higher...

Morningstar Total Portfolio Performance Attribution ... ·.....

Attribution Playbook Webinar 2

RSWUS Agency of the Future Webinar Slide Deck – Does your....

Attribution Playbook Webinar 4

Attribution Webinar 12-2014

PPC Attribution: A Portent & Hanapin Webinar

Webinar 8: Performance Measurement and Attribution