Phil Nottingham - How the best strategies start with the right metrics

@philnottingham

Boston, Massachusetts is a lovely place, but it has a

number of problems.

Chief among these problems is…

The mayor’s office had a bright idea to fix this

And it worked incredibly well!

Except there was one problem…

The app was used primarily by Boston techies, who have a lot more money than the majority

of the population

And so the data suggested that the potholes in the rich

parts of the city were causing the most problems

G A R Y K I N G , H A R V A R D U N I V E R S I T Y

“This was an example of really bad analytics and it’s even worse because it’s the kind of thing that feels like it should work and does

work a little bit.”

What they think they’re solving for

The most problematic potholes

What they’re actually solving for

Potholes near rich people

What they should be measuring

Potholes in areas with highest density of traffic

When we don’t know how to properly read and interpret

data, gathering it often causes more problems that it solves

I think this is an endemic issue in digital marketing

1 Valuing everything by ROI

3 Problems


2 Treating all metrics as KPIs

3 Problems


2 Treating all metrics as KPIs

3 Misaligning goals, channels and content

3 Problems

1

Valuing everything by ROI

(Gain from investment - Cost of investment)

Cost of investmentROI =

What you think you’re solving for

Efficiency & efficacy of spend

So, you try to put a monetary value on all activities

The most common, and egregious way of doing this is

with last touch attribution

Session 1 Session 2 Session 3 Session 4

Last Touch Attribution

When you use last-touch attribution, this is how you end up valuing the different

parts of your website…

Product Support Blog Pricing Contact

Pretty good!

Meh?Rubbish

Great!Amazing!!!

M A T T H E W Z O O K , U N I V E R S I T Y O F K E N T U C K Y

“Once something gets measured and put into a model then everything starts optimising

around that metric.”

No-one in their right minds would read this blog post and then buy a car insurance

policy, but when you value your content by how it contributes to conversions… then you

end up trying to make that happen.

A slightly better way to measure value is a linear or

time decay model

Session 1 Session 2 Session 3 Session 4

Linear Attribution



What you’re actually solving for

Conversions

But this still has issues with any activity designed for

brand building.

Lets take the example of PR

Earlier this year, my boss Chris Savage was on BBC news

Gut feel tells you that this is really good PR coverage

Here’s what the return looked like in Google Analytics

500 visits0 Conversions

~$10,000

Even with linear or even first touch attribution, this looks like

incredibly poor return

But… maybe we can measure the value of this TV spot instead as if it were a bit of TV coverage

The way TV was historically measured was via impressions and how it contributed to brand

salience (as determined by recall surveys)

This was an ok way to measure TV 30 years ago, when there

wasn’t many other marketing or advertising channels available

But today… there’s just too much noise to make such data meaningful. Brands who still

measure in this way dramatically over value TV

A N A L L T O O C O M M O N R E S P O N S E T O T H I S P R O B L E M

“Lets temporarily stop all other brand activities so we can properly measure PR…”

But we know that PR, or advertising, or other brand building activity never

works in isolation. Stopping everything necessarily reduces the efficacy of the activity you’re trying

to measure.

So we’re left with the conclusion that…

ROI is just a bad way to measure PR

Think of it like a soccer team.

#WistiaFest / @philnottingham

“ROI” in a soccer analogy is the number of goals scored.

However… you wouldn’t measure every player on how many goals they

score, or indeed even how they contribute in terms of assists.

Goal keepers are measured by saves, defenders by tackles, Midfielders by passes up the field, and strikers by

goals and assists.

Yet you need a well rounded team, with all the different players, to be

successful.

If you measure everything by “ROI”, you’re building a team of strikers.

T H E W O R S T C L I E N T S O F A N Y S E O A G E N C Y

“How much is this link worth?”

Anyone in SEO who’s had this question knows how problematic and widespread this mentality is.




Conversions

What you should be measuring

A sensible proxy for ‘value’

Effectiveness

Sum (DA^2)

Efficiency

KPIs for PR

Sum (DA^2)

Cost

2

Treating all metrics as KPIs

Everything needs to go up. We need to get more conversions, more

traffic, more links, lower bounce-rate…longer time on page etc…


Optimising breadth of value

Here’s an example from a brand I was working with recently.

They appreciated the potential value YouTube could bring to their

marketing, and so tried to optimisetheir activity for all possible value

Views

Retention

Shares

Subscribes

Click-Throughs

Conversions

TrueView Advertising

Shorten Video

Include Share CTA

Include Subscribe CTA

Include Link to Website

Track Click-Through Conversions


Optimising breadth of value

This created a frankenstein’smonster of a video that had no value.


Nothing

What you should be measuring

One specific, qualified variable

Here’s how we’ve tried to do this at Wistia

Effectiveness

View-through Conversions

Efficiency

KPIs for YouTube Advertising

View-through Conversions

Cost

3

Misaligning goals, channels and content

Here’s a strategy I was recently handed by a global insurance

business

Strategy

Industry Finance

Audience Millenials

Content Explainer Videos

Channel LinkedIn

Goal Sign-ups

Distribution Budget £1M


Integrated Strategy

Channel

ContentAudience

The problem is that the strategy had no overlap in the channel, the

audience or the content

Strategy

Industry Finance

Audience Millenials

Content Explainer Videos

Channel LinkedIn

Goal Sign-ups

Distribution Budget £1M

No amount of budget in the world can make this strategy work.


Discrete, unconnected and fragmented problems

What you should be doingCreating content on channels appropriate for

the goal and audience

Goal

Audience

Channel

Content

Distribution

Metrics

KPI

What are you trying to achieve?

Who are you trying to reach?

Where can you reach them?

How can you meaningfully engage on this channel?

How can you get your content to the right people?

What can you accurately measure?

What’s a good proxy to measure the goal?

Strategy Questionnaire

Here’s an example of how we recently applied this plan to

Facebook video

Goal

Audience

Channel

Content

Distribution

Metrics

KPI


Brand Awareness

People working on video for social media

Facebook

Competition

We created a test video to see if there was any latent demand for this

kind of content.

It worked!

Goal

Audience

Channel

Content

Distribution

Metrics

KPI


Brand Awareness


Facebook

Competition

Promoted Posts, Word of Mouth

We then encouraged further distribution with a bit of light

influencer marketing

Goal

Audience

Channel

Content

Distribution

Metrics

KPI


Brand Awareness


Facebook

Competition

Promoted Posts, Word of Mouth

Views, Audio, Retention, Likes, Comments, Shares

Page Likes

Effectiveness

Page Likes

Efficiency

KPIs for Facebook Competitions

Page Likes

Cost

This lead to a 30% increase in overall page likes over the duration of the campaign, at a cost of about

$2 per like.

• Bad Data < No Data

• Measure everything, but always optimise for just one KPI

• Only use ROI for direct response & conversion activities

• Ensure alignment of content with channel and audience

• Tie everything back to the goal

Takeaways

Thanks!Please feel free to follow me on the Twitters!

@philnottingham

Phil Nottingham - How the best strategies start with the right metrics

Marketing