Numis Securities Media Conferenc e 2011 Facts, Figures & Fictions - Johnbillett.com 1 Facts, Figures & Fictions Presentation to the Numis Securities Media Conference, London April 14 th 2011 by John Billett
Nov 03, 2014
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
1
Facts, Figures & Fictions
Presentation to the Numis Securities Media Conference, London April 14th 2011 by John Billett
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
2
7 * 7 = 48
(!)
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
3
Data has been around for ages, but there’s much that’s “new”• Scale:
• “We are moving from samples to audits”
• Speed:• “Last year to yesterday”
• Interactive: • “Output becomes input”
• Intensity: • “More pressure for immediate results”
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
4
Loads of data
1. We struggle with the unforeseen dilemma of what to do with the unimaginable quantities of data produced & received every year
2. 2011 will generate data requiring the storage of 2,700 billion iPads
3. A single USB stick holds the equivalent of 20 tonnes of printed paper
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
5
Loads of views & loads of data • “Data are becoming a new type of
raw material that’s on a par with capital & labour” (Bain & Company)
• “We shall soon be collecting data about each & every movement, transaction and more. There will be no excuse for not living lives of organised moderation every hour of every day” (Karen Canty: Enders Analysis)
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
6
Our freedom to get it wrong• These changes do not affect our
ability to misinterpret data • Indeed the apparent speed and
alleged accuracy of data could even increase that misinterpretation
• The fundamental “rules” of data have not changed
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
7
4 + 4 = ?Depends if you’re buying or selling
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
8
Buying or selling?
Gatwick Currency Exchange
Buying Selling
Euro 1.31 1.06
US 1.82 1.49
Japan 153 121
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
9
Buying or selling?
Gatwick Currency Exchange
Buying Selling
Euro 1.31 1.06
US $ 1.82 1.49
Japan Yen 153 121
Spread
23%-26%!!The “average” doesn’t exist!
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
10
Beware the Average
The parable of the statistician who drowned in a pond 5cms deep - on average
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
11
Beware the Average
Data Set #1• Average = 50. Spread/dispersion 49-51• Average Useful
• Data Set #2• Average = 50. Spread/dispersion 25-75• Average Crap
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
12
Confusion of Cause & Effect• Advertising Sales
• Massive FMCG advertiser uses “case rate” budget setting, with sales last year setting ad budget this year
• Advertising Sales
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
13
Confusion of Cause & Effect• Price of TV Advertiser demand
• Our work for ISBA & ITV proves that the better value of TV the less advertisers spend
• Company profitability• Cash Flow• Consumer spending • Confidence
• Price of TV Advertiser demand
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
14
False Accuracy
• There are no more than two significant figures
• 87.6 & 88.4 are both 88
• % calculation is meaningless on a data base smaller than 100 observations
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
15
Journalists love data fiction• “Tesco’s pre Xmas 2010 sales have
declined”• “The rate of growth fell from 3.6%
to 2.9%”• (Source: BBC Today Programme - Radio 4)
• Tesco’s growth of 2.9% (lower than competitors’ growth rates) on significantly high sales the previous year meant Tesco had increased its market share
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
16
The Direct Marketing Data Fallacy – Certainty in Fiction #1• “For the same money ad. X generated
twice the response of ad. Y so its twice as cost effective”
• This classic flawed data analysis overlooks:-
• Previous exposure patterns• Competitive activity• Other influences• Consumer experiences
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
17
The Direct Marketing Data Fallacy – Certainty in Fiction #2• “The Google -charging for last click-
proposition, equates to cost per sales creation”
• This classic flawed data analysis overlooks:-
• Previous exposure patterns• Contacts “lost” before last click• Value of non- internet exposure• Whole journey approach
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
18
The most common flaw – “The Ecological Fallacy”• “The drawing of inferences about
individual consumers based on aggregate level data analysis, results in the correct conclusions only by accident” (Robinson, Clancy, Berger et all)
• Market segmentation; targeting; clusters; mapping; promotions; etc.
• 16-34; ABC1; High Income: etc
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
19
Conspiracy Theory, Common Sense, Coincidence & Cock Up• The Spillers Flour Graders ads
never appeared in the key baking season from Nov to Jan!
• Effectiveness? • Ad Stock? • Decay?• Financial Year Starts February
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
20
Conspiracy Theory, Common Sense, Coincidence & Cock Up• P&G grew market share &
advertised in London only Monday-Thursday with Unilever only at Weekends
• Consumer receptivity?• Cheaper ad costs?• Econometric model insight?• Insufficient airtime
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
21
Attribution lies in the eye of the beholder• Chicago• 5 year time series (Republican claim)• Police numbers increased
Teenage crime reduced • 20 year time series (Democrat claim)• New abortion laws High take up
in poorer families Fewer Criminals
Teenage crime reduced
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
22
Words implying “precision” whose reality can be subject to large variation
• AVERAGE• BENCHMARK• DATA BASE• INDEX• NORM• STANDARD• SCORING• TARGET
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
23
Measurement & Importance• You can “measure” what you can
measure, but that may not be important
• Consumers’ claims about behaviour• Consumers’ intentions to act
• What is important may not be “measurable” in conventional terms
• Mixed media exposure & influence
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
24
Reject consumers’ claims & accept consumers’ behaviour• “We assume what we are being told
is not the truth” (Amelia Torode: VCCP)
• “Data based on claimed use of digital boxes indicated the end of live viewing and death of TV advertising. Reality is viewing +15% & more ads viewed” (Jeremy Tester BSkyB)
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
25
Four widespread data facts1. Chance events are often conspicuously
misinterpreted as accomplishments or failure
2. The connection between actions & results is not as direct as we might like to believe
3. We benefit from looking beyond superficial explanations
4. Never have so many data been taken out of files & left un analysed or misread
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
26
An arduous route from data to action
• “Data & insights achieve nothing unless they reach the decision-makers and are then acted upon. Fear, politics, denial etc in most companies make this a non-trivial issue” (Patrick Barwise London Business School)
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
27
Facts, Figures & Fictions Check List• #1 Averages are misleading unless
you know the range, spread & dispersion of observations around that average
• #2 Demand to know the number of observations in any data base plus the scale of any claimed “norms” or “benchmarks”
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
28
Facts, Figures & Fictions Check List• #3 When examining any data,
always appreciate who commissioned & paid for the study & adjust accordingly
• #4 Know the issues & challenges of the business whose data is under examination
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
29
Facts, Figures & Fictions Check List• #5 Investigate & encourage time
series data, not isolated one offs
• #6 Check the data you have is complete, representative & meaningful. Avoid false attribution.
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
30
Facts, Figures & Fictions Check List• #7 Are the observations fit for your
purpose? Is there unexplained variability? Randomness, bias, error & chance are alive & well
• #8 If it ain’t simple to understand – don’t believe it. Reject Google-de-goop.
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
31
Facts, Figures & Fictions
Presentation to the Numis Securities Media Conference, London April 14th 2011 by John Billett
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
32
References & Acknowledgements
• Patrick Barwise : London Business School• “Freakonomics”: Steven Levitt & Stephen Dubner• “The Drunkards Walk”: Leonard Mlodinow• “The Ecological Fallacy”: Clancy, Berger & Magliozzi. Journal
of Advertising Research• “Facts from Figures”: Michael Moroney• Graeme Hutton: Universal McCann NY• “Market Segmentation: Fallacies & Faults”: Tonks & Farr
University of Lancaster. Marketing Review • Media Tel UK : MediaPost NY• Irene Pantazis: US Association of National Advertisers• Jeremy Tester: BSkyB• Andrew Walmsley
Numis Securities Media Conference 2011
Facts, Figures & Fictions - Johnbillett.com
33
• Johnbillett.com• 22A Vincent House• Vincent Square• London SW1P 2NB• [email protected]• 07836 200 321• www.johnbillett.com