How TV Ads Influence Online Shopping Jura Liaukonyte, 1 Thales Teixeira, 2 Kenneth C. Wilbur 3 April 7, 2014 Media multitasking distracts consumers’ attention from television advertising, but it also enables immediate and measurable response to advertisements. This paper explores how the content of television advertising influences online shopping. We construct a massive dataset spanning $4 billion in advertising expenditures by 20 brands, online shopping behavior at those brands’ websites, and content measures for 1,269 distinct television commercials. We use a quasi- experimental research design to estimate how advertising content influences changes in online shopping data within two-minute pre/post windows of time. We also measure the effects within two-hour windows of time using a difference-in-differences approach. The findings show that direct-response tactics increase both web traffic and purchase probability. Information-based arguments and emotional content actually reduce traffic but increase sales among those who visit the brand’s website. Imagery content reduces direct traffic but does not affect purchase probability. These results imply that brands seeking to attract multitaskers’ attention and dollars must select their advertising copy carefully according to their objectives. Keywords: Advertising content, difference-in-differences, internet, media multitasking, online purchases, simultaneous equations model, quasi-experimental design, television. 1 Dake Family Assistant Professor, Cornell University, Dyson School of Applied Economics and Management, https://faculty.cit.cornell.edu/jl2545. 2 Assistant Professor of Business Administration, Harvard Business School, http://www.hbs.edu/ faculty/Pages/profile.aspx?facId=522373. 3 Assistant Professor, University of California, San Diego, Rady School of Management, http://kennethcwilbur.com. The authors thank comScore, the Cornell University Dyson School Faculty Research Program, Dake Family Endowment, and the Division of Research and Faculty Development of the Harvard Business School for providing the funds to acquire and build the dataset in this research. Teixeira thanks Elizabeth Watkins for research assistantship. Wilbur thanks Duke University for employing him during part of the time this research was conducted. We are grateful to Donald Lichtenstein, Chris Oveis, Catherine Tucker, the editor, area editor, two anonymous referees and numerous seminar audiences for their helpful suggestions. Authors contributed equally.
45
Embed
How TV Ads Influence Online Shopping - UMass …...How TV Ads Influence Online Shopping Jura Liaukonyte,1 Thales Teixeira,2 Kenneth C. Wilbur3 April 7, 2014 Media multitasking distracts
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
How TV Ads Influence Online Shopping
Jura Liaukonyte,1 Thales Teixeira,
2 Kenneth C. Wilbur
3
April 7, 2014
Media multitasking distracts consumers’ attention from television advertising, but it also enables
immediate and measurable response to advertisements. This paper explores how the content of
television advertising influences online shopping. We construct a massive dataset spanning $4
billion in advertising expenditures by 20 brands, online shopping behavior at those brands’
websites, and content measures for 1,269 distinct television commercials. We use a quasi-
experimental research design to estimate how advertising content influences changes in online
shopping data within two-minute pre/post windows of time. We also measure the effects within
two-hour windows of time using a difference-in-differences approach. The findings show that
direct-response tactics increase both web traffic and purchase probability. Information-based
arguments and emotional content actually reduce traffic but increase sales among those who visit
the brand’s website. Imagery content reduces direct traffic but does not affect purchase
probability. These results imply that brands seeking to attract multitaskers’ attention and dollars
must select their advertising copy carefully according to their objectives.
Keywords: Advertising content, difference-in-differences, internet, media multitasking, online
As computers have grown smaller and more convenient, simultaneous television and internet
consumption (“media multitasking”) has increased rapidly (Lin, Venkataraman and Jap 2013).
Numerous studies have reported large increases in media multitasking; among them, Nielsen
(2010) claimed that 34% of all internet usage time occurred simultaneously with television
consumption. Meanwhile, television usage has not fallen, with Americans still watching about
five hours per day. In fact, time spent with television and time spent with internet are positively
correlated at the household level (Nielsen 2011).
One might therefore suspect that television can effectively engage online shoppers. But
do multitaskers engage with television ads or does simultaneous media consumption steal
consumer attention away from commercials? Numerous studies suggest that engagement is
possible. Among them, Nielsen (2012) found that 27% of US viewers had looked up product
information for a TV advertisement, and 22% had looked up advertised coupons or deals
advertised on TV. Ofcom (2013) reported that 16% of UK consumers had searched for product
information or posted to a social network about a television advertisement.
The current paper studies how the content of television advertising influences online
shopping. It aims to contribute to the literature on cross-media effects by answering the
following questions: can TV advertising trigger online shopping? If so, how does it work and
what type of content is most effective?
Recent research (Zigmond and Stipp 2010, Lewis and Reiley 2013, Joo et al. 2014) has
used online search data to show that search engine queries to Google and Yahoo respond almost
instantaneously to television commercials. However, to our knowledge, no past research has
looked at the effects of television advertising on direct website traffic or online purchase data.
2
This paper not only establishes that online shopping responds to television advertising, it also
investigates how those effects depend on advertising content.
To uncover these issues, we merged two large databases of television advertising and
internet usage, and then created a third database of advertising content. The ad data represent
$4.1 billion spent by 20 brands in 5 product categories to air 1,269 distinct advertisements
365,017 times in 2010. The contents of these advertisements were coded to assess the extent to
which each one incorporated direct response tactics, arguments, emotional content and imagery.
Finally, the advertising data were supplemented with comprehensive, passively measured brand-
level website traffic and sales data from a daily sample of 100,000 consumers.
Advertising response studies are notoriously plagued by endogeneity. To address this, we
employ a quasi-experimental research design in conjunction with narrow two-minute event
windows (Chaney et al. 1991). For each ad insertion, online shopping variables are measured
within a narrow window of time prior to the advertisement. This “pre” period serves as a
baseline against which the ad’s effect is measured. The same variables are measured again in a
“post” window of the same length immediately following the ad’s insertion. Systematic
differences between the pre- and post-windows are attributed to the ad insertion. The
identification strategy is similar to the regression discontinuity approach of Hartmann, Nair and
Narayan (2011).
We also measure advertising effects on online shopping in broader two-hour windows of
time. In order to partial out unobserved category-time interactions, we use online shopping on
nonadvertising competitors’ websites as control variables in a difference-in-differences
regression framework.
3
We find clear evidence that television advertising influences online shopping. Direct
response content increases direct website visitation (e.g., directly using a URL) with a smaller
corresponding decrease in search engine referrals (e.g., indirectly via a search engine). It also
raises conversion probability. Arguments and emotional content reduce direct traffic while
simultaneously increasing purchase probabilities; the net result of these two offsetting effects is
positive for most brands. Imagery content reduces direct traffic and does not significantly change
purchase probabilities. In sum, the results suggest that advertisers must select advertising content
carefully according to their objectives.
The paper proceeds by reviewing literature on TV advertising and proposing a simple
conceptual framework. It then describes the data, model specification and the results. A general
discussion of the implications for television advertisers concludes.
2. Background Literature and Conceptual Framework
Our work is directly related to research on multimedia advertising effectiveness. Several recent
studies found evidence of synergistic effects between television advertising and internet
advertising on offline sales (Kolsarici and Vakratsas 2011, Naik and Peters 2009, Naik and
Raman 2003, Ohnishi and Manchanda 2012). Dagger and Danaher (2013) built a single-source,
customer-level database of ten advertising media and retail sales for a large retailer. They found
that single-medium advertising elasticities were highest for catalogs, followed by direct mail,
television, email and search, suggesting that direct-response channels are more effective at
increasing short-term sales than other advertising channels.
.
4
The sum of the evidence suggests that significant cross-media effects exist. However,
researchers are just starting to understand how the content of advertising in one medium might
influence consumers’ behavior in another. In an early effort, Godes and Mayzlin (2004) showed
that online discussions of new television programs had explanatory power in a dynamic model of
those program’s ratings. More recently, Gong et al. (2013) designed a field experiment to
measure the causal impact of tweets and retweets on ratings of a television program. They found
that the content of promotional messages on the internet influenced the number of people
estimated to view the promoted television program.
2.1 TV Advertising and Online Behavior
Television ads are valuable for generating awareness, knowledge and interest in new products. A
direct consequence is that effective television ads may lead viewers to seek out more information
about these products and brands (Rubinson 2009). To date, the most studied online behavior
among TV viewers has revolved around searching for advertised brands and products using
search engines (e.g., Joo et al. 2014).
Lewis and Reiley (2013) found that advertisements during the Super Bowl tend to trigger
online searches for the advertised brands immediately, within one minute, with smaller effects
noticeable up to an hour after the ad’s broadcast time. However, their analysis did not include
direct traffic to the brand’s website or online purchases, making it impossible to separate interest
in the ad’s entertainment value from interest in the advertised product. They suggested that
“other user data such as site visitation and purchase behavior could provide a more holistic
perspective…” This paper follows up on this suggestion.
Following this observation, we posit in Figure 1 that consumers have two major decisions
in response to TV ad exposure. First, they choose whether to visit the brand’s website or not. If
5
the brand’s website is very salient, this action may be achieved by a direct route, such as entering
the website address directly into the browser or clicking a bookmark. If the brand website is
unknown or not salient, the consumer might instead need to visit a search engine and then click a
referring link to the brand’s website. Second, upon arrival at the website, the consumer decides
whether to purchase or not.
[Figure 1 about here]
Zigmond and Stipp (2010, 2011) offered several case studies showing that large increases
in Google searches for branded keywords corresponded to the precise timing of brands’ TV ads
aired during the Olympics. They speculated that heterogeneity in search response to TV ads was
partly due to the brand and partly due to the ad content. They reasoned that new-product ads
should generate more online search while call-to-action ads should generate fewer searches and
more direct website visits. Therefore, we allow for differential effects of TV ad content on
consumers’ two major routes of visitation to the brand website. Next, we review the literature on
television advertising content.
2.2 Typology of TV Ad Content
Prior research has claimed that advertising content is a first-order determinant of advertising
effects. For example, Wind and Sharp (2009) said that “the most dramatic influence on short-
term effect is creative copy.”
Tellis (2004) summarized the advertising literature by explaining that advertising effects
can be classified as either behavioral or attitudinal. Behavioral effects act instantaneously, at the
moment of exposure, or shortly thereafter. Attitudinal effects operate by changing the
consumer’s attitudes and memory over a longer period of time. Using this simple dichotomy,
prior research has categorized ads into those that predominantly seek a behavioral response and
6
those that predominantly seek to influence attitudes. An ad need not focus on just one purpose;
many TV ads exhibit some elements of both types. However, in practice they are negatively
correlated as ad time is expensive and different tactics are used to reach these two goals.1 Ads
that primarily seek to elicit behavioral responses are normally called “direct-response” (Danaher
and Green 1997) while those that intend to cause attitudinal changes are often called “brand-
image” (Peltier et al. 1992).
Direct-response ads possess three characteristic elements. In order to elicit a behavioral
response from the viewer, they provide (i) a solicitation of a specific action(s), (ii) supporting
information to encourage a decision, and (iii) a response device or mechanism to facilitate action
(Danaher and Green 1997, Bush and Bush 1990). About 20% of the TV ads in the U.S. are
estimated to be primarily direct-response (Danaher and Green 1997, Peltier et al. 1992). The
literature has shown that these ‘gimmicks’ are indeed effective at eliciting immediate responses.
On the other hand, brand-image ads are used to reinforce or change attitudes regarding
how consumers perceive the brand. They do so by appealing to two processing mechanisms, the
cognitive and the affective system. Brand-image ads constitute about 75% of all TV ads in the
U.S. (James and Vanden Bergh 1990).
Ads that involve cognitive (or central route of) persuasion do so through the use of
arguments. These argument-based ads persuade by appealing to reason and relying on evidence
about the product, the price and brand information whereby viewers evaluate the merits of the
proposed arguments against their counterarguments (Petty and Cacioppo 1986, Tellis 2004).
1 A related literature uses similar typology and focuses on the trade-off between informative and persuasive roles of
advertising (e.g., Ackerberg 2001, Anderson et al. 2013, Bagwell 2007, Ching and Ishihara 2012).
7
Ads that appeal to the affective (or peripheral) system attempt to persuade customers of
the brand’s value either through the use of emotionally engaging content (Gross and Thomson
2007, Hajcak and Olvet 2008) or through visual imagery. We term the former emotion-based
ads as they attract attention and engage viewers by using emotion-inducing content such as
creative stories, warmth and humor tactics (Teixeira et al. 2012, Tellis 2004). On the other hand,
the use of multiple perceptual or sensory representations of ideas (predominantly visual) is
intended to excite the senses using sensory stimuli, concrete words, and vivid pictures. This
approach, in turn, evokes visual imagery processing in consumers and incites a process of
memorization, intent formation, or affect (MacInnis and Price 1987, Peltier et al. 1992). We refer
to this as imagery-based ad content.
[Figure 2 about here]
While all the tactics may be used within the same advertisement, constraints (e.g., air
time or production budget) typically require advertisers to focus primarily on one technique.
Figure 2 summarizes the four types of TV advertisements. Next, we use this classification to
develop expectations about how ad content affects online shopping.
2.3 TV Ad Content and Online Shopping
We expect the effect of TV ads on online shopping to be driven by media multitasking, an
activity in which consumers divide attention between the television set and a secondary screen,
the computer.2 Therefore, we expect the level of attention needed to process each type of
advertising content to influence how that content affects online shopping behavior (Teixeira
2 ComScore only measured internet usage only on desktops and laptops in 2010; at that time it had not yet developed
tracking technology for tablets or smartphones.
8
2014). Because there is no extensive literature on which to base formal hypotheses, we only
provide informal conjectures.
In thinking through the possible influence of TV ads on online shopping, it is necessary
to consider the role the brand’s website might play. Broadly speaking, the brand website can
serve two roles: it could be a channel for selling (e.g. providing product information and further
persuasion), or it could be a channel for order fulfillment. Ad content that stimulates interest
without providing much information would be more effective in conjunction with a brand
website that is a channel for selling. Ad content that provides extensive information would be
more effective in conjunction with a website that is a channel for order fulfillment. For example,
Hans et al. (2013) showed that some claims in text search ads are more effective for generating
click-through (promoting the site) while others generate less traffic but are better at increasing
conversions (persuading to buy). Similarly, Wu et al. (2005) found that some magazine ad
formats were more effective at generating site traffic while other formats brought less traffic to
the site but that traffic converted at higher rates. Anderson and Renault (2006) formally modeled
this trade-off; in equilibrium, a rational consumer’s willingness to incur a search cost (e.g. visit a
website) is greater when the firm provides partial information about product attributes and price
than when it provides full information.
Although we do not observe brand website content in the dataset described below, these
thoughts about the role of the website helped to shape our expectations about how TV ad content
might influence online shopping. Similar to magazine and search engine ads, TV commercials
may attempt to persuade viewers to visit a brand’s website or to make a purchase online. While
both approaches might result in a purchase, it is important to distinguish the ad’s ability to
9
generate traffic from the ad’s ability to generate sales. Next, we relate the four types of TV ad
content to their expected impacts on website visitation.
We expect direct-response ads to increase website visitations because, by their nature,
they are created to cause consumers to act immediately. Immediate action lends itself well to
media multitasking as these ads actively encourage their viewers to make use of another medium
to respond (e.g., “call now,” “go online,” “visit us,” etc.). We also expect this time of advertising
to make the web address more salient in consumers’ minds, leading to a greater impact on direct
traffic than on search engine referrals. Argument-based ads, on the other hand, make use of
content that requires heightened attention and cognitive processing. For this reason, some
viewers might not be motivated to exert the necessary effort to process the arguments in the ad
and this will reduce the likelihood that media multitaskers actively seek additional information
from the advertiser on the Internet by directly visiting the website or via search engines. This is
not to say that argument ads do not trigger interest. Rather, we expect that the desire to act
quickly is much less than from direct-response ads, which induce an impulsive act (Doyle and
Saunders 1990, Wood 2009).
As for emotion-based ads, they do not require an intense cognitive processing. We expect
emotional ads to increase both routes to website visitations by media multitaskers as they do not
require heightened attention to process the message. Further, by changing attitudes, emotions act
as a trigger for action (Gross and Thomson 2007). Lastly, while imagery-based ads also generate
affect, they can reduce people’s desire to go online as the sensory stimulation is likely to keep
viewers’ attention focused on the TV screen and less on other competing media. Thus, by
evoking strong visuals and sensory stimulation, viewers may feel less compelled to switch from
10
television, a stimulating and fast-paced medium, to the internet, a slower and self-paced medium
(Berlyne 1971). Next, we conjecture the impact of the four types of TV ad content on purchase.
We expect that, by focusing on consumer actions, direct-response ads that use the web as a
fulfilment channel will increase online purchases above and beyond that which results from more
website visits. Argument-based ads are expected to increase online purchases as well, but
because they focus on the product and brand. As for the affective-laden ads, emotion-based ads
should also increase purchases as they provide peripheral cues that entertain and persuade
viewers to evaluate the brand favorably (Teixeira et al. 2013). Contrary to the other ads however,
we expect that imagery-focus ads will reduce the viewer’s likelihood of purchasing online in the
short run as imagery offers a positive sensory experience that acts as a palliative substitute for
actual product consumption, delaying purchase (MacInnis and Price 1987). In the next section
we describe the data, sample selection and key measures used in the empirical model.
3. Data
We merge two large datasets of television advertising and internet behavior in 2010 and
construct a database of advertising content. Given the huge databases involved, the analysis
focuses on 20 brands in five product categories with extensive online shopping activity: dating,
pizza delivery, retailers, telecommunications, and travel.
3.1. Web Traffic and Transactions Data
Online traffic and transactions data were collected from comScore Media Metrix. ComScore
used proprietary software to passively track all web usage on a sample of two million internet-
connected desktops and laptops. The data contained information about the Uniform Resource
Locators (URL), date and time of each web page visited. Due to the substantial costs of data
11
retrieval, comScore randomly selected 100,000 machines each day and only retrieved internet
usage data from these machines (Coffey 2001).
ComScore reports the web browsing data at the level of the user/website session.
Consistent with standard industry practice, a new session is recorded when a user first initiates a
page view from a particular domain (e.g., Amazon.com) after not viewing any page from that
domain in the past 30 minutes. The choice of 30 minutes is commonly made because many users
stop looking at webpages without closing a browser tab, so some assumption is required about
the point at which the user stopped interacting with the site.
For each user/website session, comScore reported an anonymous user ID, the domain
name (brand website), domain name of a referral website (if any), and the exact date and start
time. Further, comScore identified paid transactions by analyzing the structure of confirmatory
URLs for all but a few brands it tracked.3
The internet usage database has several limitations that are important for interpreting the
results below. First, the data are a daily cross-section drawn from a panel, but not a panel in and
of themselves. Therefore, we analyze the data by aggregating users’ session data within specific
windows of time; we refer to traffic and transactions as the aggregate counterparts to individual
visitation/browsing and purchasing decisions. Second, the data do not track individuals across
computers (a common issue in internet usage data). Third, at the time the data were collected,
comScore only measured internet usage on desktops and laptops; it had not yet developed
tracking technology for tablet computers. In 2010, smartphone penetration was 22% and major
brands of tablet computers had just come on the market; both devices were generally less
3 Prior research in marketing has analyzed comScore data from 2002-2004 (e.g., Moe and Fader 2004, Park and
Fader 2004, Montgomery, Li, Srinivasan, and Liechty 2004, Danaher 2007, Johnson et al. 2004).
12
suitable for online shopping than desktops and laptops (Nielsen 2010). By 2014, smartphones
and tablets had become more capable and their respective penetration rates had risen to 65% and
29% (Nielsen 2014). One might suspect that the effects estimated in this paper are a conservative
estimate of the current importance of online response to television ads.
Figure 3 summarizes the online shopping data by plotting traffic and transactions within
each product category by hour of the day. In four of five categories, brand website traffic and
transactions are surprisingly flat throughout the day, from about 9 A.M. until 9 P.M., with a peak
in the early evening at 7 P.M. Eastern Time. The exception is pizza, which has a more
pronounced peak in online shopping at dinnertime.
[Figure 3 about here]
3.2. Television Advertising Data
Television advertising data were recorded by Kantar Media. Kantar continuously monitored all
national broadcast and cable networks in the U.S and identified advertisements using codes
embedded in networks’ programming streams. Each unique combination of a commercial
message, television channel, date and time is referred to as an advertising “insertion.” For each
insertion in 2010, the database reports the commercial’s duration, the brand, the date and start
time (in hours, minutes, and seconds E.S.T.), and an estimated cost of the insertion. Cost
estimates were reported to Kantar by the networks after ads aired and are commonly used by
large advertisers to plan upcoming media buys. The data also record the specific product
advertised within the advertisement, as many brands advertised multiple different products.
Finally, the database report several properties of the program into which the ad was inserted: the
“property” (defined as a national television network or program syndication company), program
13
name, program genre, the number of the slot during the commercial break when the appeared,
and the number of the commercial break within the program.4
The data included more than 750,000 insertions of 4,153 unique advertising creatives in
national networks. We dropped the bottom 5% of creatives by total expenditure, and all
insertions whose estimated cost to broadcast was less than $1,000, as these corresponded to
channels and dayparts with very small audiences. These two refinements reduced the number of
insertions by about half but eliminated just 6% of total observed ad spending. The final
estimation sample consists of 365,017 insertions of 1,269 unique advertisements accounting for
$4.1 billion of TV ad spending by 20 brands in 2010.
Like the online shopping activity in Figure 3, Figure 4 shows numerous advertising
insertions occurred between about 9 A.M. and 7 P.M. The number of ad insertions dropped but
advertising expenditures rose considerably during the prime time hours of 8-10 P.M.
[Figure 4 about here]
3.3. Research Design, Model-Free Evidence and Descriptive Statistics
We measure brand-specific shopping variables twice for each ad insertion and each window
length. The baseline rates of online shopping variables are measured in a “pre” window of time
just prior to the insertion of the advertisement. These same variables are measured again in a
“post” window of time just after the ad starts. Any systematic differences between the online
shopping variables measured in the “pre” and “post” windows will be attributed to the
advertisement itself.
4 The database did not report program name, genre, break number or slot number for 36,805 ad insertions carried by
a particular group of program syndication companies. We decided to drop these 10% of insertions from the sample.
The results of primary interest (tables 5 and 6) are essentially invariant to including or excluding these insertions.
14
The online shopping variables of interest are brand website traffic, either direct or via
search engines, and transactions. They are defined as follows.
Direct Traffic (DIR): the number of new sessions on a brand’s website that were initiated by
direct means (e.g., URL entry or clicking a bookmarked link) within a particular time window.
Search Engine Referrals (SE): the number of new user sessions on a brand’s website that were
initiated by search engine referrals within a particular time window. Six search engines (AOL,
Ask, Bing, Google, MSN and Yahoo) are included, accounting for 99% of U.S. searches.5
Transaction Count (TC): the number of new sessions on a brand’s website that are initiated
within a particular time window and where a transaction is completed within 24 hours. Purchase
decisions may take much longer than site visits, as they may be delayed by time spent reading
reviews, researching competing options or consulting other members of the household. Thus, a
one-day window was employed similarly to Blake et al. (2013).
It is important to note that the difference between sessions and pageviews (described
earlier) ensures that the same machine will not be counted in both the pre and post windows in
the two-minute data. If a given machine initiates a new session during the two-minute “pre”
window, comScore’s definition of a session ensures it will not be counted again in the two-
minute “post” window, as 30 minutes have not elapsed between pageviews.6
Several exploratory analyses were conducted using subsets of the data. In one, we plotted
traffic to brand websites corresponding to different ad creatives. Figure 6 shows Amazon.com
traffic for two distinct ads: (a) “available now” and (b) “Kindle.” The data showed a large spike
5 A limitation of this the data is that this measure does not indicate when the user initiated the search.
6 It is possible but highly unusual for a single machine to be counted in both the “pre” and “post” windows in the
two-hour dataset. If a machine’s last visit to a brand webpage is more than 30 minutes prior to an ad insertion, and
then the machine is observed to visit the brand’s webpage again during the two hours following the ad insertion,
those will be counted as one new session during the two-hour “pre” window and one new session during the two-
hour “post” window.
15
in the minute following the start of the ad and a small, enduring increase thereafter. The
magnitude of these lift patterns seemed to depend on the ad content, highlighting the importance
of more formal investigation of the impact of ad content on web visitations.
The second exploratory exercise involved plotting browsing activity within shorter time
windows for a wider selection of brands. Figure 7 illustrates this for Target and JC Penney’s.
Most of the immediate uptick in browsing activity was observed within two minutes after the ad,
with some effects persisting up to two hours after the ad. A similar pattern appeared for all of the
brands analyzed in this manner. This is how we chose the two particular window lengths of two
minutes and two hours.7 The online appendix offers a concrete example of how the online
shopping variables are measured within each of these two windows.
[Figures 6 and 7 about here]
Table 1 provides advertising and online shopping data for the 20 brands in the dataset.
The average brand created 64 different commercials to advertise 7 distinct products, and spent
$204 million to air those commercials 18,251 times. Consumers initiated 49,402 direct sessions
on the average brand’s website, with an additional 23,061 sessions coming from search engine
referrals. 6.3% of the machines that were observed to initiate those sessions completed a paid
transaction or subscription within 24 hours. Table 2 offers some back-of-the-envelope
7 Although an ideal approach would be to gauge the sensitivity of the analysis to the length of the window chosen,
this was judged to be infeasible due to computational costs. This was an unusually complex data merge; to our
knowledge, it has not previously been offered by any commercial research firm. Due to the sheer size of the
datasets, our merge routine required 3*1013
computational queries and about 45 days to run. Section 5 indicates
some agreement in the results based on the two chosen window lengths, suggesting that small adjustments the
window lengths might be unlikely to change the qualitative findings.
16
calculations showing that, under conservative assumptions, 13.5% of the online purchases in the
data may have been a direct result of in-sample advertisements.8
3.4. Television Advertising Content Data
The third dataset was created specifically for this paper by coding the contents of the 1,269 TV
advertisements. Most prior academic efforts to analyze advertising content have manually coded
a few dozen ad creatives9. Our data collection effort contained 1,269 unique ad creatives, 21 ad
content items per creative and spanned multiple brands and categories. Given the size of the task
at hand, we opted for a three-step procedure involving item coding, assessment of reliability and
classification validation.
We first used the literature to identify and define the four ad types (Direct-response,
Argument-based, Emotion-based and Imagery-based) by which each TV commercial in our
dataset to be classified. This ad typology is defined and presented in Section 2.2 and summarized
in Figure 2.
Using these definitions, we selected 21 ad content elements to code from prior academic
analyses of advertising content. All ads were coded on the basis of these items. Given the large
number of ads to be coded, we recruited ten coders and assigned each ad to only one coder who
viewed it multiple times and coded it on the basis of the items chosen. A subsample of ads was
later re-coded by a new group of six independent coders following the same procedure to
measure inter-coder reliability. Finally, we submitted the proposed classification along with the
8 We advise strong caution in interpreting these calculations as they rely on several untested assumptions and they
are intended solely as an illustration. However, they do suggest that a significant number of the observed
transactions were caused by TV ads in the sample. 9 Unusually large exceptions are Buijzen and Valkenburg (2004), who identified the presence of 41 types of humor
in 316 advertisements, and Anderson et al. (2013) and Liaukonyte (2013), who coded the product attributes
communicated by 1,571 OTC pain medication ads..
17
items pertaining to each ad type to an expert panel of 14 academics to validate or refute the
groupings. The details of each step follow.
Item selection. 21 question items were created to measure the prevalence of the four types
of content features in each ad. The features were chosen to identify direct-response elements
(e.g., call to go online, online contact information, call to purchase), arguments (e.g., product-
related, price-related, brand-related), emotion-inducing elements (e.g., story, humor, warm
feeling content) and sensory elements (e.g., visually pleasing, sensory stimulation).10
Feature coding. Ten research assistants were trained to code the advertisements. Coders
were instructed to watch each ad at least twice and then answer the 21-item questionnaire for that
ad. During coding, they could watch, pause and rewind the ad as many times as needed. If they
still remained unsure about how to code a particular ad, they were instructed to inform a research
associate. Over 99% of ads were coded completely the first time. Coders worked independently,
were paid hourly, and instructed not to work more than two hours at a time in order to avoid
respondent fatigue.
Coding reliability. A separate group of six assistants were hired to code a random sample
of 150 ads for eight of the brands (12% of the original 1,269) following the same procedure. We
dropped two survey items (“Is the product demonstrated in the ad?” and “Is the focus of the ad
more on the product or on the brand?”) due to low inter-coder reliability. The percentage match
among the remaining 19 survey items was 78%. We judged this figure to be acceptable given the
subjective nature of some of the survey items and the coders’ inability to resolve discrepancies
through discussion.
10
19 of the 21 survey items used binary response scales (presence/absence of element). For two items a three-point
scale (predominance of one element, predominance of another element or neither) was used.
18
Classification validation. In order to validate the choice of items used for each ad type,
we surveyed 14 academics from top-tier schools around the world who are experts on consumer
behavior research. We asked whether each item was applicable, somewhat applicable or not
applicable to the ad type that it was associated with. Only one of the items (“Would you judge
this to be an expensive or cheap ad to make?”) had a high rate of disagreement with the original
classification, at 50%, and was therefore dropped from the study. On average, the academics
surveyed agreed with the applicability in 97% of the remaining item/grouping combinations,
with every item-specific agreement score exceeding 85%. In the end, 18 survey items were used
to create indices based on the sum of each advertisement’s item responses within each group.
The survey items by ad type are provided in Table 3.
[Table 3 about here]
Descriptive Statistics. Table 4 describes how brands differ in their use of advertising content. For
example, Papa John’s made the heaviest use of direct-response ads in the sample, while
Victoria’s Secret ads rated the lowest on this type. However, while there are differences across
brands, standard deviations across creatives within a brand are sometimes comparable to the
standard deviations across the entire sample. In sum, every brand used every type of ad content
in its advertisements.
[Table 4 about here]
4. Model and Estimation
We model the causal effects of TV advertisements on three online shopping variables—search
engine referrals (SE), direct traffic (Dir) and transaction counts (TC)—using a system of linear
equations. Let i index advertisement insertions. Each insertion i promotes a particular brand and
19
product in a particular product category and corresponds to a particular date and time; we denote
these ib , ip , ic and it , respectively.
Let ),,( POST
i
POST
i
POST
i
POST
i TCDirSEY be a vector of the three online shopping variables
for brand ib measured within a window of time (either two minutes or two hours) immediately
following it . Let PRE
iY denote a vector of the same three variables for brand ib measured in a
window of time (of the same duration) immediately preceding insertion i.
In explaining our approach to estimating causal effects, we adapt the notation of Angrist
and Pischke (2009). We distinguish the online shopping variables we observe after an ad
insertion, denoted POST
iY1 , from the same online shopping variables we would have observed in
the same “post” time window had insertion i never occurred, denoted POST
iY0 . In other words,
POST
iY0 is the baseline level of online shopping while POST
iY1 is this baseline plus the treatment
effect of the television advertisement.
In the absence of an ad insertion, then we would expect the online shopping variables
(POST
iY0 ) to be influenced by their past realizations (PRE