Appendix starts on page 20. Tabloid Boycott Decreases Euroscepticism Florian Foos 1* & Daniel Bischof 2 1 Department of Government, London School of Economics & Political Science 2 Department of Political Science, University of Zurich * To whom correspondence should be addressed; E-mail: [email protected]. November 17, 2019 Are citizens’ attitudes towards EU-integration shaped by the tabloid media? The question whether public opinion can be a consequence, rather than a cause of me- dia reports is difficult to answer because citizens self-select into media consumption. We use a quasi-experiment, the boycott of the most important Eurosceptic tabloid newspaper, the Sun, in Merseyside county as a consequence of the newspaper’s re- porting on the 1989 Hillsborough soccer disaster – to identify the effects of the Sun boycott on attitudes towards leaving the EU. Using a difference-in-differences design and British Social Attitudes data spanning three decades, as well as official EU refer- endum results, we show that attitudes towards the EU got significantly more positive in Merseyside during the boycott. The results of this paper have important implica- tions for our understanding of media effects on public opinion, and suggest that the tabloid media played a role in influencing attitudes towards leaving the EU. word count = 4’452 Introduction The question if specific media outlets are able to shape public opinion is central to the social sci- ences and also extensively debated in the public domain and media (1–5). At least since the 17th century, democratic theorists assign a crucial role to the press, either in informing and enlight- 1
43
Embed
Tabloid Boycott Decreases Euroscepticism - Florian Foos · Appendix starts on page 20. Tabloid Boycott Decreases Euroscepticism Florian Foos1 & Daniel Bischof2 1Department of Government,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Appendix starts on page 20.
Tabloid Boycott Decreases Euroscepticism
Florian Foos1∗ & Daniel Bischof2
1Department of Government, London School of Economics & Political Science2Department of Political Science, University of Zurich
∗To whom correspondence should be addressed; E-mail: [email protected].
November 17, 2019
Are citizens’ attitudes towards EU-integration shaped by the tabloid media? Thequestion whether public opinion can be a consequence, rather than a cause of me-dia reports is difficult to answer because citizens self-select into media consumption.We use a quasi-experiment, the boycott of the most important Eurosceptic tabloidnewspaper, the Sun, in Merseyside county as a consequence of the newspaper’s re-porting on the 1989 Hillsborough soccer disaster – to identify the effects of the Sunboycott on attitudes towards leaving the EU. Using a difference-in-differences designand British Social Attitudes data spanning three decades, as well as official EU refer-endum results, we show that attitudes towards the EU got significantly more positivein Merseyside during the boycott. The results of this paper have important implica-tions for our understanding of media effects on public opinion, and suggest that thetabloid media played a role in influencing attitudes towards leaving the EU.
word count = 4’452
Introduction
The question if specific media outlets are able to shape public opinion is central to the social sci-
ences and also extensively debated in the public domain and media (1–5). At least since the 17th
century, democratic theorists assign a crucial role to the press, either in informing and enlight-
1
ening citizens (for a summary of these arguments, see: (6)), or in manipulating them to support
or oppose specific issues (3, 7). While theorists hence believe that the media are powerful in
influencing citizens, empirical research has been more sceptical (for a summary of the ”minimal
effects literature see (8)). Most importantly, researchers have faced severe methodological chal-
lenges in causally identifying the persuasive effects of the media on citizens’ attitudes (9–11).
Citizens choose which media outlets they consume (selection effects), and the media not
only set the agenda, but are also responsive to public opinion (reverse causation). Thus, even
if studies find strong associations between media consumption and public opinion, it remains
unclear whether the media are able to shape public opinion (12). While exposure to media
slant can be manipulated in the lab and even in the field (11), experiments usually only expose
citizens to media content once, and measure outcomes over a short period of time. While
we know that changes in opinions based on persuasive content are possible, particularly on
emerging issues (13), in real world campaigns, effects of one-off exposure appear to decay
rapidly (14, 15).
Therefore, one of the key challenges in the study of media effects is to identify the conse-
quences of repeated, sustained exposure to media content. When commentators worry about
the persuasive power of the media, they usually refer to sustained campaigns by a medium
meant to influence public opinion in a specific direction. Researchers have successfully used
quasi-experiments to identify the effect of exposure to a specific medium on voting inten-
tions in the United States (16). Moreover, evidence from non-democracies (17, 18) suggests
that the consumption of a medium can have important persuasive effects. Our study uses the
quasi-experimental approach to identify the effects of the sustained campaign of an important
medium, the Sun, on an issue of paramount policy importance, EU (dis-)integration.
The world’s largest Customs Union – the European Union (EU) – not only faces a severe
debt crisis but also an unfolding disintegration crisis. Most prominently, the United Kingdom
2
held a referendum asking its citizens whether or not to leave the European Union in 2016. After
a heated and polarized campaign, 51.89 % of voters supported “Brexit”. During and after the
“Brexit” campaign, observers, pundits and journalists wondered about the influence Eurosceptic
slant in the print media, particularly in the tabloid media, had on the rise of Euroscepticism in
the United Kingdom (19, 20). However, researchers have so far mostly emphasized structural
causes, focusing on left-behind and working class voters (21). At the same time, exactly these
voter groups have been most exposed to Eurosceptic tabloid media coverage since the mid
1980s, which means we face a severe challenge in identifying the causal effect of the tabloid
media on Euroscepticism.
Research design
To shed light on the question whether tabloid media consumption affects public attitudes to-
wards the EU, we use a quasi-experimental design. Our design rests on a specific historical
event, the Hillsborough sporting disaster, a human crush at the Hillsborough soccer stadium in
Sheffield (England), which led to a massive and long-lasting boycott of the Eurosceptic tabloid
“The Sun” in Merseyside county, the area surrounding the city of Liverpool in the United King-
dom. On 15 April 1989 Liverpool F.C. played Nottingham Forest in the semi-finals of the
British Football Association (FA) Cup in the Hillsborough stadium. Due to overcrowding and
an ensuing mass panic, ninety-six Liverpool supporters lost their lives, hundreds were injured
and thousands traumatized (22–24).1 The Sun’s sensationalist coverage of the disaster was
particularly one-sided and falsely claimed that “the truth” about the disaster was that Liver-
pool supporters were responsible for the chaotic escalation, and ultimately, for their own death
1Originally the match was scheduled to start at 3 pm. Yet, approximately at 2.30 pm large crowds — pre-dominantly Liverpool F.C. supporters — started gathering in front of the stadium. Police officers then decided toopen the exit gates in order to ensure that the masses could enter the stadium on time and enjoy the soccer match.This uncontrolled in-streaming of ever more people led to a overcrowding of the stadium, specifically of the sidepens (25).
3
(see supplementary materials (SM) S1.1). This version of the event was comprehensively con-
tradicted by multiple reports released by the official Hillsborough Independent Panel, which
pointed to the catastrophic mishandling of the situation by the police (26). The sustained,
misleading reporting by the “The Sun” led to an unprecedented backlash and boycott of the
newspaper in Merseyside, the home of Liverpool F.C.. The Guardian estimates that in the wake
of the Hillsborough disaster, the Sun’s circulation fell from 55’000 copies to just around 12’000
copies in Merseyside, although this figures cannot be independently verified because the Sun
refuses to release circulation figures at this level of aggregation (27). Despite the Sun’s unre-
served apology in 2012 under the headline “Hillsborough: The real truth”, which admitted that
their reporting on Hillsborough was the ‘gravest error” in the tabloid’s history and acknowl-
edged that “the people of Liverpool may never forgive us for the injustice we did to them” (28),
the boycott is ongoing to this day (29).
The occurrence of the Hillsborough disaster gives us the rare opportunity to identify the
causal effect of a sustained media boycott on attitudes towards the EU because the circulation
of the most important Eurosceptic tabloid was significantly reduced due to a sporting disaster,
which was unrelated to the tabloid’s EU coverage. Readers did not choose to stop reading the
Sun due to its EU coverage, but they did so due to a clear cause, which is exogenous to the
Sun’s anti-EU campaign. Thus, our research design addresses the methodological issues of
selection and reverse causation discussed above. Importantly, before the boycott, the Sun was
read mostly among working class voters (see SM S8), exactly those citizens who are the focus
of much of the Brexit coverage.
Given the strong anti-EU campaign of the Sun over the past 25 years (SM S1.2), we expect
that the boycott should deprive potential readers of their main source of Eurosceptic informa-
tion and opinion over the long term. This is due to a shock to both demand and supply. While
initially, the boycott was mostly driven by football supporters, their family and friends stopping
4
to purchase the tabloid in protest to the Sun’s coverage, soon the boycott was coordinated by
the Hillsborough justice campaign, and most news agents stopped stocking the Sun (30). A
clear social norm of boycotting the Sun developed, which does not only extend to reading the
paper, but also to interviews by public figures such as Merseyside MPs and celebrities who
are publicly sanctioned when breaking the norm (31, 32).2 We expect both an initial backlash
against the Sun and the major issue-based campaign it was associated with, Euroscepticism, and
a long-term effect of non-readership as former readers break their habit of consuming Euroscep-
tic information on a daily basis. Specifically, we hypothesize that as a direct consequence of
Hillsborough, Euroscepticism among the public exposed to the boycott will decrease compared
to citizens who live in areas not directly subject to the boycott. Moreover, this decrease should
be most pronounced among the most likely consumers of “the Sun”, working class citizens.3
It is important to emphasize that while “Brexit” and the EU are amongst the most salient
political issues in the UK today, in the 1980s and 1990s, the EU issue was only emerging and
reporting was overwhelmingly one-sided, dominated by Eurosceptic coverage. The absence of
an effective counter frame leaves ample room for media effects to materialise (33, 34). The
Sun’s coverage during the period of the boycott ranged from well-known sensationalist stories
about EU regulations on the shape of bananas, to criticisms of the European commission, EU
treaties, and EU leaders (35).
Our attitudinal analyses are based on the long-running and high-quality British Social Atti-
tudes (BSA) survey. We measure Euroscepticism by relying on the question asking respondents
if “Britain should continue its EC/EU membership”. Our dependent variable Leaving EU is then
coded ‘1’ if respondents answered that Britain should withdraw from the EC/EU, and ‘0’ other-
wise. We cover the years from 1983 to 2004, the last year in which parliamentary constituency2Tabloids in the UK are not usually sold via subscription, but publicly via newsagents. Especially before the
advent of the internet in the 2000s, norm violations were hence easy to police, and initial boycotts easier to enforce,leading to habit formation which likely lasted into the internet age.
3A detailed analyses of Sun readership prior to the Hillsborough disaster can be found in SM S8.
5
identifiers or postcode sectors are included in the BSA, which allow us to match respondents to
the area in which they live.
To test if the Sun boycott caused by the Hillsborough disaster led to a decrease in Eu-
roscepticism, we exploit the occurrence of the Hillsborough disaster in 1989 in a difference-
in-differences framework (DiD). Respondents who were sampled from the 15 parliamentary
constituencies located within Merseyside, directly experienced “the Sun” boycott. They form
the treatment group, while respondents interviewed in one of the remaining parliamentary con-
stituencies in the North of England did not directly experience the boycott. They form the
control group (36, 37). The treatment is the date of the Hillsborough disaster, 15 April 1989,
which divides our time series into two periods, before and after Hillsborough. 1’212 of the
48’801 BSA respondents live in one of the 15 parliamentary constituencies within Merseyside
and were directly exposed to “the Sun” boycott. Details about our identification strategy, the
sample, data, and statistical analysis can be found in the Materials and Methods section of the
supplementary materials (SM S1.3).
Results
Figure 1 shows the results of the manipulation check, the effect of Hillsborough on self-reported
print media consumption among respondents sampled in parliamentary constituencies located
within Merseyside, compared to respondents sampled from the remaining parliamentary con-
stituencies located in the North of England. Figure 1 reports results stemming from the DiD
model using self-reported newspaper readership as the dependent variable. The Sun has con-
sistently refused to release circulation data at the county level or below, and we are therefore
unable to estimate the effects of the Hillsborough disaster on actual Sun readership in Mersey-
side. While self-reports can be a function of social desirability bias, in this case this would
confirm the existence of a strong social norm against reading the Sun in Merseyside. We esti-
6
Figure 1: Effects of Hillsborough on ‘Sun’ readership and media substitution
sun
sun
no reader
no reader
- EU
- EU
+ EU
+ EU
local papers
local papers
other
other
BeforeHillsborough
AfterHillsborough
-.2 -.1 0 .1 .2Average marginal effect on newspaper readership
Did readers switch to different newspapers? Yes, to pro EU papers.
Notice: Predictions of multinomial logistic diff-in-diffs surrounded by 95 % confidence intervals.
7
mate a substantial drop in Sun readership among respondents subject to the boycott as a function
of the Hillsborough disaster of around 8 percentage-points. Self-reported Sun readership among
respondents living in parliamentary constituencies within Merseyside decreased from 18 to 6
percent, while the decline in the control group, respondents living in other Northern English
parliamentary constituencies, was much less pronounced. Notice also that Figure 1 shows a
substitution effect of Sun readership to pro-EU papers, mostly the Daily Mirror, instead of sub-
stitution to anti-EU newspapers (Daily Mail, Daily Telegraph, Daily Express). This is plausibly
the case because many readers consume the Sun based on its cultural appeal and sports cover-
age which is most closely reflected by the Daily Mirror, not the Daily Mail. While we cannot
empirically distinguish the effects of non-readership of Eurosceptic material and substitution
with pro-EU material, during the 1990s and early 2000s, which is the period covered by the
public opinion data, coverage of the EU issue was - with some notable exceptions - restricted to
the Eurosceptic tabloid press.
Figure 2 plots the percentage of respondents who support leaving the EEC/EU sampled from
parliamentary constituencies within Merseyside and the control group consisting of respondents
living in all remaining constituencies in the North of England. Before the Hillsborough disas-
ter, respondents’ opinions sampled within Merseyside constituencies followed parallel trends
on Euroscepticism than opinions of respondents sampled in other Northern parliamentary con-
stituencies. These parallel pre-trends give credibility to the assumption that in the absence of the
Hillsborough disaster, respondents in Merseyside would have followed similar opinion patterns
in Euroscepticism as the rest of Northern England. There are many similarities between Mersey-
side and the remaining North of England (e.g. voting patterns; % Irish population).4While we
estimate that Merseyside was more Eurosceptic before Hillsborough, during the boycott we es-
timate that attitudes to EU membership in the 1990s and early 2000s became significantly more4In appendix S3 we specify all remaining English constituencies as the control group. Although the pre-trends
diverge slightly between 1983 and 1984, our findings remain entirely robust to this alternative specification.
8
positive in Merseyside compared to the counterfactual.
Figure 2: DiD Graph: Trends in Euroscepticism in Merseyside and control counties before andafter Hillsborough
Table 1 reports the main findings of the formal Difference-in-Differences models. Each
model is based on the same identification strategy outlined in more detail in the appendix.
Each model in Table 1 uses a different set of specifications reported in the bottom part of the
table. The first model does not use any fixed effects or covariates, while models (2) - (6)
introduce region fixed effects, constituency fixed effects, time fixed effects by year or month of
the interview, and covariates.5
Throughout all models in Tables 1 we estimate a theoretically meaningful, large effect of
Hillsborough on respondents’ attitudes towards leaving the EEC/EU in Merseyside. Depending
5Since important covariates are not recorded in the years before 1985 (e.g. education) the main findings arebased on the years 1985-2004. Our findings are entirely robust to the inclusion of the entire time span (SM S2).
9
Table 1: DiD, Effect of Hillsborough on Euroscepticism (1985-2004)
Support leaving the EU(1) (2) (3) (4) (5) (6)
Merseyside 0.10 0.11(0.05) (0.05)
After Hillsborough -0.16 -0.01(0.01) (0.04)
Merseyside × After Hillsborough -0.12 -0.13(0.06) (0.05)
δ DiD -0.14 -0.14 -0.14 -0.15(0.05) (0.05) (0.05) (0.06)
Region FEYear FEConstituency FEMonth FEConstituency FE × YearConstituency FE × MonthControlsadj.R2 0.04 0.04 0.06 0.06 0.08 0.08adj.R2(within) 0.04 0.00 0.00 0.00 0.03 0.02RMSE 0.40 0.40 0.40 0.40 0.39 0.39Obs 9’474 9’474 9’474 9’474 9’474 9’474Nconstituencies 170 170 170 170 170 170Clustered standard errors by constituency;Controls (1985-2004): age, gender, education, religion, social class, party-ID;Constituency & time fixed effects omitted from table.
10
on the models we estimate, the effects range from a 12 to 15 percentage-points decrease in
Euroscepticism. Thus, we find a statistically significant and substantially meaningful decline
in Euroscepticism due to the Sun boycott following Hillsborough. Note that effect sizes and
significance levels are comparable once we add regional, time fixed effects and covariates.
We also report further checks in the supplementary materials. First, we report the results
using respondents sampled in all remaining English parliamentary constituencies as the con-
trol group. Re-estimating the DiD models does not change our findings (SM S3). Second,
spillover effects into adjacent counties are possible. On the one hand this would mean that we
underestimate the effect of reading the Sun on Euroscepticism since in the models reported in
table 1 these constituencies are part of the control group. On the other hand we do not find a
pattern of effects consistent with spillover into adjacent areas (SM S4). Third, the decrease of
Euroscepticism might not be unique to Merseyside, but subject to a more general pattern across
English constituencies. Hence, we randomly re-assigned the treatment into other parliamentary
constituencies in England using a (permutation test) and find that the drop of Euroscepticism in
parliamentary constituencies located within Merseyside remains statistically distinct from the
sample of estimates we created (SM S5). Fourth, we used matching on observables to address
issues of comparability between treatment and control group across space and time. Again our
findings remain robust (SM S6). Fifth, we show that there is no effect on Labour and Conser-
vative Party support. Changes in party support in Merseyside in the 1990s and 2000s reflected
broader changes across England (SM S7).
Difference-in-Difference-in-Differences
We also report the results of a Difference-in-Difference-in-Differences design which utilizes
differential subgroup propensities of reading the Sun pre-Hillsborough. Social class is the best
predictor of whether a respondent reads the Sun. Hence, the effects of the successful Sun
11
boycott should be most pronounced among working class respondents, with middle class re-
spondents who were unlikely to read the Sun in the first place acting as a control group within
Merseyside.
We report our procedure in detail in the SM S8. We find that unskilled working class re-
spondents were much more likely to read the Sun than middle class respondents before the
Hillsborough disaster. We should therefore expect that the effect of the Sun boycott should be
most pronounced among unskilled working class respondents. Figure 3 reports the marginal
effects of the three-way interaction between the DiD estimand and social class. The results of
Figure 3: Difference-in-Difference-in-Differences results for social classes
-1
-.8
-.6
-.4
-.2
0
Aver
age
mar
gina
l effe
ct o
n eu
rosc
eptic
ism
unskilledworking class
skilledworking class
middleclass
controlMerseyside
DiDiD: are likely Sun readers affected more in Merseyside? Yes.
Note: Reported are the CATEs stemming from a difference-in-difference-in-differences model interacting thestandard DiD estimand with self-reported social class (unskilled working class (baseline): “never had job”,unskilled; skilled working class: partly skilled, skilled, middle: intermediate, professionals) of BSA respondents.Plotted are point estimates (scatter) surrounded by 95 % confidence intervals (whiskers).
the DiDiD model vary as expected along social classes – we observe a large and significant
12
decrease of Euroscepticism for unskilled working class respondents, while we estimate only
very small effects for middle class respondents that were far less likely to report reading the
Sun before Hillsborough. Thus, the decrease of Euroscepticism in Merseyside after the Hills-
borough disaster reported in the first part of our analyses appears to be driven by working class
respondents.
2016 referendum
Finally, we look into differences between counting areas within Merseyside and the remaining
English counting areas in the 2016 EU referendum. Figure 4 plots remain vote shares in the
“Brexit” referendum across England. While the city of Liverpool voted significantly more “Re-
Figure 4: Remain vote share in the 2016 EU-referendum
main” than the rest of England, the “Remain” and “Leave” vote in Merseyside as a whole was
almost equally split, with a slight edge for “Remain”. However, as our previous analyses have
already shown, Merseyside was significantly more Eurosceptic before Hillsborough. Hence, we
13
use the same DiD identification strategy as in the previous analyses of the public opinion data,
described in SM:S1. We use counting level data in the 2016 and 1975 EU referenda, controlling
for observable time-variant socio-economic and political factors, and again estimate an OLS re-
gression with region fixed effects. All details about the sample, data, and statistical analysis can
be found in the Materials and Methods section of the supplementary materials (SM:S9). Con-
trolling for population age, median hourly pay, unemployment rate, non-EU and EU migration
and EU-funds per capita, the results show that following the boycott, counting areas located in
Merseyside county were significantly less likely to vote for the Eurosceptic option (“leave” in
2016 and “no” in 1975) in the 2016 Brexit referendum compared to 1975 than counting areas
located in the rest of England. Using this strategy, we estimate the effect of the boycott on the
leave vote share in counting areas within Merseyside to be around -10 percentage-points.
Discussion
Evidence from a unique quasi-experiment shows that the boycott of the most important Eu-
rosceptic newspaper - “the Sun” in Merseyside as a consequence of the Sun’s reporting on the
Hillsborough sporting disaster - led to a decrease of Euroscepticism in Merseyside, which we
estimate to amount to around 11 percentage-points. Moreover, our results suggest that “the
Sun” boycott in Merseyside might have decreased the leave vote share in Merseyside in the
2016 EU referendum. This study therefore shows that sustained media campaigns on emerg-
ing issues can have large, lasting, and ultimately, consequential effects on public opinion, and
public policy.
The paper also shows that the decline of Euroscepticism in Merseyside following “the Sun”
boycott was largely driven by a decrease in Euroscepticism among unskilled working class vot-
ers, who made up a large share of Sun readers before the disaster. We therefore contribute to
the debate about the role of the Northern working class in the Brexit vote, and highlight an
14
important non-structural factor which likely contributed to the formation of Eurosceptic atti-
tudes among sections of the public during the 1990s and 2000s. We show that public opinion
is partially endogeneous to media campaigns, which highlights the strategic failure of pro-EU
elites to provide an effective counter frame to the Eurosceptic campaign waged in the British
tabloid media over three decades. As George Osborne, the former chancellor, and one of the
key government figures in the Remain campaign wrote in 2018, ”We were too late in the day
trying to explain some of the benefits of European Union membership” (38).
References
1. L. M. Bartels, American Political Science Review 87, 267 (1993).
2. P. F. Lazarsfeld, B. Berelson, H. Gaudet, The People’s Choice (Columbia University Press,
New York, 1948).
3. W. Lippmann, Public Opinion (Transaction Publishers, 1921).
4. D. C. Mutz, P. S. Martin, American Political Science Review 95, 97 (2001).
5. J. R. Zaller, Political Persuasion and Attitude Change, D. C. Mutz, P. M. Sniderman, R. A.
Brody, eds. (University of Michigan Press, Ann Arbor, 1996), pp. 17–78.
6. S. Holmes, Democracy and the Mass Media (Cambridge University Press, Cambridge,
1991), pp. 21–65.
7. M. Horkheimer, T. W. Adorno, G. Noeri, Dialectic of enlightenment (Stanford University
Press, Stanford, 1972).
8. J. T. Klapper, The Effects of Mass Communications (Free, Glencoe, IL, 1960).
9. D. R. Kinder, Annual Review of Political Science 1, 167 (1998).
15
10. M. A. Baum, American Political Science Review 96, 91 (2002).
11. G. King, B. Schneer, A. White, Science 780, 776 (2017).
12. S. B. Hobolt, C. E. de Vries, Annual Review of Political Science 19, 413 (2016).
13. D. E. Broockman, J. Kalla, Science 352, 220 (2016).
14. J. L. Kalla, D. E. Broockman, American Political Science Review 112, 148 (2018).
15. A. S. Gerber, D. Karlan, D. Bergan, American Economic Journal: Applied Economics 1,
35 (2009).
16. S. Dellavigna, E. Kaplan, Quarterly Journal of Economics 122, 1187 (2007).
17. G. J. Martin, A. Yurukoglu, American Economic Review 107, 2565 (2017).
18. Y. Chen, D. Y. Yang, American Economic Review (forthcoming).
19. J. Martinson, Did the Mail and Sun help swing the UK towards Brexit?,
39. J. M. Ladd, G. S. Lenz, American Journal of Political Science 53, 394 (2009).
40. O. Folke, S. Hirano, J. M. Snyder, American Political Science Review 105, 567 (2011).
41. A. Fowler, A. B. Hall, European Journal of Political Economy 40, 42 (2015).
42. L. Keele, Political Analysis 23, 313 (2015).
43. E. Dinas, M. Konstantinos, D. Xefteris, D. Hangartner, Political Analysis pp. 1–10 (2018).
44. J. D. Angrist, J.-S. Pischke, Mostly Harmless Econometrics. An Empiricist’s Companion
(Princeton University Press, Princeton, 2009).
45. D. B. Rubin, Matched Sampling for Causal Effects pp. 365–382 (2006).
46. D. B. Rubin, Journal of Educational Psychology 66, 688 (1974).
47. P. R. Rosenbaum, D. B. Rubin, Biometrika 70, 41 (1983).
48. P. R. Rosenbaum, D. B. Rubin, American Statistician 39, 33 (1985).
49. R. H. Dehejia, S. Wahba, Review of Economics and Statistics 84, 151 (2002).
18
50. E. A. Stuart, K. Duckworth, J. Simmons, C. L. Barry 14, 166 (2015).
51. T. F. Becker, Sascha O., D. Novy, Economic Policy 32, 601 (2017).
Acknowledgments
Daniel acknowledges a SNF Ambizione Grant (No. 179938) during the period when this paper
was re-drafted.
Supplementary materials
Materials and Methods (S1)
Supplementary Text
Figures 5 to 10
Tables 2 to 6
References (36-45)
S1 Identification strategyS1.1 The Sun’s coverage of Hillsborough and the ensuing boycott
The Sun’s coverage of the Hillsborough disaster was particularly one-sided and falsely claimed
that “the truth” about the disaster was that Liverpool supporters were largely responsible for the
chaotic escalation (see figure 5). Based partly on false information by a South Yorkshire police
inspector, the Sun claimed that Liverpool supporters had stolen from the dead as the disaster
unfolded. According to the Sun’s source one of the dead people had “numerous wallets” on
him, and was likely “one of the Liverpool pickpockets”.6
6The Guardian: How the Sun’s ‘truth’ about Hillsborough unravelled: https://www.theguardian.com/football/2016/apr/26/how-the-suns-truth-about-hillsborough-unravelled.
19
Figure 5: The Sun’s Hillsborough coverage
Source: The Sun on 13th September 2012: We are sorry for our gravest error,https://www.thesun.co.uk/archives/news/919113/we-are-sorry-for-our-gravest-error/.
20
23 years after the incident, in the wake of the publication of the 2nd Hillsborough report
by the Hillsborough Independent Panel established by Parliament, which concluded that Liver-
pool fans were in no way responsible for the disaster7, the Sun admitted that their coverage was
“false”. The Sun apologized to the families of victims, and Liverpool supporters, and called
their Hillsborough coverage “our gravest error”, and the “blackest day in this newspaper’s his-
tory”. Their apology read “Today we unreservedly apologize to the Hillsborough victims, their
families, Liverpool supporters, the city of Liverpool and all our readers for that misjudgment.”
Despite what was clearly a commercial disaster for the paper, The Sun remained unapolo-
getic for a long period of time. This stubbornness led to a boycott of the Sun in the Merseyside
area. The boycott was not only supported by supporters of Liverpool F.C., the most popular
soccer club in Merseyside, but even supporters of Premier League rival Everton F.C. showed
their solidarity with Liverpool supporters and the Hillsborough 96, and vouched never to buy
the Sun again. Until today, this boycott is ongoing. In 2017 after speaking to several victims
of the Hillsborough disaster, the club owners, and the manager Jurgen Klopp decided to ban all
Sun journalists from entering their stadium at Anfield road and their training ground.8
S1.2 Sun’s coverage of EU
The Australian-born media mogul Rupert Murdoch bought the Sun in 1969. During most of
the period we study (1983-1997) the paper supported the Conservative party under Margaret
Thatcher (PM from 1979-1990), and John Major (PM from 1990-1997). Since the beginning of
the 1980s, the Sun has printed strong anti-EU content. For instance, on the front-page in figure
6 it takes a strong stance against EU integration in November 1990. While the Sun supported
New Labour under Tony Blair and Gordon Brown from 1997 until the 2010 General Election, it
7Hillsborough Independent Panel: http://hillsborough.independent.gov.uk.8The Guardian: Liverpool ban Sun journalists over Hillsborough coverage:https://www.theguardian.
This approach allows us to account for unit-specific time trends, meaning that treatment and
control units can follow different time trends (44).
S1.4 The British Social Attitudes survey (BSA)
Our analyses is based on the long-running and high-quality British Social Attitudes (BSA) sur-
vey. We measure Euroscepticism by relying on a question asking respondents if “Britain should
continue its EC/EU membership”. Respondents can then either answer “continue”, “withdraw”,
or “don’t know’.9 Our dependent variable Leaving EU is then coded ‘1’ if respondents answered
that Britain should withdraw from the EC/EU, and 0 otherwise. We cover the years from 1983
9From 1993 onwards the BSA introduced six answer categories to the same question: “uk leave ec”,“stay+reduce ec power”, “leave as is”, “stay+incr.ec power”, “single ec govt”, and “don’t know”. However, sincethis change in the measurement instrument does not coincide with the treatment, it should not bias our results. Allresults are robust to excluding the 1993-2004 period.
24
to 2004, the last year in which parliamentary constituency identifiers or postcode sectors are
included in the BSA, which allows us to match respondents to treatment and control areas. We
control for respondents’ gender, age, education, ethnicity, self-reported social class and party
identification. Since the BSA reports the interview dates for each respondent10, we can directly
identify which respondents were interviewed before and after the 20th of April 1989 – the day
the Sun published its front-page on the Hillsborough Disaster.
10The interview date is missing for 39 respondents.
25
S2 Including data from 1983 & 1984
Unfortunately the BSA does not report an important covariate prior to the 1985 data, namely
education. Therefore, we decide to include only data from 1985 onwards in our main analysis.
Table 2: Effect of Hillsborough on Euroscepticism (1983-2004)
Region FEYear FEConstituency FEMonth FEConstituency FE × YearConstituency FE × MonthControlsadj.R2 0.03 0.03 0.04 0.04 0.07 0.07adj.R2(within) 0.03 0.00 0.00 0.00 0.03 0.03RMSE 0.40 0.40 0.40 0.40 0.39 0.40Obs 30’830 30’830 30’830 30’830 30’830 30’830Nconstituencies 544 544 544 544 544 544Clustered standard errors by constituency;Controls (1985-2004): age, gender, education, religion, social class, party-ID;Constituency & time fixed effects omitted from table.
29
Figure 10: Difference-in-Difference-in-Differences results for social classes
-1
-.8
-.6
-.4
-.2
0
Aver
age
mar
gina
l effe
ct o
n eu
rosc
eptic
ism
unskilledworking class
skilledworking class
middleclass
controlMerseyside
DiDiD: are likely Sun readers affected more in Merseyside? Yes.
Note: Reported are the CATEs stemming from a difference-in-difference-in-differences model interacting thestandard DiD estimand with self-reported social class (unskilled working class (baseline): “never had job”,unskilled; skilled working class: partly skilled, skilled, middle: intermediate, professionals) of BSA respondents.Plotted are point estimates (scatter) surrounded by 95 % confidence intervals (whiskers).
30
S4 Spillover
Below we look into the drop of Sun readership among respondents in counties which are adja-
cent to Merseyside. It becomes visible that the region of Merseyside stands out and that if any
Figure 11: Are there spillover effects to adjacent counties? No.
Anfield Road
(0,127.4981](−13.90879,0](−50,−13.90879][−100,−50]No data
Decrease in Sun readership compared to national average
spillover effects exist, they are likely minimal.
S5 Permutation test
The decrease of Euroscepticism might not be unique to Merseyside, but driven by a more gen-
eral trend against Euroscepticism in England in the 1990s. For instance, four regions (Greater
Manchester, Lancashire, Cheshire) experienced a relatively large decrease in Euroscepticism
over the same time period. To address this concern we estimate a placebo test in space. More
specifically, we randomly re-assigned the Hillsborough event into other constituencies in Eng-
land. The upper panel in figure 8 reports the finding of this permutation test. The red vertical
line reports the effect we found for Merseyside while the density plot reports the estimated
effect for all 1000 permutations we simulated. It becomes strikingly evident that the Hillsbor-
31
Figure 12: Placebo tests: placebo in space
0
2
4
6
8
10
Den
sity
-.3 -.2 -.1 0 .1Diff-in-Diff
kernel = epanechnikov, bandwidth = 0.0097
Placebo in space
Note: Placebo in space based on 1’000 permutations, reports an ATT=-0.140 with SE(P )=0.0059 and CI:0.025/0.049
ough effect for Merseyside remains distinct and is statistically different from the distribution of
placebo effects we estimated.
Figure 13 reports the same permutation test for Northern English constituencies only.
Figure 13: Placebo tests: placebo in space
0
2
4
6
8
Den
sity
-.2 -.1 0 .1 .2Diff-in-Diff
kernel = epanechnikov, bandwidth = 0.0104
Placebo in space
Note: Placebo in space based on 1’000 permutations, reports an ATT=-0.130 with SE(P )=0.0057 and CI:0.024/0.047.
32
S6 Matching on observablesS6.1 Covariate balance statistics
Below we report the distributional differences between the treated (Merseyside after the Hills-
borough disaster) and the control group. While there is no empirical evidence to support the use
of any particular cut-off point on the standardized difference to define imbalance, Rubin (45)
suggests that a standardized difference between treatment and control group of about 0.25 is
strong evidence for imbalance. The last two columns of table 4 report the standardized dif-
ference and variance ratio (ratio of treated and control variances as a balance measure of the
second moment, where balance is defined by values close to 1.0). As can be seen according
to Rubin’s 0.25 criterium, we find imbalances between treatment and control for four variables
(university, religion, conservative and labour voter). Specifically the imbalances for voting are
severe given the low variance-ratio reported in the last column. All other variables appear to be
balanced between treatment and control groups.
Table 4: Covariate balance between treatment (Merseyside after Hillsborough) and controlgroups, BEFORE MATCHING
Treated Control BalanceMean Variance Skewness Mean Variance Skewness Std-diff Var-ratio