The use of research in public policy

The use of research in public policy

OR: why research shouldn't be like

a lamppost to a drunkard

Jim Barratt, 5 November 2009

Aims

1. Put research methods in context

2. Explore the role of research in policy development 3. Examine use of research in film policy

Outline

• Why conduct research in the first place?• What is public policy?• 'Evidence-based policy’• Types of policy-related research• Q&A

Jim who?

• UK Film Council, Head of Research• Advertising Standards Authority, Head of Policy• British Board of Film Classification, Examiner• British Film Institute, PhD

• Freelancer: Ofcom, Screen England, Skillset etc.

DCMS, BERR (DTI), DCSF (DfES), TreasuryDCAL, DENI, DETNI, Northern Ireland Screen

Jim who? (2)

…all perform some form of public service

: Non Departmental Public Body (NDPB)

: Self-regulatory body

: Private company with delegated powers

: Charity with Royal Charter

Why do research?

Research = systematic enquiry to increase knowledge & understanding

The case against…It's expensive, time consuming, and may require specialistskills and training.

‘There is nothing a government hates more than to be well-informed; for it makes the process of arriving at decisions much more complicated and difficult.’

John Maynard Keynes

Why do research? (2)

The case for…Research can:• promote better understanding• highlight underlying conditions otherwise hidden from view• bring independence of thought and perspective (objectivity)• increase transparency• encourage discussion and debate

= the Enlightenment project.

What is public policy?

A plan of action pursued by government and government-backed agencies to achieve their goals

Research can play a role in… • identifying and diagnosing a problem• informing strategy or tactics• deciding between two or more policy options• supplying an independent view

…but it can also• provide retrospective justification• serve as an alternative to action

Lamppost and the drunk

AE Housman:

"Statistics…

are like a

lamppost to a drunk…

Lamppost and the drunk (2)

…they're used more for

support than illumination"

'Evidence-based policy'

The Modernising Government White Paper (Cabinet Office, 1999) stated that government policy must be:

• evidence-based• properly evaluated • and based on best practice.

'Evidence-based policy' (2)

EBP 'helps people make well informed decisions about policies, programmes and projects by putting the best available evidence from research at the heart of policy development and implementation’.

Davies, P.T. 1999, Introducing and Monitoring Change in Dawes, M.G., Davies, P.T., Gray, A., Mant, J. and Seers, K. (eds), Evidence-Based Practice: A Primer for Health Professionals, London, Churchill Livingstone, 1st Edition.

• But new research may not be feasible in the timeframe• Answer = 'Systematic Review'

'Evidence-based policy' (3)

• Another problem for evidence-based policy is 'the uncertainty of social scientific knowledge.'• Correlation not causality

'…in many areas of policy making there is either little or no valid social scientific evidence, the consequence of which is a knowledge vacuum that other types of evidence can fill until new sound research evidence can be established.'

Philip Davies (2004), Government Chief Social Researcher's Office,

Prime Minister's Strategy Unit, Cabinet Office

A drunk loses the keys to his house and is looking for them under a lamppost. A policeman comes over and asks what he’s doing.

“I’m looking for my keys” he says. “I lost them over there”.

The policeman looks puzzled. “Then why are you looking for them all the way over here?”

'Evidence-based policy' (4)

'Evidence-based policy' (5)

“Because the light is so much better”

Factors other than evidence • Experience, expertise and judgment• Resources• Values • Habit and tradition• Lobbyists, pressure groups and consultants • Pragmatics and contingencies

'Evidence-based policy' (6)

'Evidence-based policy' (7)

'The uncertainty of social, economic and political events, coupled with the uncertainty and inconclusive nature of scientific knowledge, mean that high quality evidence often has to compete with these other factors as a basis for decision making.'

Philip Davies (2004), Government Chief Social Researcher's Office, Prime Minister's Strategy Unit, Cabinet Office

Types of policy-related research

Descriptive Analytical Predictive

National statistics Market intelligence Fact-finding/ Briefings Audit Attitudinal research Behavioural studies Forecasting Economic modeling

Types of policy-related research (2)

UK Film Council examples (www.ukfilmcouncil.org.uk/research)

• National Statistics (Office for National Statistics)e.g. Film export data

Labour market data

• Market intelligencee.g. UK Film Council Statistical Yearbook

Free sources e.g. Box Office Mojo etc.

Types of policy-related research (3)

• Fact finding/ briefingse.g. Film in the UK: Briefing Paper

• Audite.g. Audit of film education activity

A study into film development and training in the UK

Types of policy-related research (4)

• Attitudinal researche.g. IPSOS/MORI piracy research

• Behavioural studiese.g. A qualitative study of avid cinema-goers

Types of policy-related research (5)

• Forecastinge.g. Theatrical Window analysis

• Economic modelinge.g. The Economic Impact of the UK Screen Industries

www.BiggerPictureResearch.net

Q&A