Introduction to data support services and resources for public policy

Introduction to data support services and resources for public policy

Stuart MacdonaldAssociate Data Librarian Introductory Analysis of Policy course, Academy of Government's MSc public policy programUniversity of Edinburgh

8 October, 2015

Outline Data Library consultancy and resources Research Data Management (RDM) Survey Documentation and Analysis (SDA)

National data services for social sciences UK social and political surveys Population census

Non-academic data sources National statistical agencies A data future!

What is a data library?

A data library refers to both the content and the services that foster use of collections of numeric and/or geospatial data sets for secondary use in research. A data library is normally part of a larger institution (academic, scientific, medical, governmental, etc.) established to serve the data users of that organisation. The data library tends to house local data collections and provides access through various means (online or central server for download). A data library may also maintain subscriptions to licensed data products.

Data Library - http://www.ed.ac.uk/is/data-library

Data Library & consultancy Finding…“I need to analyse some data for a project, but all I can find are published

papers with tables and graphs, not the original data source.” Accessing …“I’ve found the data I need, but I’m not sure how to gain access to it.” Using …“I’ve got the data I need, but I’m not sure how to analyse it in my chosen

software.” Managing …“I have collected my own data and I’d like to document and preserve it

and make it available to others.” Reference interviews Data Library catalogue: http://

datalib.edina.ac.uk/catalogue

Data Library resources Large-scale social science survey data Country and regional level time series data Population and agricultural census data Financial data Data for mapping Resources for teaching

Opening hours: 9.30am – 5.30pm (Tues, Weds,Thurs)

Lower Ground Floor East, Main Library Tel.: 0131 651 1431 or 0131 651 1744 Email: [email protected]

Research Data Management (RDM) RDM is a general term covering how you

organize, structure, store, and care for the digital data used or generated during a research project. It includes:

• Planning how your data will be looked after – many funders now require data plans as part of grant applications

• How you deal with information on a day-to-day basis over the lifetime of a project

• What happens to data in the longer term – what you do with it after the project concludes

Why research data management is important

RDM Services RDM training:

Research Data MANTRA - Open online course aimed at researchers and others managing digital data as part of the research process - http://datalib.edina.ac.uk/mantra

Institutional data repository provision Edinburgh Datashare - http://datashare.is.ed.ac.uk/

Assistance with deposit in national archives Assistance with Data Management Plans (DMPs) Contributes to the University’s RDM Programme:

http://www.ed.ac.uk/is/data-management

Survey Documentation and Analysis (SDA)SDA allows users to discover and analyse microdata online, and to download user-defined subsets of data in formats conversant with statistical analysis packages.

Introduction to National Data Services and Support

Applied Quantitative Methods Network (AQMeN) - http://aqmen.ac.uk/

Original aim was to build capacity in the use of quantitative methods amongst Scotland’s social science community and beyond.

AQMeN is a Research Centre that aims to develop projects to improve our understanding of current social issues in the UK and provide policy makers and practitioners with independent research-based evidence to build a better future.

Funded by ESRC (2013-2016) AQMeN has three primary strands of research involving a multidisciplinary team of researchers from the UK and abroad :

• Crime and victimisation• Education and social stratification• Urban segregation and inequality

AQMeN offer practical quantitative methods training on a range of techniques developed as part of their programme of research

Data Library host and manage the AQMeN website

UK Data ArchiveFounded in 1967 and based at the University of EssexESRC / JISC-fundedHouses several thousand social science and humanities datasets Provides resource discovery and support for secondary use of quantitative and qualitative data in learning, teaching and research. More recently it has worked with environmental and medical data sources. Host to the UK Data Service which provides the following facilities:

• History Data Service• Census.ac.uk• Variable and question bank• Secure Lab• Qualibank

UK Data Service (UKDS)£17 million investment over 5 years

Funded by ESRC – new service is structured to support researchers in academia, business, third sector and all levels of government

Commenced 1 October 2012 – integrated ESDS, Census Programme, Secure Data Service, and other elements of the data service infrastructure provided by ESRC, including UKDA

A single point of access to economic and social data

Distributed service led by Univ. of Essex in collaboration with: Jisc Manchester, Cathie Marsh Centre for Census & Survey Research (Univ. of Manchester) School of Geography (Univ. of Leeds)Geography and Environment (Univ. of Southampton)EDINA (Univ. of Edinburgh)Centre for Advanced Spatial Analysis (UCL)

UKDS - http://ukdataservice.ac.uk/

UKDS features include:Data – free to download for academic purposes upon registration in formats conversant with statistical analysis packages, full/partial catalogue search, browse by subject, also links to major studies, new releases

Support – finding data, online guides, learning and teaching resources, online data analysis tools (Nesstar, Beyond 20/20), external statistical sources, FAQ

Resources – Online data browsing tools, metadata tools, qualitative tools

Advice for managing, depositing, sharing research data - best practice for ESRC researchers and beyond, preservation guidelines

News & Events – workshops, new data, publications (incl. good practice guides)

Key data

• Part of "EDINA and Data Library" division of Information Services at the University of Edinburgh

• Develops and delivers shared online services and infrastructure to support research and education in the UK

• Services free at the point of use for use by staff and students in learning, teaching and research through institutional subscription

• Focus is on service delivery and support but also undertake R&D (projects services)

delivers about 20 online services employs about 80 staff (Edinburgh & St Helens)

EDINA – a Jisc centre for digital expertise and online service delivery – http://edina.ac.uk

EDINA Digimap - http://digimap.edina.ac.uk/

Administrative Data Research Network (ADRN)- http://www.adrn.ac.uk/

Major UK social & political surveysA range of microdata sets are freely available to University researchers. These are predominantly longitudinal or cross-sectional studies.

Longitudinal surveys: the same panel or ‘cohort’ is surveyed over a period of time – change over time can be measured and collectedAdvantages:• Ability to show the patterns of variable change over time – to show ‘cause

and effect’ relationshipsDisadvantages:• Attrition over time – some subjects will no longer be able to participate (due

to death, refusal, changes in contact information) – thus cutting down on useable data from which conclusions can be drawn

• Data collected at multiple (pre-determined) points in time – cannot take into account whatever has happened in between ‘touch-points’

Cross-sectional surveys: a new representative sample is selected for each study

Advantages:• More affordable when compared to a similar longitudinal study• Allows researchers to compare many different variables at the same

time (‘snapshot’)• With fewer ‘touch-points’ (no follow up) they are also much quicker in

reaching an observational conclusion• Provided the sample size is carefully chosen, cross-sectional studies can

be helpful in representing entire populations, rather than selected subsets. This can be beneficial when considering policy change.

Disadvantages:• The single survey nature means that it may not be possible to make

conclusive observations about the direction of any association between variables.

• Do not consider what happens before or after the ‘snapshot’ is taken.

Longitudinal SurveysUnderstanding Society – the UK Household Longitudinal Study (Incorporates the British Household Panel Survey) - Commenced : 2009.Surveys 50,000 adults and 4,000 young people at over 30,000 selected addresses, who are followed up annually, even if they move home. Main topics: person’s state of health, experiences of crime, personal finances, bringing up children, involvement with local communities, working lives, personal and political views and outlook.Access: UK Data Service, http://discover.ukdataservice.ac.uk/series/?sn=200005

Growing up in Scotland - Commenced : 2005.Longitudinal study of Scottish children from infancy to their teens, with eight sweeps planned.Main Topics: Characteristics and circumstances of children and their families, housing and neighbourhood, eating habits, participation in educational, social and recreational activities, health and development, parenting styles and support, pre-school and school experience, parental health, family and social networks.Access: UK Data Service, http://discover.ukdataservice.ac.uk/series/?sn=20002

Longitudinal Surveys cont’d

Millennium Cohort Study - Commenced: 2001.Following 19,000 babies born in the UK in 2001. Main Topics: Household information, early education, schooling and childcare, child and family activities and child’s behaviour, parenting activities, child’s health, parent’s health, employment, income and education, housing and local area, older siblings’ leisure time activities, extra activities at school, classes outside school, home life.Access: UK Data Service, http://discover.ukdataservice.ac.uk/series/?sn=2000031

Other longitudinal surveys include: • Scottish School Leavers Survey (1991-)• 1970 British Cohort Study • Scottish Longitudinal Study (2007-)

Cross-sectional SurveysIntegrated Household Survey (Continuous Population Survey) - Commenced: 2009The IHS integrates the following surveys: Annual Population Survey & Living Costs and Food Survey (formerly Expenditure and Food Survey)Sample size: 340,000 individuals of which 44,950 from ScotlandAccess: UK Data Service, http://discover.ukdataservice.ac.uk/series/?sn=2000024

Scottish Household Survey - Commenced: 1999Repeated cross-sectional study with random sample of 11,000 cases.Main Topics: Household composition, housing and tenure, health, vehicles available to the household, occupation and industry of the highest income householder, household income and housing costs, tenure change, neighbourhood problems, transport and use of public transport, public services, income and employment.Access: Scottish Government - http://www.scotland.gov.uk/shs Download files for 1999-2008: http://discover.ukdataservice.ac.uk/series/?sn=2000048

Cross-sectional Surveys cont’d

Labour Force Survey (England, Wales & Scotland) – Commenced: 1979.Repeated quarterly cross-sectional study of 100,000 households, of which 7,800 in Scotland.Main Topics: Household, family structure, basic housing information and demographic details of household members, economic activity, employment status, earnings, education and health.Access: UK Data Service, http://discover.ukdataservice.ac.uk/series/?sn=2000026

British Social Attitudes Survey – Commenced: 1983.Annual repeated cross-sectional study. 3,400 cases, of which 270 in Scotland.Main Topics: Housing & home ownership, work & unemployment, health & social care, education, social security, tax & spending, welfare state, transport, environment & the countryside, law & order, civil liberties, racism & sexism, social inequality, religion, politics & governance.Access: UK Data Service, http://discover.ukdataservice.ac.uk/series/?sn=200006 or via: http://www.britsocat.com

Other cross-sectional surveys include: Family Resources Survey (1992-); British Election study (1964-); Scottish Health Survey (1995-): ONS Opinions and Lifestyle Survey (1990-)

UK population censusCensuses are comprehensive surveys, on standard topics, conducted at single points in time incl. demography, household and families, health, economic activity, travel, education, religion, housing

Aims to be a 100% head count of population: resident, absent, communal establishments, no fixed abode (e.g. 1.5% of population ‘unaccounted’ for in 1991).

Statutory basis, legally compulsory to respond, confidentiality guaranteed (1920 Census Act & 1991 amendment).

Data are collected nationally at a very local level

There is much standardisation across censuses conducted in 4 UK territories, but substantial differences do occur, both in input (e.g. religion question) and output (defining census geographies).

UK Data Service Census Support - http://census.ac.uk

Flow data

Census microdata -

Census.ac.uk does not hold the census longitudinal data. These can be found at: England and Wales (ONS LS): CeLSIUS - http://www.ucl.ac.uk/celsius/ (from 1971, 1% sample - linked to births, deaths, cancer registrations, highly confidential). Scotland (SLS): SLS-Development and Support Unit - http://sls.lscs.ac.uk/ (from 1991, 5.3% sample – c. 274,000 individuals were selected using 20 random birthdates, linked to vital events, health information, weather and pollution data, and census data from 1991 and 2011) Northern Ireland (NILS): NILS-RSU - http://www.qub.ac.uk/research- centres/NILSResearchSupportUnit/ (from 1981, 28% sample - linked to vital events, migration data, health data)

Samples of Anonymised Records (SARs) – also available for 1991, 2001)Unanonymised (All academic researchers need to satisfy the Approved Researcher Criteria)

Census geographies

Aggregate data The InFuse Tool has aggregate

data for selected variables down to local authority level (also OA for England and Wales)

includes data from the 2001 and 2011 censuses of Scotland, England, Wales and Northern Ireland

data downloaded as raw asci (csv) can be used to compare data

from the different constituent parts of the UK.

Casweb provides access to UK aggregate data for the 1971, 1981, 1991 and 2001 Censuses (down to OA level).

Government data sources National Records of Scotland (NRS) –

http://www.nrscotland.gov.uk/ Scottish Census 2011 – Data Explorer – http://www.scotlandscensus.gov.uk/ Office for National Statistics (ONS) –

http://www.ons.gov.uk/ons/index.html ONS Neighbourhood Statistics –

http://www.neighbourhood.statistics.gov.uk/ ONS Open Geography Portal https://geoportal.statistics.gov.uk/geoportal/catalog/main/home.page Scottish Neighbourhood Statistics (SNS) –

http://www.sns.gov.uk/ (to be replaced soon by the Scottish Statistics website)

NISRA - http://www.nisra.gov.uk/ Welsh Government Statistics - http://wales.gov.uk/topics/statistics/ Data.Gov.UK - http://data.gov.uk/ Eurostat – http://ec.europa.eu/eurostat

The Guardian Datastore has loads of examples of telling stories with data

Open Knowledge Foundation - School of Data - https://okfn.org/

- http://www.datacarpentry.org/ Data Carpentry

A Data future“The ability to take data - to be able to understand it, to process it, to extract value from it, to visualise it, to communicate it –that’s going to be a hugely important skill in the next decades.”

Hal Varian, Google’s chief economist.“Data is the new oil!”

Clive Humby, Assoc. of National AdvertisersSenior marketer’s summit, 2006

“The coolest thing to do with your data will be thought of by someone else”

Rufus Pollock , Founder and President of Open Knowledge

Based on a slide by Marieke Guy, UKOLN

Pics from Flickr (creative commons attribution) – creditsinclude: D Sharon Pruitt

Ask us about data

[email protected]