Harmonisation: Understanding user experiences to maximise the value of statistics Rachel Leeser Senior Research and Statistical Analyst - Social Exclusion Intelligence, Greater London Authority [email protected]Emma Emery User Engagement Manager, Royal Statistical Society (UK Stats Authority secondment) [email protected]
22
Embed
Harmonisation: Understanding user experiences to maximise the value of statistics Rachel Leeser Senior Research and Statistical Analyst - Social Exclusion.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Harmonisation: Understanding user experiences to maximise
the value of statistics
Rachel LeeserSenior Research and Statistical Analyst - Social Exclusion Intelligence,
• Reminder of Code of Practice & link between harmonisation, user engagement and maximising value of stats
• Examples of current harmonisation issues
• Rachel’s experience using income & earnings statistics
• Consideration of impact of lack of harmonisation/info about harmonisation
• What the GSS should be doing & examples of GSS harmonisation work
Harmonisation in 2015 EU peer review report - recommendation
For full details see p24 of the report here: http://ec.europa.eu/eurostat/documents/64157/4372828/2015-UK-report/d44f7d3f-64c1-4450-8a37-bfadb8542607
15.The United Kingdom Statistics Authority and the National Statistician as Head of the Government Statistical Service should strengthen their efforts to harmonise United Kingdom statistics in order to ensure coherence and comparability in the production of official statistics within the United Kingdom, over time and among producers of statistics. (European statistics Code of Practice, Principles 10, 14 and 15, coordination.)
Harmonisation: Relevant aspects of the UK Code of Practice for Official Statistics (1)
‘Promote comparability within the UK and internationally by, for example, adopting common standards, concepts, sampling frames, questions, definitions, statistical units and classifications (including common geographic referencing and coding standards). Make the reasons for any deviations from standard models publicly available.’
Harmonisation: Relevant aspects of the UK Code of Practice for Official Statistics (2)‘Ensure that official statistics are disseminated in forms that
enable and encourage analysis and re-use. Release datasets and reference databases, supported by documentation, in formats that are convenient to users.’
‘Seek to balance quality (for example, accuracy and timeliness) against costs (including both costs to government and data suppliers), taking into account the expected uses of the statistics.’
• This translates into a fair bit of work for statistics producer teams – some more than others - but is important for ensuring that statistics are as useful and valuable as possible
• Ongoing communication with statistics users essential in truly adhering to these aspects of the Code of Practice
Harmonisation in assessment reports - exampleNI Housing and Benefits stats. User feedback reported:‘Concerns raised by housing reports users included uncertainty about
the definitions of some classifications, a lack of coherence between specific statistics and those from other sources and a lack of explanation behind the data.’
‘The availability of local data and the coherence of benefits statistics with those from DWP were cited as being particularly useful.’
Recommendations:
(Sugg) ‘Provide more information about the availability of comparable housing statistics for England, Scotland, Wales and the Republic of Ireland.’
(Sugg) ‘Work with DWP to provide comparisons and analysis of benefits statistics in Northern Ireland and in GB that will aid user interpretation.’
Other examples of lack of harmonisation affecting users…(1)Some Census questions differ between UK countries – e.g
religion, resulting in questions over the impact on the data - is it why Scotland measured a substantially lower proportion of Christians & a substantially higher proportion of ‘no religion’ than E&W census?:
Other examples of lack of harmonisation affecting users…(2)
• Other Census issues – different age bands used by UK countries & timeliness varies. Analysis by age bands for whole of UK difficult.
• NHS health data e.g. waiting times not comparable between UK countries
• Differences between SOC2000 & SOC2010 causing break in time-series – can mean longitudinal series impossible e.g. Certificate of Sponsorship data
To harmonise or not to harmonise Income, Earnings, Deprivation & Poverty statistics - a user’s perspective
• Distinction between poverty and deprivation?– Child poverty measures in particular
To harmonise or not to harmonise- some questions to ponder
• What are the merits of:– flexibility vs consistency?– transparency and detail vs easy?– change vs continuity
• Local, national or international comparability?
Do they have to be either/or?
So… potential impact of lack of harmonisation/info about harmonisation:
• Confusion for users of statistics
• Difficult or impossible to answer research questions
• Time consuming for users to identify the differences & consider the implication for use
• Perception that official statistics are complicated or confusing can lead to lack of trust
• Statistics/datasets don’t get used as much as they might be if greater harmonisation – therefore, not as valuable, or having as much impact as they could be
• BUT in some cases steps to increase harmonisation may not be welcomed by users!
Examples of GSS promoting harmonisation & responding to users• ONS responded to user feedback on the 2001 Census and put in
place for 2011 outputs: Common table specifications for the key & quick statistics tables (users
previously had to do the table matching themselves) Standard bulk delivery file format for UK (users previously had to create a
common format themselves prior to analysis work)
• Personal well-being – ONS developed 4 standard questions & added them to the Harmonisation Programme Standards to encourage use by other research organisations. But EU-SILC uses different questions, meaning comparisons with other countries not straightforward.
• Disability – harmonised survey questions created following GSS project involving stakeholders. More info: http://www.ons.gov.uk/ons/guide-method/harmonisation/primary-set-of-harmonised-concepts-and-questions/index.html Will they/are they being used?
Actions for you….• Ensure external communication and researching the
uses & impact of statistics are key elements of the production process
• Encourage increased UK/international harmonisation through collaboration with other statistics producer teams
• Consider the impact of methodological changes on longitudinal series that are valuable for research
• Make use of community sites such as StatsUserNet to investigate users’ experiences of statistics & inform balancing of priorities
• Ensure info available about extent of harmonisation & comparability, and deviation from common standards
Useful references….
• GSS Guidance on assessing & improving comparability: https://gss.civilservice.gov.uk/statistics/presentation-and-dissemination/comparing-official-statistics-across-uk/
• GSS Primary set of harmonised concepts and questions: http://www.ons.gov.uk/ons/guide-method/harmonisation/primary-set-of-harmonised-concepts-and-questions/index.html
• National Statistics Harmonisation Group: https://gss.civilservice.gov.uk/about/governance-and-structure/national-statistics-harmonisation-steering-group/