Quality Assurance in Statistics

Quality Assurance in Statistics: Producer Perspectives in the Light of

International Practices

S.S. Colombage

50th Open Forum: Ensuring Quality of Survey Data Centre for Poverty Analysis

April 29, 2014

Outline

• Rationale for Quality Assurance

• International standards for statistics

• Sri Lanka’s Case

• Concluding Remarks

Rationale for Quality Assurance

• Main statistical agencies – Census and Statistics Department (CSD)

• Household income & expenditure surveys, labour force surveys • Macro-level data: National accounts, prices & wages

– Central Bank of Sri Lanka (CBSL) • Macro-level data: Balance of payments, financial sector, fiscal

sector • Consumer Finances Surveys - abandoned?

• Decision makers in public & private sectors heavily depend on statistics

• Doubts about GDP, inflation, poverty & unemployment data.

Sri Lanka: Dramatic poverty reduction vs. moderate economic growth?

Sources: Department of Census & Statistics Central Bank of Sri Lanka

Sri Lanka: Dramatic poverty reduction vs. stagnant income share of the poorest?

Sources: Department of Census & Statistics Central Bank of Sri Lanka

What is statistical quality?

Old notion: quality = accuracy

Current version: quality = accuracy +

• Relevance

• Timeliness

• Accessibility

• Clarity

• Coherence

• Comparability

• Efficiency etc.

Improving statistical quality means enhancing the fitness of data to meet the expectations of the users.

International Codes of Practices

• At international level, increased attention on statistical quality management since the mid-1980s

• Common quality management tools developed by international agencies & national statistics agencies – UN Statistical Commission – Expert Group: Generic Quality

Assurance Framework (NQAF), 2010 – Quality Assurance Framework of the European Statistical

System – IMF: Data Quality Assessment Framework (DQAF) – Statistics Canada: Quality Assurance Framework – Code of Good Practice in Statistics for Latin America & the

Caribbean

Generic National Quality Assurance Framework (NQAF) template

NQAF 4: Assuring professional independence

• Statistical agencies should develop, produce and disseminate statistics without any political or other interference from government agencies, private sector entities or persons.

• The French law on statistics states that "the conception, production and dissemination of official statistics are carried out in full professional independence".

• In Sri Lanka, no mention about autonomy of CSD, which comes under the purview of Ministry of Finance & Planning

Statistics Canada

NQAF 16: Assuring timeliness & punctuality

• Statistical agencies should minimize delays in making data available – Timeliness: How fast the data are released

– Punctuality: Whether the data are released on the dates promised.

• Sri Lanka CSD: Advance Data Release Calendar (ADRC) available on the website

• Delays – e.g. Final report of Population Census (2011) still not released

NQAF 19: Managing metadata

• Statistical agencies should provide information covering the underlying concepts, variables and classifications used, the methodology of data collection and processing, and indications of the quality of the statistical information - to enable the user to understand all attributes of the statistics.

• Metadata = Data about data • SL CSD: Provision of metadata are inadequate. • Statistics Canada

– Integrated Meta Data Base (IMDB): repository of information on each of nearly 400 active surveys

– IMDB is available to outside users of SC’s website. Each data release on the site includes hyperlinks to metadata from IMDB

• Australian National Health Information Knowledgebase: An electronic storage site for health metadata

• United States Health Health Information Knowledgebase

Quality Assessment & Reporting

• Purpose: To evaluate & prevent problems that may arise during statistical processes.

• Assess product quality in terms of relevance, accuracy & reliability, timeliness & punctuality, coherence & compatibility, accessibility & clarity

• Each of the above dimensions is assessed by using quality indicators. E.g. Accuracy: “How complete is the sample frame?”

• Quality assessments could be conducted by internal or external teams

• Statistical assessment systems are well established in several countries including Canada, Netherlands & South Korea

Sri Lanka: Positive initiatives

• LankaDatta maintained by CSD – National data repository of Sri Lanka, which holds of

over 350 data files from different surveys – LankaDatta adheres to international standards in

archiving microdata such as the Dublin Core Metadata Initiative

• General Data Dissemination System (GDDS) of IMF – Both CSD & CBSL have been participants since 2000 – Information/data are submitted to IMF under Article

IV

Sri Lanka: Shortcomings

• A National Quality Assurance Framework (NQAF) for statistics lacking in Sri Lanka

• Census & Statistics Department (CSD) & Central Bank (CBSL) are silent on quality assurance

• QA is explicit in websites of statistical agencies of some countries – e.g. Australia, Canada

• CSD & CBSL are not very receptive to user feedbacks

Concluding Remarks

• Quality assurance is not prominent in the Census & Statistics Dept., Central Bank & other statistical agencies in Sri Lanka

• Hence, credibility of data are at stake • Recommendations:

– National quality assurance framework for statistics following UN guidelines be adopted

– Quality culture needs to be inculcated – User feedbacks need to be accommodated in

statistical processes – Professional independence of statistical agencies vital