Framework for Measurement of Service Coverage Dimension of Universal Health Coverage Ties Boerma, WHO Measuring and Monitoring Country Progress towards.

Framework for Measurement of Service Coverage Dimension of

Universal Health Coverage

 Ties Boerma, WHO

Measuring and Monitoring Country Progress towards UHC Concepts, Indicators, and Experiences

Washington DC, 20 July 2012

Inputs & processes

Health Financing

Health workforce

Infrastructure

Information

Governance

Outputs

Service accessand readiness

Service qualityand safety

ServiceUtilization

Eligibility for a form of

financial risk protection

Outcomes

Coverage ofinterventions

Risk factors

Impact

Health status

Financial risk protection

Responsiveness

Monitoring and evaluation results chain

Level and distribution (equity)

Definitions

3

• Access: whether the health services that people might need are available, of good quality, and close to them

• Coverage of interventions: whether the people who need an intervention actually receive it

• Effective coverage: whether the people who need health intervention obtain them in a timely manner and at a level of quality necessary to obtain the desired effect; health gain 

• Obstacles to obtaining effective coverage: physical access, affordability, acceptability for reasons such as culture or religion, and poor service quality.  

• Financial affordability is not only instrument but intrinsic goal

Tanahashi T. Health service coverage and its evaluation. Bull WHO1978;56(2):295.  Shengelia B, Murray CJL, Adams OB. Beyond Access and Utilization: Defining and Measuring Health System Coverage. Health Systems Performance Assessment: Debate, New Methods, and New Empiricism. Geneva: World Health Organization; 2003

4 |

Measuring progress towards the service coverage dimension of UHC – key issues to consider

4

● Contents of the indicators/index

● Index or tracer indicators

● Equity dimension

● Output / proxy indicators

● Quality of services

● Measurement gaps

● Post 2015 development agenda

● Global and country perspectives

5 |

Contents of the indicators / index

5

● Proven interventions with large health impact that can be measured

● MDG related interventions / communicable diseases: MNCH, HIV/TB/malaria 

● Epidemiological transition - NCD and other interventions important : UNGA Political declaration 2011 monitoring plan; risk factors

● Injuries: emergency care, preventive measures

Indicators and targets for 2025for the global monitoring framework for NCDs

Mortality between ages 30 and 70 due to CVD, cancer, diabetes, and chronic respiratory disease

25% reduction

Hypertension25% reduction

Tobacco30% reduction

Salt30% reduction

Physical inactivity

10% reduction

Indicatorswith targets

Other country-specific indicators of NCD and related issues including social determinants of health

OtherWHO core indicators

* All indicators should be disaggregated by gender, age, socioeconomic position, and other relevant stratifiers

Policies to virtually eliminate trans fats  and to reduce marketing of unhealthy foods to children

Overweight/obesity (adult, child, adolescent)

Raised blood glucose/diabetes

Raised total cholesterol

Access to basic technologies and medicines

Cervical cancer screening

Cancer incidence, by type Multidrug therapy for CVD risk reduction

Access to palliative care

Vaccination: HPV, Hepatitis BAdult per capita consumption of  alcohol and heavy episodic drinking

Low fruit and vegetable intake

Source: WHO. A comprehensive global monitoring framework including indicators and voluntary global targets for the prevention and control of NCD. Second discussion paper. 2012.

Index or tracer indicators

6

• Tracer indicators:  selected interventions (no more than 5), target 100%, equity can be done well; disadvantage "gaming"

• Index based on intervention areas capturing the full range of services of UHC; intervention areas rather than indicators – Countdown MNCH coverage index

Source: Boerma, J. T., J. Bryce, et al. (2008). "Mind the gap: equity and trends in coverage of maternal, newborn, and child health services in 54 Countdown countries." Lancet 371(9620): 1259‐1267.

Coverage index gap: difference between poorest and wealthiest quintiles.

7 |

Equity dimension

7

● Major weakness of the health MDGs

● UHC is about equity:  should include levels and distribution according to key stratifiers. 

● Intervention coverage among the poorest (or other disadvantaged population) could be a good tracer indicator of progress towards UHC. 

● Excludes the use of statistical modeling to arrive at estimates

Source: Country profile. Countdown 2015 for Maternal Newborn and Child Health. 2012.

8 |

Input / Output proxy indicators

8

● Inputs: money not suitable as proxy for access/coverage measure, but relevant

● Outputs: level and distribution of service outputs: index (SARA)– Service availability: density of 

infrastructure (facilities, beds) and specific services; proximity

– Service readiness: a set of minimum conditions to provide services in terms of basic infrastructure, health workforce, diagnostics, treatment and universal precautions

– Service utilization: OPD or admission rates

Source: Data from NHSS China.

0

2

4

6

8

1993 1998 2003 2008

Annual number of OPD visits per person, China 1993-2008

OECD average)Range 2.5-13 (

9 |9

Maternity beds density (per 1000 pregnant women) by district, Zambia 2010.

Core health workers density (per 10 000 population) by district, Zambia 2010.

Example: service availability, 17 districts, SARA 2010, Zambia

Source: Ministry of Health and WHO. Zambia service availability and readiness assessment 2010. Summary report. Geneva. 2012.

Example: service readiness family planning, 8 districts, Zambia SARA 2008 and 2010

10

Source: Ministry of Health and WHO. Zambia service availability and readiness assessment 2010. Summary report. Geneva. 2012.

Quality of care measurement

11

• More complex, often many indicators (e.g. Agency for Healthcare Research and Quality (AHRQ) in USA over 200 indicators in use; OECD HCQI set of indicators linking quality of health services and health system performance

• Measurement more difficult; comparability an issue

• In low income setting service availability & readiness is a proxy for quality; SPA – exit interviews, observation; STI PI6 & PI7 experience

 Source: Data from NHSS China.

Five-year cancer survival rates, OECDCervix Breast

30 day hospital mortality rates after AMI, OECD

12 |

Measurement gaps & investments

12

● Issues: (1) availability of good standard measurement method (2) data collection frequency (3) data quality

● Surveys: especially good on preventive intervention coverage measures; denominator (need) problems; 3-5 year intervals

● Facility data: quality data numerator, denominator projections; annual

● UHC monitoring will need investment in monitoring

● Surveys: NCD interventions especially need work

● Facility: improvement quality, transparency, innovation

13 |

Post 2015 development agendaPossible linkages with UHC measurement

13

● Development focus:– Inclusive economic development– Sustainable development– Human development: education, 

health, nutrition– Security

● All country focus instead of low income country focus

● Likely with only few health indicators at the top, covering all areas

● Health as an impact measure of development

● Health as contributor to development / beneficiary

● Health as human right

● Health status measure, e.g. (healthy) life expectancy, as summary measure (with equity)

● Universal health coverage good candidate

14 |

Global and country perspectives

14

Global

●Few indicators, lessons learnt from the MDG monitoring

●Uniform targets

●Monitoring and reporting responsibilities clear

●Investment in measurement / monitoring

Country

●Global framework and guidance

●Monitoring and reporting responsibilities clear

●Country specificity: different epidemiology, different priority interventions for UHC – flexible coverage index or different set of tracer indicators

Table 1: Intervention areas, tracer indicators and additional indicators for coverage index.

Intervention area Tracer indicators Additional indicators Child vaccination  DPT3/penta, measles  BCG, polio, hepB 

Maternity care  ANC 4 or more visits, skilled birth attendance 

ANC 1+ visit, institutional delivery, postnatal care 

Family planning  Unmet  need  for FP  Contraceptive use 

Treatment  of sick children 

Suspected pneumonia taken to health facil ity, diarrhoea with ORT 

Suspected pneumonia treated with antibiotics, diarrhoea treated with ORS 

Malaria  Children sleeping under ITN; IPT during pregnancy 

Fever treated with antimalarials/ACT; households with IRS 

TB  TB treatment success rate;  TB case detection rate 

HIV prevention  Condom use at higher risk sex; PMTCT among HIV positive women;  ART 

Male circumcision rate 

Cancer  HPV vaccination; tobacco use*; cervical cancer screening 

5 year survival rate (specific cancer); mammography

Cardiovascular diseases 

Hypertension prevalence*  Salt intake*; hypertension control follow-up visit; AMI survival; physical activity* 

Mental health    Injuries       

Other NCD Asthma (re)admission rate; cataract surgery     

* risk factors

Coverage index, based on intervention areas with tracer indicators within each areas

16 |

Summary pointsSummary points

16

Service coverage as part of UHC can be measured and monitored but there will be measurement gaps especially for NCD (and curative care)

Coverage index and set of tracer indicators with equity a key feature; focus on intervention "areas"; support with service output data

Country-specificity including small "core" for global monitoring

Combining coverage with financial protection into index would be ideal but challenging 

Only if the resulting measure is simple and appealing, UHC stands a chance of being included at a high level in the post 2015 development agenda monitoring

17 |

18 |

Using Global Metrics to Measure Country Performance: Lessons Learned

During the MDG Countdown

Using Global Metrics to Measure Country Performance: Lessons Learned

During the MDG Countdown 

Ties Boerma, WHO

Measuring and Monitoring Country Progress towards UHC Concepts, Indicators, and Experiences

Washington DC, 20 July 2012

Tracks country-level coverage for interventions proven to reduce maternal, newborn and child mortality and associated indicators for Policies and health system strength Financial flows Equity

75 countries with highest burden are the focus

Collaboration of individuals, governments, organizations, partnerships

Produces analytical reports, country profiles, global and regional events, Lancet series, link with parliamentarians, country Countdown

Coverage indicators

Inclusion of coverage indicators driven by selection of proven interventions (continuum of care)

Plus availability of reliable data to ascertain levels, trends, equity

Use of household survey data (DHS, MICS) from 75 Countdown countries, from 1990; University of Pelotas data centre + analysis Upcoming PLoS Med series of papers to assess issues related to quality of

coverage indicators

Facility data are not used for coverage No use of model-based estimates for coverage

Coverage index: methods

• Aggregate index based on 4 intervention areas – family planning– maternal and newborn care– immunization– treatment of sick children

• Different system delivery strategies; captures overall strength of the delivery system (was original intention); can add intervention areas; equity

• The four intervention areas are given equal weight • Strong association with health status measures

Countdown 2008 Equity Analysis Group, Lancet 2008

Inputs & processes

Health Financing

Health workforce

Infrastructure

Information

Governance

Outputs

Service accessand readiness

Service qualityand safety

ServiceUtilization

Eligibility for a form of

financial risk protection

Outcomes

CoverageIndex

Impact

Health status

Financial risk protection

Responsiveness

Monitoring and evaluation results chainRelated to health system inputs/outputs and impact

Level and distribution (equity)

Indicators in intervention areas

Topic IndicatorsFamily planning Need for FP satisfied

Maternal and newborn care

Antenatal care (1+ visit)Skilled attendant at delivery

Immunization BCGDPT3Measles

Treatment of sick children

ORTCareseeking for pneumonia

Countdown 2008 Equity Analysis Group, Lancet 2008

• The overall mean = 57%– >80% in Turkmenistan and Peru– <30% in Ethiopia and Chad

• The coverage index improved by an average of 0.9 percentage point per year since the early 1990s - 2006– Based on 40 countries with more than one survey– Fastest improvements (>2 pp/year): Cambodia,

Mozambique, and Nepal– Declining index: Chad, Kenya, Zambia, and

Zimbabwe

Countdown 2008 Equity Analysis Group, Lancet 2008

Empirical data: levels and trends

Monitoring equity in Countdown

• What types stratification?– Wealth quintiles– Maternal education– Urban/rural– Gender– Ethnic groups– Etc

Photos: P Menzel

0 20 40 60 80 100

Coverage gap (% ) difference between the poorest (red) and least poor (blue) quintiles

Turkmenistan

South Africa

Brazil

I ndonesia

Zimbabwe

Congo Brazzaville

Tanzania

Lesotho

Gambia

Cambodia

Gabon

Cote d'I voire

Benin

Ghana

Cameroon

Guatemala

Rwanda

Guinea Bissau

Azerbaijan

Burkina Faso

Eritrea

Haiti

Mauritania

DRC

Pakistan

Nigeria

Lao PDR

Chad

Countdown 2008 Equity Analysis Group, Lancet 2008

Coverage gaps by country

Coverage index changes over time:levels and inequity

Table 1: Intervention areas, tracer indicators and additional indicators for coverage index.

Intervention area Tracer indicators Additional indicators Child vaccination  DPT3/penta, measles  BCG, polio, hepB 

Maternity care  ANC 4 or more visits, skilled birth attendance 

ANC 1+ visit, institutional delivery, postnatal care 

Family planning  Unmet  need  for FP  Contraceptive use 

Treatment  of sick children 

Suspected pneumonia taken to health facil ity, diarrhoea with ORT 

Suspected pneumonia treated with antibiotics, diarrhoea treated with ORS 

Malaria  Children sleeping under ITN; IPT during pregnancy 

Fever treated with antimalarials/ACT; households with IRS 

TB  TB treatment success rate;  TB case detection rate 

HIV prevention  Condom use at higher risk sex; PMTCT among HIV positive women;  ART 

Male circumcision rate 

Cancer  HPV vaccination; tobacco use*; cervical cancer screening 

5 year survival rate (specific cancer); mammography

Cardiovascular diseases 

Hypertension prevalence*  Salt intake*; hypertension control follow-up visit; AMI survival; physical activity* 

Mental health    Injuries       

Other NCD Asthma (re)admission rate; cataract surgery     

* risk factors

Coverage index, based on intervention areas with tracer indicators within each areas

Summary points

29

• Coverage index works well in Countdown to ascertain levels, trends and inequity; good way to summarize multiple interventions

• Mean coverage better understood by policy makers than coverage gap for CD – gap measure makes sense for UHC?

• Dependence on surveys a disadvantage; if facility based coverage estimates are good they should be used increasingly, at least for some indicators

– Denominator problems: also with survey-based coverage rates

• UHC coverage index will need further methodological work, including the critical evidence based selection of core interventions

• Several aspects of the CountDown model deserve consideration for UHC monitoring, including country UHC monitoring, regular global analytical reports and country profiles

Acknowledgement

30

• Countdown to 2015 colleagues, especially Jennifer Bryce and Cesar Victora