Annual Results and Impact Evaluation Workshop for RBF - Day One - Paper - Opportunities for Strengthening Quality of Health Care in Results-Based Financing Programs

Opportunities for Strengthening Quality of Health Care

in Results-based Financing Programs

March 8th 2014

Kathleen Hill, M.D.

Consultant to the World Bank

1

Table of Contents

Executive Summary................................................................................................................................3

Introduction ..........................................................................................................................................4

Quality Dimensions and Improvement Principles…………………………………………………………………………………..5

Measuring Quality of Care…………………………………………………………………………………………………………………..7

Incentivizing Quality in RBF Programs……………………………………………………………………………………………….12

Aligning RBF programs with Improvement and Health System Strengthening Efforts………………………..14

Promising Directions for Strengthening Quality of Care in RBF Programs…………………………………………..15

References………….…………………………………………………………………………………………………………………………….17

Appendices……………………………………………………………………………………………………………………………………….18

Executive Summary

2

This background paper examines current directions and opportunities to improve quality of care in the context of World Bank supported results-based financing (RBF) programs. Common quality of care gaps in low and middle-income countries (LMICs) are reviewed ranging from a lack of essential commodities to poor adherence with evidence-based standards to lack of improvement capacity in most health systems. Six dimensions of quality are reviewed including care that is safe (does no harm), care that is effective (adherent with guidelines), efficient (not wasteful), timely, patient-centered and equitable. Core principles of quality improvement are described including effective teamwork, understanding how processes of care function within a health system, use of data to track results, and an understanding and focus on patient needs. While there is no single best method to improve quality, several approaches for improving quality of care are discussed including the model for improvement and audit and feedback.

Opportunities and challenges for measuring quality of care in LMICs are explored in depth due to the importance of quality measures for RBF programs that incentivize quality performance measures. The paper explores advantages and disadvantages for a range of specific measurement methods including observation, simulation, record audit, provider questionnaire and patient interviews. Feasibility of specific measurement methods is explored within the context of common health information constraints encountered in LMICs (e.g. non-standardized patient charts, missing data, etc.) The paper explores differences between measuring quality for the purpose of routine monitoring (e.g. quarterly performance results for payment) versus periodic measurement of quality for the purpose of external verification versus measurement for the purpose of an external assessment (e.g. evaluation). The paper notes the wider range of measurement methods that are typically feasible in a one-time assessment of quality (versus routine monitoring) and notes the common challenge of overcoming discrepancies between results generated through routine supervision and results generated through external verification by an independent auditor.

The section on measurement reviews the construction of reliable quality measures in which a numerator, a denominator, and a measurement method (data source, sample size, frequency of measurement) are clearly defined at the outset. The structure of common quality of care process measures is reviewed (e.g. % adherence with minimum best practices; % cases meeting minimum standard). The paper briefly reviews integration of quality of care measures into established structurally-oriented checklists. Examples of checklists from World Bank supported RBF programs incorporating maternal and newborn care quality performance measures for hospitals and clinics are provided in an appendix.

There are a number of ways in which RBF programs may seek to incentivize better quality of care, including: (1) incentives for quality performance measures; 2) incentives for quality improvement activities; and 3) alignment of RBF programs with broader health system strengthening and quality improvement efforts. The distinct incentive approaches are discussed including the process of prioritizing specific conditions and services to incentivize (including potential phasing of priority conditions/services) and the construction of quality performance measures related to prioritized health conditions and services (e.g. incentivizing high-impact intervention “bundles”). Illustrative quality

3

improvement activities that may be incentivized are discussed including maintenance of provider certification, continuous quality improvement by facility QI teams (setting aims, testing changes, tracking results), peer to peer evaluation and structured death audits. Approaches to aligning or designing RBF programs linked to broader quality improvement and health system strengthening efforts such as national QI strategies, accreditation and workforce efforts are discussed including an example of a World Bank supported RBF program in Liberia being implemented in hospitals in parallel with a provider residency training program and a World Bank supported RBF program in Zimbabwe being implemented in hospitals and clinics in parallel with the roll out of a national QI strategy (Zimbabwe). Finally, promising directions for continuing to strengthen quality of care in RBF programs are explored at the level of country-specific directions and cross-country directions linked to an explicit learning and knowledge management agenda.

Introduction

Poor quality of health care is widely recognized as a major barrier to achieving optimal health outcomes and strong economies in low, middle and high-income settings. Despite a plethora of guidelines based on proven best practices, health care services world-wide often fail to deliver evidence-based care to patients when and where they need such care. Studies of care delivery in LMIC demonstrate widespread deficiencies in the care provided to patients. Many factors contribute to poor quality of care and weak health systems including:

• Governance and policy: Inconsistent national policies, standards, leadership and accountability mechanisms

• Financing: lack of financing for priority services, workforce and essential inputs

• Essential commodities: lack of functional supplies at the point of service delivery

• Provider competence: weak knowledge and skills and poor maintenance of provider competence after training

• Organization of care processes: poor organization of care processes and poor adherence with

• evidence-based standards.

• Health information systems: lack of routine quality of care data collection and analysis of data to improve services

• Improvement capacity within health system: lack of improvement capacity throughout health system.

Results based financing programs seek to address many of the above deficits through combinations of

4

incentives that may target commodities and basic infrastructure, adherence with best practices (clinical quality measures), timeliness and coordination of care (organization of care processes, referrals), and sometimes improvement activities. Due to strong emphasis on data and continuous monitoring and verification of performance measures, RBF programs contribute to the strengthening of health information systems.

Quality Dimensions and Improvement Principles

In 2001 the Institute of Medicine published a seminal report, Crossing the Quality Chasm, highlighting the chasm between the “the health care we could have and the care that we have” in the U.S. (Institute of Medicine, 2001.) The report emphasized the fact that “problems come from poor systems—not bad people” and that efforts to improve care must be anchored in a consideration of how systems of care operate in real-life complex delivery systems.

The Institute of Medicine report highlighted six primary aims of quality care including: 1. Safe: Care should be as safe for patients (do no harm)2. Effective: The science and evidence behind health care should be applied and serve as the standard in the

delivery of care (adherence with evidenced-based standards)

3. Efficient: Care and service should be cost effective, and waste should be removed from the system

4. Timely: Patients should experience no waits or delays in receiving care and service

5. Patient centered: The system of care should revolve around the patient, respect patient preferences, and put the patient in control

6. Equitable: Unequal treatment should be a fact of the past; disparities in care should be eradicated.

There are several useful conceptual models of quality of care. One model developed by Donabedian that continues to be widely used proposes three main categories from which information about quality of care can be drawn (see Figure 1): 1) Structure, 2) Process and 3) Outcomes. Structure describes the context in which care is delivered, including infrastructure, staff, financing and equipment. Process denotes the actions that make up health care as reflected in the transactions between patients and providers and staff throughout the delivery of health care. Processes can be further classified as technical processes (how care is delivered) or interpersonal processes (the manner in which care is delivered). Outcomes refer to the effects of health care on the status of patients and populations and are considered to be a result of inputs and processes of care.

Figure 1: Conceptualizing Quality of Health Care: Inputs, processes and Outcomes (Source: Donabedian 2005)

5

1. What is done

2. How it is done

Health services delivered

Change in

health behavior

Change in

health status

Patient Satisfaction

People

Infrastructure

Materials

(i.e. vaccine)

Information

Technology

Resources

(Inputs)

Activities

(Processes)

Results

(Outputs or Outcomes)

One approach to improving health care widely used in high-resource settings is the model for improvement (Figure 3). The model for Improvement is a change management strategy that stems from the work of William Edwards Deming. The model includes three basic questions to help structure improvement:

1. What are we trying to accomplish?

2. How will we know that a change is an improvement?

3. What changes can we make that will result in improvement?

Figure 3: Model for Improvement

A key tenet of improvement is that making care better always requires change, although not all change necessarily leads to improvement. Without “change” every system will continue to produce the same

6

results it has always produced. Or, in other words, “every system is perfectly designed to get the results it gets” (Paul Batalden.) Managing change is central to improvement efforts whether or not such efforts are prospective (e.g. defining aims and proactively testing changes to processes of care to try to reach the aim) or retrospective (e.g. auditing and examining adverse events to identify and correct root problems contributing to poor quality).

The Plan-Do-Study-Act (PDSA) cycle demonstrated in Figure 3 is one approach to managing change; the PDSA cycle guides tests of change by health care teams to determine if a change leads to improvement. Teams new to improvement benefit from supportive supervision (coaching) to identify and test changes to processes of care to improve adherence with best practices. Ideally, supportive supervision of QI teams includes integrated clinical, QI and data-management capacity-building over time. Improvement teams are typically made up of managers, front-line health care workers and staff who possess the necessary deep knowledge of their local systems to be able to identify and test feasible and sustainable changes to “usual processes” to improve care in their local setting.

While context has a strong influence on which changes may be most feasible and effective for overcoming gaps in a specific setting, categories of quality and system gaps and effective changes (solutions) are often common across settings. Diverse settings can learn from each other to overcome common quality and system gaps. Increasingly, many improvement approaches mobilize teams to work together across health system levels and geographic sites to identify, test and share successful changes for overcoming important quality and system gaps. Promoting regular shared learning among teams helps to accelerate and scale up best practices for overcoming common barriers to delivery of high quality of care.

Audit and feedback is a systematic review of care provided in relation to standards and guidelines of care. Audit and feedback alone, without action to correct problems may not improve care but can provide valuable insights into critical quality gaps in order to support change and solutions for improvement. Audit of adverse events and near-miss audits allow teams to reflect upon, understand and learn from rare, catastrophic (or near-catastrophic) events through peer review of cases that caused concern, affected patient safety, or resulted in an unfortunate outcome. Clinical audits are systematic reviews of patient charts to determine the care given in relation to the standard of care; they are done by sites for monthly monitoring and conducted externally for data validation. Team-based and individual self- assessments allow teams to assess themselves or each other through systematic chart reviews to determine the quality of care. It allows for a sustainable internal system and is potentially cost-effective.

Measuring Quality of Care: Opportunities and Challenges

Regular measurement and analysis of quality measures is a core principle of all improvement work and is a central component of RBF programs that incentivize quality performance measures. However, measuring quality is not simple in any setting as highlighted by a quote from a recent issue of the Journal of the American Medical Association:

7

“Quality measurement is in rapid flux….despite the challenges of a rapidly expanding number of quality measures, much of health care remains poorly measured or unmeasured.” Journal of the American Medical Association Nov 13. 2013

Measures of quality can encompass any of the quality dimensions discussed above, including timeliness of care, coordination of care (e.g. referral/counter-referral), clinical effectiveness of care (adherence with best practices), safety of care (adverse events), equity of care (same treatment for everyone), efficiency (not wasteful) and others. Often stakeholders think of clinical effectiveness (adherence with best practices) when they hear the word “quality”.

It is useful to consider which stakeholders need which quality of care information and for what purpose. For example facility staff may benefit from tracking quality of care process measures related to the specific services they are providing. District and regional managers may benefit from tracking performance of essential system functions at district level such as distribution of commodities and workforce, functionality of referral systems, etc, in addition to tracking a few sentinel process of care quality measures in the facilities they supervise. It may or may not make sense for a district health supervisor to track all quality of care indicators being monitored by facility health care team. In some cases, it may be preferable to incorporate a few sentinel measures of quality into routine information systems so as not to burden HMIS with too many indicators. Clients may benefit from having access to measures of client-centeredness in addition to other quality measures.

National policy makers may find it most useful to track primarily health outcome measures and a few sentinel quality indicators. Global stakeholders who use data in large part for advocacy and accountability purposes may need to track an even higher level of disease burden and health outcome measures.

Measurement Methods

There are a many methods that can be used to measure quality of care. Table 1 illustrates common methods of assessing quality, including advantages and disadvantages for specific methods. Individual methods have unique strengths and weaknesses depending on the purpose and context of the measurement exercise. It is important to tailor the method to the specific need, including the specific quality dimension being measured. For example a patient interview may be the best method to assess client-centeredness of care but may not be a reliable method to measure adherence with treatment standards for a complex disease due to the knowledge asymmetry between provider and patient. Often a combination of methods can yield a fuller picture of quality than any one single method. Measurement methods feasible for use as part of a one-time assessment of quality of care (e.g. to evaluate a program intervention) may be impractical for use for routine measurement of care in an RBF program or health care improvement intervention.

Table 1: Common Methods of Measuring Quality of Health Care

8

Measurement Method Advantages Disadvantages

Observation -Considered “gold standard”-Only method that measures performance of health service (as opposed to provider knowledge and competence which may not correlate with provider performance)-May be best method for assessing highly procedural health care tasks (e.g. surgery)

-Hawthorne effect-Resource intensive-Difficult to sustain in routine practice

Patient Interview

(e.g. exit interview; household interview)

-Client centeredness of care-May be reliable for simple clinical measures

-Recall problems-Knowledge asymmetry between provider and patients-Patient reluctance to give honest feedback for fear of negative consequences (e.g. facility exit interviews)

Death & Near-miss Audit -Targets adverse outcomes -May identify common quality deficits-Accountability

-Retrospective (after the fact)-Limited evidence for association between routine audit and improved outcomes

Simulation -Next best method after observation for complex procedural tasks and processes (e.g. emergency resuscitation)

-Resource intensive-Unclear relationship between simulated competence and actual performance

Provider Questionnaire -Assesses provider knowledge, self-reported practice and attitude

-Does not assess provider competence or performance ( knowledge and problem solving (e.g. vignettes)

Facility & Patient Records

• Individual patient

record

• Registers

• Other facility documents

-Relatively sustainable and low-cost-May encourage better documentation and point-of-care use of data for decision-making

-Records in LMICs often inadequate or absent altogether (e.g. no standardized individual patient record)-Providers/supervisors may falsely document data (intentionally or unintentionally)

Routine information system -Efficient extraction of data -Most HMIS track few (if any) quality of care measures.

Standardized individual patient records that capture patient-specific and clinical care data serve two

9

important functions: 1) support of real-time clinical decision-making at the point of care; 2) permit data extraction for calculation, aggregation and analysis of quality measures across different units of the system (e.g. provider-specific, facility-specific, district, national). Although the ideal, many health systems in low resource settings are still far from having individual medical records, and instead use registers to track patient-specific information. Such registers are often no more than columns drawn into a local notebook and may contain varying amounts of patient-specific clinical data depending on the register. Never the less, such registers can be manually adapted to capture simple routine best practices (e.g. addition of column for immediate post-partum oxytocin) while stronger patient records and more robust information systems are being developed.

Defining Indicators of Quality

Clinical quality of measures can be constructed in varying ways depending on the specific technical content, data source and measurement method and feasibility in an individual context. In most cases, clinical quality of care process indicators measure the adherence of care with proven best practices (e.g. evidence-based standards):

• % cases adherent with standards – “all or nothing adherence” (e.g. % PPH cases managed per

minimum standard; % cases pediatric pneumonia treated per standard)

• Average % adherence with minimum standards (e.g. average % adherence with newborn sepsis

case-management standards; N=30 cases reviewed)

It is very important to standardize operational definitions of quality of care indicators (performance measures) that include at a minimum: 1) clear numerator; 2) denominator; 3) source of data; and 4) frequency of data collection (see Table 2). The specific measurement method that is best suited to a particular indicator depends on a range of factors, including feasible data sources, and must be considered when constructing an indicator. For example, it is impossible to measure the quality of the highly procedural resuscitation of a newborn using a chart audit method. Instead, periodic observation of real care or simulated care using a structured checklist may be the most appropriate measurement method.

Table 2: Illustrative Quality of Care Measures: Clinical Effectiveness of Care

Quality of Care Measure

Operational Definition

Data Source/Sample(measurement method)

Frequency of Data

CollectionNumerator Denominator

% births in last month benefitting from Active Management of the Third Stage of labor for PPH prevention

# births in last month with administration of 10 units of oxytocin within

Total number births in last month

Birth register OR Partogram

(specify one)

Monthly

10

one minute of delivery of fetus and controlled cord traction and uterine massage.

Average adherence with eclampsia case- management standards

# eclampsia case management standards met

Total pneumonia case management standards

All eclampsia cases in hospital perinatal record in last month

Monthly

Measuring Quality of Care in Real-life Settings: The Challenge

Measuring quality of care is difficult in any setting. However, routine measurement of quality is especially challenging in low resource settings due to a range of factors including:

• Relative absence of quality of care (content) measures in many routine HMIS in low-resource settings

• Absence of standardized individual patient records in many facilities

• Lack of primary data to permit calculation of quality indicators (e.g. registers and individual partograms/records lack essential data; records may not be standardized or if standardized records may not include essential information.)

• Multiple competing vertical registers in facilities, often containing duplicative data (e.g. TB

register; hypertension register)

• Few routine indicators of performance of essential system functions (e.g. % maternities in

district with functional neonatal bag & mask at bedside.)

• Inadequate data management skills among providers and managers.

Although measurement methods in addition to chart audit such as observation and client/provider questionnaires may be useful for one-off periodic assessments, such methods are not typically sustainable for routine measurement of quality in low resource settings. Routine measurement of clinical quality may require a combination of measurement approaches, including adaptation of local records and/or registers, periodic patient and provider interviews, and periodic observation of care. Even when primary data is available in local records data is often inconsistent and of poor quality. Building staff capacity to capture and extract data to calculate quality measures is central to building capacity for continuous improvement in low resource settings and is a central component of all quality improvement and RBF programs.

In general it is far easier to measure adherence with routine best practices relevant for every patient than it is to measure adherence with more complex processes of care such as case management of complications. For example, a routine best practice such as administration of Tetanus Toxoid during

11

antenatal care can be tracked fairly easily by checking a “box” in a standard record or register column. It is considerably more difficult, however, to construct a simple and feasible measure of adherence with eclampsia case-management standards across a necessary continuum of care including: 1) timely accurate diagnosis; 2) stabilization and successful timely referral (primary facility); 3) prompt and ongoing treatment/monitoring (hospital); 4) discharge planning and follow up (hospital).

The emergence of electronic health records and electronic health information systems in many countries is increasing the efficiency of data management and holds great promise for standardizing and improving measurement of quality of care. However, as is being learned in high-resource settings, electronic health records and registries are not without significant challenges. Unless EHRs are pro-actively designed to be user-friendly and to capture and automatically aggregate quality of care data and indicators, they will continue to face the same challenges as traditional paper records.

Incentivizing Quality in RBF Projects

Health systems in high-resource settings have incentivized quality of care performance measures for many years. Many RBF schemes in high-resource settings incentivize a combination of clinical process measures (e.g. adherence with diabetes care standards) and outcome measures (e.g. blood glucose control in diabetes, hospital re-admission rates, post-operative death rates, etc.)

Increasingly, RBF programs in LMICs are beginning to incentivize quality-related performance measures in addition to volume of services (e.g. number of deliveries) and structural measures (e.g. availability of essential drugs). There are a number of ways in which RBF programs may seek to improve quality of care, including: (1) incentives for quality performance measures; 2) incentives for quality improvement activities; and 3) alignment of RBF programs with broader health system strengthening and quality improvement efforts.

Incentivizing quality of care performance measures

As is the case for any program that seeks to improve quality of care, RBF programs must define which quality of care measures to incentivize. In LMICs where quality of care deficits are significant across a range of clinical conditions and service delivery types it can sometimes be challenging to know where to start since it is clearly impossible to incentivize and measure everything at once.

In general, it is useful to consider the following factors, at a minimum, when deciding which clinical conditions and service delivery types to prioritize:

• Target high-burden conditions in country and local context responsible for the greatest burden

of disease in local context (leading causes of mortality and morbidity)

• Target high-burden conditions for which there is strong evidence of effective health care

interventions (preventive and curative)

12

• Align selection of priority conditions and services with government priorities; involve national

and local decision makers and experts in the decision about which areas to incentivize• When possible, consider phasing improvement priorities during RBF program implementation

since it is not possible to “improve everything at once”

Once an RBF program has defined a set of priority health conditions and services for which it seeks to improve quality, then program managers need to engage local and international experts to examine country standards and guidelines against global evidence since evidence is constantly changing. In general, review of World Health Organization guidelines is a good starting point since such guidelines are usually based on a systematic review of the evidence and are likely to be respected by country government experts. When possible, it is preferable to distill standards for priority conditions and services into the minimum highest-impact “intervention bundles” most likely to yield the best outcomes, to be understood by local providers, and to be measurable with relatively simple measurement methods. When possible, it is best to avoid long lists of standards that are difficult to measure and verify and difficult to understand among providers working in high-volume messy systems. For example, an established RBF program seeking to incorporate a greater focus on quality can add a set of quality indicators to the structural checklist.

A challenge faced by many structurally oriented RBF programs as they transition to include a greater focus on quality is the need to revise the qualifications of supervisors and independent verifiers who assess the quality performance measures. Regardless of the measurement method used (chart audit, provider knowledge/case study, observation, etc) supervisors who assess performance on quality measures and independent assessors who verify results must possess a minimum level of clinical qualifications. As RBF programs incentivize more complex clinical processes such as hospital case management of severe pneumonia the clinical qualifications of the supervisor/assessor becomes increasingly important. Because many supervisors are primarily administrative managers who no longer (or never did) practice clinical care it may sometimes be necessary to involve clinician experts in assessing and verifying quality measures in RBF programs. Another challenge faced by many RBF programs is a discrepancy between quality scores calculated by supervisors versus scores calculated by independent assessors. This challenge is may be particularly difficult when complex process of care performance measures is involved.

Incentivizing Quality Improvement and Health System Strengthening Activities

In addition to incentivizing quality of care performance measures, RBF programs can incentivize specific improvement activities at the level of the individual provider, the facility, the regional or district health management team, or other unit. Table 3 provides examples of improvement activities that may be incentivized as part of RBF programs.

Table 3: Illustrative quality improvement activities that can be incentivized in RBF programs

13

Activity to Improve Care Description

Regular facility QI team meetings Regular facility QI team meeting (review of meeting notes and action plans)

Routine collection & analysis of quality of care measures

Routine collection and analysis of priority quality measures by facility teams and /or district health management teams

Continuous quality improvement Continuous improvement led by facility teams: clear measurable improvement aims and indicators; regular tests of change and tracking of results to determine if care is (or is not) improving

Regular peer to peer evaluation Regular structured peer to peer review of medical records with feedback; regular peer to peer structured observation of procedural tasks (e.g. newborn resuscitation)

Provider professional development and or certification

Incentives tied to minimal standards of professional development and/or provider maintenance of certification

Facility accreditation Incentives for maintenance of external accreditation by facilities

Laboratory and pharmacy quality assurance processes

Death and near-miss audits Structured retrospective review and analysis of adverse outcomes, linked to an action plan

Aligning RBF with Improvement & Health System Strengthening Efforts

Many governments and donors worldwide are increasingly investing in strengthening health systems and improving quality of care. Such efforts are likely to increase in the era of universal “effective” health coverage and there are many opportunities for RBF programs to align with such initiatives, including policy, regulation and standards-setting/protocol initiatives.

Policy related to improvement falls into two distinct areas: 1) policy outlining a country’s approach to improvement (e.g. national QI policy and strategy); and 2) policy on specific clinical or non-clinical health related areas. National QI strategies typically cut across all technical areas, defining strategic objectives and implementation plans to address quality and system gaps in disease-specific programs (e.g. T.B., MNCH) and cross-cutting system functions (e.g. health information systems, supply chain and workforce). RBF programs may be designed to reinforce the implementation of a national QI strategy. For example, in Zimbabwe the second phase of a World Bank supported RBF program is being introduced in parallel with the roll-out of a national quality improvement strategy that includes pre- and in-service training and the phased introduction of an electronic medical record and DHIS2 to strengthen the country health management information system (HMIS). RBF program supervision checklists with quality indicators in prioritized clinical areas are building capacity of supervisors and front-line facility

14

health care teams to pay attention to and continually assess quality of care.

Standards are explicit statements of expected quality in the performance of a health care activity. They may take the form of procedures, clinical practice guidelines, treatment protocols, critical paths, algorithms, standard operating procedures, or statements of expected health care outcomes, among other formats. For example the WHO Integrated Management of Childhood Illness (IMCI) treatment protocol spells out specific standards of care for care of the sick child. Such clinical protocols may be incentivized in RBF programs; indeed several World Bank supported RBF programs incentivize adherence with clinical protocols.

Regulation approaches help maintain and improve quality, ensure patient safety, provide legal recognition to qualified health professionals, and verify that design or maintenance specifications are met. Regulation approaches include accreditation (facility), professional licensure and renewal, and certification and re-certification of professionals and facilities. Many countries are using regulatory approaches to try to improve or ensure a minimum level of quality and RBF programs may incentivize specific regulation activities (e.g. incentives for hospitals that maintain accreditation).

In many cases, RBF programs are being proactively designed and implemented in tandem with complementary health system strengthening activities For example, in Liberia a World Bank supported Hospital RBF program is being implemented in parallel with a hospital-based residency training program to address critical workforce gaps. Programs that incentivize availability of essential commodities such as medications and equipment can be implemented in parallel with supply chain strengthening efforts.

Global efforts in many technical areas are increasingly identifying quality of care as one essential element for achieving targeted health outcomes (e.g. MDGs). For example, the World Health Organization (WHO) Department of Maternal, Child and Adolescent health convened a three-day meeting on quality of maternal, newborn and child health in December 2013 and is planning several follow-up initiatives including an Every Mother and Every Newborn (EMEN) quality initiative to complement the soon-to-released Every Newborn Action Plan (ENAP).

Promising Directions for Strengthening Quality as Part of RBF programs

There are many promising directions for strengthening quality of care in RBF programs in low-resource settings. First and foremost is the recognition of the importance of quality and the huge quality of care deficits that undermine the performance of health systems worldwide.

Promising directions for strengthening quality in RBF programs are categorized with respect to country/local directions and cross-cutting multi-country directions:

Illustrative Directions for Strengthening quality of Care in Country RBF programs

15

• Phased introduction of clinical quality performance measures focused on high-burden diseases

with demonstrated quality of care gaps and for which there are evidence-based interventions

• Incentives for performance measures tailored to distinct system levels (e.g. primary facilities,

hospitals, district management teams)

• Inclusion of incentives for improvement activities at facility level and for improvement capacity-

building by supervisors/managers (see Table 3)

• Design of complementary RBF and health system strengthening interventions (HMIS, workforce,

supply chain, etc)

• Alignment of RBF programs with established health system strengthening and improvement

efforts (e.g. provider pre-service training; electronic HMIS, etc)

• Consideration of inclusion of incentives for quality measures related to safety (adverse events),

coordination of care (referral/counter-referral), and client-centeredness of care

• Consideration of inclusion of incentives for measures of equity of health care (accessible, high-

quality care for the poorest and most vulnerable)

• Regular participation of RBF program managers in country inter-agency technical working

groups relevant to program technical focus

• Improvement capacity-building of Government and private sector stakeholders, including

strengthening data management skills

Illustrative Directions for Strengthening Quality Across Country RBF programs

• Participation (or regular interface with technical experts) in global technical working groups and

technical conferences (e.g. Inter-agency Newborn Indicators Working Group; NCD Alliance; International Forum on Quality and Safety; etc.)

• Regular tracking and alignment of quality component of RBF programs with global initiatives in priority technical areas (e.g. WHO Department of Service Delivery and Strengthening (SDS) initiatives; UN NCD Global Action Plan and monitoring and evaluation framework; No Child Left Behind; Every Newborn Action Plan)

• Articulation of a priority learning agenda linked to an operational research and knowledge management strategy

• Regular synthesis and analysis of learning across RBF programs (knowledge management)

16

References:

Berwick DM. Lessons from developing nations on improving health care, BMJ 2004;328:1124-8.

Donabedian, A. (1997). The quality of care. How can it be assessed? Archives of pathology & laboratory medicine, 121(11), 1145-1150.

Donabedian, A (2005). "Evaluating the quality of medical care. 1966.". The Milbank quarterly 83 (4): 691–729

Donabedian, A. (2003). An introduction to quality assurance in health care. (1st ed., Vol. 1). New York, NY: Oxford University Press.Institute of Medicine.

Haynes et al. A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine. 2009; 360:491-499.

Institute of Medicine. Crossing the Quality Chasm: A new health system for the 21st century. Washington D.C. National Academy Press; 2001.

Institute of Medicine. To Err is Human: Building a Safer Health System. 1999.

Institute for Healthcare Improvement. The Breakthrough Series: IHI’s Collaborative Model for Achieving Improvement Innovation Series. Cambridge, MA: Institute for Healthcare Improvement, 2003.

Nolan T, Angos P, Cunha AJLA et al. Quality of Hospital Care for seriously ill children in less developed countries. Lancet 2001; 357:106-10.

17

Rowe AK et al. How can we achieve and maintain high-quality performance of health workers in low resource settings? Lancet 2005;366:1026-35.

Appendices: Hospital Eclampsia Chart Audit Tool Liberia RBF program

Eclampsia Chart Review TABLE

See Eclampsia Chart Review Guide

(based on quarterly review of X randomly selected charts)

Site ………. ……… Month ……………… Year……….

Chart review elements (see chart review guide for specific criteria) ; each element if recorded = 1 point

Charts

1. Evaluation 1 2 3 4 5

1. Blood pressure (BP) recorded

2. Gestational age (GA) recorded (per one of criteria indicated in GUIDE)

3. Urine protein quantified (dipstick +, ++, +++)

4. Danger signs assessed (see chart review guide)

Evaluation Score (x/4)

2. Diagnosis pre-eclampsia or eclampsia recorded (if applicable criteria met)

1. DBP > 90 and at least 2+ proteinuria pre-eclampsia (+ seizure if eclampsia)

Documentation of Diagnosis (x/1)

18

3Treatment

(a) Mild pre-eclampsia

1. BP check and surveillance for danger signs at least every 4 hours in labor and post-partum(b) Severe pre-eclampsia

1. 4 gm loading dose of MgSO4 IV ; monitor for toxicity (reflexes, urine output, respirations)

2. 5 mg MgSO4 IM every 4 hours until minimum 24 hrs after delivery or convulsion (whichever comes laterMay consider anti-hypertensive for DBP > 90 (e.g. hydralazine 5 mg or any safe in pregnancy)

3.4.Stabilize and deliver immediately if GA > 34 weeks

5.If GA < 34 weeks and patient stable administer corticosteroids to promote fetal lung maturity(c) Eclampsia

1. 4 gm loading dose of MgSO4 IV ; monitor for toxicity (reflexes, urine output, respiration)

2. 5 mg IM every 4 hours (national guideline) until minimum 24 hours after delivery or convulsion (whichever comes later)

3. Deliver within 12 hours

4. Consider anti-hypertensive (e.g. hydralazine 5 mg or any safe in pregnancy)

Treatment score (x/3)

4. In-hospital monitoring (labor & post partum)

1. Vital signs monitored- (including BP) every 4 hours

2. Danger signs assessed at least twice daily (HA, visual disturbance, epigastric pain

3. Provider note in chart at least once daily

In-hospital monitoring score (x/3)

5. Discharge

1. Vital signs documented for at least 24-48 hours (continued risk of eclampsia)

2. Pre-discharge physical exam documented with pulmonary and cardiac exam

3. Pre-discharge counseling noted : danger signs ; one week follow up ; counseling for recurrence risk future pregnancies ; FP method

Discharge score (x/4)

19

Numerator = Total of 1, 2, 3, 4, 5

Denominator = Total possible elements (14)

% adherence with pre/eclampsia standards per chart (numerator/denominator x 100)

Average % adherence with pre/eclampsia standards (all charts)

% Charts at least 80% adherence w/ pre-eclampsia standards

Guidelines for a Section of a Rural Health Center Supervision Checklist Zimbabwe RBF Program

Maternity Services Instructions for Completion of Checklist Quality Item

Routine MNH best practices; PPH & sepsis prevention/management (mother & newborn); post-partum FP

% partograms in last month completed per guideline (random review minimum 10 partograms)

-FHR, cervical dilatation, descent of presenting part, maternal BP, pulse, and temperature documented at admission and at least every 4 hours from admission until delivery

Randomly review (every 3rd partogram) at least 10 partograms completed in last month.

Calculate: % partograms completed per minimum standard.

Numerator: Total # partograms reviewed that document FHR, cervical dilation, descent of presenting part, maternal BP, pulse, temperature at admission and at least every 4 hours until delivery

Denominator: Total # partograms reviewed

% women with prolonged labour referred to higher level facility

--Review all partograms completed in past month and select out partograms for analysis in which birth occurred > 12 hrs after onset active labor (from time of admission or time of 4 cm dilation or > 12 hrs from patient-reported onset labor if admitted > 4 cm cervical dilation) (note total

20

-Review all charts meeting obstructed labour criteria in past month: active labour > 12 hours (from admission at minimum 4 cm dilation or per patient-reported labour onset if admitted > 4 cm

# in HMIS maternity section below)

-Calculate: % of partograms with labor > 12 hours in which women referred to higher level facility

Numerator: Total # partograms documenting active labor > 12 hours referred to higher level facility

Denominator: Total # partograms documenting active labor > 12 hours

% women delivered in last quarter administered uterotonic within one minute of delivery of foetus

(Active Management Third Stage of Labor (AMTSL) for PPH prevention).

-administration oxytocin 10 units IM within one minute of delivery of fetus (or misoprostol or ergomertrine, if BP normal, and oxytocin unavailable)

-Review birth register (or partograms) for all births in past month and calculate:

% total births in last month documenting administration of oxytocin 10 units IM within one minute of delivery of fetus (or misoprostol or ergometrine if BP normal and oxytocin unavailable)

Numerator: # of births in past month in which oxytocin (or another uteronic) administered within one minute (immediately after) delivery of fetus

Denominator: Total # births within past month

% newborns BF within one hour of birth Review birth register, post-partum register (or other appropriate register) and calculate:

% total births in last month documenting breastfeeding within one hour of birth

Numerator: Total # births documenting initiation of breastfeeding (BF) within one hour of birth

Denominator: Total # births in past month

% women delivered in last quarter monitored per standard in early post-partum period for early identification of danger signs

-Randomly review (every 3rd case) records post-partum register entries for at least 10 women delivered in last month

-Calculate: % post-partum women monitored per

21

(review at least 10 charts)

- vaginal bleeding, BP, pulse, respiratory rate, temperature at least every 30 minutes 1st 2 hrs after birth and then at least twice per day until discharge

standard (birth to discharge)

Numerator: Total # women delivered in past month with documentation of following items at least every 30 minutes first 2 hours after birth and then at least twice per day until discharge: vaginal bleeding (present or not), BP value, pulse value, respiratory rate, temperature value

Denominator: Total cases reviewed of women delivered in past month

22