Assessing health system performance in developing countries: A review of the literature

A

gtugsMaao©

K

C

1

0

Health Policy 85 (2008) 263–276

Available online at www.sciencedirect.com

Review

Assessing health system performance in developingcountries: A review of the literature

Margaret Elizabeth Kruk a,b,∗, Lynn P. Freedman b

a University of Michigan School of Public Health, Department of Health Management and Policy,109 Observatory Road, SPH II M3166, Ann Arbor, MI 48109, USA

b Averting Maternal Death and Disability Program, Columbia University Mailman School of Public Health,New York, NY, USA

bstract

With the setting of ambitious international health goals and an influx of additional development assistance for health, there isrowing interest in assessing the performance of health systems in developing countries. This paper proposes a framework forhe assessment of health system performance and reviews the literature on indicators currently in use to measure performancesing online medical and public health databases. This was complemented by a review of relevant books and reports in therey literature. The indicators were organized into three categories: effectiveness, equity, and efficiency. Measures of healthystem effectiveness were improvement in health status, access to and quality of care and, increasingly, patient satisfaction.

easures of equity included access and quality of care for disadvantaged groups together with fair financing, risk protection andccountability. Measures of efficiency were appropriate levels of funding, the cost-effectiveness of interventions, and effectivedministration. This framework and review of indicators may be helpful to health policy makers interested in assessing the effectsf different policies, expenditures, and organizational structures on health outputs and outcomes in developing countries.

2007 Elsevier Ireland Ltd. All rights reserved.

eywords: Health systems; Health indicators; Health metrics; Health system performance; Developing countries

ontents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2642. Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2663. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1. Effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.1.1. Health status . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.1.2. Patient satisfaction . . . . . . . . . . . . . . . . . . . . . . .

∗ Corresponding author at: University of Michigan School of Public Heal09 Observatory Road, SPH II M3166, Ann Arbor, MI 48109, USA. Tel.: +

E-mail address: [email protected] (M.E. Kruk).

168-8510/$ – see front matter © 2007 Elsevier Ireland Ltd. All rights resedoi:10.1016/j.healthpol.2007.09.003

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

th, Department of Health Management and Policy,1 734 615 3633; fax: +1 734 764 4338.

rved.

mailto:[email protected]

dx.doi.org/10.1016/j.healthpol.2007.09.003

264 M.E. Kruk, L.P. Freedman / Health Policy 85 (2008) 263–276

4.1.3. Access (availability, utilization, timeliness) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2694.1.4. Quality of care (efficacy, safety, continuity) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270

4.2. Equity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2714.2.1. Equitable health status and access to care . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2714.2.2. Fair financing and risk protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

4.3. Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2724.3.1. Costs and productivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2734.3.2. Administrative efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273. . . . . .. . . . . .

1

ipApdhdgtabf

stdppsfcaTsrp

woWhat

ccwSoaitevasatgthehtahtd

simaspd

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. Introduction

The past decade has witnessed a renaissance ofnterest in health systems in academic discourse andolicy dialogue within the development community.fter 20 years of neglect in favor of vertical healthrograms, community-based small-scale projects, andonor-directed thematic health investments, strongealth systems are again seen by policy makers andonors as essential to achieving and sustaining healthains [1–3]. This has been in part stimulated byhe Millennium Development Goals that call for thechievement of several health targets simultaneouslyy 2015—difficult if not impossible to achieve withoutunctioning health systems.

We begin with the WHO [4] definition of healthystem: “all the activities whose primary purpose iso promote, restore, or maintain health” but narrow itown to those activities, which are under complete orartial control of governments, as governments are therimary funders and in many cases providers of healthervices in developing countries. These activities rangerom direct service provision through public sectorlinics and hospitals, to population-level public healthctivities to funding community-level health education.he private sector in health is not excluded from con-ideration, however, as governments play an importantegulatory role that can influence the performance ofrivate providers.

Assessing the performance of a health system beginsith defining its goals. While there is an ongoing debaten the ultimate aims of a health system, we follow the
orld Health Organization and define the goal of a
ealth system to be the delivery of effective preventivend curative health services to the full population, equi-ably and efficiently, while protecting individuals from

orst

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

atastrophic health care costs [4]. The concept of theommunitarian underpinnings of health systems wasell captured by the UN’s Committee on Economicocial and Cultural Rights, which noted that states arebligated to ensure availability, accessibility, accept-bility, and quality of health services [5]. As core socialnstitutions, health systems also need to be responsiveo the needs and demands of the population [6]. Forxample, in a democratic society claims to health ser-ices and conditions that promote health can be seen asssets of citizenship [7]. Thus we would include patientatisfaction, public participation in decision making,nd accountability as key aims of health systems, dis-inct from the clinical and economic goals. Theseoals are similar in developed and developing coun-ries, although developing country governments whoseealth budgets are highly constrained are generallyxpected to deliver essential rather than comprehensiveealth services [8]. Essential health services are thosehat address the major contributors to death and dis-bility in countries, ranging from child and maternalealth services to prevention and treatment of infec-ious diseases to basic response to injuries and chronicisease.

Indicators of health system performance cannot beeen in isolation from their ultimate purpose and fromssues of measurement. Governments need tools to

onitor and evaluate the functioning of the system onroutine basis and to allow for more informed deci-

ions about health systems funding, organization andolicies. Performance indicators are also important toonor countries who want to document effective use
f official development assistance for health and toesearchers who generate evidence relevant to healthystem scale-up and reform. While a full discussion ofhe selection and measurement of indicators is beyond

Health

trmoodhvdtmmgtGriHW

ocontniRhpf

oc

M.E. Kruk, L.P. Freedman /

he scope of this paper, metrics need to be locallyelevant, reliable and valid as well as feasible to imple-ent. Process indicators should be causally linked to

utcomes and sensitive to change in policy. Devel-ped and developing countries may thus adopt veryifferent indicators in measuring the performance ofealth systems and different indicators may be rele-ant depending on the unit of analysis (e.g., facility,istrict, nation). There are now several major initiativeso standardize and harmonize the collection of health

etrics globally and to make recommendations on theost useful measures. These include the UN’s Intera-

ency Group on Indicators that recommends measureso track progress on the Millennium Developmentoals across countries, the WHO-based Health Met-

ics Network that is helping countries to develop healthnformation systems, and a new research Institute forealth Metrics and Evaluation at the University ofashington, among others [9–11].

(meo

Fig. 1. Framework for health syste

Policy 85 (2008) 263–276 265

This paper presents a systematic literature reviewf health system performance indicators or measuresurrently being used in the field, with a focus on devel-ping countries. Given the complex and locally specificature of health systems and the corresponding needo customize indicators for different settings, we didot set out to compare the quality or feasibility of thendicators reviewed here, which span many countries.ather, in this review, we describe the indicators thatave been applied in the field to measure health systemerformance and highlight those indicators that wereound to be in most common use.

To organize this review, and based on the definitionf the goals of a health system discussed above, wereated a framework for health systems performance
see Fig. 1). The three major dimensions of perfor-ance in our framework are effectiveness, equity, and
fficiency and the inputs are policies, funding andrganization. This framework is informed by extensive

ms performance measures.

2 Health

pscaeswh

2

E““cfWaciep

fimfsucuimlAwowi

3

ltt

Woc

seigroisiSseTo

4

4

tsmWrcucfpabiCaeier

66 M.E. Kruk, L.P. Freedman /

revious work on defining key elements of healthystem performance in both developed and developingountries [4,5,12–16]. Because the systematic liter-ture review focuses on indicators for health systemffects (outputs and outcomes) rather than healthystem inputs, the discussion below deals primarilyith the outputs/processes and outcomes/impacts ofealth systems.

. Methods

We performed a literature review of all literature innglish published since 1995 using the search termshealth system effectiveness” “health system equity”,health system efficiency” in combination with “indi-ators”, “measures” and “metrics”. We searched theollowing databases: PUBMED, Medline, ELDIS, the

orld Bank library, WHO library, ID-21, EMBASE,nd the Cochrane library. We also reviewed key arti-les, conference publications, and texts that were notncluded in the database search through discussion withxperts and by consulting the reference lists of theapers identified.

Two types of articles were selected for review. Therst were conceptual papers on health system perfor-ance measurement, which we used to assist us in

ormulating our conceptual framework. The secondet of papers selected discussed actual indicators inse in developing and to a lesser extent, developed,ountries. The latter articles either assessed indicatorssed to measure the impact of health policy changesn countries and reforms or reported on direct trials of

easurement tools. While we include major reviews orarge cross-national comparisons from Europe, Northmerica and other parts of the developed world, weere especially interested in research based in devel-ping countries. All publications meeting these criteriaere reviewed independently by two investigators for

nclusion in the review.

. Results

Our initial screen resulted in 685 peer-reviewed pub-ications from PUBMED and 168 publications fromhe other databases including those published by mul-ilateral organizations such as the United Nations, the

ccsl

Policy 85 (2008) 263–276

orld Bank, and the European Union, by bilateral aidrganizations as well as by academic institutions andivil society.

Most of the publications identified through theearch reported the results of health system reforms orvaluated specific health programs rather than exam-ning the measures used for evaluation. For example,iven our focus on the measures used rather than theesult of health reforms, we did not do a detailed reviewf the many papers examining the impact of reformsn health policy and financing around the world. Onlytudies that gave examples of actual application of thendicators in the field were included in the review.elected for final review were 118 papers on healthystem effectiveness, 90 on equity, and 97 dealing withfficiency. The indicators identified are summarized inable 1. The indicators most commonly used in devel-ping countries are indicated in bold type.

. Discussion

.1. Effectiveness

For the individual patient an effective health sys-em provides timely access to the full array of neededervices, efficacious and safe care leading to improve-ent in health, continuity of care, and respect [4,12,14].hile in theory these effectiveness characteristics are

elevant across countries, in developed countries withomprehensive health insurance and universal or near-niversal access to basic services, aggregate access andontinuity are measured less frequently than accessor disadvantaged groups, quality, safety, equity, andatient satisfaction. However, in developing countriesccess to even basic services is still one of the majorarriers to better health for much of the population ands thus a focus for policy makers and analysts [15,16]uriously, WHO excluded measures of access from itsnalysis of health system performance [4]. Technicalfficacy, safety, and continuity are variously includedn the term “quality” by some analysts, whereas oth-rs use quality synonymously with effectiveness. Theeview found that the patient’s experience of health
are, expressed as patient satisfaction, is used as an out-ome indicator of the effectiveness or quality of healthystems—particularly in developed countries whereong life expectancies and generally favorable health

M.E. Kruk, L.P. Freedman / Health Policy 85 (2008) 263–276 267

Table 1Summary of health performance indicators

Performance dimension Category Sample indicators

Effectiveness (outcomes) Health status • Infant mortality• Maternal mortality• Neonatal mortality• Incidence of low birth weight• Survival rates for lung cancer

Patient satisfaction • Being treated with respect• Quality of physician-patient communication• Length of wait for care• Administrative simplicity• Perception of access to specialists• Adequacy of time spent with physician

Effectiveness (outputs) Access to care Availability• Physicians, nurses, hospitals per 1000 population• Basic and comprehensive emergency obstetric carefacilities per 500,000 population• Percentage of population within 10 km of a clinic• Referral rates for women with obstetric complicationsUtilization• TB case detection rates• Antiretroviral treatment rates for people with advancedHIV infection• Rates of sleeping under malaria bed net (under-5)• Contraceptive coverage• Pregnant women receiving four antenatal care visits• Deliveries assisted by a skilled birth attendant• Full basic immunization rates• Screening for breast, cervical cancerTimeliness• Effective treatment for malaria within 24 h• Rapid treatment for delivery complications• Avoidable hospitalizations

Quality of care Efficacy• Use of evidence-based diagnostics and therapies (HbA1C for diabetes, aspirin for myocardial infarction, correctantibiotic for community acquired pneumonia)• Correct prescribing (dosage, duration)• Readmission rates• Rate of emergency room visits within 30 days of discharge• Rate of avoidable hospitalizationsSafety• Infection and complication rates of surgery• Case fatality rates• Rate of hip fractures in facilitiesContinuity• Frequency and rate of follow-up visits (antenatal care,antiretroviral treatment)• Treatment completion rates (TB)

Equity (outcomes) Health status (disadvantaged groups) • Mortality rates for lowest income quintile (under-five,15–49, maternal, cancer)• Mortality rates for women, immigrants, members of ethnicgroups, people in remote geographic area

268 M.E. Kruk, L.P. Freedman / Health Policy 85 (2008) 263–276

Table 1 (Continued )

Performance dimension Category Sample indicators

Fair financing • Proportion of government health financing that reachesthe poorest income quintile• Progressivity of financing methods (tax, out-of-pocket)• Extent of out-of-pocket payments, indirect payments andinformal fees for essential services

Risk protection • Proportion of population with catastrophic healthexpenditures• Incidence of impoverishment as a result of health payments

Equity (outputs) Access (disadvantaged groups) • Distance from clinic for disadvantaged populations• Utilization of essential health services by disadvantagedgroups (e.g., attended delivery, modern contraceptives,specialist visits)

Quality (disadvantaged groups) • Efficacy, safety, continuity indicators analyzed fordisadvantaged groups

Participation/accountability • Perception of exclusion/inclusion from health system

Efficiency (outcomes) Value of resources • Mortality rates per dollars invested in health care• Mortality rates for different financing structures

Efficiency (outputs) Adequacy of funding • Per capita health care spending (government, private,total)

Costs and productivity • Costs per case treated (per hospital day, per outpatientvisit)• Costs per case at different levels of care• Average length of stay• Physician-consultations per day• Cost-effectiveness ratios for specific services (compared toalternative services)

Administrative efficiency • Health worker attrition rates• Health worker morale

iinpo

4

anttdimt

wmoitoDioo

4

ndicators make measurement of the health system’smpact on health outcomes difficult. Thus the compo-ents of effectiveness we focus on are: health status andatient satisfaction (outcomes) and access and qualityf care (outputs).

.1.1. Health statusFrom a population perspective, the sine qua non of

n effective health system is an improvement in theation’s health status, although this can be difficulto document in low mortality settings where prema-ure mortality is low [17]. Health outcomes chosen to
emonstrate the effectiveness of a health system shoulddeally be ones that are most amenable to improve-
ent through the actions of the health system, ratherhan through access to exogenous factors such as clean

ettb

• Updated operational plans• Frequency of supervision, training• Waiting times for appointments

ater and education, although the various health deter-inants are difficult to disentangle. Examples of health

utcomes frequently selected by developing countriesnclude: infant mortality, maternal mortality, perina-al/neonatal mortality, low birth weight, and incidencef infectious diseases (see for example the Millenniumevelopment Goals [18]). Developed countries add

ndicators such as survival rates from different typesf cancers, although this can also be seen as a measuref the quality of care [19].

.1.2. Patient satisfactionPatient satisfaction with the health system has been

xtensively studied in industrialized countries. In coun-ries where health systems are largely funded throughax revenue or where access to health is guaranteedy the constitution, surveys of patient satisfaction can

Health

gitFoB

idpbciscccsohitttctiTqaopqthsslasaato

4

i

amdapstcAtmlpUahppcrwttaes(as

taotIatwapm3pO


ive the population an opportunity to express its opin-on about an important social program and hence abouthe ability of government to deliver on a key obligation.or example, health system performance is a key partf election year debate in countries such as Canada andritain [20,21].

Some health system features important to patientsdentified by American researchers studying armyependents included: caring behavior, provider com-etence, being treated with respect (“like I matter”),edside manner, good information sharing, efficientare process, and administrative simplicity of healthnsurance [22]. A review of published studies of patientatisfaction identified several common criteria: patient-entered care, access, communication and information,ourtesy and emotional support, technical quality, effi-iency of care, and structure and facilities [23]. Patientatisfaction is thus an important independent measuref the success of a health system. A team from Harvardas been studying public perception of health systemsn five countries since 1988 [24,25]. Adults in Aus-ralia, Canada, New Zealand, the United Kingdom, andhe United States were asked whether their health sys-ems needed minor change, fundamental change or aomplete rebuilding as well as questions about accesso needed care, access to specialists, the affordabil-ty of care (out-of-pocket payments), and quality [25].his last variable encompassed questions on overalluality of health services, time spent with physicians,dequacy of the time spent with physicians, qualityf their hospital experience, and adequacy of the hos-ital length of stay [25]. By retaining the same coreuestions over the years, researchers were able torack trends in public opinion (for example, Canadiansave become increasingly dissatisfied with their healthystem since 1988). The study design also permittedubgroup analysis, such as the views of the elderly andow income people [24]. Developing countries havelso begun to measure public satisfaction with healthystems. Mexican patients were asked to rate two par-llel health systems based on whether the staff wasble to solve their health problem, respected their tradi-ions, spoke their language, was friendly, and providedverall good quality of health service [26].

.1.3. Access (availability, utilization, timeliness)Population coverage with essential health services

s a commonly used measure of access. Service cover-

tboa

Policy 85 (2008) 263–276 269

ge is an example of an output indicator—a proximateeasure of the quantity of goods and services pro-

uced and used, whereas improved health indicatorsre an outcome (or impact) measure. Access to andopulation coverage of services are two sides of theame coin—seen from an individual and a popula-ion perspective, respectively. Access has at least threeomponents—availability, utilization and timeliness.vailability of services is a process indicator linked to

he policies, funding levels and organizational arrange-ents in each country. Availability is reported as the

evel of inputs (physicians, nurses, hospitals, clinics)er population or within a geographic area. Thus, theN process indicators for emergency obstetric care

sk countries to report functioning basic and compre-ensive emergency obstetric care facilities per 500,000opulation [27–30] and countries routinely report theercentage of population within 5 or 10 km of a healthlinic. Physician and nurse to population ratios haveecently taken on new importance in light of recentork suggesting health worker density has an impor-

ant impact on improving certain health outcomes, andhe quality of this data is improving [31]. Related tovailability is organizational access – that is the pres-nce or absence of structural/systemic barriers to care –uch as lack of referral to the appropriate level of careeven when it is available) [32]. Financial barriers toccess are discussed in Section 4.2, as they affect pooregments of the population disproportionately.

An OECD review found that industrialized coun-ries commonly use utilization of screening tests suchs those for breast and cervical cancer as measuresf access to (and equity of) services [19]. Utiliza-ion of services thus becomes a proxy for access.n the developing world, utilization indicators havessumed a greater importance with the introduc-ion in 2000 of the Millennium Development Goals,hich have encouraged countries to set targets for

nd report access to health services to demonstraterogress on reaching the MDGs [33]. Perhaps theost well-known recent utilization target is WHO’s× 5 initiative, which promised to place 3 millioneople on antiretrovirals by the end of 2005 [34].ther examples of commonly used utilization indica-

ors are: case detection rates for TB, use of malariaed-nets, access to antimalarials within 24 h of onsetf symptoms, contraceptive coverage, antenatal carettendance (and specific antenatal services such as

2 Health

to(oapnnia[riwsaaettfa

4

ttctciwiDhoaottqrrgarSf

apatfppsfhm

ipRgcqdo[vomaIotePItmriti[

rrtaf


etanus immunization), attended deliveries, availabilityf basic and comprehensive emergency obstetric careEmOC) facilities, immunization rates, and availabilityf essential drugs [1,35,36]. Again, there is consider-ble debate about how well some of these indicatorsroxy potential gains in health. For example, ante-atal care and currently available immunizations doot have large impacts on reducing the overall mortal-ty of mothers and children, respectively, yet they aremong the most commonly reported access indicators1]. The notion of timely access is increasingly beingecognized as an important feature of access althoughndicators to measure this are few in the developingorld [14]. Timely access is essential to save lives in

ome conditions (e.g., for malaria, birth complications,cute myocardial infarction) and to minimize sufferingnd disability in others (e.g., chronic illness). Thus, theffectiveness treatment for malaria is assessed throughhe provision of appropriate medicines within 24 h ofhe onset of symptoms [37]. Avoidable hospitalizationsor selected conditions can also indicate problems inccess to timely preventive and primary care [19].

.1.4. Quality of care (efficacy, safety, continuity)Perhaps the most researched component of effec-

iveness is quality of care—in its broadest sense. Whilehere is no single definition of quality, the frequentlyited Institute of Medicine’s definition is health carehat is safe, effective, patient-centered, timely, effi-ient, and equitable. Some of these components arencluded elsewhere in our framework; in this section weill focus on efficacy, safety and continuity. This last

ndicator was suggested by Judith Bruce who adaptedonabedian’s quality framework and we include itere to measure an important dimension of the processf providing care [38]. Indicators for quality of carebound, particularly in developed countries. Measuresf quality of care for children in the US include indica-ors of safety, effectiveness, patient-centeredness, andimeliness [39]. Canadian researchers evaluating theuality of surgical interventions suggest assessing theate of deaths within 30 days of admission, rate ofeadmission within 30 days of discharge, rate of emer-ency room visits within 30 days of discharge, rate of
nd satisfaction with use of home care services, andate of avoidable hospital visits and admissions [40].imilarly, American researchers use admission ratesor preventable pediatric conditions such as bronchitis,
trbr

Policy 85 (2008) 263–276

sthma, gastroenteritis to assess the quality of primaryediatric care in one community in Iowa [41]. Anotherpproach to measuring quality is to designate certainypes of care as indicative of quality and measure theirrequency in selected patient populations. For exam-le, European researchers identified 34 indicators ofrocess quality (defined as medical care based on bestcientific evidence) for 11 diagnoses and treatmentsrom review of the literature, such as annual testing foremoglobin A1C in diabetes and use of aspirin in acuteyocardial infarction [42].Measurement of quality in developing countries

s accelerating with the introduction of new healthrograms that are intended for national scale-up.esearchers evaluating the effectiveness of the Inte-rated Management of Childhood Illness (IMCI)onstructed a single aggregate measure of technicaluality of care comprised of prescribing the correctrug, prescribing it correctly (dosage, timing, durationf treatment) and explaining the treatment to the patient43]. The quality was assessed by a “gold-standard sur-eyor”, a researcher trained in the appropriate treatmentf the presenting conditions. More detailed assess-ents of assessment, diagnostic, and triage skills were

lso described in the IMCI evaluation in Tanzania [44].MCI’s evaluation also illustrates the linkages betweenutput indicators such as utilization and quality andhe final health outcome of interest. In the nationalvaluation of the impacts of the IMCI program ineru, researchers evaluated outpatient utilization ofMCI services and vaccine coverage of the popula-ion (output indicators) and nutritional status and child

ortality rates (outcome/impact indicators) [17]. IMCIesearchers in Tanzania compared the child mortal-ty rate in IMCI and non-IMCI districts, finding thathat the indicator was sensitive enough to measure thempact of IMCI even within 2 years of implementation45].

Another aspect of care quality is appropriate refer-al for complicated cases. Appropriate referral requiresecognition of a complication beyond the capacity ofhe initial provider/facility and a functioning transportnd communication system—all components of a wellunctioning health system. A team in Zambia evaluated
he appropriateness of referral for emergency obstet-ic care for pregnant and laboring women in Lusakay examining reasons for referral, timing of refer-al, percentage of Caesarean section, and case fatality

Health

rr

ptttoaocoSEph2

4

htlmavecrsiehbpc

maafimTa[i

ebcTnmicabeeesf[a

4

dnpg[ptotpmcfoeEcio

itre


ates from facility records at both the referring and theeceiving facility [46].

Continuity of care, which can be defined as the com-letion of the full treatment course or continued accesso care for chronic conditions, is an important aspect ofechnical quality. Continuity of care can be measuredhrough the number of repeat visits for a prolongedr chronic condition (e.g., antenatal care in pregnancy,ntiretroviral care for AIDS) or completion of a coursef treatment (e.g., directly observed treatment—short-ourse or DOTS for TB, antimalarials) [47–49]. Safetyf care is another important dimension of quality.afety data tends to be collected through facilities.xamples of safety indicators are infection and com-lication rates of surgery, case fatality rates, rates ofip fractures in facilities, and readmission rates within8 days [19,28].

.2. Equity

Achieving equity in health requires eliminatingealth disparities that are avoidable and unfair such ashose due to inadequate access to services, unhealthyiving or working conditions, or downward social

obility caused by ill health [50]. These disparitiesre often associated with social advantage or disad-antage as manifested through wealth, gender, race, orthnicity and therefore any measurement of equity mustapture these dimensions [51]. Improving equity mayequire a greater commitment to improve the healthtatus of the poor—not necessarily linked to achiev-ng aggregate gains [52]. Indeed, increasingly policyxperts urge governments to adopt explicit pro-poorealth policies and measure their health system successy assessing impact on the poor, rather than the entireopulation, to reverse the regressive nature of healthare delivery in many developing countries [53,54].

Health systems are of course not the only deter-inants of improved health. Adequate and equitable

ccess to education, clean water, food, and sanitation,s well as equality of opportunity and freedomrom persecution, all contribute to reducing healthnequities. However health systems are a powerful

eans for improving health equity within a country.
he two central aspects of equity in health systemsre equity in service delivery and equity in financing55,56]. Thus health services should benefit usersn proportion to need (rather than social status, for
IivN

Policy 85 (2008) 263–276 271

xample) and financing for health services shoulde progressive, requiring proportionately less inontributions from those with lowest incomes [56].his progressivity of financial contributions and theotion that people with greater health needs receiveore care is also called “vertical equity” [57]. Equity

n financing also includes protection for the poor fromatastrophic health expenses. Health equity advocatesdd that public participation and accountability areoth a means to achieving equity and in themselvesnd goals of an equitable system. Thus the sense ofxclusion from the health care system, as commonlyxperienced by marginalized groups, deprives theystem of an important voice in improving access andair financing and is in itself an indicator of inequity7,58]. We will now discuss equity in health status andccess as well as financing and risk protection.

.2.1. Equitable health status and access to careA common approach to measuring equity in service

elivery is to analyze the markers of effective-ess (comprehensiveness, access, quality, continuity,atient satisfaction, etc.) by income quintile, ethnicity,ender, geographic location or other social stratifiers15,59–61]. Daniels et al. [16], for example, pro-ose that countries define locally specific indicatorso measure reductions in geographical maldistributionf services and supplies, elimination of gender, cul-ural and other non-financial barriers to access, and therovision of an appropriate basket of services. Sucheasurement may require the development of specific

ommunity-level surveys and other monitoring tools,or facility-based data will not reflect the experiencef groups not accessing services. For evaluating thequity of the Swiss ambulatory health care system,ggli et al. propose that access, effectiveness, and effi-iency data be analyzed also for subsamples of peoplen poor health—either self-identified through surveysr through disabled insurance rolls [62].

Specific measures of equality in service deliveryncluding utilization measures such as general practi-ioner, specialist, and hospital visits were analyzed withespect to variables on ethnicity, place of residence,ducation, income, and employment in Estonia [63].
n New Zealand, utilization analysis included spend-ng on drugs and laboratory services as well as GPisits, which was regressed against Maori status andew Zealand specific measures of economic depriva-

2 Health

twbigtwadhmfribulau

4

iimassifimtgstBomm

tccmiw2e

tofistppo

4

tTetormptIttgcm[Cwt(tiiUwtiettr


ion [64]. In Pakistan, maternal health service accessas measured with antenatal care visits, institutionalirths, and use of modern contraceptives analyzed byncome quintile [65]. In Malaysia, the proportion ofovernment health expenditures on the poorest quin-ile and outpatient facility use by the same quintileere monitored [66]. Costa Rican researchers evalu-

ted equity of access by using GIS and national censusata to measure distance from the nearest clinic andospital of rural and economically disadvantaged com-unities [67]. This estimate of distance was adjusted

or the type and size of the clinic to give a more accurateepresentation of supply of facilities. GIS is a promis-ng approach to pinpointing areas where geographicarriers to access are highest. In Lao PDR, researcherssed surveys to elicit barriers to care for groups ofow socioeconomic status, identifying staff attitudesnd procedural barriers as important disincentives fortilization [68].

.2.2. Fair financing and risk protectionMeasures of equitable financing were less described

n the literature. The two most common methodologiesnvolved some analysis of the distribution of govern-

ent spending by income group (most often quintile)nd the disincentive effects of user fees on access toervices. National health accounts (NHA), a record ofources and uses of health system funds, are increas-ngly being used to assess progressivity of healthnancing methods (e.g., tax and out-of-pocket pay-ents) and the distribution of government expenditures

o different parts of the country and different socialroups [55,69]. Out-of-pocket payments at the point ofervice, which are commonly levied and pose an impor-ant access barrier for the poor, are critical to estimate.angladeshi families, for example, incurred substantialut-of-pocket costs for travel, hospital admission fees,edicines, tests, food and tips, while accessing “free”aternity services in a public hospital in Dhaka [70].The second component of fair financing is risk pro-

ection, that is protection from financial ruin due toatastrophic health expenses. The WHO [71] definesatastrophic expenses as those that demand 40% orore of the family’s capacity to pay (its effective

ncome after basic subsistence needs have been met),hereas the World Bank [73] suggests a cutoff of0% of all expenditures over a period. Others considerxpenditures catastrophic when they put a family under

o[tc

Policy 85 (2008) 263–276

he country’s poverty line. For poor families in devel-ping countries such catastrophic expenses can resultrom prolonged hospitalization or treatment requir-ng transport to a distant hospital to relatively minorurgery that is accompanied by several levels of under-he-table payments. A commonly used indicator for therevalence of catastrophic spending in a country is theroportion of households that encountered or are at riskf catastrophic expenses in a year [71,72].

.3. Efficiency

Efficiency in health systems refers to extractinghe greatest health gains from a set of inputs [71,73].his embodies the concepts of technical and allocativefficiency as well as the less tangible but no less impor-ant administrative efficiency that includes the valuef workers’ and patients’ time. Allocative efficiencyefers to directing funds to activities that will maxi-ize health gains. Technical efficiency is the highest

ossible sustained output obtained from a combina-ion of resources (e.g., workers, drugs, equipment) [74].ssues of cost and productivity of health inputs are cen-ral to a discussion of efficiency. However, extractinghe most from resources implies that the health bud-ets are fundamentally adequate. In many developingountries, particularly in sub-Saharan Africa, govern-ent health budgets are less than US$ 20 per capita

75]—too low to provide even basic services, which theommission on Macroeconomics and Health estimatedould cost US$ 30–40 per capita [8]. Thus the size of

he budget becomes in itself a determinant of efficiencyand effectiveness and equity). WHO researchers notedhat while health outcomes generally improved withncreasing health care expenditures, the sharpest risen efficiency was at expenditures below approximatelyS$ 80 per capita, suggesting health systems do notork well below this threshold [76]. In addition to

he magnitude of the budget, modes of financing cann themselves influence health system efficiency. Forxample, one study found that Western European coun-ries with national (tax financed) health systems appearo be more efficient at producing lower infant mortalityates than countries with social security systems—in
ther words, produce better outcomes at lower costs77]. The discussion below elaborates the concepts ofechnical/allocative efficiency and administrative effi-iency.

Health

4

hmdrflfcnatacm

dtacobyshRpeccwc

4

psaarwrsrsre

eswosfwfiibio

5

caihgathmTrtsilideqnsf

hip


.3.1. Costs and productivityCosts of service delivery underpin the notion of

ealth system efficiency. Cost per case is a commoneasure that usually includes the drugs, personnel,

iagnostics, hospital beds, surgical supplies and otherecurrent costs for providing a full course of treatmentor a condition. These costs can be allocated to variousevels of the system—for example, national, district,acility, and household [43]. Importantly, out-of-pocketosts (including informal or under-the-table payments)eed to be included in the cost calculation. Costs canlso be expanded to include start up costs such as infras-ructure, training, etc. This is particularly important inssessing the efficiency of interventions in developingountry settings where significant up-front investmentsay be required to scale-up health service delivery.Productivity of health care inputs is another win-

ow on efficiency. In comparing the performance ofwo branches of the Mexican health system, researchersssessed efficiency by evaluating process/output indi-ators such as physician consultations per day, hospitalccupancy rates, and length of stay, as well as the num-er of contacts with the health system per person eachear [26]. Given the large proportion of health budgetspent on hospitals, economies of scale and scope forospitals can be calculated to assess their efficiency.esearch in Vietnam, for example, demonstrated thatrovincial general and specialty hospitals were not asfficient as district hospitals in terms of their costs perase given their number of beds [78]. Similarly theosts per case for maternal health services in Ugandaere higher when performed in hospitals versus health

enters [79].

.3.2. Administrative efficiencyWhile health system efficiency is often defined in

urely economic terms, we would argue that efficientystems also maximize the value of health workersnd the value of the patients’ time. Poorly man-ged and funded systems are marked by frustration,educed performance, and eventually attrition of healthorkers, leading to reduced quality and eventually

educed demand for the services. Some markers ofuch organizational efficiency may be worker attrition
ates, waiting times for appointments, availability ofkilled personnel and drugs when needed, etc. Nigerianesearchers attempted to tap into these dimensions offficiency by auditing providers and managers in gov-
ifgg

Policy 85 (2008) 263–276 273

rnment and private primary health clinics on issuesuch as availability of updated operational plans andork schedules, regularity of team meetings, adequacyf facilities in terms of size and equipment, regularity ofalary payments, availability of written job description,requency of in-service training and overall level oforker motivation [35]. These are proximate markers

or quality of management and overall system function-ng. Poor system functioning will in part be reflectedn outcome indicators such as higher per case costs,ut will also impact other health system outcomesncluding utilization, patient satisfaction, and healthutcomes.

. Conclusion

Measuring health system performance is compli-ated by ongoing debates in the policy communitybout the purpose of health systems, the degree ofmpact of health care (versus other determinants) onealth, and the nature of health care itself (publicood versus market good). Still, most commentatorsgree that a well-performing health system is effec-ive, equitable, and efficient. The literature presentedere suggests the breadth of indicators available foreasuring each of these components of performance.here were important limitations to our analysis. We

estricted our search to literature since 1995 and tohose published in English and thus may have missedome useful earlier experiences with the application ofndicators in the field as well as those published in otheranguages. The scope of our study was limited to report-ng the experience with indicators on the ground—weid not explicitly evaluate the quality or usefulness ofach indicator beyond noting whether it appeared fre-uently in the literature (see Table 1). International andational bodies are actively engaged in studying andelecting indicators that are both good measures andeasible to apply in different settings.

Limitations considered, the framework proposedere, building on developments in health systems think-ng over the past several decades, can be helpful forolicy makers and researchers interested in captur-
ng the key aspects of health system functioning. Theramework used in concert with available data on theround and informed by policy priorities could thenuide the choice of indicators in the field to mea-

2 Health

soma

sdibciatraahsmer

A

atMUPd

R

[

[

[

[

[

[

[

[

[

[

[

[


ure current performance of systems and the effectsf change in policies, funding or organization. A com-on framework could also improve comparability of

ssessments done across different health systems.Future research in this area should focus on under-

tanding how these various indicators can be applied toifferent research and policy questions and to validat-ng the indicators themselves. For instance, it woulde important to elaborate a subset of indicators thatan best capture short term effects of changes in pol-cy or funding without being overly burdensome topply. Linking specific process indicators to difficult-o-measure but crucial impact indicators, such as aeduction in maternal mortality or fair financing, isnother area for further research. Lastly, there are rel-tively few indicators to capture the performance ofealth systems as a social institution. Operationalizinguch notions as trust, accountability, and health workerotivation and, conversely, people’s experience of

xclusion and abuse will be required to document thisole of health systems.

cknowledgments

The authors want to thank Ms. Jagila Minso for herssistance in gathering and organizing the data used inhis review. This work was supported by the Averting

aternal Death and Disability Program at Columbianiversity’s Mailman School of Public Health. Thisrogram is funded by the Bill and Melinda Gates Foun-ation.

eferences

[1] UN Millennium Project. Who’s got the power? Transform-ing health systems for women and children. Task Force onChild Health and Maternal Health. New York: UN MillenniumProject; 2005.

[2] High-level forum on the health MDGs. Summary of discussionand action points. Abuja, Nigeria: WHO/World Bank; 2004.

[3] Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder AA,et al. Overcoming health-systems constraints to achieve theMillennium Development Goals. The Lancet 2004;364:900–6.

[4] WHO. World health report: health systems improving perfor-mance Geneva, Switzerland; 2000.

[5] Committee on Economic Social and Cultural Rights. Generalcomment 14 on the right to health, E/C 12/2000/4. New York:United Nations; 2000.

[

Policy 85 (2008) 263–276

[6] Freedman LP. Achieving the MDGs: health systems as coresocial institutions. Development 2005;48:19–24.

[7] Mackintosh M. Do health care systems contribute to inequal-ities. In: Leon DA, Walt G, editors. Poverty, inequality andhealth: an international perspective. Oxford: Oxford UniversityPress; 2001.

[8] Commission on Macroeconomics and Health. Macroeconomicsand health: investing in health for economic development.Geneva: World Health Organization; 2001.

[9] Bill and Melinda Gates Foundation. University of Washingtonlaunches new institute to evaluate international health pro-grams, Seattle, http://www.gatesfoundation.org/GlobalHealth/Announcements/Announce-070604.htm [accessed on August15, 2007].

10] WHO. Health Metrics Network, Geneva, http://www.who.int/healthmetrics/en/ [accessed on August 15, 2007].

11] United Nations. Millennium Development Goal IndicatorsNew York, http://mdgs.un.org/unsd/mdg/Host.aspx?Content=IAEG.htm [accessed on August 15, 2007].

12] European Commission. Development of a methodology for col-lection and analysis of data on efficiency and effectiveness inhealth care provision. In: 4th Health Systems Working PartyMeeting. 2005.

13] Donabedian A. Methods for deriving criteria for assessing thequality of medical care. Medical Care Review 1980;37:653–98.

14] Institute of Medicine. Crossing the quality chasm: a newhealth system for the 21st century. Washington, DC: NationalAcademy Press; 2001.

15] Global Equity Gauge Alliance. Health equity—research toaction. Cape Town: GEGA; 2004.

16] Daniels N, Flores W, Pannarunothai S, Ndumbe PN, Bryant JH,Ngulube TJ, et al. An evidence-based approach to benchmark-ing the fairness of health-sector reform in developing countries.Bulletin of the World Health Organization 2005;83:534–40.

17] Huicho L, Davila M, Gonzales F, Drasbek C, Bryce J, VictoraCG. Implementation of the integrated management of child-hood illness strategy in Peru and its association with healthindicators: an ecological analysis. Health Policy and Planning2005;20:i32–41.

18] United Nations. Road Map toward the implementation of theUN Millennium Declaration, A/56/326. New York: UnitedNations; 2001.

19] Hurst J, Jee-Hughes M. Performance measurement and perfor-mance management in OECD health systems. OECD labourmarket and social policy occasional papers, No. 47, 2001.doi:10.1787/788224073713.

20] CBC. Canada votes 2006: analysis and commentary.Ottawa, http://www.cbc.ca/canadavotes/analysiscommentary/conservative plans.html [accessed on September 7, 2006].

21] Economist. Not feeling so good. London, http://www.
economist.com/research/backgrounders/displaystory.cfm?story id=616533 [accessed on October 19, 2006].
22] Jennings BM, Heiner SL, Loan LA, Hemman EA, SwansonKM. What really matters to healthcare consumers. Journal ofNursing Administration 2005;35:173–80.

http://www.gatesfoundation.org/GlobalHealth/Announcements/Announce-070604.htm

http://www.gatesfoundation.org/GlobalHealth/Announcements/Announce-070604.htm

http://www.who.int/healthmetrics/en/

http://www.who.int/healthmetrics/en/

http://mdgs.un.org/unsd/mdg/Host.aspx?Content=IAEG.htm

http://mdgs.un.org/unsd/mdg/Host.aspx?Content=IAEG.htm

http://dx.doi.org/10.1787/788224073713

http://www.cbc.ca/canadavotes/analysiscommentary/conservative_plans.html

http://www.cbc.ca/canadavotes/analysiscommentary/conservative_plans.html

http://www.economist.com/research/backgrounders/displaystory.cfm?story_id=616533



Health

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[


23] Sofaer S, Firminger K. Patient perceptions of the qual-ity of health services. Annual Review of Public Health2005;25:513–59.

24] Blendon RJ, Schoen C, DesRoches CM, Osborn R, Scoles KL,Zapert K. Inequities in health care: a five-country survey. HealthAffairs 2002;21:182–91.

25] Donelan K, Blendon RJ, Schoen C, Davis K, Binns K. Thecost of health system chance: public discontent in five nations.Health Affairs 1999;18:206–16.

26] Kroeger A, Hernandez JM. Health services analysis as a toolfor evidence-based policy decisions: the case of the Ministry ofHealth and Social Security in Mexico. Tropical Medicine andInternational Health 2003;8:1157–64.

27] Ali M, Hotta M, Kuroiwa C, Ushijima H. Emergency obstetriccare in Pakistan: potential for reduced maternal mortal-ity through improved basic EmOC facilities, services, andaccess. International Journal of Gynecology & Obstetrics2005;91:105–12.

28] Islam MT, Hossain MM, Islam MA, Haque YA. Improvementof coverage and utilization of EmOC services in southwesternBangladesh. International Journal of Gynecology & Obstetrics2005;91:298–305.

29] UNICEF, WHO, UNFPA. Guidelines for monitoring the avail-ability and use of obstetric services. New York: UNICEF;1997.

30] Borghi J, Hanson K, Acquah C, Ekanmian G, Filippi V, Ron-smans C, et al. Costs of near-miss obstetric complications forwomen and their families in Benin and Ghana. Health Policyand Planning 2003;18:383–90.

31] Anand S, Barnighausen T. Human resources and healthoutcomes: cross-country econometric study. The Lancet2004;364:1603–9.

32] Campbell S, Roland M, Buetow S. Defining quality of care.Social Science and Medicine 2000;51:1611–25.

33] UNDP. Tracking the MDGs—country progress. New York,http://www.undp.org/mdg/tracking countryreports2.shtml[accessed on August 23, 2006].

34] WHO. WHO: The 3 by 5 initiative. Geneva, http://www.who.int/mediacentre/factsheets/2003/fs274/en [accessed onAugust 23, 2006].

35] Chukwuani CM, Olugboji A, Akuto EE, Odebunmi A, Ezeilo E,Ugbene E. A baseline survey of the Primary Healthcare Systemin South Eastern Nigeria. Health Policy 2006;77:182–201.

36] Mercer A, Khan MH, Daulatuzzaman M, Reid J. Effectivenessof an NGO primary health care programme in rural Bangladesh:evidence from the management information system. Health Pol-icy Plan 2004;19:187–98.

37] WHO/UNICEF. Africa Malaria Report 2003. Geneva: WHO;2003.

38] Bruce J. Fundamental elements of the quality of care: a simpleframework. Studies in Family Planning 1990;21:61–91.

39] Beal A, Co JP, Dougherty D, Jorsling T, Kam J, Perrin J,
et al. Quality measures for children’s health care. Pediatrics2004;113:199–209.
40] Brownwell M, Roos NP, Roos LL. Monitoring health reform:a report card approach. Social Science and Medicine 2001;52:657–70.

[

[

Policy 85 (2008) 263–276 275

41] Rohrer J. Measuring health system performance from a commu-nity service perspective: the case of pediatric preventive servicesin Mason City. Clinical Performance and Quality Health Care1995;3:31–4.

42] Gandjour A, Kleinschmit F, Lauterback K, Littman V. Anevidence-based evaluation of quality and efficiency indicators.Quality Management of Health Care 2002;10:41–52.

43] Bryce J, Gouws E, Adam T, Black R, Schellenberg J, Manzi F,et al. Improving quality and efficiency of facility-based childhealth care through Integrated Management of Childhood Ill-ness in Tanzania. Health Policy and Planning 2005;20:i69–76.

44] Tanzania IMCI Multi-country Evaluation Health Facility Sur-vey Study Group. The effect of Integrated Management ofChildhood Illness on observed quality of care of under-fivesin rural Tanzania. Health Policy and Planning 2004;19:1–10.

45] Armstrong Schellenberg JR, Adam T, Mshinda H, Masanja H,Kabadi G, Mukasa O, et al. Effectiveness and cost of facility-based Integrated Management of Childhood Illness (IMCI) inTanzania. Lancet 2004;364:1583–94.

46] Murray S, Davies S, Phiri R, Ahmed Y. Tools for monitoringthe effectiveness of district maternity referral systems. HealthPolicy and Planning 2001;16:353–61.

47] Stop-TB Partnership. The global plan to stop TB: 2006–2015.Geneva: Stop-TB Partnership; 2006.

48] UNAIDS. 2006 Report on the global AIDS epidemic. Geneva:UNAIDS; 2006.

49] WHO. World health report 2005—make every mother and childcount. Geneva: WHO; 2005.

50] Whitehead M. The concepts and principles of equity and health.Copenhagen: WHO; 1990.

51] Braveman P, Gruskin S. Defining equity in health. Journal ofEpidemiology and Community Health 2003;57:254–8.

52] Braveman P. Monitoring equity in health and healthcare: a con-ceptual framework. Journal of Health Population and Nutrition2003;21:181–92.

53] Gwatkin DR, Bhuiya A, Victora C. Making health systems moreequitable. The Lancet 2004;364:1273–80.

54] Victora CG, Wagstaff A, Schellenberg JA, Gwatkin D, Clae-son M, Habicht JP. Applying an equity lens to child healthand mortality: more of the same is not enough. The Lancet2003;362:233–41.

55] Rannan-Eliya R. Affordable healthcare: how has Sri Lankaachieved low mortality rates at such low cost? Sri Lanka: Insti-tute of Policy Studies; 2004.

56] Ensor T, Ronoh J. Effective financing of maternal health ser-vices: a review of the literature. Health Policy 2005;75:49–58.

57] Culyer AJ, Wagstaff A. Equity and equality in health and healthcare. Journal of Health Economics 1993;12:431–57.

58] Loewenson R. Participation and accountability in health sys-tems: the missing factor in equity? Zimbabwe: TARSC; 2000.p. 1–27.

59] Braveman P. Monitoring equity in health and healthcare: a con-ceptual framework. Journal of Health, Population and Nutrition2003;21:181–92.

60] Gwatkin DR. The need for equity-oriented health sectorreforms. International Journal of Epidemiology 2001;30:720–3.

http://www.undp.org/mdg/tracking_countryreports2.shtml

http://www.who.int/mediacentre/factsheets/2003/fs274/en

http://www.who.int/mediacentre/factsheets/2003/fs274/en

2 Health

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[Vietnamese hospitals. Social Science and Medicine 2004;59:


61] Wagstaff A. Socioeconomic inequalities in child mortality:comparisons across nine developing countries. Bulletin ofWorld Health Organization 2000;78:19–29.

62] Eggli Y, Halfon P, Chikhi M, Bandi T. Ambulatory healthcareinformation system: a conceptual framework. Health Policy2006;78:26–38.

63] Habicht J, Kunst AE. Social inequalities in health care servicesutilisation after eight years of health care reforms: a cross-sectional study of Estonia, 1999. Social Science and Medicine2005;60:777–87.

64] Malcolm L. Major inequities between district health boards inreferred services expenditure: a critical challenge facing theprimary health care strategy. New Zealand Medical Journal2002;115:U273.

65] Siddiqi S, Haq IU, Ghaffar A, Akhtar T, Mahaini R. Pakistan’smaternal and child health policy: analysis, lessons and the wayforward. Health Policy 2004;69:117–30.

66] Bin Juni M. Public health care provision: access and equity.Social Science and Medicine 1996;43:759–68.

67] Rosero-Bixby L. Spatial access to health care in Costa Ricaand its equity: a GIS-based study. Social Science and Medicine2004;58:1271–84.

68] Paphassarang C, Philavong K, Boupha B, Blas E. Equity, priva-tization and cost recovery in urban health care: the case of Lao
PDR. Health Policy Plan 2002;17(Suppl.):72–84.
69] Bossert T, Larranaga O, Giedion U, Arbelaez J, Bowser D.Decentralization and equity of resource allocation: evidencefrom Colombia and Chile. Bulletin of the WHO 2003;81:95–100.

[

Policy 85 (2008) 263–276

70] Khan S. Hidden costs in free maternity care result in large out-of-pocket costs for patients in Bangladesh. Cost Effectivenessand Resource Allocation, 2005.

71] WHO. Report on WHO technical consultation on fairness offinancial contribution. Geneva: WHO; 2001.

72] Xu K, Evans DB, Kawabata K, Zeramdini R, Klavus J, MurrayCJ. Household catastrophic health expenditure: a multicountryanalysis. Lancet 2003;362:111–7.

73] Catastrophic health care payments. Washington, DC: WorldBank; 2003.

74] Folland S, Goodman AC, Stano M. The economics of healthand health care. Upper Saddle River, New Jersey: Prentice Hall;1997.

75] WHO. World Health Organization statistical informationsystem: core health indicators, http://www3.who.int/whosis/core/core select process.cfm [accessed on June 20, 2007].

76] Evans DBA, Tandon, Murray CJ, Lauer JA. Comparative effi-ciency of national health systems: cross national econometricanalysis. BMJ 2001;323:307–10.

77] Elola J, Daponet A, Navarro V. Health indicators and the orga-nization of health care systems in western Europe. AmericanJournal of Public Health 1995;85:1397–401.

78] Weaver M, Deolalikar A. Economies of scale and scope in

199.79] Weissman E, Sentumbwe O, Mbonye AK, Kayaga E, Kihuguru

SM, Lissner C. Uganda Safe Motherhood Programme costingstudy. Geneva: WHO; 1998.

http://www3.who.int/whosis/core/core_select_process.cfm

http://www3.who.int/whosis/core/core_select_process.cfm

Assessing health system performance in developing countries: A review of the literature

Documents