UNC Water and Health Conference 2011: Economic modeling, Professor Marc Jeuland, Duke University

It’s safe to assume: The limitations and opportunities of economic modeling

Marc JeulandSanford School of Public Policy & Duke Global

Health InstituteOctober 3 2011

It’s safe to assume: The limitations and opportunities of economic modeling

Marc JeulandSanford School of Public Policy & Duke Global

Health InstituteOctober 3 2011

NotV

The Limitations and Opportunities of Economic Modelling 3

Reflections on “evidence based” decision-making

Several key points to keep in mind1. The world is full of heterogeneity: Static

problem2. We will almost never know enough to predict

(economic) outcomes reliably: Uncertainty problem

3. Economic (and thus behavioral) outcomes from interventions may be unstable over time, or visa versa: Dynamic problem

10/03/2011


The world is full of heterogeneity: Static problem

• This is the main insight I have reached based on economic modeling

• In retrospect, it is an obvious point: Historic failure of supply-driven models (“White

Elephants”) Evidence from the field is mixed on effectiveness, despite

all of the meta-analyses that say 30-40% Economists know that preferences vary enormously

(problem of unobserved heterogeneity, which is non-random)

10/03/2011


How did I reach this obvious conclusion

• Modeling of various types of preventive health interventions in the sector Started with water/sanitation interventions and

cholera vaccines Extended framework to other kinds of

interventions

• And of course working in the field, interviewing and listening to households

10/03/2011


Economic Modeling Framework

• Construct model of costs and benefits• Populate model with parameters (based on

local data or, in this context, from global evidence base) Ranges to reflect uncertainty Correlations to reflect the fact that things do not

always vary independently (admittedly more art than science)

• Simulate the living daylights out of the model

10/03/2011


Example: A general typology of costs & benefits

10/03/2011

Costs Examples Benefits Examples

Capital (“hardware”)

Cost of physical investments

Morbidity & mortality

Benefits from reduced incidence of and mortality from disease

Program (“software”)

Cost of implementation: Marketing & promotion; NGO/government staff time

Time savings Benefits of reduced time spent collecting water

O&MCost of replacing / cleaning of equipment, including time

Aesthetic gainsBenefits from use of additional water that are not health-related; improved cleanliness

Learning Familiarization costs Improved social standing

Benefits of improvements in household status

Inconvenience Costs related to undesi- red behavior change Environmental Benefits from reduced

environmental contamination


Example: Model equations

10/03/2011

Source: Whittington et al. (2008). “The Challenge of Improving Water and Sanitation Services in Less Developed Countries.”


Example: Model parameterization

10/03/2011

Parameter Value [Min-max range] Sources (if applicable)Household size 5 [4 – 6] N/ANumber of adults per hh 2 [1 – 3] N/AMarket wage (US$/day) 1.25 [0.5 – 2] N/AValue of time/market wage (%) 30% [10 – 50] (Jeuland, Lucas et al. 2010)Diarrheal disease incidence (cases/person-yr) 0.9 [0.4 – 1.4] (Whittington et al. 2009)Diarrheal disease case fatality rate (%) 0.08 [0.04 – 0.12] (Whittington, Hanemann et al. 2009)Diarrheal disease cost of illness (US$/case) 6 [2 – 10] (Whittington, Hanemann et al. 2009)Malaria incidence (cases/person-yr) 0.3 [0.02 – 0.6] (Snow et al. 1999; Mueller, Wiseman et al. 2008) Malaria case fatality rate (%) 0.2 [0.05 – 0.35] (Snow, Craig et al. 1999; Mueller, Wiseman et al. 2008) Malaria cost of illness (US$/case) 26 [12 – 40] (Russell 2004)Cholera incidence (cases/1000 persons-yr) 2 [0.1 – 3.9] (Jeuland et al. 2009)Cholera case fatality rate (%) 1.75 [0.5 – 3.0] (Jeuland and Whittington 2009)Cholera cost of illness (US$/case) 50 [15 – 85] (Jeuland and Whittington 2009)Value of a statistical life (US$) 30,000 [10,000 – 50,000] (Whittington, Hanemann et al. 2009)Discount rate (%) 4.5 [3 – 6] N/A


Example: Model parameterization (cont.)

10/03/2011

Parameter Handwashing Total sanitation

Chlorination Biosand filters Cholera vaccination

Benefit parametersReduction in diarrhea cases (%) 45 [25 – 65] 30 [10 – 50] 37.5[25 – 50] 40 [20 – 60]

Reduction in malaria cases (%) - - - - -

% of aesthetic benefits that are health-related 25 [0 – 50] - - - -

Round trip travel time to defecation site (min) - 15 [10 – 20] - - -

Number of trips to defecation site per day - 1 [0.8 – 1.3] - - -

Cost parameters

Capital cost ($) 3.5 [2 – 5] 20 [10 – 30] 8.5 [5 – 12] 75 [60 – 90] a 3 [1.4 – 6.6]

Transportation/distribution cost ($) - - - 25 [15 – 35] a Included above

Program software cost (% of upfront expenses) Same as CLTS 30 [20 – 40] Same as CLTS - -

Initial time expense: uptakers (hours) 40 [20 – 60] 10 [5 – 15] 1 [0.5 – 1.5] 8 [4 – 12] a 1.5 [0.5 – 2.5]

Initial time expense: nonuptakers (hours) 10 [5 – 15] 3 [2 – 4] N/A - -

Operation and maintenance cost ($/yr) 3 [2 – 4] 5 [2 – 8] 4.4 [3.2 – 5.6] - -

Water collection time (hr/20L jerrican) 1 [0.1 – 2] - - - -

Water needed for washing (L/person-day) 0.8 [0.25 – 1.4] - - - -

Number of filter washes/yr - - - 6 [2 – 10] a -

Ongoing community time expenses (hr/hh-yr) - - - 2 [1 – 3] a -

Ongoing household time expenses (hr/hh) - Per year:10 [5 – 15]

- Per wash:0.25 [0.2 -0.3] a

-

Time out of operation after maintenance (days) - - - 5 [3 – 7] a -

Project lifespan (yr) 1.5 [1 – 2] 3 [2 – 4] 2 [1 – 3] 8 [6 – 10] 3 [2 – 4]


Example: Model outcomes (on average)

10/03/2011

$-

$0.50

$1.00

$1.50

$2.00

$2.50

$3.00

Cost

s and

Ben

efits

(U

S$/h

h-m

onth

)

Benefits

Costs

Medium Uptake and Usage

Source: Whittington et al. (2011). “Setting Priorities, Targeting Subsidies among Water, Sanitation, and Preventative Health Interventions in Developing Countries.”


Example: Model outcomes (variability)

10/03/2011

Source: Whittington et al. (2011). “Setting Priorities, Targeting Subsidies among Water, Sanitation, and Preventative Health Interventions in Developing Countries.”


This is by no means limited to WASH interventions

10/03/2011

Source: Jeuland and Pattanayak (2011). “Benefits and costs of improved cookstoves.”


We will almost never know enough to predict outcomes reliably: Uncertainty problem

• “Gold standard” evaluations are discipline specific or reduced form Discipline-specific: Perhaps understand disease

epidemiology really well (incidence and case fatality rates)

Reduced form: Measure outcomes really well, but don’t know how we got there

• Preventive health interventions are inherently “messy”

10/03/2011


We will almost never know enough to predict outcomes reliably: Uncertainty problem

10/03/2011

Example from a study I am participating in on “scaling up” improved cook stoves


Economic outcomes from interventions may be unstable over time: Dynamic problem

• Suppose you think we can precisely know the values of all the many different parameters that drive outcomes (which I don’t believe we can)– Disease epidemiology– Technological characteristics– Preferences for improvements

• Are these stable over time?

10/03/2011


Common sector narrative (Not unique to WASH)

10/03/2011

Economic development

Health gains

Donor or government investment


Common sector narrative (Not unique to WASH)

10/03/2011

Economic development

Health gains

Donor or government investment

One may have doubts about this narrative.However, if any of these linkages are true, dynamic changes will occur


A call for humility…

10/03/2011

“We should be very tentative about how we understand the world. That doesn’t mean you don’t do things. You’ve got to do things, but you’ve got to recognize that you may be wrong. We don’t know enough. And so it is terribly important to recognize that you can be wrong, and to be, therefore, very susceptible to modifying the theories you hold in light of new evidence.”

- Douglas North (1993 Nobel Laureate in Economics)


Potential directions1. Wat/san researchers can make a distinct

contribution in the production of knowledge about processes

2. We are unlikely to get policy on specific preventive health interventions “right” at the high level; local decision-makers will better respond to local realities

3. Local decision makers however can benefit from increased capacity for systematic thinking about the value of interventions

4. Externalities may justify resource transfers, but these should probably not be “paternalistic”

10/03/2011


Theory AND evidence:Provision of public goods

1. Tiebout model: When preferences vary, public goods are best provided at the local level

People can “vote” for these locally (participatory or demand-driven development)

Incentives maintained by the threat of moving out (which may include seasonal migration)

2. Model makes some restrictive assumptions, but there is empirical support for it in the developed world

3. However, relevance to developing world may be limited, particularly in rural areas

10/03/2011


Thanks!

10/03/2011

UNC Water and Health Conference 2011: Economic modeling, Professor Marc Jeuland, Duke University

Education

economic outcomes

wash1003201117the limitations

wash1003201118the limitations

messy1003201114the limitations

model100320116the limitations

uncertainty problem

economic modelingin

model outcomes variability