1 Caroline van den Berg and Sasha Danilenko Water week, February 28, 2007 Baseline and Target values: What have we learned so far?
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Caroline van den Berg and Sasha Danilenko
Water week,February 28, 2007
Baseline and Target values:
What have we learned so far?
2
Introduction
Guidelines on Results Measurement in Bank projects
To include baseline values for project outcomes by first ISR To determine target values for outcomes/results
What have we learned so far?
3
Baseline values for (intermediate) outcomes
Households have better access to sustainable improved water supply and/or sanitation services (household perspective)
Households have better access to improved WSS services (by income quintiles)Increase in average water consumption by residential hh% hh that have adopted good hygiene behavior (e.g. handwashing) at key times
Service providers are providing WSS services more efficiently (service provider/utility perspective)Reduction in pollution levels (e.g. BOD levels) in water at all monitoring stations downstream (water quality perspective)
4
Key Informant Interviews
Community Interviews Review of
administrative records
One-Time Survey
Impact Evaluation (Experimental Design)
CensusRigor/Difficulty
(Cost/Time/skills)
Focus GroupInterviews
Formal MethodsRapid Appraisal Methods
Direct Observation
Validity Reliability Credibility
Choice depends
1. Issue to be examined
2. Quality of information needed
3. Timeframe in which information is needed
4. Costs
Panel Surveys
5
Some thoughts on baseline data
Use of baseline data is directly linked to project design
What to do?Where to intervene?
Retrofitting of baseline values is a complex matter at best“Think outside of the box”
There is often more data available than you think:It is most likely not available in the MOW
6
Target values
Baseline indicator +
desired level of improvement
= target performance
0
10
20
30
40
50
60
70
80
90
100
Yr0 Yr1 Yr2 Yr3 Yr4 Yr5 Yr6
7
When setting target valuesBalancing between easy to achieve and impossible to realize: reality check on indicatorsKey factors influencing setting target values for basic indicators
Expected funding and resource levels – existing organizational capacity, personnel, funding, facilitiesPast performance to determine the realism of targetExternal environment – the economic and political environmentThe relationships between the different indicatorsThe type of indicators selected
8
Illustration of target values
Looking at indicators, baseline values and how to set target values Focus on
improving the efficiency which utilities provide WSS servicesImproving quality of service
Data from International Benchmarking Network (IBNET) at www.ib-net.org
9
IBNET website
www.ib-net.org
10
Illustration 1: past performance
Source: IBNET
Moldova: staff ratio by 1,000 people servedcompared over time and with region
0
2
4
6
8
10
12
Quartile 1(best) Quartile 2 Quartile 3 Quartile 4 (w orst)
1999 2004 ECA low -income 2004
11
Illustration 2: political context
Rezina, Moldova: hours of supply per day
0
4
8
12
16
20
24
1997 1998 1999 2000 2001 2002 2003 2004 2005
Source: IBNET
12
Illustration 3A: the impact of reform
Impact of reform in Moldova: consumption and metering
0
50
100
150
200
250
300
1996 2001 2004 2005
lcd
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
met
erin
g ra
tio
3_1_TotalWaterProdLCD 4_1_TotalWaterConLCD 7_1_Metering
Source: IBNET
13
Illustration 3B: the impact of reform
Moldova: consumption and price
0
50
100
150
200
250
300
1996 2001 2004 2005
lcd
0.00
0.20
0.40
0.60
0.80
1.00
1.20
US$/
m3
Total water production Total water consumption Billed tariff (US$/m3)
Source: IBNET
14
Illustration 3C: the impact of reform
Moldova: Staff ratio per 1,000 population
2.40
2.50
2.60
2.70
2.80
2.90
3.00
1997 1998 1999 2000 2001 2002 2003 2004 2005
Staf
f rat
io
Source: IBNET
15
Illustration 3D: the impact of reform
Moldova - Operating Cost Coverage Ratio
-0.20
0.400.600.801.00
1.201.401.601.80
2.002.20
1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
Source: IBNET
16
Illustration 3E: the impact of reform
Moldova: subsidies to consumers through underpricing of services (in US$ million) and the
operating cost coverage ratio
0
5
10
15
20
25
30
2000 2001 2002 2003 2004 2005
US$
mill
ion
0.000.200.400.600.801.001.201.401.60
Ope
ratin
g co
st
cove
rage
ratio
Subsidies Operating cost coverage ratio
Source: IBNET
17
Illustration 4A: indicator selection
Brazil: operating cost coverage ratio
-0.20
0.400.600.80
1.001.20
1.401.601.80
2.002.20
1996 1997 1998 1999 2000 2001 2002 2003 2004
Source: IBNET
18
Illustration 4B: indicator selection
Moldova: non-revenue water
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
NR
W in
%
0
10
20
30
40
50
60
70
NR
W in
m3/
km/d
ay
6,1 Non Revenue Water, % 6,2 Non Revenue Water, m3/km/ day
Source: IBNET
19
Some thoughts on target values
Setting targets without baseline data is difficult at bestThe indicators that are selected will be affected by the different interactions that take placeThe choice of indicators will affect the likelihood of achieving targets
It is more difficult to predict how composite indices are likely to behaveVolatility in indices can be smoothened out with e.g. moving averages
20
Some thoughts on target values
Target values are also be affected by events that are not directly linked by the project
External factors can have a big impact in the short run,And in the long run
21
Conclusions
Setting baseline and target values is an integral part of project design
How to design a project without baseline values? Retrofitting is not the way to go
There are many different options to measureDifferent indicators can measure similar-like project outcomesUnderstanding of past performance is a pre-requisite for understanding how to set targets
Related Documents
