Sub-National Monitoring and Impact Evaluation
Kai KaiserPREM Public Sector & Governance
DPWG 6th Annual MeetingBrusselsMay 2011
The World Bank Page 2Public Sector & Governance
Motivation
• With Decentralization, Sub-National Governments Increasingly Responsible/Pivotal for Sectoral Outcomes
• Higher levels of government provide bulk of SN finance– Own source mobilization important for revenue
mobilization & accountability effects• Concerns about SN Outcomes
– Monitoring– Influencing
The World Bank Page 3Public Sector & Governance
Decentralization – A Many Splendored Thing
• Devolution, Delegation, Deconcentration• Fiscal, Administrative, Political Decentralization• Different Motivations & Drivers
– Decentralization Theorem/Subsidiarity– Nation Building (Glue versus Solvent?)– Political Economy
• Design and implementation of system• Accountability for performance
The World Bank Page 4Public Sector & Governance
Promoting Results– Diagnostic Frameworks & Instruments
• Recognizing Political Economy of Decentralization• Service Delivery Accountability• Results Chains
– SNGs & Public Expenditure Tracking– Tensions
• Country Donor Motivations & Drivers• Donor Donor Motivations & Drivers• Vertical Results “versus” Horizontal-Downward
Accountability Institution Building– Illustration: Intergovernmental Performance Grants– Selected Resources
The World Bank Page 5Public Sector & Governance
A Results Framework
Decentralization Framework
Internal use of Performance Information
Sub-national Jurisdiction Sub-national Government
Inputs Process Outcomes Outputs
Central Government
Central Government Transfers
Sub-national Government Own Resources
Political Support
Sub-national Government Institutional Capacity
Resource Management Perspective
Ss. Delivery and Community Development Perspective
Measures of Effectiveness
Downward Accountability
Upward Accountability
Measures of Efficiency
Measures of Process
Performance Monitoring
System
E x t e r n a l
u s e
o f
P e r f o r m a n c e I n f o r m a t i o n
Citizens Voice & Democratic Participation
Unbundling Decentralization Dimensions
Policy Making Who sets the main policy guidelines for a service (e.g., free primary education as a national policy)?
Standards Setting Who sets the standards (e.g., national tests versus local building standards)?
Administration/contracting Who is the primary government authority that administers services on a day to day basis?
Financing Who provides the financing for services? Capital investments, wages, O&M?
Service deliveryWho actually produces/delivers the service (e.g., this can often
be private)?
Regulation Who regulates a service (e.g., professional accreditation in health)?
Monitoring & Evaluation Who monitors and evaluate a service (e.g., how can local citizens provide feedback)?
The World Bank Page 7Public Sector & Governance
What Gets Measured Gets Done?
Central Government(Cross-Cutting/
Sectoral Agencies, Intermediate)
….. (Average)
SNGi
500++
Heterogeneity Ranking (e.g., systems, human, fiscal capacity, baseline performance )
SNGi#1
…..
Observed Performance
Indicators
Observed Performance
Indicators
Observed Performance
Indicators
Observed Performance
Indicators
Observed Performance
Indicators
The World Bank Page 8Public Sector & Governance
Indonesia Decentralization Results?-1
00
01
00
200
300
ch
an
ge
in pa
ved
ro
ad
%
-100 0 100 200 300 400 500change in real Infrastructure expenditure %
Fitted values ch_paved
-5
00
50
100
150
ch
an
ge
in
acce
ss to cle
an
w
ater %
-100 0 100 200 300 400 500change in real Infrastructure expenditure %
Fitted values ch_clean_wtr
The World Bank Page 9Public Sector & Governance
Bosnia-HerzegovinaMunicipal PFM and Service Delivery Benchmarking
0
5
10
15
20
25
30
BE
RK
OV
ICI
ST
OLA
C
MO
ST
AR
KA
LES
IJA
IST
OC
NI
ST
AR
I
BU
ZIM
VIS
EG
RA
D
ZV
OR
NIK
RIB
NIK
BR
EZ
A
JAB
LAN
ICA
SIR
OK
IB
RIJ
EG
LJU
BIN
JE
OD
ZA
K
DO
BO
J-JU
G
SA
MA
C
NO
VO
SA
RA
JEV
O
TU
ZLA
BA
NJA
LUK
A
ILID
ZA
02468101214161820
Service Outcome Score PFM Score (28+3 indicators)
Citizen Satisfaction Survey & PFM PEFA Type Benchmarking in 20 Pilot Municipalities (of 142) in 2007Source: From Stability to Performance: Local Governance and Service Delivery in Bosnia and Herzegovina (draft report)
Having the “ideal” counterfactual……
Y1 (observedl)
Y1
* (counterfactual)
Y0
t=0 t=1 time
Intervention
10
allows us to estimate the true impact
Y1
Impact = Y1- Y1*
Y1
*
Y0
t=0 t=1 time
11
Source: Skoufias
The World Bank Page 12Public Sector & Governance
Strategies in Practice
• Pure Randomization Challenging– Political Economy of Decentralization
• Matching, Difference in Difference, Discontinuity• Layering Interventions• Natural Experiments• Commit of Baseline Platforms
– Across SNG Engagements– Across Time
The World Bank Page 13Public Sector & Governance
• Can higher levels of government leverage specific transfers to sub-national governments to enhance local “performance” and “results”?
• What link of “results” chain to target?– PFM Process versus Outcome?– Compliance versus “Stretch” Reform
• Instruments & Interventions– Prevailing country flows & systems– WBG instruments/policies (e.g., new P4R OPs)
Application: Performance Grants
The World Bank Page 14Public Sector & Governance
Selected References
Bird, Richard, (2010), Taxation and Decentralization, Economic Premise Note, No. 38 Eaton, Kent, Kaiser, Kai, and Smoke, Paul (2010), 'The Political-Economy of Decentralization Reforms: A Development Partner Perspective', Washington, DC, pp. 114 Dumas, Victor and Kai Kaiser, (2010), Sub-National Performance Monitoring Issues and Options for Higher Levels of Government, Washington, DC: Public Sector Group, June pp. 51 World Bank, (2004), World Development Report: Making Services Work for Poor People, Washington, DC, pp. 271 World Bank, (2009), Implementing Public Expenditure Tracking Surveys for Results: Lessons from a Decade of Global Experience, Washington, DC, pp. 16
Q & A