The World Bank’s Involvement in GRPPs: The Current State of GRPP Evaluation Practice (2011 Evaluation Week)

The World Bank’s Involvement in GRPPs: The Current State of GRPP

Evaluation Practice

Chris GerrardEvaluation WeekOctober 24, 2011

What are GRPPs?

► Programmatic partnerships in which:• The partners dedicate resources (financial,

technical, staff, reputational, etc.) toward achieving agreed objectives over time.

• The activities of the program are global, regional, or multi-country (not single country) in scope.

• The partners establish a new organization with shared governance and a management unit to deliver these activities.

► 40 percent are located in the Bank, 35 percent in other international/partner organizations, and 25 percent are free-standing independent legal entities.

The Bank Plays Multiple Roles . . .

. . . But Is Not a Major Source of Funding

Relationship Between GRPPs, Trust Funds, and the DGF

Bank-AdministeredTrust Funds

DevelopmentGrant Facility

Global and RegionalPartnership Programs(shared governance)

What Are The Programs Doing?

Principal Activities

Location of the Secretariat

TotalIn the

World BankIn Another

PartnerIndependent Legal Entity

Knowledge Networks 11 31 11 53

Technical Assistance For NPGs 20 4 6 30 For GPGs/RPGs 1 1 2

Financing Country-Level Investments

For NPGs 3 3 For GPGs/RPGs 13 3 6 22Financing Global Investments for GPGs 1 2 4 7

Total 48 41 28 117

How Much Money Are They Spending?(Millions of U.S. Dollars)

Principal Activities

Location of Secretariat

TotalIn the

World BankIn Another

PartnerIndependent Legal Entity

Knowledge Networks 52 68 52 171Technical Assistance

For NPGs 184 3 94 282 For GPGs/RPGs 14 241 255Financing Country-Level Investments For NPGs 291 218 For GPGs/RPGs 970 124 4,045 5,139Financing Global Investments for GPGs 580 66 180 826

Total 2,077 274 4,612 6,963

GRPPs Are Concentrated in a Few Sectors . . .

. . . And a Few Vice Presidencies

GRPPs Challenge the Bank’s Country-Based Model

► They challenge the Bank’s traditional financial accoun-tability mechanisms. Their legal and governance arrange-ments do not always confer clarity on how collective responsibility for results is supposed to work in practice.

► Bank is dedicating more and more senior management time to the governance of these programs.

► Bank expects programs’ activities to be linked to the Bank’s country operations. But the way in which GRPPs plan their activities is different from the way in which country teams plan theirs.

GRPPs Present Challenges for Evaluation

► Programs have shared governance and accountability for results – to achieve something collectively that individual partners could not achieve by acting alone.

► Programs evolve over time – do not usually have a fixed end-point.

► Programs operate at multiple levels – global, regional, national, and local.

► Programs are diverse in size, age, sectoral focus, objectives, and activities (knowledge, technical assistance, investments).

There Is a Clear Trend Toward Stakeholder Models of Governance

No. of programs

Inter-national/ regional organi-zations

Donor countries

Private foun-

dations

Low- and middle- income

countries

Civil society organi-zations

Commer-cial

private sector

Share-holder models

24 24(WB 24) 17 5 2 1

Stake-holder models

90 72(WB 60) 62 22 55 49 27

Total 114 96(WB 84) 79 27 55 51 29

Most Programs Have Been Evaluated, But Transparency & Accessibility Have Been Weak

► Of the 93 programs that are five years older or more, 77 have had evaluations.

► However, only 30 evaluation reports have been posted on the programs’ external Web sites and only 12 programs have posted a “management response” to the evaluation.

► By way of comparison, 57 programs have posted their charters on their external Web sites.

Independence, Quality and Impacts – Based on 20 GRPPs Reviewed since 2006

Principal Activities

Location of the SecretariatIn the

World BankIn Another Partner

OrganizationIndependent Legal

EntityKnowledge, advocacy & standard-setting networks

CGAPIAASTDMAPS

ADEA (AfDB)GISP (CABI-Nairobi)ILC (IFAD)PARIS21 (OECD)ProVention (IFRC)

GDN (New Delhi)GFHR (Geneva)GWP (Stockholm)

Providing country-level technical assistance

Cities AllianceMDTF-EITIPRHCBPTFSCB

Financing investments (global or country-level)

CEPF (CI)Stop TB (WHO)

Development Gateway (Washington, DC)Global Fund (Geneva)MMV (Geneva)

Evaluation Independence

► 14 of 20 evaluations were effectively independent throughout the evaluation process:• Organizationally and behaviorally independent of

program management• Protected from outside interference• Avoided conflicts of interests

► Independence of 2 evaluations was compromised during one stage of the evaluation process: commissioning, team selection, oversight, or review

► Independence of 4 evaluations was compromised at more than one stage

Evaluation Quality

► The quality of the evaluations was satisfactory in 8 of 20 cases in terms of evaluation scope, design, methods, and feedback

► The evaluations had moderate shortcomings in 9 cases, and significant shortcomings in 3 cases

► Most common issues adversely affecting quality were:• An unclear terms of reference• Insufficient budget and time• Weak M&E frameworks for the program• Lax evaluation methodology and tools

Evaluation Impacts on the Programs

► Evaluations have had notable impacts on the programs.

► Most common impacts have been:• Half the programs modified their

governance arrangements in some way, such as establishing an executive committee.

• Almost half the programs revised their strategies.

• About one-quarter improved their M&E systems.

Thank You

www.globalevalutions.org