IEG Global Partnership Programs: Addressing the Challenge of Evaluation Christopher Gerrard Global Programs Coordinator, IEG-World Bank March 31, 2006.

IEG

Global Partnership Programs:Addressing the Challenge of Evaluation

Christopher Gerrard

Global Programs Coordinator, IEG-World Bank

March 31, 2006

www.worldbank.org/ieg/gppp/

IEGGlobal Partnership Programs – An Important & Growing Line of Business• What are Global Partnership Programs?

• Why are they growing?

• How important have they become?

– The World Bank is involved in about 110 global programs and 40 regional programs, which together spent around $3 billion in FY05

• What do we know about them?

– UNDP, Office of Development Studies (1999, 2003, 2006)

– UN Vision Project on Global Public Policy Networks (2000)

– International Task Force on Global Public Goods (2005)

– IEG Review of the Bank’s Involvement in Global Programs (2002, 2003, 2004)

IEGBilateral Donors Are the Principal Partners at the Governance Level

0 10 20 30 40 50 60 70

Number of Programs (Global & Regional)

Canada

U.K.

U.S.A.

Netherlands

France

Norway

EU

Switzerland

Japan

Sweden

Germany

Denmark

Italy

Australia

Donors participating in more than 15 programs

IEGAre Global Programs an Efficient Use of Scarce ODA?• “Letting a thousand flowers bloom” in the 1990s

provided opportunities to learn about using global programs to address the challenges of globalization

• But selectivity and oversight have been weak

• Most have been donor-driven

• Most are advocacy/technical assistance programs supporting national public goods – although global public goods programs still command the major share of expenditures

• Global-country linkages have been weak – and incentives to foster such linkages also weak

• The bottom line: Undermanaged partnerships pose significant reputational risks for program partners

IEGThe Number of Global Program Evaluations Is Increasing• There is also growing acceptance that evaluation and

audit should be viewed as functions of the programs’ governing bodies

• But there is no agreed-upon methodology – either substance or process

• And so different donors, including the Bank, are acting on their own:

– The Bank’s Development Grant Facility requires evaluations every 3-5 years

– IEG is now including global programs in its regular evaluation and reporting practices

– The Bank’s Quality Assurance Group (QAG) has initiated quality-at-entry reviews

IEGMany Evaluation Challenges Remain

• Programs vary in size, scope, duration, objectives, activities, complexity, and number of partners

• How to apply results-based evaluation methodologies to advocacy and technical assistance, as opposed to investment programs?

• How to aggregate country and global-level activities?

• What do relevance, efficacy, and efficiency mean in the context of global programs?

• What to do when programs do not have adequate internal monitoring processes to assess results?

• How to assess the quality and effectiveness of governance and management?

IEGA Common Set of Evaluation Principles Would Be Desirable• Similar to the OECD Principles of Corporate

Governance

• Governments are demanding accountability – if we don’t do this together, we will continue to do it separately

• The lack of agreed-upon principles – both substance and process – is hampering efforts to assess relevance, efficacy, and efficiency of programs

• And leading to over-evaluation and duplication in some cases

• Why should the Bank – by default – define what constitutes generally accepted principles for the evaluation of global programs?

IEGA Proposal

• To establish a working group, under the auspices of the DAC Evaluation Network

• To develop generally accepted principles for the evaluation of global programs – both substance and process

• A suggested membership, drawn from:

– OECD members (including the OECD secretariat)

– UN organizations

– MDBs (including World Bank)

– Foundations

– Developing countries

– Global programs (management)

IEG

Background Slides

IEGWhy are global programs growing?

• Growing awareness of the need for collective action to provide global public goods – R&D on food crops and diseases of the poor

– Mitigating & adapting to global climate change

– Mitigating the spread of communicable diseases

• But also:

– Dissatisfaction with traditional aid mechanisms

– New actors, advocates, and constituencies

– New ICT technologies for facilitating networks

IEGOther major reviews of global partnerships• UNDP, Office of Development Studies: Global

Public Goods (1999), Providing Global Public Goods (2003), and The New Public Finance (2006)

• U.N. Vision Project on Global Public Policy Networks (2000)

• IDS (Sussex) and Swedish Ministry for Foreign Affairs, The Future of Development Financing (2004)

• International Task Force on Global Public Goods (2005)

IEGAn important and growing line of business for the Bank

• The World Bank is involved in about 110 global programs and 40 regional programs, which together spent around $3 billion in FY05

• About $1.9 billion from Bank-administered trust funds

• About $140 million from the Development Grant Facility (DGF) and $20 million from the Bank’s administrative budget

• Global programs also provide supplementary resources of about $100 million a year to the Bank’s administrative budget

IEGInternational organizations contribute legitimacy and expertiseInternational orgs. participating in more than 6 programs

0 10 20 30 40 50

Number of Programs (Global & Regional)

UNDP

WHO

OECD

FAO

UNEP

UNICEF

IFAD

UNESCO

ILO

UNCTAD

UNFPA

UN-Habitat

IEGGates Foundation is bringing a corporate perspective to evaluation

Foundations participating in more than 3 programs

0 2 4 6 8 10 12 14

Number of Programs (Global & Regional)

Gates

Rockefeller

Ford

ConservationInternational

McArthur

IEGThere are a few large and many small programs

0%

20%

40%

60%

80%

100%

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75

Number of Programs (Global Only)

Cumulative Share of Total Expenditures

GFATM

GEF

CGIAR

IEGMost programs are advocacy/know-ledge/technical assistance programs

0 10 20 30 40 50 60 70 80 90

Financing country-levelinvestments to delivernational public goods

Financing country-levelinvestments to deliverglobal public goods

Financing R&Dfor new products

& technology

Implementing internationalconventions, rules,standards, & norms

Donor coordination& priority setting

Globaladvocacy

Supporting national-levelpolicy, institutional reforms

& capacity-building

Knowledge sharing& dissemination

Number of Programs (Global and Regional)

Networking andTechnical Assistance

FinancingInvestments

IEGEnvironment and Health have largest number of programs

0 5 10 15 20 25 30 35 40

Number of Programs (Global Only)

ESSD

HDN

INF

WBI

PREM

DEC

FSE

In-House SecretariatExternal SecretariatIndependent Legal EntityOther

Vice Presidency

IEGWhat is the World Bank Doing?

Evaluation

Selectivity & Oversight

Governance, Management& Financing

Strategy

IEGStrengthening selectivity & oversight

• Bank established a central unit in 2004 – the GPP Group in the CFP Vice-Presidency – which is:

– Putting in place new business processes for global and regional programs.

– Refining and overseeing the application of Bank-wide criteria for selectivity and oversight

– Assisting thematic Networks and geographical Regions in program design, financing, and progress reporting

– Clarifying roles, responsibilities, and accountabilities of Bank staff serving on global program boards

IEGFocusing on key strategic issues

• Proactively managing the proliferation of theme-driven advocacy/knowledge programs

• Forging better linkages between global programs and developing country priorities

• Seeking new sources of financing for GPG programs – beyond ODA allocations

IEGWorking with partners to improve governance and management

• To strengthen shared accountability for results at the program level

• To enhance effective representation of developing countries on global program boards

• To develop and implement guidelines on conflicts of interest

• To view evaluation and audit as functions of the governing body, not program management