Evaluation Office EVALUATION THE INDEPENDENT EVALUATION OFFICE OF THE UNITED NATIONS DEVELOPMENT PROGRAMME REFLECTIONS ON CONTRIBUTION TO TRANSPARENCY,

Evaluation Office

EV

ALU

ATI

ON

THE INDEPENDENT EVALUATION OFFICE OF THE UNITED NATIONS DEVELOPMENT PROGRAMME REFLECTIONS ON CONTRIBUTION TO TRANSPARENCY, LEARNING AND ACCOUNTABILITY

June 2013Indran A. NaidooDirector

2

Demand from Members states and management. The Evaluation Policy of the UNDP clarifies roles and responsibilities for

evaluation The Evaluation Office (EO) of the UNDP, the largest in the UN system, produces

independent evaluations at the corporate, programme and country level. It extends its influence by managing the UNEG Secretariat, producing guidance

and standards, engaging with networks and supporting evaluation capacity building across the globe

Setting the stage

Evaluation Office3

1. Mandate and functions

Reports directly to the Executive Board of the UNDP

Supports the Administrator in her substantive

accountability function

Operates within the UNEG Norms and Standards and

ethical guidelines, and engages stakeholders in the

conduct of evaluations to ensure transparency, learning

and accountability

1999EO was

established

2004GA 59/250 Resolutions

for UN System

2005UNEG

Norms & Standards

2006 UNDP

Evaluation Policy and

Independence

2011 UNDP

Evaluation Policy (revised)

4

Evaluations are planned and answer questions within the Strategic Plan - has been RELEVANCE, EFFECTIVENESS, EFFICIENCY and SUSTAINABILITY

DO UNDP INTERVENTIONS MAKE A CHANGE?

EO staff lead all evaluations, drawing on advisory panels and using engagement processes with evaluands and stakeholders as a means to enhance credibility.

As an independent office, the Director signs off on all evaluations.

In practice this means…

Evaluation Office5

2. Contributions to other organisational tier support and professionalasation)

• Setting standards; Guides; Assessment of evaluation quality Setting standards; Guides; Assessment of evaluation quality

UNDP: Support to decentralized evaluation function and products

• Norms and standards; Guidelines; Methodology; Peer Reviews; UNEG Secretariat. (www.unevaluation.org)

United Nations Evaluation Group: Coherence in UN system-

wide evaluation

• Regional: AfrEA, RELAC, Malaysia Eval Society, IPEN

• International: NONIE; ECG; IDEAS; IOCE; Eval Partners

Professional Networks: Advancing Development Evaluation

6

Commitment to evaluator professionalisation

Ability to engage in a supportive yet independent manner

High levels of methodological skill and content expertise

Strong strategic and communication skills

For evaluators it requires …

7

Has produced over 100 evaluations since 2000, and 80 country level evaluations. All have a management response.

Management uses evaluations to review policies, programmes and approaches – shown in 92%

uptake on recommendations. Independence respected, and

evaluation on the agenda at key top management meetings.

Board allocates significant time to engage with evaluation findings, management responds fully.

The Evaluation Office

8

The EO engages fully as evaluation processes are as important as the “big report”

Learning can come through accountability processes, these are not opposed to each

Stakeholder workshops held at country level, led by EO, with government as key stakeholder.

Transparency and stakeholders

9

Globally, across a broad mandate In complex situations where

interventions are difficult to embed and hard to measure

Implications for evaluation: difficult to work out additionality in terms of attribution or contribution

Context plays an important role, not easy to superimpose systems to measure results, but not impossible. Involves high levels of engagement to agree on what constitutes success

Where UNDP operates

Evaluation Office10

UNDP Programmes and Operations

177 countries in 5 Regions

National

Goals and

Priorities

United Nations

Development

Framework

UNDP Focus Areas

Poverty and MDG Democratic

governance Crisis prevention and

recovery Environment and

sustainability development

Programs

•Global•Regional•Country•Others

Projects

Non-Project operations (advisory, advocacy, standard setting/

normative, coordinatio

n, mobilizatio

n)

11

Evaluations need to be context specific, yet meet evaluation norms and standards.

At the country level they inform the next UNDP programme, and draw on lessons from the last programme period

Aggregation for synthesis needs to consider the question of scale, variability and the challenges of validity

Stakeholder workshops demonstrate to government UNDP commitment to transparency and accountability

What does this mean?

12

Credibility goes beyond the tools and methods. It rests on the leadership that directs evaluation processes. This comes about through transparent and logical evaluation plans and processes which ensure engagement opportunities throughout and across the spectrum to reduce bias

Results must be engaged publically Independence is central for the

reasons above, to ensure credibility and authenticity of reports

Independence, a credibility question

Evaluation Office13

Transparency

Defined in UNEG norms and standards Consultations with Stakeholders

TORS > Inception Report on Scope, Design and plan for data collection and analysis > Stakeholder Meetings > Draft Reports

The Audit Trail <> significance for mutual understanding and final decisions by EO

Public Access All UNDP plans and evaluations in the Evaluation

Resource Center (l) EO evaluations in the EO website Management response and tracking system (ERC) Ratings on quality of decentralized evaluations

Evaluation Office14

Independence - Structural

Functions, and staff - organizationally independent from operations and policy units and decision making. Executive Board

Director reports to the Executive Board (2 terms and no re-entry into UNDP)

Board approval of programme of work and budget (independent of programme budget)

Reporting Evaluation reports are the

responsibility of the Director Transmitted directly to the Board

following review and comment by management

Senior managers safeguard the independence: EO has access to all records and information

Executive Board

UNDP Administra

tor

Evaluation Office

Director

Evaluation Office15

Examples from …2013

As of June 2013 the Evaluation Office of UNDP presented eight independent evaluations to the UNDP executive board, which will contribute towards the development of the new

UNDP Strategic Plan.Five Regional Programme Evaluations

Evaluation of UNDP Strategic

Plan

Evaluation of South-south and Triangular

Cooperation

Evaluation of Global Programme of UNDP

Evaluation Office16

Country Programme Evaluation (Assessment of Development Results – ADRs)

Evaluation Office has conducted 80 ADRs since 2002

17

Engaging with decision-makers – engagement for quality and credibility (management responses and update)

Statistics on uptake – 92%

Acting on evaluations

18

Support of professional networks, associations and events, to profile evaluation:Supportive of IDEAS, Barbados 2013 (key notes, panels courses)Supportive of IPDET – course presentationSupportive of EvalPartners – management boardSupport to continental and regional networks (AFREA, APEN, …Hosting of the National Evaluation Capacity workshops, next San Paulo, September 2013www.NEC2013.org

Global influence

19

Evaluation is not an event, but a difficult journey requiring constant push, reappraisal, strategising to ensure relevance. It does not occur naturally and requires drivers

An evaluation function must be independent, which can be supported when it is both inward and outward focused. The outward – through events like these brings in critical insights necessary for ongoing revision. An outward orientation is the only way that one get professionalisation that is necessary for evaluation.

Conclusions

Evaluation Office20

Thank Youwww.undp.org/evaluation