IAOD Evaluation Section, the Development Agenda (DA) and Development Oriented Activities
Julia Flores Marfetan, Senior Evaluator
2
1. Introduction to the Independent Evaluation Function
2. Services provided to the Development Agenda3. Independent evaluation of the Development
Agenda and development oriented activities4. Types of Evaluations5. Evaluation Approach for the Development Agenda
and development oriented activities
IAOD Evaluation Section
Evaluation DefinitionEvaluation is an objective, impartial
and systematic assessment to provide useful feedback about an activity, initiative, project, program,
strategy, policy, unit, organization or sector, whether financed from
regular budget or extra budgetary resources
4IAOD Evaluation Section
Evaluation in WIPO• Where did all start?
– Recently no real evaluation activity since 2004– Evaluation was perceived as a program management
functionWhat has IAOD and its Evaluation Section done since 2007?
– The WIPO Evaluation Policy since 2007– The Evaluation Section was set-up by the DG in 2008 within IAOD with a dedicated Evaluation Team– Focusing the evaluation function in their core business “Independent Evaluations”– Independent Review of the PPR process– Validation of the 2008 PPR– Having dedicated evaluation team within the Evaluation Section – 2 post were created– Training to improve existing monitoring and evaluation frameworks– Development of self-evaluation guidelines for program staff– Improving the previous PPR and making it more performance oriented
5IAOD Evaluation Section
• What are the Evaluation Section future priorities?– Undertake high quality independent evaluations of development oriented
activities in form of country, thematic, strategic or program evaluations– Develop WIPO Evaluation Strategy and Guidelines – Contribute to enhance accountability, learning and transparency by delivering
relevant information to our stakeholders specially Member States
May help Member States in building their capacities to undertake evaluation exercises
Including impact evaluations depending on resources provided. This is important for
The country focus of the DA and DOA.
Design, manage and undertake independent evaluations of the
Development Agenda and development oriented activities
Other Support Services
CORE SERVICES: INDEPENDENT EVALUATIONS
Support for capacity building
Ensure that monitoring and performance management of
the development oriented activitiesis of good quality
Validation
8IAOD Evaluation Section
EVALUATION SECTION SERVICES
• Independent evaluations contribute towards a UN system identifying ways to improve delivery of development oriented activities to the beneficiaries; overcoming weaknesses and building on strengths from a strong evidence base
• Independent evaluations assist Organizations to take a deep and detailed look at internal systems and development oriented activities, and examine what practical steps need to be taken to improve them
• According to the UNEG norms and standards the evaluation function and its products should be independent, credible, and useful for learning and accountability purposes
• It is central to the credibility and reliability of evaluation• By increasing independence in evaluation, we can decrease
certain types of bias including extreme conflict of interest • Evaluation quality without independence does not assure
credibility and does not protect learning
10IAOD Evaluation Section
Whz are independent evaluations necessary?
Assuring the Independence of Evaluations
Organizational Independence
The ES is part of IAOD and it performs its function independently from other
WIPO Management functions to ensure impartial and independent reporting
Behavioural Independence
The evaluation reports are based on evidence and stakeholders are consulted
at the various stages of the evaluation process. The budget for the evaluation is
managed solely by IAOD
Protection from outside interference
IAOD Evaluation Section is be responsible for designing and
executing the evaluation and evaluatorsresults will not be subject to
overruling or influence by any external authority
Avoidance of conflict of interestIAOD will assure that the evaluators undertaking the evaluation will in no manner have an official,
professional, personal or financial relation with any WIPO program and that he/she
does not have any current or previous involvement with the development oriented activities or the
entity being evaluated at a decision making level, or in a financial management or accounting role;
or seeking employment with the Organization.
11IAOD Evaluation Section
• Are we doing the right things? And are we doing them right?
• To know if we are achieving an effect (+/-) on the intended beneficiaries
• To contribute to learning • To support accountability and transparency to stakeholders
specially Member States.• To inform the CDIP, other stakeholders and management To
inform the planning, programming, budgeting, implementation and reporting cycle
• To contribute to the improvement of the performance of on-going and future activities and to the enhancement of institutional and development effectiveness.
Why do we need to evaluate the DA and DOA? (1)
12IAOD Evaluation Section
• To enhance the generation and use of value-added evaluative information for:
•Decision-making process concerning the improvement of present and future activities
•Policy formulation and review by Member States
•Management oversight by the Director General
13IAOD Evaluation Section
Why do we need to evaluate the DA and DOA? (2)
• Establishing a program theory: how the intervention will benefit the beneficiaries and the potentials for side-effects
• Setting up a monitoring framework which includes expected results targets, baselines, SMART KPIs including systems and data base for data collection
• Allow comparison: before and after the intervention• Plan the evaluation at the start of the intervention • Define the evaluation approach, methodology and
allocated a budget and time for it
14IAOD Evaluation Section
Requirements for evaluating the DA and DOA?
What will guide the evaluation of DA and DOA?
• The WIPO Evaluation Policy approved by the DG in 2007 guide the undertaking of the independent evaluations
• The Evaluation Policy lays a framework for the evaluation function of WIPO
• The Evaluation Policy provides a comprehensive framework for planning and conducting the evaluation following international best practice, taking due account of the Norms and Standards for Evaluation in the UN System adopted by the United Nations Evaluation Group (UNEG), and other international good practices.
• The Policy is applied to all independent evaluations
15IAOD Evaluation Section
Relevance
Are the Organization’s development oriented activities suited to the
priorities and policies of the target group?
Effectiveness
Are the development oriented achieving its
objectives?
Efficiency
Is the Organization for the implementation of its development oriented
activities using the least costly resources possible in order to achieve the desired
results?
Impact
What positive and negative changes are being produce by a development oriented
activities, directly or indirectly, intended or unintended?
Sustainability
Are the benefits of the Development Agenda likely to continue after the Organization’s funding has been
withdraw?
Coverage
which groups are included or excluded from the development oriented
activities, and the differential impact on those
included and excluded?
Coherence
Are other policies and programs affecting the implementation of the
Development Agenda?
Co-ordination
How are the development oriented activities harmonizing efforts with WIPO programs,
other aid agencies and alignment with country priorities
and systems?
DA and DOA
Are we doing the right things?
Are we doing them right?
Where is the Evidence?
16IAOD Evaluation Section
What are the evaluation criteria applied to the evaluation of DA and DOA?
• Evaluation will examine the processes and contextual factors to understand why, and to what extent, expected results have been achieved, and what impact they have had.
• Evaluation will recommend and suggest better ways of achieving expected results.
• Evaluation will identify good practices and lessons learned.
What other information will the evaluation deliver?
17IAOD Evaluation Section
Independent Evaluations vs Costs
Country Level Evaluations
Program Evaluations
ThematicEvaluations
Systemic Evaluations
Organizational Assessments
Project LevelEvaluations only Above Sfr 1 mio
0
1
2
3
4
5
6
Mill
ion
USD
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Sugg
este
d %
for e
valu
atio
n
Size of Program Suggested % for Evaluation
19IAOD Evaluation Section
Summative EvaluationsFormative Evaluations
DA and DOA evaluation timing
Development Intervention Starts/ about
To start
Development Intervention
Is in the middle Of implementation
Development Intervention is completed
Time
Stage of theDevelopment Intervention
Type ofEvaluation
Ex-ante evaluation Mid-term evaluation Ex-post or final evaluation
DA and DOA
20IAOD Evaluation Section
Utilization-focused evaluation approach
1. ToRs drafting
Participatory planning withKey stakeholders and partners
2. Evaluation process
Supporting the evaluation team with aResource or learning group
3. Draft findings/ Draft report
Consultation with key stakeholdersAnd workshops
4. Evaluation report
Dissemination to a wider public
5. Follow-up day
Assisting program managers with the Implementation of recommendations
22IAOD Evaluation Section