More Enabling and Less Controlling! Rio De Janeiro, 6-7 November 2008 Lichia Saner-Yiu CSEND, Geneva, UNFCCC Meeting on Experiences with.

More Enabling and Less Controlling!

Rio De Janeiro, 6-7 November 2008

Lichia Saner-YiuCSEND, Geneva, www.csend.org

UNFCCC Meeting on Experiences with Performance Indicators for Monitoring and Evaluation of Capacity-Building in Developing Countries

Objectives of Presentation

To urgently draw attention to the need to review CB performance indicators in the context of an institutionalised M&E processes in the partner countries

To clarify that monitoring should not be “mini-evaluation” but rather a management tool!

To propose a monitoring tool which will support greater effectiveness of the CB projects/programme!

Field Experience

Capacity Building for the Public Administrative Reforms – China (1988-1997), Slovenia (1994-1996) etc.

Building Business Training Centres in Russia (1995-1997)

Building Negotiation Capacities in the International Arenas: UNFCCC, WTO, FTAA.. (2000-2003)

Strengthening management capacities of UN Agencies & INGOs (ILO, UNICEF, WHO, UNHCR, Action Aid …)

Thematic Training and short term consulting inputs for 20+ years

Continue to teach in different universities on topics related to institutional performance management and transformation

Contexts

Decision 2/CP.7 The Paris Declaration for Aid Effectiveness

(2005) The ACCRA Agenda for Action (2008) calling

for accelerated progress in areas of country ownership, inclusive partnerships and achieving development results. Specifically, “Developing countries will strengthen their capacity to lead and manage development”

Framework | Scope of priority areas (UNFCCC, 2007)

9. Research and systematic observation

10. Development and transfer of technology

11. Decision-making and participation in international negotiations

12. CDM13. Article 4.8 and 4.9 (LDC

Work Programme, NAPA)14. Education, training and

public awareness15. Information and networking

1. Institutional CB (National Focal Point)

2. Enabling environment3. National Communications4. National climate change

programmes5. GHG inventories6. Vulnerability and adaptation

assessment7. CB for implementation of

adaptation measures8. Assessment for

implementation of mitigation options

The Challenge of Capacity Development (CD)

“Capacity Development is a fundamental component of development and aid effectiveness and a key element in achieving the Millennium Development Goals” (OECD/DAC 2006).

In particular CD is critical to achieving objectives of ownership, aid alignment and mutual accountability, as set in the Paris Declaration on Aid Effectiveness (March, 2005).

However, the traditional approach to CD (as a technical process or transfer of knowledge or institutions from North to South) rarely leads to the expected results i.e. sustainable capacity within partner countries.

Increase in funding is not sufficient to address the challenge of capacity development. We need a fundamental change in development practice and the way that we look at the role and function of M&E.

The Challenge of Capacity Development (CD)

The need for Adequate CD

Focusing on capacity as an endogenous process (led but not exclusively by country actors) including M&E components.

Agreeing at country level on clear (institutional) capacity objectives.

Making effective use of existing capacities and harmonising external support within the MEA framework.

Monitoring outcomes from the perspective of beneficiaries (OECD/DAC Recommendations).

Enhancing institutional learning and knowledge creation through M&E processes for all development partners.

Parameters

Shortcomings of Current Monitoring Practices

It is more of controling than for enabling.

It relies more on self-reporting, than evidence and documented records.

It remains short in providing opportunities to the partners and external (non-national) stakeholders in taking charge of the CD process and in being held accountable for what has been achieved and what has not.

Shortcomings of Current Monitoring Practices -2

The process for the QA of TBCD remains exogenous rather than endogenous.

The learning remains at best temporal, at worst short-lived on the account of the partners since there is no mechanism to ensure institutionalisation of learning.

Monitoring remains as activities to be checked off, rather than an integral part of the day-to-day process and mangement system

Confusing Use of Terminology

Mixing terms of “evaluation”, “inspection”, “supervision” and “monitoring”

Monitoring in the parlance of the QM is about “ensuring the CB process is being managed and implemented as required so as to provide objective evidence that the CB process is effective in meeting requirements.”

Monitoring, therefore, should involve reviewing the entire CB process at each stage of the project cycle (from design to implementation) and not just mid-term review or end-of the pipe evaluation.

Evaluation About the past. Who remembers the W’s after 3

or 4 years of project duration? Lessons learnt for whom? Project partners

(beneficiaries, donors, experts) move on to new jobs

Very costly and very heavy for beneficiaries (evaluation tourism)

Against Paris Declaration (e.g.EIF: multiple evaluation methods, LDC overwhelmed with compliance duties)

OECD Survey on Monitoring and Paris Declaration (2008).

Les than 10% of countries have sound frameworks to monitor and assess development results. While some progress has been made since 2005, an enormous effort will be required to meet the target of 35% by 2010

Urgent Need to Invest Less evaluation, more monitoring! Change our mindset and approach

to monitoring Invest in a management information

system that monitors and enables

In order to accelerate the CB and the attainment of UNFCCC on "preventing dangerous anthropogenic

interference with Earth's climate system“ and MDG 7 –

Environmental Sustainability

Proposal by Engel, Land, Keijzer (2006)1

A balanced framework to define the term “Capacity Building” in clear terms (capability to produce development results, to self-organise and act, to adapt and self-renew, relate& create adequate operating space, and to achieve coherence/vision/ strategy)

An excellent framework to develop indicators for baseline studies, progress assessment and benchmark

Yet, there remains a missing link – an institutionalised management system and database to support the emergence of these essential capabilities through practice and to facilitate effective management decision making and self regulation through evidence and data

1. “A balanced approach to monitoring and evaluating capacity and performance”

CSEND Proposal

Capacity Building model

of Monitoring

The Three-Tiered Monitoring Framework at the global level

Tracking Aid for CB Flows

Donor Self-Assessment

National Self-Assessment

Quantitative Information

Qualitative Information

UNFCCC CBReview

CSEND’s Non Fiduciary AfT Monitoring Framework (within borders)

Tracking CB Implementation Process

Verification of Participating Donor In-Country Inputs

Verification of Executing Agency In-Country Inputs

Documentation of Input-Throughput- Output & Outcome data

Validation of AfT Processes

Knowledge Creation

Continuous Improve-ment

Based on Paris Decla-ration Principles

Verification of National Inputs

Project Cycle

2. Initiating & formulating

project proposal

3. Appraising &

approving project(at country and global levels)

4. Implementing

project

5. Evaluating project

outcome

Monitor

1. Defining strategic interests & needs

of a DC

CB programming

process(framing the

scope)

Global

National

What Is Needed for An Effective Monitoring System -

M&E protocol Monitoring methodology (focusing on

organisation, management and institutional collaboration)

Third party verification according to the agreed CB objectives and measure

A matrix structure and delineating levels of responsibility for monitoring

Institutional actors and roles Level of responsibilities regarding monitoring Periodicity of monitoring actions Inputs needed and from Outputs produced Monitoring criteria & indicators Records

Example of M&E System (EIF)

EIFActors

Level of Responsibilities regarding Monitoring

Periodicity

Inputs Outputs Monitoring Criteria & Indicators

Records

NSC Oversight on the Use of EIF resources for the country

Setting national policy regarding trade development and performance targets

Supervise the performance of National Implementation Organs and mechanisms

6 months National Trade Development Plan and Priorities

PRSP Country Strategy and Programme Priorities if existing

DTIS and Action Matrix

NIU quarterly implementation reports (both technical and financial)

NIU process audit reports

Country EIF programme, strategies, workplans

Semester Country EIF Implementation Review (Tier 1 & 2)

Annual Country EIF Programme Performance Report

Monitoring Criteria Compliance to

National EIF programme formulation procedure

Compliance to National EIF project approval procedure

Decisions on Tier 1 workplan and Tier 2 programme priorities

EIF Country Programme Monitoring Records

NSC audit records

Request for Corrective or Preventive actions

Corrective action report

It aligns CB with DC/LDC country development and ministerial performance objectives within the context of national strategy

It provides a process management tool for learning transfer, continuous improvement of capacity building processes and outcome (higher pay-off)

It gives ownership and accountability to the beneficiary countries and counterparts

It supports intermittent programme evaluation (monitoring means collecting in-depth data and rich information source)

It facilitates multilevel knowledge creation and knowledge management amongst donors, beneficiaries and experts.

Added Value of a CB based monitoring management system Safeguards against sub-optimal use of resources Ensures result-based accountability of the

capacity building function Documents quantifiable data for process

improvement Anchors CC agenda within recipient country’s

development agenda, planning and implementation processes

Knowledge management in the hands of ALL partners

Vision to Achieve

Capacity building should assist developing countries to build, develop, strengthen, enhance, and improve their capabilities to achieve the objective of the Convention through the implementation of the provisions of the Convention and the preparation for their effective participation in the Kyoto Protocol process (italic added).

Thank You !!