APR Workshop 2010-M&E-Integratng monitoring and evaluation in project implementation-Pakistan-A.Karim

Integrating M&E

into theProject

Implementation

APR Workshop: M&E Session

1- 4 Nov 2010

Nanning, China

Abdul Karim

Rush

• COSOP 2010-14

• CPMT

• Portfolio

• Project Implementation

• Locally defined

• Practically evidenced

• Collectively targeted

COSOP 2010-14: SOs

The COSOP supports the government’s strategies and initiatives for rural poverty reduction.

Strategic Objectives :

(i)Enhancing the access of poor rural men and women to productive assets, skills, services and improved technologies, and

(ii)Strengthening the capacity of the rural poor to engage in and benefit from local development process.

Target Areas :

• High incidence of poverty.

• Areas with geographic disadvantages.

Targeting Approach:

Encouraging the self-selection of poor and vulnerable households:

(i)small farmers with limited landholding and livestock ownership,

(ii)landless tenant farmers whose survival strategy depends upon a combination of on-farm and off-farm wage employment, and

(iii)women headed households and women within poor households with little access to resources, services and assets of their own. Gender equality and the empowerment of women are a special area of focus, as is the challenge of climate change.

(iv)Greater efforts are made to pursue the innovation, policy dialogue, and knowledge management agenda.

COSOP 2010-14: Targets

Country Programme Management

CPMT: The main instrument for country programme management.

• EAD, government’s focal point agency,• Key ministries (e.g., Finance, MINFA, etc.), • Donor and UN agencies, • NGOs and civil society, • Private sector, • IFAD headquarters and field staff• A semi-annual country portfolio review meeting, the

forum for portfolio management and project performance review.

Country Office

Greater emphasis on supporting the effective implementation of the ongoing projects,

Instruments:- Project supervisions, - Follow-up missions, - Quarterly project status monitoring system,- Knowledge management.

Particular focus: - Portfolio’s project-at-risk (PAR) rating - Status of the problem projects.

Portfolio

Particulars Commitment (USD)Disbursement till Sep

2010 (USD)

a) On-going Projects

PROG FOR INCREASING SUSTAINABLE MICROFINANCE (2007-13)

35,000,000

9,939,997

MICROFINANCE INNOVATION & OUTREACH PROGRAMME (2006-12)

26,460,000

15,146,578

SOUTHERN FATA DEV PROJECT (2001-11)

17,150,000

4,459,000

AJK COMMUNITY DEV. PROGRAMME (2004-12)

21,770,000

17,217,821

CROP MAXIMIZATION SUPPORT PROGRAMME (2009-15)

18,400,000

-

b) Project in Pipeline    SOUTHERN PUNJAB POVERTY ALLEVIATION PROJECT (2010-16)

40,200,000

-

GWADER LASBELA LIVELIHOODS SUPPORT PROJECT (2011-17)

30,000,000

-

Locally defined

Monitoring

• A continuing process of gathering information systematically on pre-defined indicators;

• Providing progress on objectives, utilization of resources and achievement of goals;

• Regular tracking of inputs, activities, outputs, outcomes and impacts.

Evaluation

• A process of determining significance of a development activity;

• Determining relevance of objectives, efficacy of design and implementation;

• Sustainability of results;

• Enabling incorporating lessons learned into decision making process;

Locally defined

M&E System in Projects: AJKCDP

• The PD is the overall in-charge of the affairs;

• M&E Specialist leads the team of M&E Section under the PMU;

• MOs at district level;

• MIS established;

• Flow of Information;

• M&E Tools;

• Project Baseline in 2003

Different Levels of Information Flow

Program Management Unit

District Program Offices

Field Unit

Community Organization

Implementation Mechanism

Implementation & completion of a scheme

Field Units(to follow up & feedback)

Presentation of a scheme in District Coordination Committee for approval

District Program Offices(to plan, process, coordinate

& facilitate

Field Units (for recommendation after social

appraisal)

Demand from CO (based on VDPs)

Govt. Line Departments/

Service Providers(for estimation,

preparation, technical guidance,

monitoring & billing etc.)

NGOs & Clusters

Management Information System

• Monitoring Formats (MP0 to MP6) for data collection on monthly basis;

• MP0: Baseline; • MP1: Activity status at the grassroots level;• MP2: Field Unit performance;• MP3: Service providers;• MP4: M& E activities at district level;• MP5: professional staff’s activities;

• MP6: Data Compilation, Processing (report generation, Input, output) & Dissemination of Information, Identification of Bottlenecks & Issues, Management’s decisions to address the issues.

RIMS Reporting

• Bench mark study (2003);

• Impact study (planned in 2010);

• Output reporting of NRM activities by Government Line Departments was different from the information of COs/beneficiaries. Hence the district offices reporting after discussion at district coordination committee;

• Calculation of project beneficiaries was a challenge initially and after wards COs were involved to avoid duplication.

Head of Line Deptts./ Service Provider

State level NGOs (Rural Support Program )& IOs

Ext. staff of LDs/ SPs

NGOs / clusters

Dist. Heads of Service

Providers

PMU

DPOs

Field Units

COs

Ministry of Planning & Development

Coordination Mechanism

Monitoring Tools

• Logical framework• Surveys • Institutional Maturity Index (IMI)• Periodic Physical & Financial Review

- Monthly - Quarterly - Annual

• Monthly Coordination Meetings• District Coordination Committee• Field Visits • Program Technical Committee • Program Steering Committee

M&E System in Projects : MIOP

Development Goal:

“Reduce poverty and improve livelihoods of rural households”

•The COO PPAF is the PD/overall in-charge of the project

•GM, CED looking after management

•GM, ERD leads the M&E team responsible for progress reporting

•MIS planned

•Flow of Information

•M&E Tools – Quarterly, Bi-annually Prog. Reporting, monthly review

•RIMS-Baseline Survey in 2009 (MIOP)

• Knowledge Management Portal (www.ppaf.org.pk/km)

M&E System in Projects : PRISM

Development Goal:

“poverty reduction, promotion of economic growth and to improve the livelihoods of rural households”.

• The COO PPAF is the PD/overall in-charge of the project• GM, CED looking after management • GM, ERD leads the M&E team responsible for progress reporting• MIS planned• Flow of Information

• M&E Tools – Quarterly, Bi-annually Prog. Reporting, monthly review

• RIMS-Baseline Survey in 2009 (MIOP)• Knowledge Management Portal (www.ppaf.org.pk/km)

Monitoring Framework: MIOP & PRISM

• Uniform monitoring mechanism:

- All POs to follow,

- Pre-mission data by POs,

- Examination of financial and operational performance,

- Book-keeping and financial management,

- Beneficiary feedback, interaction with clients,

• Concluding meeting with top-management and critical members

• A back-to-office report to unit head, COO and CEO and circulated.

• PO Management invited for further discussion for any serious weaknesses or flaws in the systems or operations at the PO level.

• M&E a multi-tasking arm of PM;

- More involved in planning & implementation than the M&E

• Considered a conventional tool for control and accountability purposes;

- Meant for mainly highlighting issues/weaknesses (NADP).

• An integral part of the management;

- Rely on Quantitative results insufficient to provide feedback for generating learning and leading improvements.

- Hesitant to engage in learning process thus lack self reflection and dialogue.

- Whenever M&E used for learning, project performance improved (AJKCDP).

Practically evidenced

• Results achieved are disseminated only through reports;– Of the 2979 COs (AJKCDP) that are involved in the programme, some 1200

or 40% of them are regularly submitting progress reports.

– Experience sharing at country level had been a useful tool.

– Publications, NLs, radio, TV, documentaries, special editions in Newspapers, are not extensively used for dissemination.

• Lessons learned are rarely shared; – WDGs were revived through the ‘Mehram’ in NADP.

– Two important aspects of the MIS need attention.

» 1st, the lack of output data (which is most often available in the files of the district GLD offices) severely constrains the ability of management, the PSC and IFAD to assess the results of the multifarious programme activities.

» 2nd, the M&E team tends to present all the collected data in progress reports rather than reviewing and summarising the data into information. As a consequence the progress reports tend to be very heavy and somewhat difficult to navigate

Practically evidenced

Practically evidenced

• Pacifying hardcore Issues

- Issues pertaining to communities are normally pacified to avoid weaknesses - women development, quality of community infrastructure, micro-credit, etc;

• Bargaining effectiveness of solutions

- Putting core issues under the carpet mostly lowers the effectiveness of the solutions and failing to resolve issues for longer convert into chronic;

• Interaction and consultation with communities

- All participatory projects must start with developing a community interaction strategy through broad based consultations with the communities and local opinion makers (irrespective of SAR).

• Knowledge Management/Experience Sharing not yet common amongst the programmes & projects

• Inter-regional experience sharing workshops are important source of getting benefit from the results and for up-scaling.

Practically evidenced

• M&E Fails to emphasize:

» Societal and attitudinal change is a long haul process and projects must start with soft areas and soft interventions through a path of least resistance (NADP).

» Extremely difficult start-up phase, owing to a very hostile operating environment generated by ill-informed propaganda and local cultural sensitivities vis-à-vis project approach (NADP).

• M&E needs to learn how to prioritize:

» Bigger things must be attempted only when the project credibility has been sufficiently established and reasonable amount of goodwill created (NADP).

» Most project interventions benefitting individual households should be designed and scaled to facilitate self targeting (NADP).

Collectively targeted

• Results gained/sustained

- AJKCDP which remained under a seriously “at-risk” project for long turned around to one achieving the majority of its targets.

- A number of path breaking interventions with very high impacts have been demonstrated with quite high replication rate (fruit orchards, off-season vegetables, breed improvement, etc).

- Unless there is an umbrella support organization to follow up on project initiatives, very few of these COs/WDGs would survive for long (NADP).

- Female staff is must for effective extension activities in areas like Diamer to access the women who play a major role in both crops and animal husbandry