Frame work for Monitoring & Evaluation under IWMP Workshop for Effective Implementation of IWMP DOLR Govt. of India 7 th &8 th January 2014.

Frame work for Monitoring &Evaluation under IWMP

Workshop for Effective Implementation of IWMP

DOLRGovt. of India 7th &8th January

2014

Components to be Monitored under M&E:

1. Concurrent Process Monitoring

2. Input – Output Monitoring

3. Impact Evaluation

4. Thematic/Special Studies

5. Sustainability Assessment

6. Case Studies/ Success Stories

7. Feedback and Dissemination Mechanism for Learning

8. Learning’s and Adoptability

9. Envisaged Deliverables/Outputs

ProcessMonitoring

Input-Output Monitoring

Impact Assessment

M&E

Learning & Documentation

M&E is an integral part of the project’s day to day operations rather than a periodic offline activity

Major Processes to be Monitored under IWMP

Separate indicators (Both qualitative and quantitative) to be developed for each of

above processes

Entry point activity (EPA)Sensitization and awarenessCBO formation & functioningCapacity buildingTransparencyAction plan preparation (Net plan & Complete action plan)Action plan implementation (Activity implementation)Flow of funds and procurement proceduresIncome Generating Activities (Livestock / Skill based & Micro enterprises activities) Institutional functioning- NGO’s, CBO’s & Government dept. or any other agencies. Demonstrations if anyWithdrawal strategyConvergence (with other projects)Post project O&M strategySustainability

1. Concurrent Process Monitoring:

Project Stages and Process Monitoring

“Indicators to be redefined as per the stage of the project”Weightage to be shifted as per project time chart

Indicators

• Prioritize

• Identify new

• Continue or discontinue old

• Make changes (if needed)

• Operationalise

A

I Stage

capacitation

II Stage

Implementation

III Stage

IV Stage

Consolidation

OB

C

D

Initiation

Process Monitoring Cont….

CBO Ist Six months

FormationMeetings AttendanceSavings

IInd six months

CapacitationSavingsMeetingsLending

IIIrd six months

Financial transactionBook keeping

IVth six months

IGAValue additionsProfit earnedSustainabilityemployment

SHG

(Indicators)

Collection of Data via Physical Surveys

Process Data and GIS analysis as applicableEx: graphs / tables

Interpret Data using MIS. And other wiseEx: Trends Graphs / Patterns/ GIS outputs

Use Findings for Learnings and Corrective Measures Ex: To change practice, inform policy, etc.

Documentation of best practices and strategies for up scaling

Process Monitoring carried out in Five-Stages

In An Effective M&E the five stages are carried sequentially

A Format for Process Monitoring (Observation) Report - IWMP

Issues observedSituational Status

(What is happening in the field)

Reasons(Why it is

happening)

Suggestion / Scope for

improvement (What is desirable)

Stakeholders to

be involved

CBOs Establishment & Functioning

IWMP Implementation

Transparency

Capacity Building

IGA & Micro Enterprises


Sl. No.

Activities / Processes

Situational Status Brief Analysis of the Status Observed

01 CBO training

S1, S2, S3, U1, U2, U3, E1, E2 & E3 trainings are completed in Bagdal sub watershed.

All the trainings are completed.Overall participation level in S1, S2 & S3 trainings was about 86.45%.Overall participation level in U1, U2 & U3 trainings was about 69.37%.Overall participation level in E1, E2 & E3 trainings was about 58.33%.Details pertaining to the trainings have been provided in the previous reports.

02Activity wise

Documentation

NGO level documentation is made. Photo documents of Grama sabha, Jatha & Street play have been maintained.S1, S2, S3, U1, U2, U3, E1, E2 & E3 trainings documents are maintained.Register book and photo documentation is also maintained.

03 Baseline survey Baseline survey of 2390 households is completed.

Entire 2390 household’s data has been entered into the system.

04 EC functioning & EC office establishment

There are 4 ECs in Bagdal watershed.Offices have been opened for all these 4 ECs in Bagdal sub watershed.

Office related logistics were provided in the month of August to 4 EC offices viz Bagdal, Khashempura, Bairanahalli & Honnadi in this sub watershed.Books are not maintained in the EC office.Documents are not maintained in the EC.

05 S & WC Work S & WC Work has been carried out in this Sub-WatershedVisited Bagdal sub watershed on 19/01/2013

Visited Sy. No. 65 (Lakshmana) where 300 rmt of bunding work has been done instead of 600 rmt as per net plan.Bund dressing has not been carried out.Bund carried out is not as per specification.

Taluk – Bidar BIDAR DISTRICT Sub-Watershed – BagdalIWMP – 1/2009-10 NGO – Santeshwari

Eg: Monthly Report Extract


Reports on Monthly/ Quarterly basis

Reflect the actual ongoing field status.

All the process and activities will be monitored at random basis (at

least 30% of Project site in a quarter)

Watershed wise reports will be consolidated at the district level

Report would also identify the key issues requiring immediate

attention

Flag-off issues to be taken up as special studies or thematic

studies.

Highlight the particular processes observed, the situational status,

the reasons for the same and the institutions or individual

associated in it.

Also suggest field based remedial measures to improve upon.

Process Monitoring Methods & Reports

2.INPUT-OUTPUT MONITORING

Under the concurrent input-output monitoring, the financial and physical

progress of the project would be monitored.

Aspects such as fund release from the PIA to grass root level, actual

condition/status of physical achievements, awareness creation, CBO’s

capacity building, etc will be monitored using specific indicators.

This would be done through the MIS deployed and operationalised by the

project.

M&E unit would verify the MIS data integrity through field verification on

sample basis through process monitoring.

Project will ensure flow of MIS data to M&E for analysis.

The MIS data thus received would be analyzed and the report would be made

available to the top management once in six months.

3. Impact Evaluation

Address the overall impact on Natural Resources & Socio -Economic aspects

APPROACH

With &

Without

• Before (Baseline) • During (Midterm) • End of the Project• Post Project

Household level Short term Impact Community / Village Long term Micro / Sub Watershed

Before &

After

Continues….

Continues….

Impact Evaluation Cont….

Sampling Approach

Sub watershed Micro-Level (Micro watershed)

Village-Level (Household Survey)

State


At the outset, as a part of impact assessment, Baseline Survey would be

conducted in the sampled watersheds.

This analysis will provide a bench mark enabling a comparative analysis at

the end of the project.

The degree of project goals and objectives achieved would be assessed as

against this bench mark.

For a better comparison, ‘control samples’ outside the watershed (i.e.

untreated area with the similar topographic and socio-economic condition)

also be analyzed.

Continues….

Baseline Study:


At the End of the Project, M&E will also carry out the impact assessment to

establish the net impact of the project on the socio-economic and natural

resource indicators.

Data would be collected with respect to the same samples collected during

the baseline.

A comprehensive comparative analysis to the baseline status would be made

in this report.

End Project Impact Study:

The major perceivable impacts would be:-

•Hydrological – Ground & Surface water, Drinking water status, etc.

•Soil Status – Runoff, infiltration and moisture content, nutrient capacity, etc.

•Agriculture and allied – Cropping pattern, cropping intensity, land

reclamation, productivity and diversity

•Institutional Aspects – CBOs, PRIs, any other

•Socio-economic – Income level, Employment status, Migration, Women

Empowerment, Educational status, Living condition, Livelihood options, etc.

•Environmental – Land use land cover change, a forestation, biomass and

carbon sequestration and

•Spillovers / Externalities (if any) – Political involvement, linkages with other

projects, financial linkages, etc.


Sampling Strategy for Impact Assessment

Sampling will be Based on Ridge, Middle, Valley Concept

25% of the projects will be studied

Households will be selected using PPS Method

Detailed questionnaire schedule covering relevant

indicators will be used for Baseline, Midterm and End of the

project enabling comparative analysis

Data will be collected by the M&E field staff

Satellite imageries will be used for spatial analysis


Indicators for Impact Assessment

Natural Resources

Surface Runoff

Water Resource Development

Ground Water level/Yield

Change in Irrigated Area

Crop Diversity

Crop Yield

Crop intensity

Fodder Availability

Afforestation

Climate Change & Biodiversity

Land Use Change

Socio Economic

Sustainability of Structures /CBOs/IGA

Micro Enterprises

Employment Opportunity

Migration Status

Economic potential of Household Income

BPL Family

Animal Husbandry

Impact on Milk yield


M&E will also conduct certain thematic studies on key issues as identified.

Some of the thematic areas include Capacity building, Investment pattern;

CBO participation, women empowerment, gender issue, level of participation,

Income generating activities; Run-off and land cover transformation, success

stories, etc.

4. Thematic/Special Studies:

The basic principle governing the thematic studies are that

(i)it should answer specific questions in the short run and help in policy

derivations for next phase.

(ii)should reflect both positive and negative aspects of the project.

(iii)should be demand driven and locally identified (local specific) by the PIA.

It is proposed that about one or two of thematic studies to be

conducted per year. Continues….

Investment Pattern

Social Inclusion

Women Empowerment

Capacity Building

Equity Issues

Common Property Resources

Income Generating Activities

Live stock

Short term Impacts

Sustainability

A few possible Thematic / Special Studies

• Run off Estimation

• Land use / Land cover changes

• Ground Water recharge

• Soil Fertility

• Water Quality

Thematic/Special Studies cont…..

A Few New Thematic Areas (Suggestive)

Community Based Activities Poverty Mapping Sustainability Micro

Enterprises Input & Output estimate for

a given Micro watershed

Soil & Water Conservation Activities

Soil erosion and Run-off

estimation Crop – Water Budgeting

Hydrology

Groundwater level and

quality Water Harvesting Structures:

Potential of Water availability

and its utility.

Natural Resources

Reclamation of degraded

land Agriculture Productivity Biomass Estimation CDM supportive studies

(Assessment of Carbon

Credits)Environmental Impact Assessment and Climatic Change Studies Watershed impact on overall ecosystem and climate

Thematic/Special Studies cont….

Post project sustainability of assets generated and also the institutions

created is very essential.

The level of involvement various institutions particularly the CBOs are crucial

for the sustainability.

In the long run the maintenance of the assets has to be taken up by the

community on their own.

Particularly the Common Property Resources (CPRs) needs to be maintained

by the community.

It requires a support system built in by the project during the project period.

The withdrawal strategy needs to be evolved on these lines.

The M&E studies need to focus on the successful and effective functioning

institutional arrangements for replication at a larger scale over time.

The withdrawal strategy adopted in the IWMP-I will be assessed for evolving a

strategy in the next phase of the project.

Selected 10 per cent of the institutions from SHGs, CIGs, and AGs will be

studied assessing their sustainability after completion of IWMP –I projects.

5. Sustainability Assessment:

Case studies to identify the specific project impacts would be carried out from

time to time.

These case studies will reflect the immediate outcomes and outputs of the

project with respect to specific area or sector.

The case studies reflect both positive and negative aspects.

This will help to consolidate on the success stories of the project and to

improve upon the weak areas if any.

At least one case study will be prepared per project per year.

6. Case Studies/ Success Stories:

7.Feedback and Dissemination Mechanism for Learning: Along with respective State departments identified M&E agencies evolve

suitable dissemination mechanism for learning at different levels.

It will assist department in reviewing the M&E observations /findings, evolve

compliance mechanism and provide feedback for decision-making.

Dissemination of the findings through learning events is proposed to ensure

that available and accumulated knowledge is fed back into an operational

learning process that provides appropriate and timely information to relevant

stakeholders, including management, to allow informed participation and on-

going corrections/adjustments.

The regular feedback of Process Monitoring Reports will provide online

corrections in the course of project implementation itself.

The regular dissemination of information both from Process Monitoring and

Impact Assessment if any would enhance the corrective options.

The thematic studies carried out at a larger scale discretely would support the

policy corrective measures to be adopted as a future strategy.

Annual learning workshop would be organized at each district level and at the

project level (State Level) to exchange the M&E findings arrived at through

various monitoring and assessment mechanisms.

The End of Project reports would lead to a State level Workshop disseminating

the major findings which would guide in developing new policy requirements if

any.

Based on the experience of various States a National level workshop

(Annual/Once in two years) can be planned.

8. Learning’s and Adoptability:

The following outputs / reports will be envisaged during the project period, from

the M&E Sector.

•Inception report

•Process monitoring reports (Monthly/Quarterly, Half yearly/ Annually)

•Baseline & Impact report (for the sampled watersheds) - Preparatory phase

Evaluation report, Work phase Evaluation report and Final Evaluation report.

•Input – Output reports (Annually)

•Thematic reports (Discrete)

•Case studies/Success stories

9. Envisaged Deliverables/Outputs:

Discrete monitoring

Concurrent monitoring

Impact Assessment

Input Output Monitoring

Process Monitoring

Self Evaluation/ AssessmentIndependent

M&L

(Frequency)

Pre treatmentBaseline

Mid –term impact

End of project

Various levels through MIS

Data

Continuous Monitoring

Yearly

By CBOs (SWS-EC, AG &

SHG) facilitated by NGO

Monitoring & Evaluation (In Nutshell )

Quarterly

Participatory

Monitoring

Community and household level

Post Project

In addition Thematic Studies, Case Studies and other related studies (Special) if anywill be prepared as directed by VWDA

M&E is a part of the project not an outsider

M&E is a facilitator of learning and is not for policing

M&E is an effective management tool

M&E ensures very high level of transparency

MEL &D is the eyes and ears of the project

Conclusion:

How an M & E can be successful?

Good Integration & Co operation between M&E & Project Management

M&E Information should be unbiased, relevant, timely & accurate

M&E documentation should address or resolve problems

M&E staff should have good skills and experience

Monitoring and Evaluation should be integrated – Single agency

Project Evaluations and Assessments – at frequent intervals

Information Dissemination – Upward & Downward

M & E also plays a facilitator’s role for all stakeholders

M&E should be an integral part of the project

“M&E is not - & cannot be - a substitute for good management. For M&E to be success – it needs to be driven by manager’s need for information, their use of information and their

desire to create a learning environment”

Road ahead

• Put in place an independent M&E agency—Feb-2014 (ToR and Bid documents uploaded on DoLR Web site)

• Ensure the critical elements of Concurrent –Process monitoring in real time

• Arrange M&E workshop at SLNA level (April 2014• Realise importance of learning and policy linkages• DoLR to contract directly Regional Independent M&E Agencies

Note the Change in the receptive level and Body language of DoLR- “Reciprocate”

Proposed MEL&D Structure

• Natural Resource Scientist• Hydrologist• Social Scientist• Statistics Scientist • Remote sensing/GIS/IT Scientist• Documentation Specialist

District Coordinator(1 for 3 districts)

Natural Sciences / Social Sciences

Field Staff (2 for each District)Social Sciences

Natural Sciences

*Project Director (M&E)HEA

D Q

UA

RTER

AT P

RO

JEC

T LE

VEL

Organizational Structure

* A senior person among the scientists will be act as Project Director

ANNEXURE –III

Multi-faceted Indicators of Impact at Different Spatial Scales(In Case of Soil Conserving Technologies)

IndicatorLevel

Farm Household Watershed

Biophysical

Rate of erosion

Soil fertility status

Vegetation cover

Crop yields

Areas abandoned due

to high erosion

Food produced

Access to water and fuel

Quality of drinking water

Quantity of drinking water

Slopes stabilized

Rainfall data

Rate of siltation

Quantity of water in reservoir

Area under tree cover

Social --Awareness on environmental

aspects

Rate of immigration

Conflict for access to land

and water

Income redistribution

Access to natural resources

Economic

Fertilizer use

Rate of profits

Level of risk

Level of diversification

Income level

Level of food security

Level of assets

Infrastructure network

Biodiversity level

Siltation cost

ANNEXURE – IV

A Format for Process Monitoring (Observation) Report - IWMP

Issues observed Situational Status

(What is happening in the field)

Reasons

(Why it is happening)

Suggestion / Scope for improvement

(What is desirable)

Stakeholders to be involved

CBOs Establishment & Functioning

IWMP Implementation

Transparency

Capacity Building

IGA & Micro Enterprises

ANNEXURE – V

EXPECTED / ESTIMATED OUTCOME

List of Expected/Estimated Outcome – to be compiled district wise

Sl. no Particulars Unit of measurementPre-project

status (Baseline)

ExpectedPost project status

Actual Outcome

1 Status of water table Depth in ft Increased water table (Specify in feet)

2 Ground water structures repaired/ rejuvenated

Numbers Increased

3 Quality of drinking water Hardness Reduced 4 Availability of drinking water Distance from the

house to source (m)

Duration (months)

Reduced Increased

5 Increase in irrigation potential Pumping hours Yield (gallon/hr)

Increased pumping hrs Increased yield

(gallon/hr)

6 Change in cropping/land use pattern

Area in ha (Crop wise Shift from - to

Wasteland reclaimed

Shift to Agro-forestry/Horticulture

Wasteland reduced (area in ha)

7 Area under agriculture crop Area in ha Increase (10-15%) 8 i. Area under single crop Ha Increase (18-20%) 9 ii. Area under double crop Ha Increase (10-15%)

10 iii. Area under multiple crop Ha Increase (10-15%) 11 Net increase in crop production

areaHa Increase (20-25%)

12 Increase in area under vegetation

Ha Increase (10-15%)

13 Increase in area under horticulture


14 Increase in area under fuel & fodder


15 Increase in milk production Number/Yield Increase (20%) 16 No. of SHGs Number

Savings (Rs) Increase

Increase (25-30%)

17 Increase in no. of livelihoods Number Diversification

Increase (10-15%)Increased

18 Increase in Income Rs Overall by 30-35% increase

19 Migration Numbers Reduced 20 SHG federations formed Numbers

Savings (Rs) Increase

Increase (20-30%)

21 Credit linkage with banks No. of institutions Amount Rs.

Increase

22 Resource use agreements Number of MOUs Executed

Increase

23 Collection & management Number of institutions Regular 24 Summary of lessons learnt

Project Goals &Objectives

Key Performance IndicatorsLevel of data

collectionMethods Frequency

Improve the productive potential of watersheds

Changes in household income - Area under Agri. - Agri. Income / Ha

Household Baseline (once) Survey, Periodic Assessment

Annual

Increment in crop yield Household/micro watershed

Baseline (once) Survey, Periodic Assessment

Crop Seasons

Ground water recharges 1. Change in GW level & yield 2. Drinking water availability 3. Change in cropping intensity4. Change in irrigated area

Household/micro watershed

PRA Survey, Measurement

Baseline and End of the Project

Soil Erosion – Soil loss & Run-offHousehold/micro watershed

Field observation PRA

Annual

Change in Cropping Pattern Household/micro watershed

PRA / RRA, Survey Annual

Changes in milk, fuel wood & fodder production Household/micro watershed

Household Survey, FGD, PRA Survey

Annual

Treatment of non-arable land - changes in forest cover, forest produce, output from revenue land

Micro watershed Sample Survey & RS&GIS Annual

ANNEXURE - VILOG MATRIX FOR KEY INDICATORS TO BE ASSESSED UNDER IWMPThis is only an indicative format for operationalization of project assessment through ME&L activities. However, this is not a final document; it can be used as guide to conduct various activities under ME&L. An improvement on this matrix can be made as per the actual field requirements. (For Neeranchal project additional indicators will be added and details would be worked out in the format after inputs from various stakeholders)

When You Have Measurement, You Have

PERFORMANCE

When You Have NO Measurement, You Have

EXCUSES

What Gets Measured, Gets Managed Well

Frame work for Monitoring & Evaluation under IWMP Workshop for Effective Implementation of IWMP DOLR Govt. of India 7 th &8 th January 2014.

Documents

process monitoring indicators

concurrent process monitoring

scaling process monitoring

monitoring evaluation

stage implementation

input output monitoring

physical surveys process

project stages