World Change Startswith Educated Children. ®
Monitoring & Evaluation
World Change Startswith Educated Children. ®
Presentation Flow
• Glimpse on M&E • M&E at RtR • GI & CI • GSD• Data Trail for Quality Assurance • Ground Rule – There is no Ground Rule is the
Rule Every One is encouraged to
A(lert) S(eek) K(now)
World Change Startswith Educated Children. ®
• When or Where is the Last Time you have used an M&E Mechanism in Your
LIFE Other than Programme
World Change Startswith Educated Children. ®
TERMINUS
Where Does It Stand Now ?
World Change Startswith Educated Children. ®
Authorized Service Centre ?
Passengers ?Driver ?
Time-keeper at the
terminus ?
Whose Concern ???
World Change Startswith Educated Children. ®
Driver
The car is in good running condition !!
Road is good !! Running at good
speed !!Passengers
Authorized Service Centre
Time-keeper at
the terminus ?
No Jerking !! Feeling Comfortable !!
Driver is not drunk !Is the car good enough
for this kind of a
road ??
Need to redesign some
of the parts ??
How
far
is the
car ??
How
long
will
it tak
e ??
Who are
they ?
What Concerns
World Change Startswith Educated Children. ®
DriverPassengers Authorized
Service Centre
Time-keeper at
the terminus ?
Who monitors ? Who evaluates ?
Is monitoring and evaluation same ?
World Change Startswith Educated Children. ®
Monitoring & Evaluation (As defined by World Bank)
Monitoring Evaluation
A continuing function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing development intervention with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds.
Thus monitoring embodies the regular tracking of inputs, activities, outputs, outcomes and impacts of development activities at the project, program, sector and national levels. This includes the monitoring of a country’s progress against the millennium development goals (MDGs), or other national measures of development success.
the process of determining the worth or significance of a development activity, policy or program ….. to determine the relevance of objectives, the efficacy of design and implementation, the efficiency or resource use, and the sustainability of results. An evaluation should (enable) the incorporation of lessons learned into the decision-making process of both partner and dono
World Change Startswith Educated Children. ®
Indicator& DataIndicator
A quantitative or qualitative variable that provides a valid and reliable way to
measure achievement, assess performance, or reflect changes , Progress
connected to an intervention.
An indicator provides a sign or a signal that something exists or is true. It is
used to show the presence or state of a situation or condition. In the context of
monitoring and evaluation, an indicator is a quantitative metric that provides
information to monitor performance, measure achievement and determine
accountability.
Data
specific quantitative and qualitative information or facts that are collected and
analyzed thru indicators .
World Change Startswith Educated Children. ®Indicator Designing Flow Chart
World Change Startswith Educated Children. ®
Types of IndicatorsMost of the indicators are classified as Quantitative or Qualitative Indicators which are used as per the programme needs . There are different school of thoughts on what should be nature of an Indicator
SMART SPICED
S- Specific S-Subjective
M-Measurable P-Participatory
A- Achievable I-Interpreted and Communicable
R- Realistic/ Relevant C-Cross Checked and Comparable
T- Time Bound / Timely E-Empowering
D-Diverse and Disaggregated
Source : http://www.toolkitsportdevelopment.org/html/resources/40/408CC56F-509A-40D8-BE46-D7EEB4261F97/10%20Indicators.pdf
World Change Startswith Educated Children. ®
Qualities of a Good Indicator
UsefulnScientifically Robust Valid Reliable Sensitive Specific
Understandable
Accessible
Ethical
Yep!!!!!!!!
My Indicators are GOOD Indicators
World Change Startswith Educated Children. ®
Input Process Output Outcome / Impact
Fund Flow
TrainingSupplies
StrategyService PackageActivitiesAdvocac
y
Management Targets
Quantitative &
Qualitative
Change in the
CommunityQuantitative & QualitativeOrganization/ National Goal
Monitoring Evaluation
System Box
World Change Startswith Educated Children. ®
1. Know : what you
are suppose
d to achieve
2. Know : what you are
supposed to do
3. Selection
of key indicators
4. Data Collection
5. Analysis and
Interpretation
6. Action7. Share
Steps
World Change Startswith Educated Children. ®
Data
Dat
a
Information
*&^+
Books / bOys/ Girls !@#$%
Books Related to RR
Girls Related to GEP
Raw Vegetable
s
Eatable Cooked
Food
Analysis
Presentation
In 2012 , 850 RRs have been established in 8
states.
3000 Girl Scholars are supported with
Academic Support
World Change Startswith Educated Children. ®
Whether things are in order
Whether desired changes take place in the community
Continuous Usually at one or two years interval
Internal Internal /External
Corrective Effectiveness of strategy
Management Decisions Lessons Learned
MIS Data /Checklists ~ Registers, Forms, reports
Population Based Sample Survey/Cohort Analysis/ Participatory Appraisal/ Civil Registration System
How Many ? What %age ?
Monitoring Evaluation
World Change Startswith Educated Children. ®
Monitoring and Evaluation Framework
Process data-4
Analyze data-5
Ensure data quality-3
Show results-6
Share results-7
Receive feedback from
users-8
Change policies and
perspectives-9
Change actions-10
Identfy issues for examination- and
Define Role as per program Demand- 1
Collect data-2
Ideal Flowof Information
in a Functioning PerformanceMonitoring
System
World Change Startswith Educated Children. ®
Role of RM&E in Program Cycle
Baseline/Research/Assessment
Planning
Implementation &
Monitoring
Evaluation
World Change Startswith Educated Children. ®Project Data Flow
AP&B Document
Targets as Forecast Status
General Ledger
GSD
MIU/Quarterly Report
Global Indicators Reports
Donor Report
Budget
Project Progress (standard & data quality) Reports
Finance Reports
World Change Startswith Educated Children. ®
Program Monitoring SystemRtR
RtR PM/PD/CD
Field Facilitator
Moblizer/LMC/SMC
• Partner NGO• District /PA
PO/PA
RtR- RO/GO
Feed Back
1
2
3
4
Reporting
5
World Change Startswith Educated Children. ®
Goals, Objectives, and Indicators
• All programs have global goals, objectives and indicators
• All countries are collecting global indicator data for all programs & projects since 2008
• This data gets into the Global Solutions Database (GSD) for GI reporting
• Data Quality reports to check errors, outliers, exceptions etc.
World Change Startswith Educated Children. ®
Summary of GIs
PROGRAM NO. OF GIS QTRLY
ANNUAL
RR 11 5 6
SR 9 2 7
GEP 11 2 9
LLP 6 2 4
ALL PROG 37 11 26
World Change Startswith Educated Children. ®
Summary of CIs
PROG
BD KH IN LA NP SA LK VN ZAM
RR 7 2 5 8 9 5 1
LLP 6 2 3 3 3
SR 2 2 3 3 2
GEP 3+6 5+5 7+11 2+2 7+8 5+5 3+4 6+4
ALL PRO
G
24 16 23 18 30 5 13 10 10
World Change Startswith Educated Children. ®
RR GIs & CIs
Indicator Frequency
RR1 Number of libraries established Quarterly
RR2 Number of local language and/or English books provided to libraries Quarterly
RR3 Percentage of libraries where school personnel are trained by Room to Read and/or a Room to Read partner
Quarterly
RR4 Average number of days project sites receive training (REVISED GI) Quarterly
RR5 Number of books checked out by library users Quarterly
RR6 Number of targeted children enrolled in schools with a RtR library Annually
RR7 Number of targeted teachers at schools with a RtR library Annually
RR8a Average number of minutes per week each student/class can access the library during designated library/reading periods
Annually
RR8b Average number of minutes per week each student/class can access the library during non-designated library/reading time
Annually
RR9 Percentage of libraries with book classification systems in place Annually
RR10 Percent of libraries where books are classified according to reading level (NEW GI)
Annually
RR11 Percentage of libraries that have functional check-out systems Annually
World Change Startswith Educated Children. ®
•Percentage of RRs where the reading period is held regularly •Percentage of RRs where the CMC is trained •Percentage of RRs which have active CMCs •Percentage of RRs where the community has contributed in terms of events – organization and material support •Percentage of RRs where the students are checking out books as per the reading levels
RR CIs
World Change Startswith Educated Children. ®
GEP GIs & CIs
Indicator Frequency GEP1 Number of GEP participants in 2011 QuarterlyGEP2 Percentage of GEP participants who dropped the program in 2011 QuarterlyGEP3 Number of GEP participants who completed the final year of the program in
2011Annually
GEP4 Percentage of eligible GEP participants who advanced to the next grade AnnuallyGEP5 Percentage of eligible GEP participants who transitioned school levels AnnuallyGEP6 Number of GEP participants who received academic support AnnuallyGEP7 Number of GEP participants who attended life skills training AnnuallyGEP8 Percentage of GEP participants whose guardian(s) has participated in Girls
Education meeting(s)Annually
GEP9 Percentage of GEP participants who received mentoring and stayed in school each school year (NEW – for Enhanced approach only)
Annually
GEP10 Percentage of GEP participants who received material support and stayed in school each school year (NEW – for Enhanced approach only)
Annually
GEP11 Percentage of GEP participants in targeted grades who participated in life skills activities
Annually
World Change Startswith Educated Children. ®
GEP CIs
Percentage of girls transition from 5 to grade 6 (annual)Percentage of girls transition from 8 to grade 9 (annual) Percentage of girls transition from 10 to grade 11 (annual) Percentage of girls with less than 50% attendance (monthly) Number of girls participated in development camp (annual) Percentage of girls who went on exposure visit (annual)Percentage of field staff (SMs/PCs) trained on thematic areas (annual)
World Change Startswith Educated Children. ®Annual M&E Activities (Typical)
• Need Assessments to identify new project sites• Site visits by field staff to monitor progress & provide
hands-on support plus feedback to PO/PA & above• Covering all active project sites at least once a quarter• Monthly and Quarterly Reviews by RtR and NGO staff to
examine plans and progress/identify and overcome bottlenecks
• Pilot studies to test new program approaches• Country specific research/evaluations as needed• Annual/quarterly collection of GIs and entry into GSD• Measurement and sharing of CI progress with program
teams and initiate corrective measures• Verify GSD data quality during field visits with source
information on project progress
World Change Startswith Educated Children. ®
GSD purpose and processes
• Global Solutions Database (GSD) a Web based program management system of salesforce.com
• All program staff expected to use GSD are indicated by HR and given password by Administrator
• GSD power user in a country is the first point of contact with support from Database Administrator at RO
• Each GSD user is expected to:– Keep login active– Use reports for review and action– Update information as necessary– Check data quality reports– Share /discuss results with team members
World Change Startswith Educated Children. ®M&E and the GSD
• Complementary roles in program management
• M&E caters to local CO needs while supporting GSD to meet global needs of information gathering and reporting
• Field information through M&E System flows into GSD
• Both mutually contribute to program quality improvement
World Change Startswith Educated Children. ®Data Quality
“It is the process of ensuring that the Data flow from source or the first point of generation to the final source is same, consistent , accurate and assured Corrective in data”
World Change Startswith Educated Children. ®Example
World Change Startswith Educated Children. ®Report Sent by NGO - SO
ALARIPPU, HaridwarProject year- 2009 RR (Phulwari) Usage of books Month :- October - December 2011 Block- Bhagwanpur
S.N. Name
of the schools
Check in and Check Out DetailBy Children By Teachers By Community TOTAL
Books Readers Books Readers Books Readers Books ReadersDec
Bhudhwa Shaheed
646 59 28 2 216 23 890 84
Nov
Bhudhwa Shaheed
555 61 0 0 175 17 730 78
Oct
Bhudhwa Shaheed
311 56 0 0 52 9 263 65
1512 176 28 2 443 49 1883 227
World Change Startswith Educated Children. ®
World Change Startswith Educated Children. ®
Any Questions????
World Change Startswith Educated Children. ®
Thank Q………