The Most Significant Change ProLearn Project in India Inka Píbilová Impact Evaluation Conference in Wageningen, 25 March 2012.

The Most Significant ChangeProLearn Project in India

Inka PíbilováImpact Evaluation Conference in Wageningen, 25 March 2012

A „tsunami relief“ project 2004 first tsunami relief in Andra Pradesh India funded by

individual donors of ADRA Czech Republic 2007 needs assessment with Education Office among 20

schools 2008 – 2009 infrastructure development in 8 schools to

increase and retain the number of children– Classrooms construction– Teacher training– Learning aids– Community sensitisation– Children participation

Evaluation Objectives• Show evidence of the project impact on access to

education, increased education quality and child-friendly learning environment, thus achieving learning outcomes and higher literacy.

• Show evidence of attitudinal changes of community, school staff, children and government representatives in education.

• Assess sustainability of the “Pro Learn” project after ADRA withdrew support.

• Draw lessons learnt and recommendations for improvements in future projects with respect to planning, implementation, monitoring and evaluation.

Evaluation Methodology - Factors

1. Logical matrix did not correspond to reality2. Audit of the construction done, client interested in

impacts and sustainability. 3. Donor did not have experience with evaluations

and did not have a specific method in mind. 4. Lack of funding to conduct the evaluation. 5. Lack of baseline data. 6. Field mission restricted to 3 weeks

Term

s of

Ref

eren

ce –

Obj

ectiv

es,

scop

e, s

take

hold

ers,

que

stio

ns,

budg

et, s

ched

ule,

out

puts

, use

. Desk study  Interviews

The Most Significant Change

Community conversations

Observations

Pre

limin

ary

findi

ngs

& c

oncl

usio

ns Finaldebriefing of project team and community volunteers

Communication with the Project Partners

Draft evaluation report F

inal

eva

luat

ion

repo

rt

Inception phase

1 month

Field research1 months

Reporting phase

2 months

Initi

al b

riefin

g an

d in

cept

ion

Evaluator Selection

Evaluation Methodology

The Most Significant Change

http://www.mande.co.uk/docs/MSCGuide.pdf

The Most Significant Change

http://www.mande.co.uk/docs/MSCGuide.pdf

The Most Significant Change

http://www.mande.co.uk/docs/MSCGuide.pdf

How was the technique applied

MSC Training of community volunteers

Collection of the MSC stories by volunteers

Selection of the MSC with

community volunteers and project team

MSC drawings by children

How we selected the „MSC story“

What were the findings

What were the findings

Communication of findings• Field mission closed after evaluation finished - key

communication had to be done at the end of the evaluation. • The local community was to a big extent illiterate - community

volunteers were expected to debrief the respective communities.

• Headquarters in India unavailable for debriefing – done by Skype, donor organisation debriefed back in Europe.

• Donor organisation decided to delay the communication and finally did not published the evaluation report.

• Though a short debriefing was held with District Education Office, despite the original plan findings were not officially communicated to school management, teachers and students.

• Stories had already been used in local media.

Utilization of findings• Utilisation was affected by dismissal of the whole project

team at the end of evaluation and lack of capacities of the heasquarters in Delhi.

• The government school staff did not own much the project (teachers´ absenteeism was a local issue) - principals did not attend the final debriefing and did not receive the report.

• Evaluation report did not reach the authorities, though the interviews revealed a good awareness on the project.

• Evaluation report was used only internally in ADRA CR, especially general lessons learnt.

Any questions?Inka Píbilová

[email protected]