VVSG Vorming Event Most Significant Change (MSC) Technique 1 St 2010 Introduction to MSC Technique Cecile Kusters, Centre for Development Innovation, Wageningen.

VVSG Vorming EventMost Significant Change (MSC) Technique 1St 2010

Introduction to MSC Technique Cecile Kusters, Centre for Development Innovation, Wageningen UR, The Netherlands

Adapted from:

Rick Davies - MandENEWS Jessica Dart – Clear Horizon

Most Significant Change (MSC) Form of qualitative, participatory M&E

Storytelling technique

Based on ‘stories’ of significant change

Developed by Rick Davies 1996 - Bangladesh

Now used in numerous development programs and in the public sector

MSC – exercise (1)

Each person to think about a story (beginning-middle-end) that illustrates the most significant change that has taken place in the way you view and carry out (or support) monitoring and evaluation as a result of being engaged in the city-to-city linkage (or if this is difficult as a result of your engagement in international development).

Draw a story board (on cards) to develop your story: Draw 4-6 boxes or use 4-6 cards How would you introduce your story? Draw a sketch

in the first box/card. Draw a simple image (stick figures, quick sketch, no detail needed, no words)

Continue rapidly with the outline of the story. Draw these in the boxes/cards.

MSC - exercise (2) Turn to your neighbor and ask:

Tell me how you (the storyteller) first became involved with monitoring and evaluation (especially more qualitative M&E methods or methodologies) and what your current involvement is with these methods/methodologies is

From your point of view, describe a story (beginning-middle-end) that illustrates the most significant change that has taken place in the way you view and carry out (or support) monitoring and evaluation as a result of being engaged in the city-to-city linkage (or if this is difficult as a result of your engagement in international development).

Why was this story significant for you?(Get the details)

Document the MSC story (use the story collection format): Contact details, incl. who documented the story (name, position

location, date) exposure/engagement with more qualitative

methods/methodologies description (who, what, where, when) explanation (why is it significant)

Then….

Then let your neighbor ask the same questions to you.

Then come to a decision about which of the two stories you both think is most significant, and identify why you both think so. You may have a number of reasons.

Do the same with another pair: share each your 1 selected story and select one out of the 2 MSC stories. Document why you chose this as the most significant of the 2 MSC stories.

Do the same with another group of 4 people.

Share your selected stories plenary

Plenary selection of stories

Share: name; title of MSC story; what was the MSC and why was this selected as the most significant of stories; comments about the story

Select: secret balloting – write name of the person and explain why you choose this as the most significant of all selected stories.

Selection chart

Name Title of MSC story

What the MSC is about

Comments about the story

Selection process

Brief reflection after the group work

How do you feel about the process? What have you learned? What did you like? What was difficult? Link to Zanzibar experiences (e.g.

facilitating storytelling and voting)

“Most Significant Changes”

Why stories? People tell stories naturally - indigenous

Stories can deal with complexity and context

People remember stories

Stories can carry hard messages /undiscussables

But stories not known for accuracy/truth

Use of stories in MSC Collection of stories + systematic,

collective interpretation = story telling can be effectively harnessed for participatory monitoring and evaluation and learning

Because interpretations tell another story & process has beneficial outcomes for evaluation utilisation

MSC

Form of qualitative, participatory M&E

Based on ‘stories’ of significant change

Developed by Rick Davies 1996 - Bangladesh

Now used in numerous development programs and in the public sector

MSC

• Creates space for stakeholders to reflect, to make sense of complex changes

• Provides dialogue to help make sense of each other’s values

• Facilitates dynamic dialogue ie. “what do we really want to achieve and how will we produce more of it?”

• Excellent for participatory programs with diverse, complex outcomes & multiple stakeholders

The core of MSC

A question: “In your opinion what was the most significant change

that took place in ….over the … months”• [describe the change and explain why you think it is significant]

Re-iteration of the same kind of question “Which of these SC stories do you think is the most

significant of all?”• [describe the change and explain why you think it is significant]

Explaining MSC in stages

1. Defining Domains of Change2. Define reporting period3. Collecting SC stories4. Selection of collected SC stories5. Feedback of the choices made6. Verification7. Quantification8. Meta-monitoring and secondary analysis9. Re-settings of MSC system

Before you begin – questions for fit and purpose

1. Why have you decided to use MSC?2. In what ways does your programme lend

itself to MSC?3. How will it complement your existing M&E

system?4. What are the main benefits you hope to

get from using MSC?5. Is there management support and/or an

organisational learning culture?

1. Defining “domains”

Opposite of SMART indicators? Like newspaper sections: sports, finance,

leisure, business, etc Defined by how people use them

Examples: “changes in peoples’ lives” “changes in relationships with our partners” “changes in government policy on HIV/AIDS”

Defining domains…

Not essential but Can help structure the selection process Can help focus on goals of concern Their use tells us how what goals mean to

participants

Options Open window domain Negative changes domain

Zanzibar case

‘Change in people’s lives as a result of being engaged in the program’ (agriculture and livestock)

Example “Before training my cassava tubers were very thin, light and small in size. After training, cassava tubers are big, heavy and many per plant. Banana and potatoes also are bigger. Indeed my income has increased to the extent that I have started building a new house and I am planning to buy a milking cow”.

Example of negative change “First I could use the land of my brother for farming. Now that both he and I are engaged in the program (farmer-field-school) he wants his land back so he can practice his new knowledge and skills”

2. Set the reporting period

“In your opinion what was the most significant change that took place in ….over the … months

Period used by NGOs varies from 2 weekly, to monthly, to three monthly, and yearly. Three monthly is most common

Time demands on staff is the main constraint on frequency

Zanzibar case

Annually, spread over the year.

3. Collecting SC stories

From those closest to the event’s of concern. But do not exploit people’s unpaid time

Basic format: Description (who, what, where, when) Explanation (why is it significant) Who documented the story (name, position

location, date) Option: Recommendation

Collecting SC stories…

Reminder: Key parts of the question “Looking back over the last month…” “…what do you think was…” “…the most significant…” “…change…” “…in the quality of people’s lives…” “…in this community?”

Zanzibar case

Story collection format

Zanzibar case- overview field work

1. Introduction 2. Name game3. Explain the

process

4. Farmers develop story board for

MSC story

5. Farmers tell MSC stories. Confidentiality;

involvement; MSC; reason; link to progr;

Film stories

6. Farmers select the most significant

MSC storyDocument the selection

process

7. Develop a story board with farmers for the selected SC

story

8. Film the selected story (being told

again, integrating comments)

9. Take the cutaway shots

10. Review the footage with

farmers

11. Edit the movie with farmers

12. View the 4 selected SC stories and select one

story as the most significant.

Document the selection process

13. Reflect on the process with

farmers

4. Selecting SC stories

S

A G M

B C D E F H I J K L N O P Q R

Funder meeting

State meetings

Region 1 Region 2 Region 3 Region 4

Story tellers

feedbackflow ofstories

Selecting SC stories…

Task is to read through and identify the most significant of all the submitted SC stories. Take one domain at a time

Need to decide who to involve: story providers, their superiors, their peers,..

Need to decide whether to predefine selection criteria, or let them emerge through discussion of SC stories

Selecting SC stories…

Must (not optional) Document what SC story was selected Why it was selected Process used to make the selection

• Participants• Their preferences

[Subjectivity is made accountable through transparency]

Zanzibar case

- -

5 Farmers select 1 story

District: 3 groups each one story. Select one out of 3 stories

1 farmers’ group selects 1 out of 2 stories

National: 1 story form each of the 9 district. Select 1 out of 9.

5. Feedback

To immediate providers of SC stories, on what was selected, why selected and process used

Enables adjustment of focus of MSC next time around

A motivational factor

Weakest point in all M&E systems, including MSC

Zanzibar case

Feedback: from ASFT (Agric Services Facilitation Team) with national farmer forum to DFT (distrcif Facilitation Team) & district farmer forum to FFS (Farmer Field Schools). Also directly from district level back to farmers once a story has been selected.

Resource centre can be used for sharing. Representatives at district level can provide the feedback to farmers. Quarterly newsletters for farmers can also be used.

6. Verification of SC stories

What Factual content & interpretation of facts

Why Encourages some discipline in reporting Enables elaboration and further learning

When When SC story first enters system When selected as MS of all SC When SC stories are publicly used

Zanzibar case

All selected stories will be verified

Verify: the source of the story; the most significant change; reasons for choosing this as most significant of changes for this person; comments on the story itself.

Who: Storyteller and FFS facilitators.

When: verification at farmers’ level – after selecting 1 per group. It has to be verified before going to the next level. ‘

Visit the family of the story teller, have a discussion around the most significant change indicated by the farmer and the story behind it. Capture photos, video of e.g. the new house, the chickens.

7. Quantification

Within the SC story Number of people, events, etc involved

As once–off follow-up to SC story How many other cases like this known

Within meta-monitoring (see next) How many other SC stories like this

8. Meta-monitoring and secondary analysis

Keep all SC stories on record

Meta-monitoring (Recommended) of Changes in numbers of SC stories, who provides them,

whose SC stories are selected, changes in percentage of negative stories

Secondary analysis (Optional) by Categorising and counting of types of changes reported,

and types of explanations given, at different levels

Zanzibar case on secondary analysis and quantification Will feed into the M&E system – link to logframe/theory of

change.

Add domains to the story collection format. The facilitator can tick the most significant change in one of these domains. The domains include: income, food quality, food quantity, housing, transport, health, education, decision making, leadership. Additional domains to be added by the district (define these before starting the whole process) plus ‘open’ domains e.g. to capture negative changes.

Combine with the selection process at district and national level: quantifying the number of stories on e.g. improved income as the most significant change.

A format is to be developed for the facilitator to fill in the number of stories per domain.

Should be in line with selection process at district and national level.

9. Re-setting of MSC process

Frequency of reporting

Definition of domains to use

Who sorts SC stories into domains

Selection process design: participants & process used

Feedback and follow up

Where to use MSC

Not as a stand-alone method Alongside indicator based systems To identify unexpected changes To engage people in analysis of change To involve a wide range of people To focus on outcomes rather than outputs

Questions about starting

1. How will you get ‘buy in’ from the people who will be involved in creating/selecting SC stories?

2. How will you expose people to MSC – what training, if any, is needed?

3. Where can you begin – is there a small pilot that you can test first?

4. Who are the best people to capture the first SC stories from?

Finding out more about MSC

Original MSC paper (n’th version) is at http://www.mande.co.uk/docs/ccdb.htm

MSC Mailing list is at http://groups.yahoo.com/group/mostsignificantchanges

Rick Davies at [email protected]

Combining MSC with PV?

Participatory Video – see also http://insightshare.org/browse/category/monitoring-evaluation

It takes training and commitment to engage in MSC-PV

Webbased support is needed to ensure that videos with stories are shared. Sharing can be done easily e.g. youtube or bliptv .

Important that it al fits within an overall approach for monitoring and evaluation!

© Wageningen UR

Thank you for your attention!