CAFÉ Project Evaluation

PowerPoint Presentation

CAF PROJECT EVALUATIONSilva Ferrettiindependent consultant

try and test different approaches and tools for qualitative analysis, which could also be owned and employed by the staff

emphasize participation within the process, by the staff which had accompanied it, and by other stakeholders which were invited to discuss ideas and findings.

be grounded and make the evaluation an opportunity to connect to the ground, and to hear the voices of the project beneficiaries and primary actors rather than just checking adherence to the logframe.

document the methodology, so that it could be shared organization wide

emphasize the communication angle of the evaluation, to create a product which could capture the interest of other people, beyond these directly managing the project

THE EVALUATION SPACE

Evaluation does not (should not!) happen in a desertSTOP AND THINK. TAKE STOCKEvidence, data (e.g. from monitoring)Stories, anecdotesPerceptionsIdeas

Often evaluation proposal ask to tell us allEvaluation is NOT a substitute for monitoringEvaluation is not research4

Evaluation as sense making(this is why a framwork matters)Systematize, learn, communicateProcesses, paths of changeDrivers, obstacles, optionsEmerging frameworks

Evaluation are an opportunity to stop and think, and make sense of change. They create a space for people involved in the programme to think about the programmeIN CONTEXTS WHERE THINKING TIME IS A RARE RESOURCE5

Focus ondeliverables

TimeframesFor change

DevaluingoverheadsQuality?

Sustainability?

Learning?

Alternatives?

AppreciativeHighlight achievementsEmbrace challenges

Complexity orientedhow / connections(not only what / results)

Active listening

In-depth conversationsGenuine curiosity

CommunicationDiverse audiencesDiverse media

VoicesAmplifying the voicesof key stakeholders

LearningCapture innovationRethink, look forward

Deep Deep (thick data)Rather than broad

ParticipativeEmerging practicesOwnership

Open dataSharing evidenceEnable further analysis

8

Active listening

Curiosity

Fishing for ideas9

Active listening

Open interviews

What change?Any example?Why? How?Tell me moreWhat else?

-> A-ah! moment

WITH THE FRAMEWORK IN MIND

Fishing for ideas10

Depth

Big / thick

From: https://medium.com/ethnography-matters/why-big-data-needs-thick-data-b4b3e75e3d7#.icfixhljq MEASURE is not the SILVER BULLET

UNDERSTANDINGMATTERS!

Dynamics,Patters,Key factors

Fishing for ideas11

Depth

Data to analysis:The platform

Fishing for ideas12

Complexityoriented

Exploring relations

https://kumu.io/embed/38cab429768fc637c5480462e35e4c37 ActivitiesStakeholdersCapitals

Complexityoriented

Applying complexity thinking

www.issuelab.org

Exploring relations(mapping, navigating)Complexityoriented

ActivitiesStakeholdersCapitals

Complexityoriented

A framework on which tosystematize evidence

https://prezi.com/3dqtzfd0kwhf/cafe-presentation-with-your-tube-videos/?utm_campaign=share&utm_medium=copy

There are many frameworks. What matters is:The fit with the organization experience: a good framework capture itStability. Better to stick with one an DEEPEN rather than always changing it (including its aspect)A good framework recognize that RESILIENCE IS POWER16

Complexityoriented

Which framework?

Grows with use, but stay simpleSticks, is a practical lensAcknowledge that resilience is power

There are many frameworks. What matters is:The fit with the organization experience: a good framework capture itStability. Better to stick with one an DEEPEN rather than always changing it (including its aspect)A good framework recognize that RESILIENCE IS POWER17

Complexityoriented

The lenses

100 km to.. The distance does not matters, when you do not know the territory. 18

Complexityoriented

. and the maps

479 miles

100 km to.. The distance does not matters, when you do not know the territory. 19

Participation

Who is involved?

Team involvementOutsider feedbackOpen Space Technology

Participation

Who owns it?

Disepersed ownership (e.g. with open space technology: hard to 21

Open data

Evaluation bloghttps://cafeevaluation.wordpress.com/

Open data

Georeferenced evidence

https://www.google.com/maps/d/edit?mid=1741PT13-KpJx4rzfg1z8ACTNHJE

Please use the evidence to derive diverse insights!23

Voice

Why the videos?

Narrative with the voice of peopleCapture soft, human dimensions of resilienceOpen new possibilities for reporting, analysis sharing

Consider how video are collected, and how easier is to learn to capture videos rather than report writing.journalistic reporting (rigorous!)24

Communication

The mother report

SystematizedevidenceWhat else?For whom?By whom?

Different products for different audiences25

Learning

Acquire a method(and linked tools)https://youtu.be/JrnXswYD8sE

Learning

Capturing innovation

Discussing possibilities, ideas and ways forwardA forward-looking orientation

RigorFIT FOR PURPOSE: contextualizing the framework /identifying ideas

REALITY CHECK: timing / scope / existing information

HONEST, OPEN: on limitations / sampling (purposive) / evidence

FACILITATIVE: socially constructed shared understanding/ on-going validation

INNOVATIVE: testing new tools for social analysis

Complexityeasy methodsOpen source softwareAt hand technologyAnalysis / ThinkingFacilitationSpaces for reflection

PracticalityTOOLS Open source software, mobile phoneRETHINKING PRACTICES For example: Why not video reporting? INVESTMENT ON STAFF: exposed then what? Follow up on skillsTOO MUCH TO DIGEST: Layers, recognition of diverse needs : from the STRUCTURE toDIGEST. MAXIMIZING THE ADDED BENEFITS: material for communication, training participation (e.g. monitoring, lessons learnt), advocacy