1 of 19Evaluating an Information ProjectDefining Content
© FAO 2005
IMARK Investing in Information for Development
Evaluating an Information Project
Defining Content
2 of 19Evaluating an Information ProjectDefining Content
Learning ObjectivesLearning Objectives
At the end of this lesson you should be able to:
use a Logic Model to identify the if-then causal relationships in your project;
understand how this Logic Model can be used to decide what you want to measure.
3 of 19Evaluating an Information ProjectDefining Content
IntroductionIntroduction
In order to examine the issues involved in the planning of an evaluation, we are following a scenario: Dr Kumar, the Director of publications at SMAU University, must evaluate a newsletter.
Dr Kumar and his colleagues have held the first Evaluation Management Committee meeting, to reach an agreement on the focus of the evaluation.
The EMC conducted a preliminary analysis of the RICE NEWS project, examining Needs, Beneficiaries, Activities, Outcomes.
Also, a SWOT analysis identified potential risks and benefits of the evaluation.
The next step will be to define what to measure, and how.
4 of 19Evaluating an Information ProjectDefining Content
Deciding what to measureDeciding what to measure
During the EMC meeting, we agreed that an impact of RICE NEWS is “more farmer knowledge about fertilizer applications”.
How can we measure that? Should we ask how many farmers have read RICE NEWS?
Or how many farmers actually knew more after reading it?
5 of 19Evaluating an Information ProjectDefining Content
Deciding what to measureDeciding what to measure
It is important to decide first what to measure, and then how. Therefore:
1. agree on what issues to cover in the evaluation (e.g. farmer knowledge)
2. agree on what data to collect and how to do it.
...we can use something called a “Logic Model”..
6 of 19Evaluating an Information ProjectDefining Content
What is a Logic ModelWhat is a Logic Model
A Logic Model is a tool that helps understand the if-then relationships in our project.
We can use it to understand “what leads to what”...
7 of 19Evaluating an Information ProjectDefining Content
What is a Logic ModelWhat is a Logic Model
What inputs in the project have led to what activities?
What activities have led to what outputs?
What outputs have led to what impacts (or outcomes)?
8 of 19Evaluating an Information ProjectDefining Content
The Usefulness of a Logic ModelThe Usefulness of a Logic Model
These are some of the questions that a Logic Model can answer:
Are we clear as what the project was supposed to achieve?
What do we really want to measure? Are we focusing on the right things?
Are we likely to be able to collect good data about the issues we want to include in the evaluation?
9 of 19Evaluating an Information ProjectDefining Content
Analyzing the if-then relationships in the projectAnalyzing the if-then relationships in the project
To build a Logic Model, start by preparing some worksheets:
•the first one deals with the relationship between inputs and activities;
•the second with activities and outputs; and
•the third with outputs and outcomes.
10 of 19Evaluating an Information ProjectDefining Content
Analyzing the if-then relationships in the projectAnalyzing the if-then relationships in the project
Let’s take the first worksheet: inputs and activities.
What resources have been used for RICE NEWS?
What activity has each input contributed to?
11 of 19Evaluating an Information ProjectDefining Content
Creating your own logic modelCreating your own logic model
Let’s think about our resources, or inputs, one at the time. Let’s start
with funds coming from the Ministry of Agriculture...
...we used that money to buy some equipment for the production of RICE NEWS
12 of 19Evaluating an Information ProjectDefining Content
Creating your own logic modelCreating your own logic model
This is the first table: inputs and activities.
SOURCE TYPE OF INPUTS CONTRIBUTION TO WHICH ACTIVITIES?
Ministry of Agriculture Funds Purchase of publishing equipment
Mikuni State Agricultural University
Publication professionals
The Newsletter production process
Mikuni State Agricultural University
Scientists Research
The extension department
Extension agents The Newsletter distribution
13 of 19Evaluating an Information ProjectDefining Content
Creating your own logic modelCreating your own logic model
This first table is useful, to:
evaluate if the RICE NEWS project has been organized efficiently (a “performance” evaluation);
focus more on “impact”, and see how inputs and activities have contributed to outcomes. It will be necessary to use this first table, as well as the next two.
14 of 19Evaluating an Information ProjectDefining Content
Creating your own logic modelCreating your own logic model
Now the second table.
For each activity noted in the first table, let’s ask:
What has been the main output?
Who have been the main beneficiaries of that output?
15 of 19Evaluating an Information ProjectDefining Content
Creating your own logic modelCreating your own logic model
Here's the second table: activities and outputs.
ACTIVITIES OUTPUTS BENEFICIARIES
Research. Articles submitted to RICE NEWS.
Articles accepted.
The UniversityScientists
The Newsletter production process.
Number of trained staff.Participating students.
The Publication Department.Students.
Commercial Firms.
Purchase of publishing equipment.
Number of PCs, printers, presses etc.
The University.The Publication Department.
The Newsletter distribution.
Number of copies distributed.
Extension agents.Farmers.
Agribusiness.
16 of 19Evaluating an Information ProjectDefining Content
Creating your own logic modelCreating your own logic model
And finally, the third table. Outputs and Intended Impacts (or outcomes):
OUTPUTS INTENDED IMPACTS
Articles submitted to RICE NEWS.
Articles accepted.
More relevant research.More scientists promotions.
Number of trained staff.Participating students.
More farmer knowledge of new developments in rice
production practices.
Number of PCs, printers, presses etc.
More cost-efficiency in the production of RICE NEWS
Number of copies distributed
Increased farmer commitment to put
knowledge into practice.
17 of 19Evaluating an Information ProjectDefining Content
Creating your own logic modelCreating your own logic model
A Logic Model for planning can help separate activities, from outputs, from impacts (or outcomes). By doing so, you can decide whether the main focus of your evaluation will be on performance, or impact, or both.
18 of 19Evaluating an Information ProjectDefining Content
Creating your own logic modelCreating your own logic model
You have your Logic Model. You are ready to use this Logic Model as a basis for formulating your detailed evaluation questions.
19 of 19Evaluating an Information ProjectDefining Content
Summary
To agree on the detailed content of an evaluation, it is useful to build a “Logic Model”.
A Logic Model is a tool that helps understand the if-then relationships in a project.
A Logic Model can therefore help focus on what you really want to measure.
The major benefit of using a Logic Model for planning your evaluation is to separate activities, from outputs, from impacts (or outcomes).
It can also help evaluate if the objectives of each of these areas have been met.