This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
sMrab;RtYtBinitükarGnuvtþn_kargarrbs;RkumRbwkSaXuMenAeLarrbs;kmµviFI sila . karRbmUlTinñn½ysMrab;ÉksarpÞúkTinñn½yCak;lak;TaMgenH RtUv)aneKrMBwgfanwgRtUv)anerobcM
Community Based Rural Development Project (CBRDP) Rural Development Program (RDP)
Kampong Thom & Kampot Cambodia
Monitoring & Evaluation Manual
Supported by Gesellschaft für Technische Zusammenarbeit (GTZ)
International Fund for Agricultural Development (IFAD) German Development Service (DED)
Internal Paper 27 Modification 2
March 2006
Community Based Rural Development Project (CBRDP) Rural Development Program (RDP)
Kampong Thom & Kampot Cambodia
Monitoring & Evaluation Manual
A guide for M&E officers and –focal points
Supported by Gesellschaft für Technische Zusammenarbeit (GTZ)
International Fund for Agricultural Development (IFAD) German Development Service (DED)
Internal Paper 27 Modification 2
March 2006
Abbreviations ADESS Agricultural Development Support to Seila AusAid Australian Agency for International Development CAAEP Cambodia-Australian Agricultural Extension Project CAU Contract Administration Unit of the ExCom CBRDP Community Based Rural Development Project CBO Community Based Organization CC Commune Council CD Community Development CDA Chief District Agriculture CMDG Cambodian Millennium Development Goals CMT Component Management Team DAE Department of Agricultural Extension DCD team District Community Development team DFT District Facilitators Team DIW District Integration Workshop DOT District Outreach Team DRD Department of Rural Development ExCom Executive Committee [of the PRDC] FP Farmer Promoter GTZ Deutsche Gesellschaft für Technische Zusammenarbeit HRD Human Resources Development IFAD International Fund for Agricultural Development LAU Local Administration Unit of the ExCom LTC Local Technical Committee (includes VTC) MAFF Ministry of Agriculture, Forestry and Fisheries M&E Monitoring & Evaluation M&E CT M&E Core Team MVHL Most Vulnerable Household List NGO Non-Governmental Organisation NRM Natural Resource Management OA Office of Agronomy OAE Office of Agricultural Extension PDA Provincial Department of Agriculture PDP Provincial Development Program Kampong Thom PLG Partnership for Local Governance PRA Participatory Rural Appraisal PRASAC Support Programme for the Agricultural Sector in Cambodia PRDC Provincial Rural Development Committee PSU Project Support Unit PTST Provincial Technical Support Team RC Result Chain RBM Result Based Monitoring RDP Rural Development Program Kampong Thom & Kampot Seila A Khmer word meaning ‘foundation stone’ and the name of a RGOC program for
poverty reduction through improved local governance TA Technical Assistance TSO Technical Support Officer TSU Technical Support Unit UNOPS United Nations Office for Project Services VLA Village Livestock Agent VTC Village Technical Committee WUC Water User Committee WUG Water User Group
i
Foreword
The CBRDP/RDP management is pleased to share with M&E officers and focal points, Technical assistance staff (TA) and others staff members of CBRDP/ RDP this M&E Manual: A guide for M&E officers- and focal points. This manual aims to provide readers with the basic concepts behind Result-Based Management, an M&E system supported with M&E plan, methods and tools for data collection, data storage and reporting that will help to better articulate whatever the project, program or their unit/ components wants to achieve.
CBRDP/RDP has been pioneering monitoring activities for institutionalizing this manual within components, sub-components, ExCom units for the last few years. This has taken three years of consistent and unrelenting effort in applying and reviewing of methods, tools and practices, introducing new understanding of result chains and ensuring project implementation are much more participatory and transparent by involving M&E results.
We hope that all these efforts will help to contribute to develop results that are more sustainable and meaningful for the people who benefit from the project. This is not easy task and much works remains to be done and could be only fulfilled as long as decision makers use the M&E results effectively for that purpose. Therefore, this manual has to be used and applied. The best M&E system is not good, if it is not implemented and followed and, if it is implemented well, management has to use the data, otherwise it remains an exercise in itself, just is a manual only, and it doesn't achieve what we want.
The manual evolves from support of different consultants in 2002 and 2003 and the experiences of the M&E task force to date. It is designed in a user-friendly manner that can be easily understood in various components' contexts. It is our hope that the manual will become a useful guideline rather than a reference when component staff members and TA of components are looking for guidance to design, manage and monitor their activities and results or report on results achieved.
This manual has been a collaborative effort by many people and we would like to take this opportunity to acknowledge their contribution. Within CBRDP/RDP, our thanks to Dr. Georg Deichert, Advisor for Agriculture, NRM and M&E, who spearheaded this effort. To Mr. Ten Samrach, Advisor for Training and M&E, who facilitated the process of manual development, compiled everything together and kept track of all changes both English and Khmer version and finalized the manual. To Mr. Song Sophal, Planning and M&E officer of PSU/MRD, Mr. Hy Vichet, M&E Program Assistant in Kampot and Mr. Lean Kunlam, M&E Program Assistant in Kampong Thom, who provided their valuable inputs and practical experiences of all M&E evolutions in PSU and at the provinces.
We are grateful to Dr. Eberhard Bauer, M&E consultant, who developed jointly with us the M&E approach, indicators and the respective monitoring methods and tools. He assessed the M&E plans and provided training and backstopping to all M&E task force members and M&E focal points.
This manual could not have been prepared without the generous contribution of those individuals. This M&E manual will undoubtedly continue to evolve over time. We welcome any experiences you may have in the manual or suggestions and ideas you may have in improving the handbook. We have attached a feedback sheet in the annexed section for this purpose.
ii
We hope that the manual will not only useful and practical, but also enjoyable to read.
Phnom Penh, March 2006
Dr. Angelika Fleddermann Senior Advisor/Teamleader Rural Development Program (GTZ-CBRDP) Kampot/Kampong Thom
Dr. Chan Darong Deputy Director General Director of PSU/CBRDP/IFAD Ministry of Rural Development
iii
Table of Content
Abbreviations ......................................................................................................................... i Foreword................................................................................................................................ ii Table of Content................................................................................................................... iv Overview ................................................................................................................................ 1 Chapter 1: Introduction to the M&E Framework of the CBRDP/RDP ............................... 4
1.1. Objectives and target group of the M&E Manual ......................................................... 4 1.2. Requirements of the CBRDP/RDP M&E system ......................................................... 5 1.3. Relation to the Seila Planning and M&E framework .................................................... 5 1.4. The planning basis for M&E......................................................................................... 6
1.4.1. The logical framework ........................................................................................... 6 1.4.2. The annual plan of operation ................................................................................ 6 1.4.3. Monthly work plans ............................................................................................... 6 1.4.4. Monitoring & Evaluation Plan ................................................................................ 6 1.4.5. Monitoring and Evaluation work plan (integrated into plan of operation) .............. 7
1.5. M&E levels and relation to the IFAD manual ............................................................... 7 1.5.1. Level 1: Activity monitoring (Chapter 2) ................................................................ 7 1.5.2. Level 2: Output monitoring (Chapter 3)................................................................. 8 1.5.3. Level 3: Monitoring of Impacts (Chapter 4) ........................................................... 8
1.6. M&E management in CBRDP/RDP ............................................................................. 9 1.6.1. Monthly M&E meeting in each province................................................................ 9 1.6.2. Quarterly M&E meeting on national level.............................................................. 9 1.6.3. Permanent management backstopping on M&E................................................... 9
Chapter 2: Monitoring Level 1: Monitoring of Activities ................................................. 10
2.1. Systematic Work Planning as Basis for Activity Monitoring ....................................... 10 2.2. Tasks for M&E Focal Points and Managers in Activity Monitoring............................. 14
3.1. M&E Plan ‘Decentralized Management’..................................................................... 17 3.2. M&E Plan ‘Enabling Commune Councils’ .................................................................. 19 3.3. M&E Plan ‘Community Development and Gender Mainstreaming’............................ 23 3.4. M&E Plan ‘Infrastructure incl. Land Titling and Irrigation’ .......................................... 32 3.5. M&E Plan ‘Agriculture Extension’............................................................................... 39 3.6. M&E Plan ‘Farmer Training & Demonstrations’ ......................................................... 44
Chapter 4: Monitoring Level 3: Outcomes and Impacts.................................................. 50 Chapter 5: Monitoring Methods and Tools....................................................................... 56
iv
5.1. Monitoring of Human Resources................................................................................ 56 5.1.1. Monitoring quantitative aspects of training and Target Group Events ................ 57 5.1.2. Monitoring of quality of Training and Target Group Events................................. 59 5.1.3. General Staff Assessment and Implementation of Recommendations............... 62
5.2. Monitoring of institutional performance using SWOT................................................. 66 5.2.1. Background to SWOT-Analysis........................................................................... 67 5.2.2. SWOT analysis as the key method for monitoring institutional performance...... 68 5.2.3. Follow-up of addressing weaknesses ................................................................. 70
5.3. Monitoring of Micro-Projects and -Processes ............................................................ 70 5.3.1. The ‘Project Book’ ............................................................................................... 72 5.3.2. End-of Project Evaluation.................................................................................... 72
5.4. Monitoring at Beneficiary Level .................................................................................. 75 5.4.1. The Beneficiary Impact Assessment (BIA) Survey ............................................. 76 5.4.2 The Beneficiary Monitoring Workshop ................................................................. 78 5.4.3. Management Information Workshop ................................................................... 80
Chapter 6: Storage, Retrieval and Analysis of Data and Information ............................ 83
6.1. Information on physical progress (quantity) ............................................................... 83 6.1.1. General aspects of a database system............................................................... 83 6.1.2. Overview of CBRDP/RDP data files.................................................................... 85
6.2. Information on quality................................................................................................. 86 Chapter 7: Reporting System ............................................................................................ 90 Background Documents .................................................................................................... 91 Annexes ............................................................................................................................... 92
Annex 1: Revised Logical Framework of the CBRDP................................................93
Figure 1: Result Chain................................................................................................. 2 Figure 2: Overview of Monitoring at Beneficiary Level .............................................. 82
v
Overview
This manual should primarily guide the M&E officers and –focal points of the CBRDP/RDP and its components to fulfil their duties for programme internal monitoring.
The present manual is complete but will undergo regular updating. Methods, tools and formats presently in use by the different components in the two provinces for physical progress monitoring have been streamlined, agreed upon and integrated into the manual. The manual has to be seen as a flexible tool with adjustments to be made on the basis of accumulating experiences during implementation. The manual describes more or less the compulsory part for the M&E system. Each aspect of the M&E systems allows, however, to add optional monitoring and evaluation tools to the system. In this way, the manual has to be seen also as a snap shot (status of that moment) of a continuous learning and improvement process.
The design of the presented M&E system focuses on a “Result Based Monitoring (RBM)” approach, and therefore requires a basic understanding of this approach. RBM means looking at logframe-hierarchies of goals as a “result (or impact) chain (RC)”, whereby the results at the higher impact level (outcomes, direct and indirect benefits) become more the centre of attention compared to the lower levels of the chain (inputs, activities, outputs). An example of a result chain is shown in the figure below. Although RBM has a strong focus on higher level impacts, it should not be misunderstood, that monitoring of inputs and activities becomes obsolete. Input, activity and output monitoring remain important for day-to-day management and steering of a project. The challenge for the M&E system for higher level impact monitoring is to develop appropriate and effective tools to measure direct and indirect benefits, and to allow with reasonable justification to attribute these measurements to the project interventions.
Another important change with the RBM is the different understanding of terms used. While the term “result” was used synonymously with “output”, the understanding of “results” in result chains is much broader and includes higher level impacts. In this M&E manual, the terminology used is based on the RBM approach.
Another clarification of terms refers to “forms” and “formats”. In this M&E manual “forms” are understood as actual hard copy paper forms, which can have different designs, lay-outs or formats.
1
Figure 1: Result Chain
Long-term impacts (indirect benefits)
Medium-term impacts
Outcomes (direct benefits) (medium-term results)
Use of outputs (short-term results)
Outputs
Activities
Inputs
Impa
cts
Res
ults
Man
agem
ent r
espo
nsib
le
Attribution Gap: change caused by other factors
The manual is structured in to 7 Chapters:
• Chapters 1 explains the objectives and target groups of the manual, the requirements of the manual, the relation to the Seila M&E framework, the planning basis for monitoring within the CBRDP/RDP, the different M&E levels and the connection to the IFAD M&E manual and agreements made concerning M&E management within CBRDP/RDP.
• Chapter 2 describes methodologies for activity monitoring. Activity monitoring is done mostly informally on the basis of regular work planning and serves mainly day-to-day decision making.
• Chapter 3 describes methods and formats for output monitoring which is the core of the manual. ‘Outputs are the direct products or services delivered by the project’ (IFAD, 2002); e.g. CCs enabled to do proper planning, LTCs enabled to maintain wells, farmers enabled to use better farming practices etc.). Output indicators are therefore the key for monitoring what we are going to achieve with all our efforts. Chapter 3 describes for each output indicator of each component methods and formats to be used for monitoring.
2
• Chapter 4 describes impacts, which are ‘changes in the lives of the rural poor or, more specifically, improvements in their well-being’ (IFAD, 2002). Impacts are the direct or indirect consequence of our outputs. This could be target groups adopting our services (do proper commune planning, use wells, use better varieties etc.), products and, with adoption, improve their lives (better health, more income etc.).
• M&E methods and tools which are only relevant for tracing one indicator of one component are described directly in the Sections of Chapter 3 where monitoring of the different components is described. Methods and tools which can be used for tracing several indicators from different components are described in Chapter 5.
• Chapter 6 describes procedures for data storage, analysis and retrieval while
• Chapter 7 closes the manual with reporting procedures.
The M&E officer and focal points of ExCom, the M&E focal points of the different components and the M&E provincial TA of CBRDP/RDP will have to plan all M&E activities annually within the annual operational planning exercises of the CBRDP/RDP. Planning basis are the indicators – mainly on output level – the information needed for this purpose and the methods and tools to be used. This information can be found in the M&E plans attached to each component in Chapter 3.
The M&E focal points of the different components and of the different units within ExCom will have to plan ‘their’ M&E-exercises on the basis of the descriptions of methods and tools in Chapter 3 and 5 and seek assistance from the M&E-officers CAU/ExCom and M&E TA.
The M&E officers CAU/ExCom and the M&E provincial TA will support the M&E focal points of the components and monitor the implementation of planned M&E activities. In addition, the M&E-officer, CAU ExCom will initiate and coordinate overall M&E activities across the components (like the beneficiary impact assessment and beneficiary monitoring etc.).
The M&E national TA will give M&E support in all these activities where necessary and coordinate M&E activities beyond the provincial level.
The reader might notice that this manual does not yet contain M&E tasks to be taken over by the communes themselves. Such tasks are, for the moment, left to the Seila commune M&E system. However, all M&E activities of the CBRDP/RDP at commune level should include the M&E focal points of the commune councils as part of the M&E team.
3
Chapter 1
Introduction to the M&E Framework of the CBRDP/RD
1.1. Objectives and target group of the M&E Manual The purpose of this manual is to describe
• the CBRDP/RDP’s overall Planning and Monitoring framework and
• procedures, methods and tools for monitoring the CBRDP/RDP’s activities, HRD, institutional performance, outputs, use of outputs, outcomes (direct benefits) and impacts (indirect benefits).
The manual is mainly made for the M&E officers and –focal points in ExCom and the different project components in both provinces as a guideline for tracing defined indicators for the different level objectives (= different levels in result chain) above. The tasks of the different M&E officers within CBRDP/RDP are as follows:
• The M&E officer and two M&E focal points within the CAU/ExCom who are responsible for monitoring of contract and overall project implementation and assist in the preparation of reports to ExCom and PSU. This Sub-section is mainly responsible for monitoring indicators and milestones related to output 1 in the CBRDP/RDP planning framework (see section 3.1 and annex 1).
• The M&E focal point within the LAU/ExCom is responsible for monitoring the implementation of the decentralized regulatory framework on commune level (particularly the commune planning process, legality issues etc.). The M&E focal point is responsible for monitoring indicators and milestones related to output 2 in the CBRDP/RDP planning framework (see section 3.2 and annex 1).
• Each project component has one or two M&E focal points from within the institutional structure of implementing partners in Kampot and Kampong Thom respectively. These M&E focal points are responsible for monitoring indicators and milestones related to their respective component (‘Community Development/Gender Mainstreaming’ for Output 3, ‘Infrastructure’/ ‘Irrigation’/‘Land Titling’ for Output 4, and ‘Agriculture’ for Outputs 5 and 6; see sections 3.3 to 3.6 and annex 1).
In addition to guiding M&E officers and focal points, the manual should also give component managers the necessary insight into Monitoring & Evaluation and encourage management to recognize the potential value of monitoring information for decision making, steering implementation and strategic planning.
4
1.2. Requirements of the CBRDP/RDP M&E system The M&E system for CBRDP/RDP should first and foremost serve the information needs of the CBRDP/RDP. The M&E system will be designed not only to work until the CBRDP/RDP ends but should, from the beginning, as much as possible, be designed in a way that local governments on all levels can continue using the same system for their own development planning and implementation thereafter as well. The following requirements have to be taken into consideration when defining the different system elements:
• The system should satisfy all information requirements for management decision making and reporting of the CBRDP/RDP and should be tracking all important indicators specified in the logical framework (and result chain) and the contracts between the PRDC/ExCom and the implementing agencies.
• As these indicators do reflect national priorities for decentralized rural development anyway (CBRDP/RDP is a government project), the system will automatically suit the M&E requirements of implementing agencies (Line Departments) for their long term M&E needs as well.
• The system should be institutionalised into above mentioned implementing agencies (Government Departments) from the beginning of its design. PRDC/ExCom and line department management staff should be the ‘owners’ of the system and its procedures from the beginning. TAs should understand their role as advisors.
• The system should be as simple as possible and build on existing M&E systems as well as on local communication habits (e.g. verbal reporting should not necessarily be replaced by written reporting if verbal reports do yield sufficiently reliable information).
• The system should not aim for technical sophistication in sampling procedures and survey designs. Rather, one should aim at simple but purposeful sampling procedures and simple survey designs and be satisfied with having approximate and relatively reliable results, which are needed for decision making. Sometimes M&E can be better and more efficient if the findings are approximately wrong rather than precisely right.
• The system should yield a minimum of gender specific and poverty related information. Poverty related information is not only a high priority for donors, but also a high priority for measuring the CMDG.
• And finally, M&E results should be presented in contents and method in a way ‘attractive’ for decision makers and managers at different levels. Boring results or results presented in a boring way are less likely to be taken up for decision making than ‘highlights’ for the ‘users’.
Even if these characteristics aim at simplifying the system as much as possible, there will be heavy demand for capacity building in M&E. System's development and initial implementation should, therefore, from the beginning, be combined with intensive training of relevant officers (ExCom, M&E-focal points).
In addition, close, competent and regular backstopping of relevant M&E officers/focal points by national and international TA will be necessary during the years to come (such backstopping is anyway foreseen to be provided by GTZ personnel on site).
1.3. Relation to the Seila Planning and M&E framework Seila is a national ‘program for poverty reduction through improved local governance’ covering, since recently, all provinces of Cambodia. Therefore, also the two target provinces of the CBRDP/RDP, Kampot and Kampong Thom, will follow Seila procedures in the fields
5
where both, CBRDP/RDP and Seila are active. These are mainly the provincial and commune level development planning processes within the Seila-implementation.
Seila, however, covers only part of the CBRDP/RDP planning framework (basically outputs 1: ‘Decentralized Planning’; outputs 2: ‘Commune Planning Process’ and output 4: ‘Infrastructure’). The CBRDP-components ‘Community Development’ and ‘Agriculture’ are not covered within Seila. In addition, financial and contracting procedures of the CBRDP/RDP are partly different to the Seila procedures. And finally, some of the methods and tools developed by Seila for monitoring purposes (e.g. contract data base and project information data base) are not compatible with CBRDP/RDP requirements and would have to be adjusted in a time consuming process.
Therefore, CBRDP/RDP management decided to develop an own M&E system tailored to the needs of the CBRDP/RDP but closely related to the Seila procedures wherever feasible. The CBRDP system will also yield the information needed by Seila from ExCom and the different programme components. Close cooperation will be kept with Seila in order to cooperate with respect to M&E wherever useful.
1.4. The planning basis for M&E The CBRDP/RDP planning frame has three main elements:
1.4.1. The logical framework The logical framework describes the main objectives of the CBRDP/RDP for the project’s lifetime and the indicators, which specify the objectives in more detail (Annex 1 presents the revised logical framework for the CBRDP/RDP). The logical framework is the overall strategy of the CBRDP/RDP and the basis for the annual plan of operation.
1.4.2. The annual plan of operation The plan of operation is developed annually on the basis of the logical framework and monitoring results/reflection of the project components. The component plan of operations are approved by PSU and technical assistance and merged into a project plan of operation.
1.4.3. Monthly work plans While the annual plan of operation is compulsory, different project components use different kinds of work plans (not compulsory). Community Development in Kampot uses monthly work planning while Infrastructure in Kampot uses fortnightly work planning for their day-to-day operations and activity monitoring. Other components do not use systematic written work planning but discuss field programmes verbally. The need to prepare a monthly work plan is obvious for monitoring, and a standard format for such a work plan will be introduced later in this manual (see section 2.1).
1.4.4. Monitoring & Evaluation Plan The M&E plan specifies information requirements, frequency and methods of data collection for tracing the different indicators. The M&E plan helps M&E officers and focal points to identify the different sources of information and to plan for M&E activities throughout the year.
6
1.4.5. Monitoring and Evaluation work plan (integrated into plan of operation) The M&E work plan is derived from the M&E plan and integrated into the annual plan of operation and not a separate document. The M&E plan assures that the different responsible officers do not forget the initiation of M&E activities at the right point in time.
1.5. M&E levels and relation to the IFAD manual The M&E system proposed here for the CBRDP/RDP is following the basic principles described in the IFAD guide for project M&E (IFAD, 2002) and closely related to the Seila M&E manual (PLG/Seila, 2003) wherever appropriate. As proposed in the IFAD guide, the basis for monitoring are the indicators in the logframe and the M&E matrix (IFAD, 2002, page 5-3), which is called M&E plan in the CBRDP/RDP (and M&E work plan in PLG/Seila, 2003) and has a slightly different form. In addition, in its final version, the CBRDP/RDP M&E plan will also be integrated into the CBRDP/RDP annual plan of operation by adding a time frame and clear responsibilities to the different M&E exercises.
The IFAD guide is general in nature and, therefore, very valuable as a guide for the direction and understanding of M&E in general. Also, the ‘toolbox’ for data collection presented in Annex D of the IFAD guide is a valuable source when looking for appropriate data collection and tools. On the other hand, the IFAD guide can neither be specific and operational enough for the local situation in each country nor for each project.
Therefore, the IFAD M&E guide was used as the general framework and underlying philosophy for the CBRDP/RDP M&E system but with adjustments to the local situation in Cambodia and operationalized for the specific needs of the CBRDP/RDP.
Different terms will be used in this manual. These terms will, therefore, be defined here to make everybody understand what we are talking about. Throughout this manual, three different M&E levels are distinguished. The first level refers to the activities, the second level to outputs & institutional performance of implementing agencies (partner organizations) and the third level to use of outputs, direct and indirect impacts on higher levels.
1.5.1. Level 1: Activity monitoring (Chapter 2) As the term indicates, this level addresses the activity level of the different planning frames. Activity monitoring means to regularly collect information about the progress of the different components and comparing actually implemented activities with planned activities.
The main objective of activity monitoring is to track project progress against planned targets, to detect plan deviations and find the reasons for them, to find out ‘irregularities’ from the data and to follow-up on them.
Activities are usually monitored on the basis of the milestones in the plan of operation. Collecting data for monitoring milestones is mostly done during routine reporting exercises and reporting to decision makers mostly during management meetings, often only verbally.
Information from activity monitoring is primarily of quantitative type, and used for day-to-day decision making and for activity reporting. Often, no written records are kept except information needed for the monitoring of physical achievements within the framework of output monitoring (see Chapter 3). Most of the quantitative information from the activity monitoring are stored in several specially designed Excel data bases.
7
1.5.2. Level 2: Output monitoring (Chapter 3) ‘Outputs are direct products or services delivered by the project’ (IFAD, 2002, pages 2-6), particularly its partner institutions on target group level. Outputs/services/products are e.g. CC members knowledgeable in how to do commune development planning; no. of water wells functioning and LTCs in a position to maintain them; appropriate recommendations advice and inputs are readily available and accessible for farmers, etc. It is important to note, that ‘outputs’ measure more than the successful completion of activities! The main objective of output monitoring is to assess in how far and to what quality the products/services are provided by the project and its implementing agencies (partner organisations).
Collecting data for monitoring outputs is mostly done by using data from activity monitoring for analysis, by special surveys and by comparing the quality of services according to quality criteria.
Information from output monitoring is mainly used for learning lessons for regular re-planning of programs but also for policy planning on national level.
Human resource development and institutional performance are also outputs of the project on intermediary level (implementing partners like line departments) and not on the level of the final target groups (farmers, villagers, commune councils)
The main objective of monitoring human resources development and institutional performance is to assess in how far the support of the project in terms of resources invested in people and institutions for training, advise, equipment given has changed knowledge, skills and behavioural pattern of staff and, consequently, efficiency and effectiveness of implementing institutions. Tracking related indicators is important for the long term sustainability of impacts as a result of project implementation particularly if institutional development is a priority (as with the CBRDP/RDP).
1.5.3. Level 3: Monitoring of Impacts (Chapter 4) IFAD (2002, page 2-6) specifies impacts as ‘behavioural changes…as a results of the project’ (direct or lower level impacts) and ‘changes in the lives of the rural poor…’ (indirect or higher level impacts).
Impact monitoring goes beyond activity and output monitoring, but relies to a large extent on the results of activity and output monitoring. From the planning logic, impacts are the consequences of outputs achieved. Impact monitoring is usually not done regularly but need specific surveys. These surveys can be conducted project internally by project staff or project externally.
The main objective of this M&E level addresses the question of what is now the result of our activities and outputs on the communities when utilizing the outputs.
Direct (lower level) impacts or outcomes are for example that proper commune development plans are developed regularly, water wells are sustainably functioning, farmers’ yields in agriculture increases with newly adopted recommendations. With the Result chain approach, there is often a level just below the outcome level, which describes the “use of the outputs” that leads to the outcome. In the above example, that the people use the water from the wells, or that the farmers adopt the recommended innovative farming practices are typical for the “use of output” level.
Indirect (higher level) impacts are for example that income increases with proper economic investment planning in the commune, incidence of diseases and child mortality decreases
8
with the use of clean water; household income increases with the use of improved farming practices.
Internal impact monitoring usually focuses on the direct impacts and leaves the monitoring of higher level impacts for external evaluations or national surveys to be done in longer intervals. Usually, it is sufficient to trace direct impact (outcomes) for efficient project steering and management.
1.6. M&E management in CBRDP/RDP The following coordinating mechanisms were agreed upon for proper M&E management:
1.6.1. Monthly M&E meeting in each province All M&E focal points, under the chairmanship of the M&E-officer CAU/ExCom will meet monthly. They will discuss progress and problems arising in M&E within the province. The provincial M&E TA, national M&E TA, the GTZ provincial coordinator as well as the M&E officer of the PSU will participate in this meeting and give advice as necessary.
1.6.2. Quarterly M&E meeting on national level All M&E officers, M&E focal points and M&E TA (provincial, national) will meet quarterly under the chairmanship of the PSU director. They will discuss overall progress and problems arising between the province and other important matters concerning M&E. The GTZ senior advisor as well as the two GTZ provincial coordinators and selected other TA will participate in this meeting.
1.6.3. Permanent management backstopping on M&E The M&E national TA will have a key function in advising PSU and GTZ programme management in all matters arising from M&E (see Terms of Reference in Annex 2).
9
Chapter 2
Monitoring Level 1: Monitoring of Activities
Activity monitoring is done by tracing activities/milestones defined in the plan of operation and in more detail in the work plans. Deviations from the plans are regularly reported by the M&E officers to managers who, in turn, use this information for regular decision making and steering of project implementation. Often, the results of activity monitoring are reported to managers informally for discussion of deviations and decision making during meetings.
Activity monitoring serves mainly two purposes:
• Activity monitoring gives feedback to managers for day-to-day decision making. Main questions are: Are the activities carried out as planned? If not why not? What immediate decisions are necessary to make the programme ‘run smoothly’?
• Activity monitoring provides important basic data for output monitoring (e.g. number of men and women trained, number of commune councils advised, number of wells built, number and composition (gender) of LTC established, amount of improved seed rice distributed to how many men and women farmers etc.).
As mentioned above, the planning and conduct of activities should be routine procedures in any organisations, especially for those involved in development activities. Even though the implementation of activities is running smoothly in an organisation, activity monitoring forms an essential part of the M&E process, also when it is performed routinely without much special attention.
Activity monitoring is particularly important in situations when implementing organisations still have problems in simple day-to-day management (like the use of meetings for information exchange and decision making, delegation of responsibilities etc.). This is the case with most of the CBRDP/RDP partner organizations. Therefore, activity monitoring will receive high importance within the M&E system of the CBRDP/RDP with the main aim to more smoothly manage the different components of the project.
2.1. Systematic Work Planning as Basis for Activity Monitoring Although the project has a plan of operation for each component per province per calendar year, systematic written work planning needs to be followed as well. Although monthly activities and milestones are planned during meetings and reporting is often done verbally during follow-up meetings, there is a real need also systematically monitor activities (milestones). From experiences made in Kampot it is, therefore, proposed that all components in future develop written monthly work plans. The following format is proposed for monthly work planning and reporting (see Table 1).
10
11
In addition to this format another very simple but useful format for daily work planning and re-call for individual officers and field teams (see Table 2) should be used. This second format helps the individual officers or field teams to keep track of their own activities on a day-to-day basis and allow them easy reporting at the end of the month on the basis of the daily re-calls. In addition, it allows superiors to make un-announced supervisory visits to the sites where individuals and field teams are supposed to work at a certain point in time. Daily re-call on these formats would, of course, not mean elaborate reporting but a 5-minutes recall at the end of the day on achievements and deviations in e.g. 10 key words. This time is very likely more than compensated for the sake of easy reporting at the end of the month!
Table 1: Monthly Work Plan and Report for teams on district level (all components)1
Month: Indicator/Milestone Week of the
month
PO Number and Activity
Planned Achieved Place Resp 1 2 3 4 5 Remarks Explain plan deviations
1 The white fields would be used for planning at the beginning of the month, while the shaded fields would be empty during the month and used for reporting purposes at the end of the month on the basis of the results from Table 2
12
13
Table 2: Daily Work Plan and Re-Call Format for Individuals2
2 The white fields would be used for planning at the beginning of a month/week while the shaded fields would be used for daily recall. At the end of the month, the individual officer would report on the basis of his/her daily summaries. Small team would sit together and report on their daily summaries
It is proposed that all components try the suitability of these formats for their monthly planning and reporting and for day-to-day management. Senior officers of ‘Community Development’ and ‘Roads & Water Supply’, Kampot, are resource persons and would be the ideal trainers for a – say – one day training course on the appropriate use of such procedures and formats.
2.2. Tasks for M&E Focal Points and Managers in Activity Monitoring As mentioned above, activity/milestone monitoring would be done by the M&E Focal Points for each component on a regular basis. The M&E focal points have the following tasks:
• They will define, on the basis of the plan of operation, the information needed from all implementing officers, the time frame for information delivery and the mode of delivery (Should the officers report on progress during a management meeting or component meeting? If yes, in which form and what should be reported? Should reports be verbal or written? What specific information should be reported verbally, what information in writing? , etc.)
• They will, in advance of key management meetings, summarize certain information which is not reported by the implementing officers themselves and present the information either during the management meeting or before to the manager on the request of the latter.
• They will, if necessary, report proactively special events to the manager if management decision making is necessary immediately (e.g. outbreak of an epidemic pest, urgent staff issues etc.)
• They will summarize all important information for reporting to other authorities according to standard reporting procedures
• They will document management decisions made and secure follow-up of decision making (are decisions really implemented?)
14
Chapter 3
Monitoring Level 2: Outputs
As mentioned in Section 1.5 ‘outputs are the direct products or services delivered by the project’. The CBRDP is supposed to ‘produce’ 6 outputs, each of them specified by a set of particular indicators. There are additional indicators to be monitored from the RDP program design, also for non-CBRDP outputs. Monitoring of outputs means tracing all indicators on output level, quantitatively and qualitatively.
Output monitoring is the key issue in project monitoring as the outputs are the preconditions for reaching the planned impacts. Most of project’s monitoring efforts are, therefore, spent on output level. With the result chain approach becoming stronger, this focus is shifting more to the “use of output” and outcome level.
Chapter 3 presents a number of methodologies for systematic output monitoring. The Chapter is divided into 6 Sections; one Section for each of the outputs. Each output refers either to one component or a sub-component.
Each Section is structured in the following way:
The first page presents the M&E plan with the following information:
• First and second column: show the source of each indicator and its level in the project planning matrix (CBRDP log-frame or RDP offer) and using codes (OP for Output; OC for outcome; OB for objective; and SG for strategic goal.
• Third column: indicators (full text version) including quantitative and quality measurements. The quantitative information indicates, which type of data are required from the activity monitoring and reporting.
• Fourth column: Details (source and value) on base line information. This is especially necessary for indicators expressed in percentages.
• Fifth column: quality of physical achievements. This is the most important column for monitoring because the quality of our outputs (products and services) is the key issue for its contribution to our next higher levels of objectives (use of outputs, outcome and indirect impact). High numbers of physical achievements (e.g. commune development plans, wells, field demonstrations, input availability, etc.) are useless if the quality is not good enough (e.g. useless plans, broken down wells, wrong demonstrations, low quality inputs etc.). In most cases, the quality of outputs (services, products) demands the key efforts in output monitoring. Therefore, the main focus of this manual is related to monitoring the quality of our services and products.
15
16
• Sixth column: specifies the frequency of monitoring exercises related to each indicator
• Seventh column: indicates where we can find the methods and tools to trace each indicator. Methods and tools, which are commonly to be used for several or even all components, are described in Chapter 5 while methods and tools used within one particular component only are described directly after the M&E plan of the respective component (see below).
All following pages describe methods and tools to be used for monitoring each of the indicators only relevant for the particular component.
Responsibilities and a time frame for monitoring activities are not specified in the M&E plan but in the annual plan of operation.
M&E information which has to be collected by the respective components internally, is indicated in bold and italics; all other information will either be collected by others (e.g. trainers in the case of training, ExCom in the case of staff assessment, survey team in the case of the beneficiary impact assessments, facilitators in the case of beneficiary monitoring workshops etc.). The M&E focal point of the respective component will be responsible for the collection of those information in bold only.
M&E tools and methods in the ‘reference column’, which are not highlighted, are described in Chapter 5 as these are methods and tools relevant for several components. All methods and tools which are needed for one component only are printed in bold and explained directly after the M&E matrix for the respective component.
17
3.1. M&E Plan ‘Decentralized Management’ Output 1: Management structures and procedures for decentralized management of development activities are in place on provincial levels. CBRDP
Log-frame Code
RDP Offer Code
Indicators for CBRPD output 1 Sources/ Value of Baseline Data
Quality Aspects of Indicators
Frequency Methods/Tools to measure as references in M&E manual
OP1.1
1. No. of PRDC members, ExCom Unit members, DFT/PFT members and LAU management staffs who have successfully increased their capacity. Targets: 70% of trainees reach at least 60% score (according to the test for each training course).
Members PRDC CAU FU LAU PFT DFTKampong Thom 8 6 4 2 8 31Kampot 8 7 4 2 8 35
Success rate of trainings Quality of trainers Quality of trainees
Each training; Occasionally
Monitoring of training section 5.1.1 and 5.1.2
OP1.2 2. % of staff who have increased their capacity as shown by staff evaluation results. Target: 80% and 80% of the recommendations from the annual staff assessment is implemented within 6 months after staff assessment.
Capacity of staff defined by performance evaluation; Implementation of proposals
Annually; Six months after assessment
Staff assessment section 5.1.3
OP1.3 3. Contracts between ExCom and line departments are established and executed according to contract specification. Target: 90%.
Quality of contract implementation
Annually Contract evaluation
OP1.4 4. PRDC/ExCom implement an annual self-assessment of their internal management (including the results cooperating partners’ satisfaction assessment) and services performance (SWOT) and follow-up appropriately. Target: 50% of weaknesses identified are addressed within 6 months after self-assessment.
Strengths/Weaknesses/ Opportunities/ Threats identifiedNo. of weaknesses addressed after 6 months
Annually;
six months after assessment
SWOT analysis section 5.2
18
Monitoring methods and tools for ‘Decentralized Management’ This chapter describes tools and methods, which are listed in the ‘methods and tools’ column of the previous table. However, only those tools and methods are being described here, which are specific to this particular component only. Tools and methods, which are being used for several components, are described in detail in chapter 5.
Contract evaluation This tool will be used to measure the following indicator:
"Contracts that are established between ExCom and line departments have been executed by overall 90% for each contract according to the respective contract specification". One important step to measure the quality and quantity of contract implementation is the so-called “end-of-contract” report. This end of contract report shows the annual achievements of the implementing line department against the targets as specified in the indicators of the contract. Comparing quantitative, qualitative and financial achievements and targets of the various contract activities should allow to generate one figure on the overall percentage of contract implementation. This figure should be agreed by both contract partners and is to be used as measurement for the above indicator.
3.2. M&E Plan ‘Enabling Commune Councils’ Output 2: Commune Councils are enabled to prepare commune plan with village participation to manage their implementation and operation. CBRDP
Log-frame Code
RDP Offer Code
Indicators for CBRPD output 2 Sources/ Value of Baseline Data
Quality Aspects of Indicators
Frequency Methods/Tools to measure as references in M&E manual
OP2.1
1. No. of Commune Council members who successfully increase their capacity to perform their tasks, based on defined criteria. Targets:
Members 2004 2005 2006 2007 RemarksKampong Thom
400 450 500 500 Out of 575councillors.
Kampot 260 280 300 320 Out of 542councillors.
Success rate of trainings Quality of trainers Quality of trainees
Each training; Occasionally
Monitoring of training
Section 5.1.1 and 5.1.2
OP2.2 2. CBRDP supported Commune Investment Plans are of high quality and followed up with high quality monthly reports (according to NCSC standards). Target: 50% (increase every year).
Quality of CIP and of monthly reports according to quality standards.
Annually CIP and CC report evaluation
OP2.3 3. No. of commune investment plans prepared. Targets: Kampong Thom annually 81. Kampot: 92 annually
Record of CIP Annually Data aggregation of CIP
OP2.4 4. No. of commune investment plans implemented. Targets: Kampong Thom annually 81. Kampot: 92 annually.
Record of CC report at district integration workshop
Annually Data aggregation of CC report
OP2.5 5. % of commune councils raising local resources. Targets: 90% No. of communes have raised local funds at the required amount.
Annually BMW
Section 5.4.2
19
20
OP2.6 6. % of commune councils raising external resources from at least one source. Targets: Kampong Thom: 50% respectively. Kampot: 40% respectively
No. of communes have raised external resources
Annually BMW Section5.4.2
Monitoring methods and –tools for ‘Enabling Commune Councils’ This chapter describes tools and methods, which are listed in the ‘methods and tools’ column of the previous table. However, only those tools and methods are being described here, which are specific to this particular component only. Tools and methods, which are being used for several components, are described in detail in chapter 5. CIP and CC report evaluation This tool will be used to measure the following indicator: "CBRDP supported Commune Investment Plans (7 districts in K, 4 districts in KT) and their respective CC monthly reports are of high quality (according to NCSC standards)". The target is that 50% of CIP and CC reports are of high quality, and the percentage of high quality plans and reports should increase every year. The baseline figure still needs to be determined.
The first part of the indicator (quality of commune development plan) will be monitored in two steps as follows:
In a first step, LAU will, at the end of the year select, by random, some of the commune investment plans and compare them with established NCSC quality standard structure as presented below.
3-year rolling CIP structure (see commune planning guideline, in the hands of PFT/DFT for more details) The commune development plan (CDP) shall be developed for a period of five years, and shall be reviewed every year to update the three-year rolling commune investment programme (CIP) which is adapted the following standard structure: Foreword
Introduction
Commune Map
1. Commune development situation analysis a) Economic b) Social Sector c) Natural Resources and Environment d) Administration and Security e) Gender
2. Immediate Objectives and strategies for investment
a) Immediate Objectives b) strategies for investment
3. Three-year rolling Investment a) Budget for Investment Implementation b) Budget summary by source of funding Annexes In a second step, the content of the CIP will be assessed according to a score from 1 to 10, whereby 10 means a 100% quality score. (P)LAU staff will do this scoring, and would need some guidelines and training on quality aspects in order to perform this scoring.
The monitoring of the availability and quality of monthly CC reports requires additional monitoring activities. (P)LAU will regularly monitor the incoming CC reports with regard to
21
22
time, structure and quality in an Excel spreadsheet. (P)LAU staff will read through each report and score the quality from 1 to 10 for each of the 5 topics of the report separately (see CC report structure below). (P)LAU staff writes the score on the paper copy of the CC report with signature and date. Somebody else could be responsible for entering the information into the computer spreadsheet file. This Excel file will allow at any given time to retrieve the information for the above indicator.
Monthly CC report structure (see the report structure in the hand of PFT/DFTs for more details) a. Commune Administration
a-1 Commune Councilors' situation a-2 Commune council meeting a-3 Deikas and other decisions issued by commune council a-4 Other committees meeting a-5 Changing of commune clerk a-6 Main problems, events happened in the commune
b. Commune Development Plan
Commune development planning activities c. Commune Finance
Revenue & Expense Statement d. Commune Project Implementation d-1 Development activities implementation and other services provision d-2 Implementation as agent-function d-3 Extension of information to local people d-4 Commune positive experiences and problems in cooperation with NGOs, private sectors and
Community Based Organizations e. Capacity Building and Support e-1 Training activities for commune councilors and commune clerk e-2 Training activities for other committee members and other villagers e-3 Other capacity building measures (exposure trip, workshop, …) e-4 Commune problems in cooperation with district and provincial level (LAU, Treasury, etc.) e-5 Backstopping support and other field visits.
23
3.3. M&E Plan ‘Community Development and Gender Mainstreaming’ Output 3: Village population and organizations of civil society are enabled to undertake self-help activities and to play an active role in local self-administration. Gender awareness is increased within all line departments and communities and women are enabled to play a more active role in local governance. CBRDP
Log-frame Code
RDP Offer Code
Indicators for CBRPD output 3_CD Sources/ Value of Baseline Data
Quality Aspects of Indicators
Frequency Methods/Tools to measure as references in M&E manual
OP3.1
1. DCDT/PCDT team members who successfully increased their capacity to perform their tasks, based on defined criteria (per capacity building measure; according to training plan). Targets: 80% of trainees reach at least 65% score (according to the test for each training course).
Members PCDT DCDTKampong Thom 4 22 Kampot 8 33
Success rate of trainings Quality of trainers Quality of trainees
Each training; Occasionally
Monitoring of training section 5.1.1 and 5.1.2
OP3.2 2. CD staff increase their capacities as proven through staff evaluation results. Targets: 80% and 80% of the recommendations from the annual staff assessment are implemented within 6 months after staff assessment.
Capacity of staff defined by performance evaluation Implementation of proposals
Annually; Six months after assessment
Staff assessment Section 5.1.3
OP3.3 3. CBO/Village Network leaders and committee members who receive training successfully increased their capacity to manage their CBO transparently. Targets: Kampong Thom: 94 CBOs (450 persons) annually. Kampot: 162 CBOs (810 persons) annually. 2004: 60%, 2005: 65%, 2006: 70%, and 2007: more than 70% increased capacity; and 35% of committee members are women.
Success rate of trainings Quality of trainers
Quality of trainees
Each training
Occasionally
Monitoring of training Section 5.1.1 and 5.1.2
BIA & BMW Section 5.4
OP3.4 4. All water supply and road projects have an LTC with at least 30% of committee members are women. Targets: Kampong Thom: LTC: 2004: well 360, road 6;
Establishment/ Functioning of committees data filed updated.
Annually CBO/LTC evaluation including the establishment
24
Kampot: LTC 2004: well 245, road 105; 2005-2007 "according to RIIF".
BIA & BMW section 5.4
OP3.5 5. All community management groups have women in leadership positions; 10% of the groups have women leaders.
Establishment/Functioning of committees data filed updated.
Annually CBO/LTC evaluation including the establishment BIA & BMW section 5.4
OP3.6 6. No. of people (beneficiaries) belonging to community based groups by type of group. Targets: Kampong Thom: 2004: RB: 10,500; CA: 23,500. LTC: 2004 well: 30,000; road: 30,000; 2005-2007 "according to RIIF". Kampot: RB: 40,000, cash association: 2,000, catering club: 7,000. LTC: 2004 well: 30,000, road: 30,000; 2005-2007 "according to RIIF"
Establishment/Functioning of committees data filed updated.
Annually CBO/LTC evaluation including the establishment
OP3.7 7. Community Development Component implement the annual self-assessment (including beneficiary impact assessment) of their internal management and services performance (SWOT) and follow-up the results. Target: 50% of weaknesses identified are addressed within 6 months of self-assessment.
Strengths/Weaknesses/ Opportunities/ Threats identifiedNo. of weaknesses addressed after 6 months
Annually;
Six months after assessment
SWOT analysis Section 5.2
Monitoring methods and –tools for ‘Community Development’ for each indicator This chapter describes tools and methods, which are listed in the ‘methods and tools’ column of the previous table. However, only those tools and methods are being described here, which are specific to this particular component only. Tools and methods, which are being used for several components, are described in detail in chapter 5. CBO/LTC evaluation including establishment Several indicators for this output (e.g. No. 4, 5, and 6) require monitoring of the establishment and functioning of CBOs and LTCs. A number of forms have been developed to collect relevant information. Some information are being collected with a form on village events, other forms are more specific on the establishment of a CBO or LTC, or on the functioning of a CBO or LTC. Especially the forms on functioning are different for different CBOs (like Rice Bank, Cash Association, etc.) and different LTCs (road, well) because of their different criteria for functioning. Three types of computer databases are used to store and process the relevant data from the field, namely:
• a village event database, • a CBO/LTC establishment database, and • a number of CBO/LTC functioning databases.
These databases together with some information from the infrastructure database should provide all necessary information for CBO/LTC related indicators. Below are samples of the data collection forms are being used. The first form (CD-A1) is a general form for recording of village events. The second form (CD-A2) is used in addition to CD-A1, if during this village event a CBO or LTC was established and a committee was elected. The third form (CD-A3) has to be used in addition to CD-A1, if the draft MVHL was presented and discussed during the village event. Another set of forms (CD-A4.1 to CD-A4.x) will be used to monitor the performance of respective CBO or LTC. This form should be filled by the CD staff at least once in 3 months for each CBO or LTC during their regular field visits to those village organizations. CD-A1: Village Event Minutes
1. Village event number: …………. 2. Type of event: ................... 3. Location: Village, Commune, District 4. Date: ……………. 5. Facilitators:
Name Organization
Role in this event
6. Participants
Total Number
Female Type of participants separately by village
(villagers, CC members, PBC, ...)
Invited Attending Invited Attending
25
7. Was the most vulnerable list presented or discussed during this event? a. Yes b. No
If yes, please attach separate form on Results from the establishment of MVHL.
The following information should be added on a free paper:
8. List of topics raised (including time frame) 9. Documents provided 10. Key issues raised during the meeting, important questions raised by the participants,
answers provided (not in the existing document); 11. Key results or decisions of the meeting including follow up steps of activities. 12. Annex all relevant documents to this minute such as: List of most vulnerable families,
result of committee election (see annex sample), Work group summary paper, village plan, activity plan, etc.
Remark 1: Key issues and key results that are explained in the document to be annexed to this minute, do not need to be described in point 10 & 11: just mention the reference annex. Example: List of most vulnerable families has been established (see joint document). Remark 2: The secretary and the chairman of the event should sign the minute.
CD-A2: Annex 1 to CD-A1 Village Event Minute: Result of committee election Annex to the village event number: …………………… Name of the committee: …………………. District: ……………….Commune: ………………….. Village: …………………..
1. Direct beneficiaries of the project for which the committee is responsible Village Families
Total number of
persons Female
2. Number of committee members to be elected Male: Female: Total:
3. List of candidates a. Number of candidates
Male: Female : Total: b. Female candidates list
No. Name Age No. of votes received
Rank
c. Male candidates list
No. Name Age No. of votes received
Rank
26
4. Committee elected
a. Number of elected members Male:……………………Female: ……………………..Total:……………….. b. Committee members list
No. Name
Sex Role in the community Role in the committee
Others
5. Signature and endorsement:
(This result of committee election should be signed by the secretary, the chairman of the event, the elected committee chief and endorsed by commune councillor for official use).
CD-A3: Annex 2 to CD-A1 Village Event Minute: Result of the establishment of Most Vulnerable Family List Annex to the village event number: District: ………………………..Commune: ………………………..Village: …………………….. 1. Village statistics: Total families: …………………..Total persons: ……………………. Total female: ……………
2. Most vulnerable families
Family members No. Name of head of family
Sex Age Total Female
Reasons of vulnerability Others
3. Signature and endorsement
(This result of the most vulnerable family's list establishment should be signed by the secretary, the chairman of the event and endorsed by commune councillor for official use).
27
CD-A4.1: Model Questionnaires for LTC functioning evaluation
1. Name of DCD: ... 2. Date: ...
3. District: 4. Commune: 5. Village:
6. Type of LTC: Road Well No 6 Well Afridef Combined well Pond
7. Total committee member: ........... 8. Women: .........
9. Did villagers use the road/well according to the user's rules?
5= Very good 4=Good 3= Fair 2= Basic 1= Weak
10. Are the committee internal rules applied?
5= Very good 4=Good 3= Fair 2= Basic 1= Weak
11. Have the expenses been done according to the rules?
5= Very good 4=Good 3= Fair 2= Basic 1= Weak
12. Has the accounting book been filled in properly?
5= Very good 4=Good 3= Fair 2= Basic 1= Weak
13. Does the LTC committee meet regularly with all members and good results? (Check the attend list, date of last meeting)
5= Very good 4=Good 3= Fair 2= Basic 1= Weak
14. Are the members regularly informed by the committee?
5= Very good 4=Good 3= Fair 2= Basic 1= Weak
15. Does each committee member fulfil his/her function and role properly?
5= Very good 4=Good 3= Fair 2= Basic 1= Weak
16. Are the administrative documents managed properly?
5= Very good 4=Good 3= Fair 2= Basic 1= Weak
17. Are the maintenance tools kept properly?
5= Very good 4=Good 3= Fair 2= Basic 1= Weak
18. How much cash is available at the moment? ..................................................................
19. Do you have laterite/spare parts in reserve?.......................................................................
28
29
20. How much cash have you spent since January 2005?.......................................................
21. Did you use any laterite/spare parts since January 2005?.................................................
22. How is the condition of the road/well?
5= Very good 4= Good 3= Fair 2= Basic 1= Weak
23. Are the committees members involved in CC activities and participate in CC meetings?
5= Regularly 4= Often 3= Sometimes 2= Rarely 1= Never
Other forms (CD-A4.2 ... CD-A4.x) for data collection in terms of functioning of other types of CBOs, such as Rice Banks, Cash Associations, Catering Clubs, etc, need to be made according to the requirements of CD component.
30
Gender Mainstreaming: Gender awareness is increased within all line departments and communities and women are enabled to play a more active role in local governance. CBRDP
Log-frame Code
RDP Offer Code
Indicators for CBRPD output 3_ Gender mainstreaming Sources/ Value of Baseline Data
Quality Aspects of Indicators
Frequency Methods/Tools to measure as references in M&E manual
OP3.8 8. 50% of women actively participation in local planning process, project management and decision making
Quality ofwomen's participation
Annually Analysis of data from village event records BMW
section 5.4.2
OP3.9 9. No. of gender focal points in Commune Council and Line Departments who successfully increase their awareness on gender issues. Target: 80% of trainees reach at least 60% score, from 2004 and annually.
Focal Points
Communes Districts Line Departments
NGO
Kampong Thom
81 8 27 0
Kampot 92 9 33 5
Success rate of awareness events Quality of trainers
Quality of trainees
Each training;
Occasionally Monitoring of training
Section 5.1.1 and 5.1.2
OP3.10 10. No. of gender training activities completed: (i) gender focal points; (ii) NGOs and commune councils; and (iii) individuals. Targets to be developed in the AWPBs.
TG event file updated
Annually Counting of all training in TG data file.
31
Monitoring methods and –tools for ‘Gender Mainstreaming’ This chapter describes tools and methods, which are listed in the ‘methods and tools’ column of the previous table. However, only those tools and methods are being described here, which are specific to this particular component only. Tools and methods, which are being used for several components, are described in detail in chapter 5.
Analysis of data from Village event records To get the information for the indicator on the participation of women during the planning process, the village event records, which are being collected with the form CD-A1, will provide the quantitative data for the participation of women in events. To what extent women are involved in management and presumably in decision-making would come from the data in the database of functioning CBOs/LTCs. The quality aspect of the women’s participation would be analysed by beneficiary monitoring workshops (BMW), which is described in Section 5.4.2.
The gender M&E focal point will liaise with the LAU M&E focal point to assess the respective databases for the analyses.
32
3.4. M&E Plan ‘Infrastructure incl. Land Titling and Irrigation’ Output 4: Support services (technical and community development) for the construction of rural infrastructures (roads, water supply, irrigation schemes, land titling) are delivered and Local Technical Committee/Water User Committee enabled to manage on going maintenance needs. CBRDP
Log-frame Code
RDP Offer Code
Indicators for CBRPD output 4_Rural Road and water supply Sources/ Value of Baseline Data
Quality Aspects of Indicators
Frequency Methods/Tools to measure as references in M&E manual
OP4.1 1. No. of TSS members successfully increase their capacity to perform their tasks, based on defined criteria (type of officers per capacity building measure, according to training plan). Target: 20 in Kampot and 20 in Kampong; 80% of trainees reach at least 60% score (according to the test for each training course).
Success rate of trainings Quality of trainers Quality of trainees
Each training; Occasionally
Monitoring of training section 5.1.1 and 5.1.2
OP4.2 2. No. of staff who increase their capacities as shown by staff evaluation results and recommendations from the evaluations implemented. Targets: 80% and 80% of the recommendations implemented within 6 months after staff assessment
Capacity of staff defined by performance evaluation; Implementation of proposals
Annually; Six months after assessment
Staff assessment Section 5.1.3
OP4.3 3. Km of roads constructed according to quality standards as set in the Seila manual. Targets: Kampong Thom: up to 2004: 100 km. Kampot: up to 2004: 100 km. For 2005/2006/2007 figures to be determined once project priorities are determined under the RIIF.
Quality ofconstruction according to quality standards
End of construction;
Annually
End-of project evaluation Section 5.3.2;
BIA & BMW Section 5.4
OP4.4 4. No. of water points (wells) drilled/dug for drinking water according to quality standards as set in the Seila manual. Targets: Kampong Thom: Up to 2004: 360; Kampot: Up to 2004: 245. For 2005/2006/2007 figures to be determined once project priorities are determined under the RIIF.
Quality ofconstruction according to quality standards
End of construction;
Annually
End-of project evaluation Section 5.3.2;
BIA & BMW Section 5.4
OP4.5 OB2.3 5. 75% of public works activities (infrastructures) and other small projects assisted by service providers satisfy the government-defined quality criteria like Seila Technical Manual (results of
Quality ofconstruction according to quality standards
End of construction;
End-of project evaluation Section 5.3.2
33
regular technical evaluations).
OP4.6 6. No. of households served by wells (water points). Targets: Kampong Thom: accumulated target end 2004 for water points: 5,500 households. Kampot: accumulated target end 2004 for water points: 4,000 households. For 2005/2006/2007 figures to be determined when the commune councils determine their priorities for use of the RIIF and the numbers of households will be increased accordingly.
LTC functioningdata file updated
Annually Data aggregation of number of households
OP4.7 7. Rural Infrastructure Component implements an annual self-assessment (including the results of beneficiary satisfaction assessment) of their internal management and services performance (SWOT) and follow-up upon the results. Target: 50% of weaknesses identified are addressed within 6 months of self-assessment.
Strengths/Weaknesses/ Opportunities/ Threats identifiedNo. of weaknesses addressed after 6 months
Annually;
Six months after assessment
SWOT analysis section 5.2
Monitoring methods and –tools for ‘Roads and Water Supply’
This chapter describes tools and methods, which are listed in the ‘methods and tools’ column of the previous table. However, only those tools and methods are being described here, which are specific to this particular component only. Tools and methods, which are being used for several components, are described in detail in chapter 5. The indicators 3, 4, 5 and 6 require monitoring of quantity and quality of rural infrastructure projects at large. Basically the following tools will be used to collect the necessary information:
• planning and bidding documents,
• log sheets (as they exists for well constructions),
• field supervision protocols,
• end-of-project evaluation,
• technical audits,
• various component data bases, and
• BIA.
Some of the above tools are described in detail in the following.
Log sheets Log sheets are work progress reports, which are maintained during the drilling or construction of a well. They document the work progress with regard to technical specifications. They provide data for the ‘well data base’, which stores information on all wells constructed within the project. Log sheets usually are maintained by the contractor, and the project support staff should assure to receive a copy of this log sheet, when the construction is completed.
Field Supervision Protocol (and/or project book) Field supervision protocols provide information from each supervision visit to any construction site by the technical support staff. These protocols can also serve as a project book. They provide first of all information for managing the process of construction and allow to take corrective action immediately. In addition, these supervision protocols provide useful information on the performance of the technical support staff.
The procedures and methods for site inspection are as follows:
• conduct information collection every week or every month from construction site activities, by technical support officers with the support from TA (if any available).
• the information collected will be communicated to TSU management.
• this information will be used to ensure construction of civil work according to technical specification.
In principle, the site inspection is done according to the sample formats (see annex 8).
End-of-Project Evaluation
34
35
The end-of-project evaluation is based on the SEILA standard of decentralized project implementation. It involves a standardized form (see section 5.3.2). Although the form needs to be filled once at the time of handing over the project to the commune council, much information in this form goes well beyond this purpose. The form is to be filled by the technical support staff when the commune council formally takes over the ownership of the completed project. The information from this form is processed through SEILA databases such as the PID (Project Implementation Database).
Technical audit The technical audit sometimes is considered the same as the end-of-project evaluation. However, in the context of this M&E system the technical audit is a separate exercise, which should be undertaken once a year for each newly constructed well, road or structure. It serves as the main source of monitoring the technical quality of rural infrastructure projects. The data from the technical audits are processed and stored in separate Excel databases depending on the type of infrastructures. The technical audit measures, whether the implemented projects fulfil the required technical quality standards. Because the technical audit is to be done one time for each project, the data also supply quantitative information, such as the number of respective infrastructure projects in an area. The technical audit will be conducted by a team from the technical support unit (TSU) and may include staff from technical assistance or outside consultants.
The technical audit focuses on the quality of the “hardware” (technical construction); no effort is made to also assess the processes during site selection, technical planning, and implementation of the project. It is very important, that DFTs and members from Community Development participate during the end-of-project evaluation. This is a very critical issue not only because CD has a crucial function for the establishment and strengthening of the LTC but also because CD should evaluate the processes during project planning and implementation.
It is felt very important to also assess these aspects not only to know whether a functioning LTC has been established (a very important aspect) but also to know in how are women and other groups have been involved in decision making during the project cycle, whether the powerful people have pushed through their ideas for the location of the well or the road, whether also the disadvantaged have access to the well etc. etc.
The BIA will provide additional quality information from the perspective of the beneficiaries. A methodology for evaluating these processes is described in Section 5.4.1.
Land Titling CBRDP
Log-frame Code
RDP Offer Code
Indicators for CBRPD output 4_ Land titling Sources/ Value of Baseline Data
Quality Aspects of Indicators
Frequency Methods/Tools to measure as references in M&E manual
OP4.8 8. No. of households provided with long-term security of tenure of land. Targets: Land titles issued: Kampot 2004: 6,500. Kampong Thom 2004: 3,500; 2005: 5,000.
Annually Summary of statistics
Monitoring methods and –tools for ‘Land Titling’ Indicator 4.6: Land titles issued related to IFAD funded infrastructure The number of land titles issued is the result of simply recording the number of land titles issues on each handing over of land titles. The M&E focal point will just sum up the number of land titles issued at the end of the year and compare with targets. As of mid 2005, there will be no more monitoring of this component, because the IFAD support for this component has been completed.
36
37
Irrigation CBRDP
Log-frame Code
RDP Offer Code
Indicators for CBRPD output 4_Irrigation Sources/ Value of Baseline Data
Quality Aspects of Indicators
Frequency Methods/Tools to measure as references in M&E manual
OP4.9 9. Sufficient PDWRAM staff with suitable skills (technical and community facilitation) to perform defined tasks. Targets: staff evaluations show that 80% of staff increased their capacities and 80% of the recommendations from the annual staff assessment implemented within 6 months after staff assessment.
Capacity of staff defined by performance evaluation; Implementation of proposals
Annually; Six months after assessment
Staff assessment section 5.1.3
OP4.10 10. Technical planning and supervision of project irrigation schemes is undertaken on behalf of commune councils on time (including projects under RIIF) and according to professional standards.
Planning,supervision and support according to defined quality standards.
Annually End-of projectevaluation section 5.3.2
BIA & BMW Section 5.4
OP4.11 11. All irrigation schemes constructed have a functioning WUC dealing with O&M and water distribution. Targets: Kampong Thom: 2004: 7 schemes "11 WUCs?". Kampot 2004: 2 schemes "2 WUCs"; 2005 according to RIIF. At least 50% of the members of the all the WUC assisted by the project are women in Kampong Thom and 35% in Kampot.
Water usercommittee functioning according to defined quality standards.
Annually BIA & BMWsection 5.4
OP4.12 12. Ha. of newly irrigation schemes rehabilitated/constructed. Target: Kampong Thom: 2004 (2 schemes): wet season 400 (dry season 413). Kampot: 2004 (1 scheme): wet season 750 (dry season 300).
Annually Summary of statistics
OP4.13 13. No. of water user committees newly formed/ strengthened. Targets: Kampong Thom: 2004: 11; Kampot: 2004: 2.
FWUCestablishment file and TG event file updated
Annually Data aggregation of number of FWUCs established and trained.
38
Monitoring methods and –tools for ‘Irrigation’
For this component technical ‘hardware’ and qualitative ‘software’ performances need to be monitored. Although ‘Irrigation’ is considered a separate component, the monitoring of the indicators uses the same tools and methods as for other infrastructure projects. The functioning of Farmer Water User Communities (FWUC) will be monitored with the same tools and methods as other CBOs (see output 3).
39
3.5. M&E Plan ‘Agriculture Extension’ Output 5: Demand oriented services to support the crop and livestock production in place. CBRDP
Log-frame Code
RDP Offer Code
Indicators for CBRPD output 5 Sources/ Value of Baseline Data
Quality Aspects of Indicator
Frequency Methods/Tools to measure as
references in M&E manual
OP5.1
1. Number of CMTST-, DOT-members who successfully increased their capacity to perform their tasks, based on defined criteria (type of officers per capacity building measure; according to training plan). Targets: 80% of trainees reach at least 60% score.
Members CMTST DOTKampong Thom 10 56 Kampot 10 49
Success rate of trainings Quality of trainers Quality of trainees
Each training Occasionally
Monitoring of training section 5.1.1 and 5.1.2.
OP5.2 2. 80% of staff who increased their capacities as proven through staff evaluation results. 80% of the recommendations from the annual staff assessment implemented within 6 months after the assessment.
Capacity of staff defined by performance evaluation; Implementation of proposals
Annually; Six months after assessment
Staff assessment section 5.1.3
OP5.3 3. Services supplied are increasingly in response to demands expressed in the Commune Investment Plan (2004: 50%; 2005: 60%; 2006: 70%; 2007: 80%)
Quality ofadjusting service delivery to demands of villagers
Annually Monthly Reports DOT Expenditures of PDA Village demands from CIP in General Village Data base Person days from TG participation
OP5.4 4. No. of private agricultural service providers (e.g. VAHWs, FPs) participating in capacity building measures to a level which increases farmer demand for their services. Targets:
Annually Counting of all extension events in TG event data file.
OP5.6
6. No. of animals vaccinated. Targets:
Animal vaccinated
2004 2005 2006 2007
Kampong Thom
39,000 46,000 62,000 77,000
Kampot 9,000 73,000 73,000 73,000
Record ofvaccination from VAHWs
Annually Data aggregation on animals vaccinated
OP5.7 7. Each Agriculture Component (incl. District level) implements an annual self-assessment (including the results of beneficiary satisfaction assessment) of their internal management and services performance (SWOT) and follow-up upon the results. Target: 50% of weaknesses identified are addressed within 6 months after self-assessment.
Strengths/Weaknesses/ Opportunities/ Threats identified; No. of weaknesses addressed after 6 months
Annually;
Six months after assessment
SWOT analysis
Section 5.2
3 To be supplemented by small special surveys on e.g. the FP-, VLA etc. approaches and their effectiveness/outreach to farmers
Monitoring methods and –tools for ‘Agricultural Extension’ for each indicator This chapter describes tools and methods, which are listed in the ‘methods and tools’ column of the previous table. However, only those tools and methods are being described here, which are specific to this particular component only. Tools and methods, which are being used for several components, are described in detail in chapter 5. Indicator 5.3: Services supplied are increasingly in response to demands This indicator measures the extent by which the provincial agricultural department carries out its services according to the demands in Commune Investment Plans. The focus is therefore on the total service package of PDA within CBRDP/RDP (as 100%) and/or including all otherwise funded services (CBRDP/RDP + all other sources of funds as 100%). Because a livestock service is difficult to compare with a service in rice production, quantification of service delivery is problematic. The quantification of services delivered could be done in monetary terms. Other options are to measure this indicator in person-days of extension staff being actually in the field, or the person days of target groups participating in field events. The indicator compares how many of the services provided are in response to the commune plans and how many aren’t. To obtain the necessary monetary data one has to monitor the expenditure budget of operational cost by technical sub-sector and compare these on village or commune basis with demands expressed in commune plans or other plans (such as AEA = Agro-Ecosystem Analysis). The target group event data file and work plan reports (see section 2.1) from field staff provide further quantitative data for this indicator. The individual field visit protocol will also result quantitative data on the number of person-days that the staff spent in the field, if this field visit notes are being used consistently by the staff.
Indicator 5.4: Number of private agricultural service providers participated in capacity building measure to a level which increases farmers' demands for their services The first part of this indicator (Number of private agricultural service providers participated in capacity building measure) can be measured by the general procedure for monitoring of target group events and formal trainings (section 5.1.1 & 5.1.2). The target figures for each year include the number of newly established service providers as well as refresher trainings of existing private service providers, mainly VAHW and Farmer Promoters.
Also, the second part of this indicator of increasing farmers’ demands for private agricultural services (farmer promoters, village animal health workers, inputs suppliers), will be assessed during the beneficiary assessment survey (BIA) and consecutive beneficiary monitoring workshops (Section 5.4).
While the quality of services can be monitored with the BIA from the beneficiaries’ point of view, targets need to be identified with regard to qualities, that the service providers are expected to perform. For example, the VAHW should be able to treat animals so that a certain percentage of clients are fully satisfied with the service. Similarly, the farmer promoters are expected to conduct farmer group extension, so that those farmers in the groups are satisfied with the services to a certain percentage.
An increase in the demand for the services can also be measured through increased number of services provided, that is the number of existing service providers (and which are included in the annual target figure of the indicator table) should all show an increased number of activities compared to their previous year activities. In order to calculate this, one has to monitor the volume of service provision from each service provider. For the VAHW, his/her pocket book provides all information on his/her business activities. Often, the information
41
from the pocket book are summarized in the monthly report from the VAHW to the district veterinarian.
Further information for this indicator can be gathered through specific surveys. The following briefly describes these different tools.
VAHW's Pocket Book and Monthly Report At the moment, the regular income of all village livestock agents is monitored by the agricultural component using the VAHW’s pocket book with the structures as below:
Each VLA has a kind of project book and record the following information:
The following two small sample surveys are proposed on an annual basis:
Annual farmer promoter survey The component should select by random, in each province, a few communes where a farmer promoter works (and which have also at least one VAHW in operation; see below). The outreach of each farmer promoter and the impact on the commune/village will be assessed in open discussions with (i) the farmer promoter himself, (ii) the farmers in the village where the farmer promoter works and (iii) one neighbouring village. The discussions with these groups should centre around the following questions:
• What did the farmer promoter do after he/she received training and inputs for his village and in the neighbouring village? How many farmers did he reach with what activities?
• How many farmers (male, female) participated in trainings at the farmer promoter’s demonstration fields? Which other advice did farmers get from the farmer promoter?
• How many farmers (male, female) applied which innovative practices learnt from the farmer promoter? What was the experience with these innovative practices?
• What was the benefit of the farmers when applying innovative practices learnt from the farmer promoter?
• What additional advise/inputs would farmers need from the farmer promoter?
• What did farmers pay for the services they got from the farmer promoter? What would they be prepared to pay for additional services and inputs?
• How does the farmer promoter value the support from agriculture (training, extension, inputs, and materials)? Which additional support would the farmer promoter need from agriculture? (Ask FP only).
• What are the motives for the farmer promoter to give advice to other farmers? Which benefit does the farmer promoter get from training/advising others? (Ask FP only).
42
43
Annual village animal health worker (VAHW) survey During the annual farmer promoter survey (see above) a few questions will also be asked concerning the outreach and quality of the village animal health worker services. Discussions should centre around the following questions:
• How many households with animal keeping (according to male and female respondents) use the services of the VAHW? How regularly? For what purpose (kind animals, diseases)?
• How many households with animal keeping (according to male and female respondents) do not use the services of the VAHW? Why not?
• What is the experience of animal keepers with the VAHW (which services are good, which ones are not so good)? Which additional services would animal keepers need?
• How do users of the services see the benefit of the services compared to the costs?
Outreach and efficiency of farmers associations and private input suppliers will be assessed during the beneficiary impact assessment survey and the subsequent beneficiary monitoring workshops.
Indicator 5.5: No. of extension events attended by target households The number of participants in target group events can be directly calculated from the target group event monitoring file (see chapter 6). A target group event is defined as any event involving different types of target groups. Events may be farmer classes, field days, CBO committee trainings, LTC trainings, or village meetings. They normally last half or one day only. If they last longer than one day they are counted as a separate event. Target groups may include committee members, VAHWs, Village Network members, CC members, or ordinary villagers.
Indicator 5.6: No. of animals vaccinated This indicator can be directly measured by counting the number of vaccinations being performed. In connection with the log-frame, only vaccinations in connection with vaccination campaigns are considered here. These imply vaccinations for cattle and buffaloes, only, as no vaccinations are being done for other animals normally. Therefore, it would not include poultry vaccinations irrespective of the bird flu situation and control strategy. Cattle and buffalo vaccinations could be for any disease, but mainly against H.S (Haemorrhagic Septicaemia) or FMD (food and mouth disease). For a full immunization of an animal, two vaccinations are recommended for one animal. Therefore if a second vaccination campaign is done within a year in the same village, one would more or less cover the same animals, and should make necessary remarks and adjustments to the data presented.
44
3.6. M&E Plan ‘Farmer Training & Demonstrations’ Output 6: Farmers are enabled to improve their subsistence production and to expand their market oriented production CBRDP
Log-frame Code
RDP Offer Code
Indicators for CBRPD output 6 Sources/ Value of Baseline Data
Quality Aspects of Indicators
Frequency Methods/Tools to measure as references in M&E manual
OP6.1 1. No. of households reporting livestock losses. Targets: Kampong Thom & Kampot: based on data collected by VAHWs, all households report decrease of annual livestock losses of at least 3% (cattle and buffalo only).
Record of cattle died from VAHWs
Annually Data aggregation on cattle died. BIA & BMW Section 5.4
OP6.2
2. No. of people trained by gender and sector. Targets: (female 40%).
Annually Data aggregation on people trained by VAHWs or FPs.
OP6.4 4. No. groups of farmers (associations) that improved their capabilities in selling their products. Targets: Kampong Thom: 6 (2 organic rice asso. 1 VAHWs asso. and 3 seed producer asso.). Kampot: 12 (1 cooperative, 4 seed producer asso., 3 new VAHWs asso. and 4 old VAHWs asso.).
Success rate of trainings
Quality of trainers
Quality of trainees
Amount and kind of products sold
Each training
Occasionally
Annually
Monitoring of training
Section 5.1.1 & 5.1.2
Farmer Association Survey
45
Accounts of associations
OP6.5
5. No. of demonstrations and associated activities that have been fully implemented according to guidelines and quality standards. Targets: Demonstrations (crop plots, animal husbandry demonstrations) include establishment of demonstration, 3 farmer classes and 1 farmer day with at least 50% female participants.
No. of fully established4 and implemented demonstrations according to the guidelines for successful demos5
After each demonstration
Annually;
Supervisory records;
BIA & BMW
Section 5.4
4 Demonstrations to be laid out and monitored according to established procedures and guidelines for successful demonstrations 5 Demonstrations to be implemented and crop cuts made according to established procedures and guidelines for crop cuts
Monitoring methods and –tools for ‘Trials & Demonstrations’ This chapter describes tools and methods, which are listed in the ‘methods and tools’ column of the previous table. However, only those tools and methods are being described here, which are specific to this particular component only. Tools and methods, which are being used for several components, are described in detail in chapter 5. Indicator 6.1: No. of households reporting livestock losses Data for this indicator will have to come from the recordings of the Village Animal Health Workers (VAHW), which report monthly to the district animal health and production officer. The data would have to be processed (calculating the percentage) at provincial level. A respective database file needs to be set-up at provincial level for these data from VAHWs. The data should allow calculating the percentage in decrease of livestock losses from the total number of households in the village, as well as from the total number of clients of the VAHW only. Details of the data base are in the Chapter 6. Indicator 6.2: No. of people trained by gender and sector (Targets: 40% females) The data for this indicator are directly available from the records on target group events, whereby topic and number of participants are recorded by gender for each target group event (see format in Annex 7). The data allow only calculating the total number of participants, but not the number of people trained, because same people could be participating in different events. It is proposed to estimate the difference between individual people and participants according to the invitation policy and type of event. For example, during 3 farmer classes in connection with a field demonstration, one would assume that the people in the 3 farmer classes are all the same. The difference between “No. of people trained” and “No. of participants” should be clear when presenting and interpreting the data. Indicator 6.3: No. of people accessing technical advisory services facilitated by the project (from private service providers only; Targets 40% female) This indicator measures the extent, by which villagers from the target area actively demand and use services from private service providers. Private service providers in the project/program context are Village Animal Health Workers (VAHW) and Farmer Promoters (FP). The information will be collected through the annual BIA and BMW. The percentage of females accessing the services could be biased, depending on the percentage of females participating in the BIA and BMW. Indicator 6.4: No. of groups of farmers (associations) that improved their capabilities in selling their products Information for this indicator will be collected partly through the general procedure for monitoring of formal training (section 5.1.1 & 5.1.2) for the number of male and female trained in the field of marketing and/or product quality management. However the fact of attending trainings doesn’t fulfil the requirement of this indicator yet. Monitoring of information about produce sold in term of quality, quantity and kind will be assessed during the beneficiary assessment survey and consecutive beneficiary monitoring workshops (Section 5.4). Additional information should be collected from the accounting books of the associations. Informal groups of farmers might not have any accounting books, and the indicator should therefore concentrate on the formal association and co-operatives. Basically the indicator
46
addresses the following 3 types of associations of the project, namely the organic rice producers, rice seed producers and VAHW pharmacy associations. Indicator 6.5: No. of demonstrations and associated activities that have been fully implemented according to guidelines and quality standards (protocols) Monitoring the implementation of field demonstrations requires a combination of different monitoring tools. Firstly, the guidelines and standards (= protocols) for field demonstrations have to be clearly defined, as they form the basis for the monitoring work. The provincial departments (PDA) have now prepared these protocols, and which are different for different types of field demonstrations, e.g. a pig raising demonstration will have a different protocol from a SRI demonstration. One important tool to collect qualitative information for this indicator is the BIA. In order to monitor correctly the implementation of field demonstrations, the facilitators of the BIA must be well aware of the protocols for the respective demonstrations! A special component survey would be another tool to compliment information for this indicator. The combination of General Village Data Base and Target Group Data Base provide quantitative information on the number of extension events in connection with a demonstration, as well as the on the participation of females. Typically a complete demonstration could have 5 target group events: identification of demonstration farmer (1), farmer class (3) and field day (1). Monitoring quality of demonstrations also require collecting technical information for each demonstration. The extension worker should collect this information during his/her regular visit to the demonstrating farmer using a supervision field visit form and a field demonstration book, where he/she records the detailed technical information from each demonstration. As the information occurs at different times, the extension worker ideally reserves a double page in his/her field book for each field demonstration. The types of information (parameters) to be collected for different demonstrations are shown as examples below. If the extension worker finds it difficult to collect all these information, he/she should provide a “project book” for the demonstrating farmer, for him/her to record all important events in connection with the demonstration. The types of information would differ depending on the protocols, and major types of demonstrations are
Further details on the respective parameters are described in chapter 6.
Data Collection Parameters for (Rice) Demonstrations
Demo Id No: running number (3 digits) within year, e.g. 00106, 00206, ...)
Name of demo farmer
Sex of demo farmer
47
Village
Commune
District
Type (crop or livestock species)
Description of demonstrating technology (i.e. a new variety, SRI elements, certain recommendations of ecological chicken raising, etc...)
Date of demo farmer selection
Area/size of demo (plot)
Area/size of comparison (plot)
Area of seedbed (if applicable)
Soil type of demo plot
Soil type of comparison (plot)
Inputs received by demo farmer (type and amounts)
Date of receiving inputs by farmer
Type and amount of seed used (demo and comparison plot)
Date of sowing/planting (or date of starting the demo for livestock/poultry)
Germination Percentage
Date, type and amount of all fertilizer applications (including for seedbed, and manure and chemical fertilizer)
Date and method of transplanting (description must be in relation to SRI elements)
Date and type of crop management events like weeding, irrigating, pest control etc. (description must be in relation to SRI elements)
Date of harvesting
Harvesting data (preferably crop cut information; see also separate box on crop cuts), but minimum total yield estimate from demonstration (and comparison!) plot
For a livestock or other kind of demonstrations certain parameters are not applicable or need to be modified accordingly!
48
Guideline for Crop Cuts Crop cuts from demonstration plots are done also since years and quite substantive information is available on the yields of different rice varieties. Crop cuts are done according the following guideline: A crop cut should take place during a Field Day, one crop cut per village is sufficient, i.e. not every field day has to include a crop cut nor need a crop cut be done for each demonstration plot. It is highly recommended to have one demonstration plot and one control plot close by, preferably a neighbouring field from the same farmer. Control plots should be chosen at the start of the season and should allow a meaningful comparison with the technology demonstrated (e.g., if demonstration for improved variety, control plot should have local variety, if demonstration plot has one seedling per hill, control plot should have the common local practice). Within the field, crop cuts should be carried out at 2 locations, which are not close to the border of the field. Each crop cut covers 2 m x 2 m = 4 m2, i.e. per field 8 m2, and per demonstration and control plot 16 m2 altogether. Finally, the yield per one ha is calculated separately for the demonstration and control plot, written in the Crop Cut Form and send to the provincial office. Formula: (yield crop cut in kg) x 1250 = wet yield in kg per ha, dry yield will be calculated at the provincial office, if no moisture measurement was performed in the field directly.
49
Chapter 4
Monitoring Level 3: Outcomes and Impacts
Output monitoring is the key for programme internal monitoring (because outputs are our products, services where we are responsible for; see introduction to Chapter 3). However, we also have to observe whether our products, services are really accepted by our target groups and sustainably applied (direct benefit: outcome (formerly purpose level describing the use of outputs); see matrix below). Finally, it is necessary to check whether this application really contributes to a ‘better live’ (indirect benefits at higher level of the result chain (formerly impact at goal level); see matrix below).
In most cases, higher level (medium and long term) impact (or goal level) monitoring is not the task of project internal monitoring. Usually, tracing of higher impact level (goal level) indicators is done in national census, evaluations at the end of project implementation etc... However, in the case of the CBRDP/RDP, the yield measurements and adoption rates (agricultural component) would even allow an approximation of the additional income induced by the project (goal level indicator 4; see M&E plan below).
The major sources for impact monitoring are the results from output and outcome monitoring. However, additional special surveys are necessary. The M&E plan for the direct benefits (outcomes, former purpose or objective level (see table below)) specifies these surveys.
The following M&E plan describes tools and methods for the indicators at outcome (objective, purpose) level. Most tools have been described already under the output monitoring, or will be described under the following chapter 5. of “Monitoring Methods and Tools”.
Describe infrastructure maintenance survey for OB8: The following procedures are proposed to assess the regular use and maintenance of infrastructure facilities:
• Participants of the beneficiary monitoring workshops should be encouraged to report on all cases known where CBRDP supported infrastructure is dilapidated or is not in use any more to a considerable degree (number of users by defined Seila standards). The number of reported cases will be compared with the number of constructed infrastructure per year.
50
• An annual infrastructure maintenance survey should be made covering all dilapidated infrastructure facilities to assess the causes of the problem and to analysis problem solutions.
• This infrastructure maintenance survey on broken infrastructure should follow a similar line of thinking and interview guidelines as presented in Section 5.3.2.
Describe agriculture component survey for OB9 and OB10: The sustainable adoption of CBRDP facilitated improved technologies by farmers is the key issue in agricultural extension and agronomy. Therefore, this issue has also been tackled as a key issue within the Department of Agricultural Extension with its offices at provincial level. With the support of the Cambodian-Australian Agricultural Extension Project (CAAEP) a methodology for annually assessing sustainable adoption of agricultural technologies was developed and introduced. This methodology is documented and provides a starting point for an adoption survey for CBRDP/RDP.
It is proposed that the CBRDP/RDP cooperates with the Offices of Agricultural Extension in the application of the methodology in the two provinces as well. However, the methodology has to be adjusted to the needs of CBRDP/RDP.
The CAAEP methodology foresees three elements of adoption assessment, namely (i) 6-monthly monitoring sheet of effectiveness of field program, (ii) annual adoption monitoring sheet of field program, and (iii) annual diffusion monitoring sheet. While the first two data sets are collected in village group meetings in those villages, where project extension activities (farmer classes, etc...) took place, the diffusion survey data are to be collected in neighbouring villages, where no extension activities took place.
For CBRDP/RDP the following adjusted methodology is proposed. For reasons of simplicity and practicability, only one monitoring sheet should be used. This monitoring sheet has to cover the following aspects.
The technology or innovation that has been promoted needs to be clearly identified, as well as the level of detail. This may be straight forward for the promotion of a new rice variety, for example for SRI, should the recommended technology refer to the separate elements of SRI, such as transplanting one seedling, age of seedling at transplanting etc..
For each identified technology, one will have to monitor the following questions in village group meetings:
a) How many of you heard or learnt about this technology?
b) How many of you believe you have skills to apply the technology?
c) How many of you applied the technology already (first hand experiences)?
d) What are your experiences with the technology? (++, +, 0, -)
e) How many of you plan or will apply the technology in the next season or months?
f) How many of you will continue to use this technology?
(From c to f) separate for Female Headed Households (FHH)
The survey should be conducted annually in at least 10% of villages, where project activities took place during one of the previous two years.
The agriculture component survey could be combined with the need to collect information on the use of and the satisfaction with public and private agriculture services. In order to get the information for the respective indicator, the survey should include the following principle questions:
51
52
a) from whom did you buy your last farm inputs like - seed - seedlings - fertilizer - farm tools - other (specify)?
b) from whom did you use agriculture services and at what cost? - animal health services - animal breeding services - extension/advice
For each of the above questions, the farmers should be asked, to what degree they were satisfied with the farm inputs and services. The data collection has to be specific for most vulnerable households and female headed households respectively.
The data collection can be done in group discussions or with individual household questioning. For CBRDP/RDP, a combination of the two methods is proposed in such a way, that the collection will be organized in a village meeting, but each participant will fill his/her own questionnaire. Villagers could sit together in small groups up to 4 persons in order to support each other in the filling of the questionnaire, while a facilitator moves around and provides help to the understanding.
M&E Plan for Outcome and Impacts
CBRDP Log-frame Code
RDP Offer Code
Indicators for CBRDP strategic goal Sources/ Value of Baseline Data
Frequency Methods/Tools to measure as references
in M&E manual
Information needed
SG1 1. Poverty incidence reduced by 50% between 1990 and 2015
National socio-economic survey (triangulation of external data).
Data from Commune Database, World Bank poverty assessment, and national census in 2008 (?).
SG2
2. The number of Most Vulnerable Families in selected project villages is reduced by 30% in 03/08.
Annually MVHLs Sample survey by using criteria the same as those used in 2004. (small in 2006 and full scale in 2007)
MVF lists.
SG3 3. % of male and female children < 5 years malnourished (weight for age), chronically malnourished (height for age) and wasted (weight for height) compared to the level at programme start-up. (Stunting of children at appraisal < 5 years of age 36% in Kampot and 49% in Kampong Thom).
36% in Kampot and 49% in Kampong Thom
2007 Wasting and Stunting survey
Needs to be further discussed with IFAD
SG4 4. Increase in the number of cattle (or other assets) owned by target group households with data disaggregated by gender of the household head.
Annually Agriculture component survey (small in 2006 3rd quarter, and full scale in 2007)
Data from commune database (MoP).
CBRDP Log-frame Code
RDP Offer Code
Indicators for CBRPD Objectives Sources/ Value of Baseline Data
Frequency Methods/Tools to measure as references
in M&E manual
Information needed to apply tools
OB1 1. No. of target group households with improved food security (months per year) with data disaggregated by
Annually Household Survey Household survey in villages
53
gender of the household head. Target number of households 39 150: Months with insufficient food per year reduced: 2004: 6 months; 2005: 5 month; 2006: 4 months; 2007: 3 months
where there are rice banks, roads, wells and agriculture programs, and use information of rice banks.
OB2
2. No. of households reporting increased production/yield increases (at least 50% increase). Targets:
Households 2004 2005 2006 2007Kampong
Thom 4,500 5,400 6,500 7,800
Kampot 5,500 6,000 6,500 7,000
Annually Agricultural component survey
OB3 OB2.1 3. Households and Commune Councils report satisfaction about frequency and accessibility and quality of services by government and non-government organizations. Targets: Satisfaction: at least 50% of households and at least 75% of CC report satisfaction (qualitative assessment every year through Beneficiary Impact Assessment and quantitatively in 2007). Standards of satisfaction to be defined.
Annually, BIA & BMW in 2005 and 2006 Section 5.4.
2007
Citizen survey; CC performance assessment survey
Projection of no. of households and commune councils using project supported GO and NGO services (from output assessments)
OB4 OC1 4. 50% of households and CBOs/village networks report satisfaction with the service of CC (Confirmation that the implemented development measures meet their most important needs and are beneficial to their social and economic development and the consideration of people’s priorities in Commune Development Planning, are considered ); qualitative assessment every year through Beneficiary Impact Assessment and quantitatively in 2007). Standards of satisfaction to be defined
Annually BIA & BMW in 2005 and 2006. Section 5.4.
2007
Citizen survey
54
55
OB5 5. 80 % of target Commune Councils fulfilling their designated tasks (according to respective prakas) to the satisfaction of PLAU.
Annually
PLAU CC assessment Realization of planned activities by commune councils
OB6 OB1.1 6. 50% of commune councils appropriately involve the population and their village organizations in exercising their duties (survey of the population and evaluation of documents, e.g. pertaining to the course of commune-level meeting, to the consideration of the population’s priorities for commune development planning, to the disclosure of the annual financial planning).
Annually 2007
BIA & BMW in 2005 and 2006. Section 5.4. Citizen survey (small in 2006 and full scale in 2007)
OB7 OB1.2 7. 60% of commune councilors/commune councils consider the co-operation with the decentralized/ deconcentrated administrative institutions on Province and District level effective, transparent and efficient (survey among commune councils (-lors?).
Annually 2007
BMW in 2005 and 2006 Section 5.4.2 CC performance assessment survey
OB8 OC2 8. 75% of projects supported infrastructure being regularly maintained through functioning LTCs/ FWUC (farmer water user committees). (780 water supply system and 200 Kms of tertiary rural roads till the end of 2004, from 2005 depending on annual planning of the commune councils; own data collection).
Annually
Analysis of CBO/LTC functioning data file; end-of-project evaluation. Infrastructure maintenance survey (small in 2006 and full scale in 2007)
Data from CBO/LTC functioning data file; end-of-project evaluation
OB9
OC3;OB2.2
9. No. of farmers adopting technologies recommended by the project (at least 30% are female headed HH); Targets: (accumulated)
Annually Agriculture component survey (small in 2006 3rd quarter, and full scale in 2007)
OB10 OC4 10. 50% of all households make use of at least one privately-offered service (e.g. sale of improved seeds and fruit trees, veterinary services) (own data collection)
Annually Agriculture component survey
Chapter 5
Monitoring Methods and Tools
5.1. Monitoring of Human Resources Section 5.1 deals with monitoring human resources development (HRD) for staff of implementing agencies. Human Resources Development for beneficiaries (farmers, villagers, commune councils) is also dealt with here.
Human Resources Development (formal training, on-the-job training, advice, backstopping etc.) is a key issue within the CBRDP. The main objective is to enable government officers for more efficient and effective services provision to target groups.
Human resource development is an output of the project on intermediary level (implementing partners like managers and extension staff of line departments) but not on the level of the final target groups (farmers, villagers, commune councils).
As methods and tools for monitoring HRD are the same for all components, no separation of methods and related tools is made in this Chapter.
There are three different monitoring exercises concerning HRD:
• Monitoring quantitative aspects of formal training (staff and target group training) on individual and group basis. This monitoring is on activity level and provides information on who received what kind of training for how long and where. (See also HRD data bases in Annex 7.
• Monitoring quality of formal training (staff and target group training) where the quality of trainers and the impact of training on participants’ knowledge and skills is regularly assessed on the basis of each training course (Section 5.1.2)
• General staff assessment where all CBRDP staff is assessed annually concerning their general performance over the year (Section 5.1.3)
Capacity building is one of the key issues in the CBRDP because capacities are, in general, very low in all fields of interventions, particularly concerning the ‘soft skills’ like planning, M&E and other management skills. Therefore, tracing the outcome of capacity building measures on implementing staff is of crucial importance.
The indicator concerning formal training measures for all components and sub-components (1.1, 3.1, 4.1, and 5.1), in principle, reads as follows:
56
Number of officers from a certain staff category who successfully increased their capacity to perform their tasks, based on defined criteria (type of officers per capacity building measure; according to training plan); 80% of trainees reach at least 60% score (according to the test for each training course). What does this mean?
A training plan for each component reflects the capacity development requirements for each staff. Accordingly, training courses are tailored to the capacity development needs or staffs are sent to training courses which match with capacity development needs.
The indicator above demands that, for each training course, 80% of the trainees reach at least 60% of the score. This necessitates that we assess the specific subject-knowledge of the trainees after the training course through a simple test and find out whether the trainee fulfils the requirements of the job concerning the subject taught to at least 60% or not. The main monitoring method will be the test for each course.
Before this monitoring of quality of training, the quantitative aspects also need regular monitoring.
5.1.1. Monitoring quantitative aspects of training and Target Group Events The quantity refers to the number of person-days of male and female staff members having been trained. To get these data the trainers will be asked to fill the general form of Report on HRD for staff, which is being used for all components for formal staff training. In case training is provided at national level or abroad, an individual staff member should complete this report, or one of staff members should be responsible for it if they join the same training. After filling it, the staff has to submit the form to M&E focal point in the component or sub component in the same way as the trainers of internal trainings report to the M&E focal points. To ensure completeness of data, DSA payments should be linked to the availability of the completed of the training.
Below is the template of the HRD report form. It covers different kinds of training events, including formal training classes, workshops, seminars, study tours etc., which can be analysed separately according to need. Also because every workshop has an element of training inbuilt, this form should therefore be filled for participating in workshops as well. The topic of the training event will be coded during data entry into the computer, so the data can be analysed according to topic or content of training. Report on HRD for staff
Duration: …………………………...………………………………………………………………… Start date: ……………………………………….…………………………………………………… End date: ……………………………………………………………………………………………… Place: ………………………………………………………………………………………………… Source of fund: ……………………………………………………………………………………… Trainer/Organizer: …………………………………………………………………………………… Topic: ……………………………………………….………………………………………………… Type: Training course Workshop Study trip other List of individual staff members participating
No. Name Function and/or organisation Sex Number of days
1 2 3
57
Each M&E focal point per component will use the data from this form to enter into the HRD for staff data file (see chapter 6 and annex 7). All analyses can be done on the basis of individual staff members.
Similar to the staff training, the quantity of target group events, refers to the number of person-days of different target groups and sub-groups (male and female), with events such as village meetings, class trainings, or the like have been carried out. Such groups can be commune councillors, village development committee, community-based organizations' committee etc. or ordinary villagers. For these types of events with target groups two kinds of forms are available to record the necessary information. While one form (Report on target group event B1) is more suitable for events that last longer than one day, the other form (Report on target group event B2) is more frequently used for one day or half day events. Templates of both forms are presented below. Both forms contain the same type of information and feed the same database, namely the target group event database (see Annex 7). The main difference of this data compared to the HRD data is, that while HRD data are monitored individually, target group data are monitored in numbers within group, but differentiated by gender only.
Likewise to the HRD report form, the target group event forms should be submitted to the M&E focal point in each component on a regular basis, but at least monthly.
In case GTZ or partner staff joins any target group event as participant, they would have to fill additionally the HRD report form, because their participation would be recorded as an individual HRD event in the HRD database.
Each M&E focal point per component will use the data from this form to enter into the target group data file (see annex 7).
Report on target group event (especially for event lasting more than 1 day (B1))
Duration: ………………………………. Start date: ……………………. End date: …………………. Place: …………………………….. Trainer/Organizer: ……………………… Topic: …………………….. Type: Training course Workshop Study trip other Table of participating target groups
Number of Female and Total Participants
Day 1 Day 2 Day 3 Day 4 Day 5 No Kind of Target Group6
Female Total Female Total Female Total Female Total Female Total
1 2 3
Total
6 The different kinds of target group are CC, CC sub-committee, VDC, LTC, GTZ staff, partner staffs, etc. For all kinds of staff, who in target group events as “trainees”, they have to fill the report on HRD for staff training separately.
58
Report on target group event (B2) 1. Date: ………………………………………………..…………………………….. 2. Start time: …………………………………….End time: ………………………. 3. Place: ............................................................................................................ 4. Total number of participants: …………………………………………….……… 5. Male: ………………………………..Female: ……………..…………………….. 6. Facilitator: ...................................................................................................... 7. Topic discussed: ............................................................................................ 8. Documents and inputs provided (free or as loans): ………..…………………. 9. Considerations and remarks: ……………………………………………………. 10. Signature of reporting person
The two forms B1 and B2 for collecting target group event information are only two examples for this data collection. Each component can use their own forms, if they like, as long as they make sure, that the forms contain all necessary data for monitoring target group events. The form “Village Event Minute” (see chapter 3.3 – Monitoring methods and tools for Community development) from the community development component is another example for collecting information on target group events. There is another form for interactions of service providers with individual or informal small target groups. The individual field visit form records information on the interaction with the target group, but serves mainly management to assess staff performance in terms of quantity and quality of field activities. A template of this form is given below. The data from this form are processed in a separate data file, except for the qualitative aspects, which could be scored by management for an advanced use of this tool of staff management. Individual field visit report 1. Date: …………………………………………………………………………….….. 2. Start time: ……………………End time: …………………………….…………… 3. Place: …………………………………………………………………….…………. 4. Name of farmer and address: …………………….……………………………… 5. Name of staff/extension worker: ………………………………………………………. 6. Topic discussed: ............................................................................................. 7. Documents and inputs provided to the farmer (free or as loan): …………………… 8. Consideration or remarks: ……………………………………………………………… 9. Signature of reporting person
5.1.2. Monitoring of quality of Training and Target Group Events The quality of all formal trainings (staff and target groups) refers to in-class trainings and will be assessed in two ways:
• Comparing the level of knowledge and skills gained by participants during the training by testing knowledge and skills before and after training.
• Assessing the quality of the trainer by occasional observation of the training by an experienced trainer and by feedback from the participants.
Quality of Trainees: Comparing level of knowledge and skills gained during training
The trainer of each training course will:
• develop test questions for each course which reflect the main topics of the course including an evaluation roster based on defined criteria (the trainer will define these criteria),
59
• conduct a test at the beginning and at end of each course by letting each participant answer the test questions by writing. For in-class trainings of target groups, the trainer should let the participants answer the test questions in groups (per committee, per village, etc.). The table below provides an example for a test format.
• analyse the results of the test and categorise the trainees into those who reach 80%, those who reach between 50 and 80% and those who do not reach 50% of the score (template below provides an example for a summary sheet), and
• hand over individual test results and summaries to the M&E focal point of the component.
Potential Format for Testing Training Success
Training Course Topic Date: Participant:
Score in % of total acc. to criteria
Test question 1:
Answer of the trainee:
Test question 2:
Answer of the trainee:
Test question 3:
Answer of the trainee:
………..
………..
Test question x:
Answer of the trainee:
Total score
Summary Sheets for Test Results per Course
Training course:
Number of participants: Trainer: Date:
Number of participants reaching 80% score or more:
Number of participants reaching between 50% and less than 80%:
Number of participants reaching less than 50%:
The M&E focal point will:
• inform trainers in advance of training courses about the need for the test, tell trainers to develop the test design and the criteria for test evaluation and instruct the trainer on the use of the reporting formats and the summary sheets per course (see form templates above), and
• collect test evaluation sheets from the trainers and analyze summaries per type of course annually according to the template below,
• discuss results and possible improvements with the trainers,
• make results available to decision makers and propose possible changes,
60
• keep individual test evaluation sheets in the personnel files of the staff as one basis for general staff performance evaluation (see Section 5.1.3) and
• occasionally assess the qualities of trainers (or to assure that an experienced teacher or trainers assesses the quality of the trainers) by observing the training process according to guideline criteria/questions as outlined below.
Summary Sheet for Test Results per Type of Course
Training Course:
Date/Trainer Total trainees 80% score or more 50%-80% score less than 50%
Grand Total (no.)
Grand Total (%)
Quality Assessment of Trainers The following are some guiding criteria and questions for occasional observation of trainers:
• In how far are the objectives of the training clearly defined?
• How well are the training contents structured?
• Are the contents presented in a way understandable to participants? Are the contents relevant to the participants’ expectations?
• Is the background material understandably prepared and presented? How relevant is the background material for the trained topic?
• Is the trainer more lecturing or more discussing issues with participants? In how far does he/she use questions for initiating participation? In how far is he/she asking for feedback from participants to encourage participation?
• In how far does the trainer include practical exercises and involve participants in practicing?
• In how far does the trainer use life examples, models, and pictures to get the message clearly to the participants?
• In how far does the trainer use other participatory methods like brainstorming, role plays etc.?
• How well does the trainer visualize contents and involve participants in visualization during the training session?
• How are the trainers’ body language, gesture, mimics, and voice modulation? In how far does he/she use simple words and understandable language or not?
This list is not exhaustive but at least can serve as a starting point. Other points can be added as requested. A scoring scheme will have to be developed to assess the qualities of the trainers. A scoring from 1 (very poor) to 5 (very good) is proposed for each of the guiding
61
criteria/questions of the assessment. The minimum requirement would be to reach 50% of the maximum score.
Trainers who do not fulfil the minimum requirements (contents and methods) would have either to go for capacity building measures themselves or to be replaced by other, more competent trainers.
The second way to assess the quality of trainers is done by the participants, who will answer at the end of each course the following questions in writing (Mood Barometer):
• What did you like during the training course (contents, methods, frame conditions)?
• What did you not like during the training course (contents, methods, frame conditions)?
• What should be changed for other courses?
The feed back information from participants will have to be made available to the respective management unit. The M&E focal points are responsible to collect these feedbacks, and to analyse and present them to management.
5.1.3. General Staff Assessment and Implementation of Recommendations Besides the monitoring of capacity building measures during particular capacity building exercises (mostly trainings), all staff working within the framework of the CBRDP will be assessed annually according to a specific procedure defined by the CBRDP (for details see procedure below).
Staff assessment will result in recommendations for each staff in terms of further capacity building needs, the need for re-posting and others.
The indicator concerning staff assessment for all Components and Sub-Components (1.2, 3.2, 4.2, 4.9, and 5.2), in principle, reads as follows:
80% of staff increased their capacities as proven through staff evaluation results and 80% of the recommendations from the annual staff assessment are implemented within 6 months of staff assessment What does this mean?
The indicator above spells out the following demands concerning monitoring:
• 80% of the staff increased their capacities in general with the efforts of the project during a time span of one year (training courses, on-the-job training, cooperation etc.). That means we assume that 20% of the staff probably will not have the basic capability or the opportunities to develop further with respect to their job performance.
• 80% of the recommendations spelled out for staff on the basis of annual staff evaluations are implemented within 6 months. That means that further training measures, re-postings will have been implemented within 6 months from staff assessments.
In principle, the general staff performance evaluation is done according to Seila's procedures and formats which are sometimes modified to meet the requirement of the management of each component. Although the Seila standard has to be maintained, there is flexibility in changing or adding criteria while keeping the total score the same (see Performance Evaluation Form below). This allows staff assessments to be more technical and component specific.
Potential procedures and formats for staff performance evaluation according to SEILA 1. OBJECTIVES OF THE EVALUATION.
62
An overall objective of the performance evaluation for CBRD-Project staff is to summarize the staff’s performance during the review period in relation to the agreed performance plan and job description. This review will improve effective work done by ExCom, Components and Subcomponents. The specific objectives are to:
1. Increase the effectiveness of human resource management system of the ExCom, Components and Subcomponents.
2. Improve the staff performance and their job description. 3. Identify the needs of staff development. 4. Identify the issues in the staff management system.
2. PROCEDURE OF EVALUATION. 2.1 Information. The meeting between PLG/GTZ, Chief/Deputy of ExCom, Permanent member, Components and subcomponents should be organized to inform the purposes, procedures and forms of staff performance evaluation. After the meeting, the Permanent member and/or Components and subcomponents will forward the information to all staff by holding a meeting or in a written letter. 2.2 Self-evaluation. Each staff is expected to do self-evaluation. The form used for self-evaluation is the one used by the panel. This result from self-evaluation will be used as one of references for the panel discussion to finalize the decision. This process is to allow staff to measure and expresses his/her own performance. Each staff member completes the form and addresses in closed envelope to the ExCom and/or Components and subcomponents for the panel. 2.3 Composition of Panel. The panel composes of Permanent member of ExCom, Components and subcomponents (or branch heads), Deputy Chief of ExCom can also be a member (option), and a PLG/GTZ representative (can be deputy or selected staff). It is additionally suggested to include one other advisor (and not an assessor) from concerned PLG/GTZ staff from technical sector/unit: for example if the panel makes an evaluation of staff from Secretariat Finance Unit - a PLG Provincial Finance Advisor should be called to advisor to the panel. 2.4 Management/Panel review. An important purpose of the management review will be for managers to reach agreement on those staff members who will be recommended for reassignment, training or discharge from employment. Upon receipt of a filled form from each panel member, the Permanent member enters the actual score of each staff into the table (table below). The Chief conducts the meeting with panel members to ensure that there is consistency in appraisals and ratings and to reach agreement on recommendation (the panel filled in the last column of summarize actual score table). The Chief submits a table with written statement explaining the result of the review of each staff performance to the Chief/Deputy of ExCom for approval. If necessary the Permanent member shall present the results of the recommendation of each staff reviewed at the ExCom meeting before approval. 2.5 ExCom Review. Upon receipt a written statement with detail explanation of each staff performance review from the Permanent member, the Chief/Deputy of ExCom will review and approve the recommendation. The Chief/Deputy of ExCom will then issues a letter address direct to staff to inform him/her the result (contract extension or discharge from employment). Staff shall have a change to defend or explain to the management or to panel if the result is not satisfactory. 3. FORM OF THE EVALUATION.
63
There are 4 forms that will be used to review the performance of ExCom and Components and subcomponents staff. The first form is to be filled by staff member for self-evaluation, the second form to be filled by TA (Technical Assistant), the third to be filled by Unit/chief and the fourth one to be used by the panel. The four forms are similar format with five main areas that are focusing on (see below). Each area is breaking down to four or five different points and each point scored 4 or 5. Each area has sub-total score 20. Total score for the five areas is 100. 3.1 Technical Expertise (four points) The extent to which staff member keep up to date with technical knowledge in their fields and the level of proficiency achieved as they apply the knowledge to their jobs. (1.1) Focus on knowledge and understanding of each staff toward his/her technical expertise. (1.2) Each ExCom, Components and subcomponents staff needs to have facilitation skills to performance his/her own given tasks effectively (it is considered important for all units within ExCom/ Components and subcomponents). (1.3) The ability to express thoughts clearly, accurately in writing. (1.4) Presentation: the ability to express thoughts orally in a clear, concise and accurate and effective manner. Each of the four points scores 5. Total 20 points. 3.2 Initiative (four points) The ability to use judgement, produce creative solution to problems and take appropriate level of initiative to apply his/her skills. (2.1) Focus on ability to use judgement within his/her own work when assisting villagers/local committees. (2.2) Able to work independently, and (2.3) Able to solve problems creatively. (2.4) Level of initiative to apply skills with a minimum of supervisory intervention from own supervisor and/or less facilitation support from PLG/GTZ staff. Each of the four points scores 5. Total 20 points. 3.3 Communication (four points). (3.1) Ability to listen within team meeting or during any working session or conducting facilitation meeting with local committees or villagers. (3.2) Able to use language appropriately with both villagers/local committees. (3.3) The ability to delegate work to subordinate and with local villagers or concerning partners. (3.4) Dissemination skills: able to disseminate verbal/written information to local committees/villagers effectively. Each of the four points scores 5. Total 20 points. 3.4 Personal relations and interaction with local community (five points). The ability to establish and maintain harmonious work relationship with colleagues, supervisor, and villagers (4.1) Focus on attitude and behaviours toward working partners and villagers. (4.2) Assess level of participation. (4.3) Adaptability and (4.4) flexibility of staff member within working environment both working team and at district, commune and village levels. (4.5) Assess ability of staff to work as team. Each of the five points scores 4. Total 20 points. 3.5 Management/commitment (five points). (5.1) Quality of works: focus on staff's work that is accurate, thorough, reflects sound judgment, meets appropriate standard and with reasonable time constraints (5.2) assess the level of ability to delegate work to concerned team members (if any) or those he/she is working with. (5.3) Responsibility: readiness to accept any given tasks or additional duties. (5.4) Punctuation: on time and able to meet most given tasks/assignment within time limit. (5.5) Monitoring ability: focus on ability and preparedness of staff to monitor within his/her own given responsibility. Each of the five points scores 4. Total 20 points. 4. REFERENCES OF THE EVALUATION. Additional materials/references that are commended to used for staff performance review are as follows:
1. Job description or Term of Reference. 2. Staff development policy. 3. Personnel regulation and rules (if any) 4. Staff contract. 5. Self-evaluation.
64
Here is the common form for staff performance evaluation. Even though this form used by different people, the content in the table is remaining the same. Performance Evaluation Form Name : .................................................................. Functional title : …………................................................... Entry of duty : ..............……….............…........................ Period of review : from ................... to …......................... No Points to be focused on: Please tick only one in
each point 1 Technical expertise: Max. score 20 1 2 3 4 5 1.1. Knowledge and understanding of own tasks 1.2. Facilitation skills. 1.3. Writing and reporting skills. 1.4. Presentation skills. 2 Initiative Max. score 20 1 2 3 4 5 2.1. Ability to use judgement. 2.2. Work independently. 2.3. Problems solve creatively. 2.4. Level of initiative to apply own skills. 3 Communication Max. score 20 1 2 3 4 5 3.1. Listening skills. 3.2. Language proficiency (Khmer, …) 3.3. Dialogue skills with local/villages. 3.4. Dissemination skills. 4 Personal relations and interaction with local community. Max. score 20 1 2 3 4 4.1. Attitude and behaviours. 4.2. Participation. 4.3. Adaptability. 4.4. Flexibility. 4.5. Ability to work as team. 5 Management/commitment. Max. score 20 1 2 3 4 5.1. Quality of works. 5.2. Ability to delegate. 5.3. Responsibility. 5.4. Punctuation. 5.5. Monitoring ability.
Grand Total Actual Score Grand Total Max. Score 100 Comments by: (Appraiser)….....…................................................................................................................ ............................................….................................................................................................................. ............................................….................................................................................................................. Signature of: (Appraiser) .........................…..............Date: ............................................ Sample Table of Actual Score for all staff members: No Staff name TA Unit/Chief Panel Average Self-
The table of actual scores shows us the performance of each individual staff in percentage of a maximum of 100 scoring points, and separately for each scoring party of the assessment committee. The higher the percentage, the better the performance of the staff. The assessment has to reveal an average score of at least 50% in order to be able to renew the contract with the project. For staff assessed between 50% and 60% special measures should be taken in order to improve their performance in the future. Basically there are two types of recommendations/ decision: • First recommendation/decision for keeping staff-members to continue their contract
(>50%), provide additional training to them (>50% and <60%) and/or to give any chance to them to get a promotion (>90%), etc. and
• Second recommendation/decision for discontinuing the contract of the staff member as project partner staff.
Respective component management is responsible to follow-up implementation of decisions arising from staff assessments. Component management can use M&E focal points to collect respective information. Component management should present the results of follow-up activities in an ExCom meeting not later than 6 months after staff assessment took place. The presentation should be made using the same table format as above (Table for actual scores of staff members) by adding an additional column for Follow-up of the Recommendations. This will make sure in how far the actions planned during the staff performance evaluation have been addressed by management. After the presentation in ExCom, the Permanent member will be responsible to aggregate the follow-up tables from each component into one table. When presenting the follow-up information, the table should contain answers to the following questions:
• Has this staff-member been discharged? If not, does s/he still continue with and receive allowance from CBRDP?
• If s/he continues with the project, does s/he have a new contract and for how long, and why?
• What happens after ending of the new contract of less than six months?
ExCom is responsible to inform PSU on the follow-up actions and results of staff assessment decisions. 5.2. Monitoring of institutional performance using SWOT
Section 5.2 deals with monitoring institutional performance of implementing partners. While capacity development for individual officers improves the knowledge, skills and attitudes of staff, it doesn’t automatically improve effectiveness and efficiency of service delivery of organisations and institutions to target groups. Therefore, the strengthening of institutions and organisations needs special attention to achieve CBRDP/RDP’s objective for more effective and efficient delivery of services to target groups. As the CBRDP/RDP- implementation strategy depends on local agencies, it is essential to strengthen these agencies – an important area of CBRDP/RDP intervention – and also to monitor whether our institutional development efforts yield the wanted results.
Improved performance of organisations and institutions is an output of the project on intermediary level (implementing agencies like ExCom and line departments) but not on the level of the final target groups (farmers, villagers, commune councils).
This M&E system proposes one tool only for the monitoring of institutional performance, namely the SWOT Analysis. The following describes this tool whereby organisations,
66
institutions or smaller unit are assessed with regard to Strength, Weaknesses, Opportunities and Threats.
The indicator concerning institutional performance for all Components and Sub-Components (1.4, 3.5, 4.5, and 5.6), in principle, reads as follows:
Institutions implement the self-assessment (including the results of beneficiary satisfaction assessment) of their internal management and services performance (SWOT) and follow-up upon its results (50% of weaknesses identified are addressed within 6 months of self-assessment) What does this mean?
The indicator above spells out the following two demands for monitoring:
• Each implementing partner organization (e.g. ExCom, components etc.) and/or sub-organizations (e.g. LAU, DCD Offices, District Agriculture Offices etc.) reflects on their internal management during a workshop along the SWOT methodology and particularly analyzes weaknesses in services provision to target groups
• Strategic solutions for the identified weaknesses are worked out and implemented latest 6 months after this workshop.
5.2.1. Background to SWOT-Analysis SWOT does not only aim at finding information, it also wants to raise awareness about existing problems and promote initiatives, so that the participants may agree to solve those problems together. But it does not want to stop at solving one problem. Development is a process and development methods have to aim at establishing processes conducive to development. The SWOT method is based on the following rationality:
• Life is a sequence of activities in the past, present and the future. This applies to individuals as well as to the life of organizations. Successful past activities are likely to be remembered. We call these our strengths (successes, achievements etc.) although we know that the level of success might not have been 100%, and that different individuals may judge successes differently.
• Other activities in the past proved to be failures, errors, mistakes, etc... We will call these our weaknesses.
• When we are able to further build on our strength and upon our successes we can use the opportunities (chances, potentials) which we have for improvements.
• Weaknesses may not harm us too much if we learn from them and avoid them in future. But if we let them occur often they will be a threat (risk, obstacles, and hindrances) to us and we might not reach our objectives if we do not try to overcome these threats. When we analyse the threats, we will often find that many of our today’s threats exist because we did not alleviate weaknesses during the past. These weaknesses consumed time and money in the past and today. The question arises whether there are possibilities to overcome our weaknesses. This starts with awareness about the weaknesses, including their causes and effects, and then reducing, avoiding or eliminating them. This way we save time, money and other resources. But saving them is only one advantage. We can benefit even more by investing them in new activities with more chances of success.
67
Diagram of the SWOT analysis: Past Now Future
Strength Successes and achievements which we experiences in the past
Opportunities Chances, potentials, objectives which we have for improving in the future
Weaknesses Failures, errors, mistakes which we experiences in the past
Threats Risks, obstacles, hindrances on our way to a better future
This basic concept of SWOT is elaborated into several steps, designed to focus and structure the discussions, to reach common understanding of its causes and effects and to agree on the actions to be taken by whom to remove or to alleviate the problem. SWOT thus smoothly crosses from problem analysis to planning and allocation of responsibilities.
The method has been used for development work since the early nineties in discussions on village level and to a lesser extent in agricultural research stations in a number of countries.
Participants’ capacity and attitude building is only one central objective of the support of participation and self-help initiatives – the one which starts the process moving. Even more important is to keep the process sustainable: By establishing a permanent learning process which enables the participants to solve their recognized problems better and with increasing independence. This process must incorporate the five components awareness building, problem solving discussions, planning of solutions, implementation and reflection – and each of them may require external support. This process cannot be established simply by the introduction of appropriate methods. It requires cultural changes, incorporating a new openness and transparency, a new utilization of experiences and mistakes, and a new feeling of responsibility for others. Change cannot be handled without appropriate tools. But tools alone are not sufficient for change. Similarly, participation can only be established if suitable working methods are available, but applying participatory methods does not necessarily lead to attitudes which promote participation. A development process ‘from below’ needs micro-projects as well as participatory methods, but it will only become sustainable if a learning culture can be established.
5.2.2. SWOT analysis as the key method for monitoring institutional performance SWOT is a working tool for organisations or part of it for self-evaluation. Basically people of an organization come together in a workshop and discuss and analyse their situation within their organization. SWOT can be used for large organisations (e.g. Department of Agriculture), medium size organizations (e.g. ExCom) or small organization (e.g. a District Facilitation Team). It is important that an experience facilitator guides the SWOT process.
There are many ways how to facilitate the self-evaluation process of an organization and facilitators conversant with the methodology and the local situation will use the method
68
flexible. Also, the methodology can be combined with different other tools (like Venn-Diagrams from the PRA toolbox etc.).
The following steps have proven to be promising at the beginning and are proposed as a starting point for CBRDP partner organizations (one day for smaller organizations; two days for larger organizations):
Step 1: Introduction to the SWOT workshop
• Facilitator explains objective (improve the services of our organisation by self-analysis of our strengths/successes & weaknesses/failures, opportunities/potentials for improvements & threats for failing in future etc.)
• Facilitator explains the methodology and procedures (see below)
Step 2: Plenary: Brainstorming with cards
• Where are we strong? What were our successes last year? What do our clients like with our services? In which aspects does our organisation function properly?
• Where are we weak? What were our failures last year? What do our clients not like with us? In which aspects does our organisation not function properly?
• Prioritizing strengths/weaknesses: Which are the most serious weaknesses which prevent us from providing good services to our clients? What are the ‘key strategic issues’ we have to tackle if we want to improve? Such prioritising is important as we usually can not tackle all problems at the same time. We need to know these problem areas which jeopardize project progress most seriously and those strengths which are least utilized. Then we can decide where to start.
Step 3: Purposive inputs Purposive inputs should be given if we have to deal with problems where we do not feel competent to find solutions. Here, we might call experts to enlighten us before we proceed with finding solutions to our problems.
• Individuals might present inputs and results from monitoring exercises which highlight and clarify ‘key strategic issues’ to enlighten participants for the following group work on certain issues with potential value for finding strategic option
• One issue to be presented and discussed could be the results from the annual staff assessments; others could be the results from specific surveys on the functioning of LTCs (in the case of infrastructure), farmer promoters (in the case of agriculture) etc. etc.
Step 4: Group Work (for each ‘strategic key issue’ one group)
We might decide to tackle the most serious problems separately in groups. We might even decide to tackle only the most serious problem now and postpone others to the next meeting or the next year….
• Which opportunities/potentials do we have to improve on our weaknesses and to build on our strengths? What strategic options do we have? Which objective, strategy should we follow in future?
• What are the threats, risks, obstacles which might prevent us to reach our objectives and strategies? How can we overcome these threats, risks, obstacles?
69
Step 5: Presentation of group work results in plenary and discussions
• Working groups present results of their work and clarify issues
• Agreements are made on actions to be taken (action plan)
SWOT should be used flexibly and according to the specific situation. There is no need to conduct all steps during one workshop/meeting. We could have several meetings covering one step during each meeting. We might also combine the different SWOT steps with regular staff meetings in order to reduce the additional work load for SWOT.
SWOT is a rather simple and straightforward method for self-analysis of organizations. However, the SWOT workshop needs thorough preparation (Venue? Material? Inputs?) and facilitation. When ‘doing’ SWOT the first time, professional support should be sought for moderation; the second time, moderation could be done by an experiences moderator from the CBRDP staff and the third time moderation could be taken over even from within the organization which is doing the self analysis.
The SWOT-analysis should be conducted annually. This would allow the organization to compare progress over time – which is one of the main objectives for monitoring.
An ideal time for the analysis would be the end of the operational year. The workshop should be conducted immediately before the annual planning for the next operational year. Workshop results would also be a valuable input to the annual plan of operation.
5.2.3. Follow-up of addressing weaknesses Conducting the SWOT analysis is only the first part, the implementation of solutions resulting from the SWOT analysis another part. Unless such follow-up actions are being implemented, no significant improvements in the performance of the organization can either be expected or monitored. Therefore it is important to monitor the follow-up of results of the SWOT analysis.
The actual SWOT analysis ended with identifying strategic solutions, the implementation of which need to be monitored. Management should therefore report to ExCom (and ExCom to Governor) at least once every 6 months the progress of implementation of actions resulting from the previous SWOT analysis. If no sufficient follow-up progress can be reported after 2 reporting periods (12 months), no further SWOT analyses should be done, unless other tangible changes occurred to justify another SWOT analysis. Management of respective units are responsible for the implementation of follow-up activities and for the reporting of progress. They may get the support from M&E focal points to document the progress. Reports on the progress should go to ExCom and PSU.
5.3. Monitoring of Micro-Projects and -Processes Micro-projects in the components Infrastructure (e.g. roads & wells, irrigation schemes) and Agriculture (e.g. demonstrations) is another key area of intervention of the CBRDP/RDP. Also, the initiation and management of a self-help organization in the component Community Development as well as the Local Planning Process in the component Decentralized Planning can be seen as ‘soft’ micro-projects: the initiation and support of processes on target group level.
The monitoring of micro-project implementation (‘hard’ or ‘soft’) on target group level is one of the most important aspects of monitoring in the CBRDP/RDP. Micro-project planning, -implementation and – monitoring happens mostly on village level and is supposed to benefit
70
people directly – if done properly. The monitoring of micro-project implementation is largely happening at output level and primarily used for operational management of implementation.
The following procedures are proposed for monitoring:
• The basis for micro-project planning and –implementation is always the quality criteria/technical design of micro-projects. This is the benchmark for monitoring. Each micro project or process needs a clear standard specifying the minimum quality criteria for each micro-project design (e.g. clear steps and quality criteria for the Local Planning Process or Self-Help Organizations, clear guidelines for the planning and implementation of a well, a road, a demonstration etc.)
• Micro-projects are supervised by technical staff during project implementation. Supervisory protocols should trace the implementation (particularly during the critical stages e.g. when the iron is laid out and before the concrete is added etc.). This Supervisory Protocol consists of two parts. Part one is the Individual Field Visit Report as described in chapter 5.1.1, which each officer needs to complete whenever he or she is conducting a field visit, whether it be for supervision or monitoring purpose. Part two of the Supervisory Protocol is the Site Inspection Protocol, which is specific to the component or sub-component. A site inspection protocol for a road construction looks different for a well construction than for an agricultural demonstration. Even a rice demonstration site inspection protocol will be different from the one for a pig demonstration. As these site inspection protocols monitor detailed technical aspects of the micro-project, each respective component is responsible to develop suitable site inspection protocol formats for their respective micro-projects. Some examples of site inspection protocols are presented in Annex 8. The procedure to collect information on the functioning of CBOs is also done with a site inspection protocol designed for that purpose. Information collected with site inspection protocols enter into respective databases, which are being maintained by the respective components. The M&E focal points of the component play a key role in maintaining these databases (see also chapter 6 for further details on databases).
• A ‘Project Book’ is an advanced alternative or addition to the site inspection protocol. Each micro-project would have a ‘project book’ on-site where all events in connection with the project (supervisory visits, arrival of materials, report on problems and solutions etc.) are noted down. Preferably the book would be kept with the local organization responsible for the micro-project (the CC in case of the LPP, the self-help-group in case of Self-Help Organizations, the Local Technical Committee in case of infrastructure projects, the farmer in case of demonstrations etc.). These organizations/committees/individuals would increase their ownership of the micro-project and they will enhance their responsibility to continuously watch project implementation. The project book will be used for evaluating the process of project planning and implementation on the local level, but it would be more difficult to aggregate the information from the project book for analyses on higher levels of monitoring. The next chapter describes more details on a ‘project book’.
• Each micro-project should undergo an end-of-project evaluation where qualified staff evaluates the quality of the micro-project before ownership is formally accepted by the beneficiaries. Although the ‘end-of-project evaluation’ derived from rural infrastructure projects, it might be a helpful tool also for monitoring other contracted directly implemented micro-projects or processes. Chapter 5.3.2 describes more details on the ‘end-of-project evaluation’ tool.
• Probably the most important tool to assess benefits of micro-projects and processes is an annual beneficiary assessment, because it addresses impacts at a higher level of the result chain. The tool of a Beneficiary Impact Assessment (BIA) assesses in how far the population benefited from different project interventions. Because of the overall
71
importance of beneficiary impact assessments for impact monitoring, a separate chapter (5.4) provides the details for this tool.
5.3.1. The ‘Project Book’ The ‘project book’ will be a notebook with specified headings or with a free format. The advantage of such a project book is that not only supervisory staff and visitors can record information in this book, but also the owner of the project themselves, too. The main important parameters to record in the project book are:
• Date
• Activity
• Observation (issue, problem, etc)
• Solution, Advise, Recommendation,
• Agreements, Responsibilities for follow-up actions,
• Name and signature of recording person
It is recommended to start always with a new page for every new day of recording. If the book is provided with a free format, the owner will have to make sure, that all important parameters are being recorded. This demands a lot of discipline from the project book owner.
If these parameters are recorded consistently by all parties involved, the project book provides a very good picture of the quality and time of the process of this micro-project implementation.
It will be difficult to perform an aggregated analysis from many project books. Therefore the main purpose of the project book lies in the local management of micro-project implementation.
5.3.2. End-of Project Evaluation Each micro-project (water well, roads, small irrigation schemes etc.) should be evaluated at the end-of-implementation stage. This is done at the moment at least for the infrastructure micro-projects and limited to the technical aspects (quality of the construction itself). During a technical evaluation the beneficiaries/clients are questioned to check if they are satisfied with the output of civil work e.g. well construction
The procedures and methods for this end-of-project evaluation are as follows:
• conduct end-of-project evaluation once every six months for all projects that were completed during the proceeding 6 months. The M&E focal point conducts the end-of-project evaluation with beneficiaries at the respective site, with support from TA (if available).
• the information collected will be analyzed and the results will be communicated to TSU (in case of wells, roads)/Irrigation sub-component (in case irrigation scheme) and ExCom.
This information gained from the end-of-project evaluation can be used to adjust/improve the implementation of civil works in the future.
In principle, the technical evaluation is done according to the SEILA's formats which are sometimes modified to meet the requirement of the unit or component.
Technical Evaluation Report
72
Technical Evaluation of CBRDP Works contract
Province: District: Commune: A. Information About the Project Project Name : One Project can be implemented in one contract, or more than one contract. Enter the number of contracts for this Project implemented in previous years, implemented this year, and still to implement in the future. Number of contracts completed in previous years Number of contracts being implemented this year Number of contracts still to implement in future years B. Information about the Contract Contract Number: Contractor:
TSO Name of Technical Supervisor: Private
Description of Project Outputs
Contract Cost:
Quantity Quality Comments No Description Village Planned Achieved Good Fair Bad 1 2 Completion Date
Planned Date: Actual Date: Comments: Describe any problems that occurred during construction of the works
Good Acceptable
Evaluate the work of the contractor
Unacceptable Good Acceptable
Evaluate the quality of service provided to the Commune by the Technical Supervisor
Unacceptable C. Environmental Issues Was an Environmental Analysis carried out for this Project? Yes No
If Yes, has the Environmental Management Plan been correctly implemented?
Yes No
Comments
Have there been any bad impacts on the environment as a result of the project?
Yes No
Comments D. Land Ownership Issues Was a Land Acquisition Report prepared for this project? Yes No
If Yes, were the land compensation agreements implemented correctly?
Yes No
Comments Has any land user made any complaint about losing land because of implementation of this project?
E. Overall Assessment of the Project How much benefit will this project bring for the people in the commune? Give a score from 0 (no benefit) to 10 (very high benefit).
Score
F. Recommendations for Action Action to be taken Reason for action Who is responsible?
73
G. Who was consulted during the Technical evaluation? Commune Chief Contractor Commune M&E Focal Point Technical Supervisor Others
H. Members of the Technical evaluation Committee and Signatures
Name Position Signature
Date of the Technical evaluation:
The second part of the end-of-project evaluation – which does not exist at the moment – should be informal and separate interviews with the village community and the LTC about the processes happening during the project cycle along the following interview guidelines:
Interview guidelines for the process during the end-of-project evaluation Participation in project planning
• How many men/women/youth/elderly people have been participating in the original needs assessment? How many of these groups have been participating in technical planning? How many of them have been participating in project implementation?
• What were the main problems during planning and implementation of projects? How were the problems solved? Who helped in problem solving?
• What should be made differently if another project is to be planned and implemented in the village (lessons learnt/best practices)?
• What should be made the same way if another project is to be planned and implemented in the village (lessons learnt/best practices)?
• Was the project the first priority of the community? If now, which priority was it?
Beneficiaries and benefits of micro-project
• Which groups benefit from the project? Who uses the infrastructure? How often is the facility used by different groups? (Specify numbers according to different people).
• Is there any visible impact from the infrastructure (fewer diarrheas, more income earning opportunities etc.)?
Securing long term benefits
• How does the facility look? How is the condition of the facility? Is regular maintenance visible or does the facility look rather not cared about at all?
• What kind of maintenance systems are in place? In how far are they working?
• What kind of problems does the community have with maintaining the facility? How are these problems solved?
74
• What would we do differently next time concerning maintaining the facility? What would we do again the same way? (Lessons learnt and best practices).
Effectiveness of services providers
• Which functions did the LTC perform well, which not well? What problems were coming up? How did the LTC solve these problems? How satisfied is the community with the LTC?
• How did the TSU perform during project planning and –implementation? Which functions did TSU perform well, which not well? How satisfied are people with the support of TSU?
• How did the contractor perform during project implementation? Which functions did he/she performs well, which not so well? How satisfied are the people with the contractor?
• Concerning the demand for services: what would people do different next time, what would they do in the same way? (Lessons learnt and best practices).
The results of this interview will give valuable hints on the appropriateness of the planning and implementation process, the participation of different groups, the methodology of site selection.
Lessons learnt can be drawn for planning and implementing future infrastructure projects.
5.4. Monitoring at Beneficiary Level M&E activities of the CBRDP/RDP during the last years concentrated on the tracing of activities and consequent physical achievements (number of commune councils trained, number of wells and roads constructed, number of farmers supplied with inputs or supported with demonstrations etc.). Some of the qualitative information is also available on outputs and outcome (direct benefit at purpose level) of the CBRDP/RDP. However, it is not enough for regular reporting. To cover qualitative aspects on output and outcome level, special data collection exercises are still highly needed.
Therefore, particular annual data collection exercises are planned to allow an approximation of the CBRDP/RDP’s outputs and outcome (lower impacts7). One of the methodologies for this purpose is the beneficiary impact assessment survey covering 13 of the planning framework’s indicators on output and outcome/purpose levels (the number of these 13 indicators are presented in column 4 of the interview guideline in Annex 3). This survey has to be seen in close connection with the subsequent beneficiary monitoring workshop.
By this way, the beneficiary impact assessment survey, together with the beneficiary monitoring workshop, can give sufficient information to trace the CBRDP/RDP’s indicators almost 30% on output level and all indicators on outcome/purpose level. The remaining indicators on output level are monitored with more formal methods (see Chapter 3 and 5 of this manual).
7 When we talk about outcome (lower impact), we mainly mean the indicators on purpose level of the project planning matrix (the goal level indicators can only be traced in longer intervals and possibly at the end of the project’s lifetime). Outputs and the use of outputs are, in the narrow sense, not yet impacts of the project as such. However, most of the outputs and the use of outputs can also be seen as lower level impacts of project activities. Example: Increased capacities of staff and farmers are a typical output of the CBRDP and not an impact. However, an increased capacity of staff and farmers is a clear impact of the project activities related to staff and farmer training. Therefore, we call the tracing of outputs also impact assessment within the framework of the beneficiary impact assessment survey.
75
The beneficiary impact assessment survey as well as the beneficiary monitoring workshop can only cover the subjective impressions of a small sample from the (potential) beneficiaries. It is, therefore, necessary to combine these subjective assessments with the data on the complete physical progress (see above) from the CBRDP/RDP Excel data bank.
Both, the complete data on physical progress and the subjective impressions from a smaller sample, allow projections on the likely impact of the CBRDP/RDP related to the indicator covered by the survey and the workshop.
5.4.1. The Beneficiary Impact Assessment (BIA) Survey The beneficiary impact assessment survey will cover all lower impact areas which can only be covered by directly discussing potential impacts with village communities themselves on village level.
The objectives of the beneficiary impact assessment survey are
• To assess changes induced by the CBRDP/RDP on target group level during intensive discussions with members of the target groups and
• To estimate impacts of the CBRDP on the basis of these discussions and on the basis of physical progress monitoring data.
The most appropriate method for these discussions is felt to be interviews with different sub-groups of beneficiaries and non-beneficiaries in their villages. An interview guideline has been developed for this purpose.
The following considerations give a brief outline of the different procedures necessary for the planning, conduct of the survey and for data analysis and reporting. The interview guideline is attached to this section as well (see below).
Procedures for the beneficiary impact assessment survey
Step 1: Plan and design survey • Form the core team (3 M&E officers, 3 M&E TA) and select 4 additional promising M&E
focal points for the survey. This allows you to work in 4 sub-teams with 2 officers each (plus the overall coordinator).
• Screen the indicator you want to cover and skip/add as needed
• Design interview methodology along an interview guideline (see also the BIA interview guideline in the Annex 3).
• Test interview guideline in 1 village (the data from this villages can be used for data analysis later on if reliable and useful)
• Be clear about how to make notes from the interviews
Experiences from other survey show that often the interview guidelines are longer than necessary. When designing interview guidelines, the following questions should always be kept in mind for each single question in the guideline:
o Does the question really contribute to the objectives of our survey (in the case of the Beneficiary Impact Assessment Survey: Does the question help us to contribute meaningful information for the indicators we want to follow-up?)
o If no, skip the question!
76
o If yes, can we get similarly reliable information easier (different questions, different methods)?
o If yes, change! In general, interview guidelines are made for a specific survey and purpose – here for getting sufficiently reliable information for our indicators. If these indicators change, we have, of course, to revise and adjust the interview guidelines accordingly.
Step 2: Prepare the survey • Select 5-10 villages per province according to coverage of the different field programmes
of the CBRDP
• Prepare all basic data available on selected villages (particularly data on the different CBRDP programmes which were conducted in this village: all records related to these programmes. This allows you to go deep into the subject during your interview and to relate to actual programmes).
• By involving the district staff member from CD and Agriculture component, brief village authorities of selected villages and ask for cooperation. If cooperative, set a date for the survey (1 day per village).
• Divide the survey team into 4 sub-teams and all of them will work together in a selected village in 1 day. Each sub-team will be responsible for interviewing the group concerning a specific project, e.g. CBO, Agriculture or Infrastructure. These roles should be regularly rotated.
Step 3: Pre-investigation visit to selected villages Each village chosen for the survey should be visited before the survey for the following purpose:
• Check the basic data you collected from secondary statistics (see step 2)
• Assess the suitability of the village for investigation (is the village an average village or is there anything which makes it extraordinary compared to other villages? If the village is completely different than the rest, it will give us little ideas about the general situation in the area and should be replaced by a ‘normal’ village for this survey)
• Collect obvious additional data which are important as preparation for the survey
• Discuss with village elders and officials the idea of the survey and seek their approval
• Agree with the village population a data for the survey which is appropriate to the village population
Step 4: Conduct the survey • Each village should be covered in ½ days, maximum 1 day. The rest of the day per
village will be used to reflect team internally on the discussions and results during the day
• It is proposed that the most experienced member of the sub-team conducts the interviews during the first day while the second experienced member of the sub-team takes down the notes. If there is a third member, he or she will mainly watch, observe and learn from the others. Roles can be changed at a later stage
• Survey methodology will follow the interview guideline below
77
Step 5: Analyse data and prepare user friendly report • The coordinator will collect the information related to each indicator form the survey sub-
teams in intensive discussions during three days. He/she will decide him/herself how he stores the information (either on wall chart or in a notebook). The ‘wall chart’ may also directly be prepared in the computer (See also chapter 6).
• During this process, hypotheses will come up in his/her ‘head’ related to the different indicators. He/she will discuss these hypothesis with the rest of the survey team and the whole team together will get consensus on how the situation is likely to be concerning this indicator
• The coordinator will then alone write a summary assessment for each of the indicators
• At the same time he will relate the information to the statistical data available from the components and try to project also quantitative progress on this basis
• The first report will be made in a way easily to be presented during the beneficiary monitoring workshops and during management information workshops/meetings.
Step 6: Give feedback of survey results to communities • Results of the survey will be fed back by representatives of all communities during the
beneficiary monitoring workshops (one per district or per two districts (see section 5.4.2))
• Participants will discuss and agree/disagree on the results thus verifying results or modifying them.
Step 7: Discuss survey results with CBRDP and component management • Survey results and feedback from commune representatives will be communicated to
and discussed with CBRDP/RDP and component management during a one-day-management workshop (see section 5.4.3).
• Consequent recommendations will be discussed and corrective action agreed upon, if possible.
Step 8: Final reporting • The results of the beneficiary assessment survey including the feedback from commune
representatives and result of CBRDP/RDP-management information workshop will then comprehensively summarized in an annual BIA (assessment survey) report and distributed to donors, line departments and other interested parties
• The final report should only be made after all forthcoming Beneficiary Monitoring and Management Information Workshops and it will include the discussion results of these workshops.
5.4.2 The Beneficiary Monitoring Workshop The Beneficiary Monitoring Workshop will on the one hand verify or correct the results from the beneficiary impact assessment survey.
On the other hand, the workshop will cover information needs for the tracing of additional indicators on output and purpose level, which can be covered best by discussion related issues with representatives from all communes.
78
The objectives of the Beneficiary Monitoring Workshop are
• To feedback the results of the Beneficiary Impact Assessment survey and to verify or correct impressions and impact estimations together with representatives of all communes and
• To discuss with representatives from all communes their opinion about additional indicators on purpose and output levels.
Each workshop should have not more than 50 people. It is proposed to invite from each commune several people including commune councillors, the survey team and selected M&E focal points.
The commune councillors should be the knowledgeable members of the commune councils (knowledgeable means having participated in the local planning process, knowing about local and external resources raised by the commune councils, knowing the commune and the provincial development plans etc.)
No project staff except M&E focal points should be invited to the workshop because the questions dealt with in the workshop partly concerns their performance as perceived by villagers.
Procedures for the beneficiary monitoring workshop Step 1: Plan and Design workshop • Form the core facilitation team (3 M&E officers, 3 M&E TA). This allows you to work in
maximum 5 groups during the workshop
• Define the existing information you want to discuss with participants (from beneficiary impact assessment survey and other monitoring results)
• Decide on time plan and methodology for the workshop (see the Potential programme/ methodology for the Beneficiary Monitoring Workshop in the Annex 3).
• Make a list of materials needed (inputs, charts, paper, felt pens etc.)
Step 2: Prepare the workshop • Decide on the number of participants to be invited
• Decide on suitable venue for the workshop
• Prepare all inputs (= monitoring results) for presentation and discussion
• Prepare all needed materials
• By involving the district staff member from CD and Agriculture component, invite participants and inform them about objective and contents of the workshop.
Step 3: Conduct the workshop • Introduction of participants, objectives, programme and methodology
• Conduct of workshop according to programme (possible programme outline see below)
• Closing of the workshop with information on follow-up, management decision making, provincial workshop etc.
79
Step 4: Analyse data and prepare user friendly report • The facilitators will document the information from the workshop and correct the results
from the beneficiary impact assessment survey according to the assessment of participants
• Together with the results from the beneficiary impact assessment survey related to all indicators on output and purpose levels, the CBRDP/RDP M&E core team (3 M&E officers, 3 M&E TA) will write an annual BIA Report (assessment survey report).
5.4.3. Management Information Workshop The results of all Beneficiary Impact Assessment and Beneficiary Monitoring Workshop will be summarized and presented during a provincial level workshop to the Project Director, the Governor, the ExCom permanent member, PSU, TA, Managers and M&E staff of all components and discussed in detail. Recommendations will be worked out for strategic changes, re-planning and management decision-making on this occasion. Project director and governor will chair this workshop.
The objectives of Management Information Workshop are as follows:
• To present the methodologies and results of Beneficiary Impact Assessment and district Beneficiaries Monitoring Workshop.
• To discuss critical issues with management of each component and draw conclusions
• To facilitate decision making on problem solutions and follow-up implementation.
Procedures for the Management Information Workshop Step 1: Plan and Design workshop • Clarify the role of different M&E task force members from different organizations in
conduction of this workshop.
• Define the existing information (from BIA and BMW) to be discussed with participants
• Decide on time plan and methodology for the workshop
• Make a list of materials needed (inputs, charts, paper, felt pens etc.)
Step 2: Prepare the workshop • Decide on the number of participants to be invited and make sure that all managers of all
components will participate in the workshop.
• Decide on suitable venue for the workshop
• Prepare all inputs (methodologies and results of BIA, BMW) for presentation and discussion
• Prepare all needed materials (charts, paper, felt pens etc.)
Step 3: Conduct the workshop • Introduction of participants, objectives, program and methodology
• Conduct of workshop according to program
80
• Closing of the workshop with conclusion, problem solutions and follow-up on management decision-making.
Step 4: Analyse data and prepare user friendly report • The facilitators will document the information from the workshop to be integrated into the
BIA report. However, the result from the management information workshop cannot always be used for correcting the results of beneficiary impact assessment and beneficiary monitoring workshop, for it is sometimes totally contradictory with the findings at the beneficiary level. Therefore, it should be part of the report but in a separate chapter.
81
Figure 2: Overview of Monitoring at Beneficiary Level
Storage, Retrieval and Analysis of Data and Information
When we talk about storage and retrieval of monitoring information, we mainly mean data on the physical progress of the project (numbers of people trained/reached, inputs distributed, planning processes facilitated etc.). These data represent the main output of the M&E system for the “lower end” of the result chains in CBRDP/RDP.
With the monitoring system introduced for the physical progress (quantity), we still need, of course, information on the quality of the physical outputs on higher result levels of the result chain. Qualitative methods are, in most cases, more suitable for measuring quality of results at all levels (from outputs to impacts) than quantitative methods. Therefore, the focus with the introduced monitoring system will be on assessing the quality of results at all levels mainly with qualitative methods (see also Chapter 5 above). But again here, we can use the statistics from the Excel data files to get an overview and select by random or by purpose communes, villages or target groups for the application of our qualitative methods (BIA, sample/adoption surveys, etc).
The following sections describe the database system for quantitative monitoring (6.1) and tools for qualitative monitoring (6.2).
6.1. Information on physical progress (quantity) Quantitative data are preferably stored in computerized databases. A database system has to be seen as an integral part of an M&E system. 6.1.1. General aspects of a database system A well functioning data base system must consider the following steps: a. design of database file structures (depending on analyzing requirements), The design of suitable databases starts with the identification of variables. The selection of variables depends on a number of aspects, such as whether the data are occurring regularly (i.e. date of weeding) or one time only (i.e. date of birth). After defining which variable to be included, each variable in a database needs to be classified for its type (i.e. date, number, text ...) and the possible values allowed for this variable. For example, numbers and text should not be entered together as one value of a variable. When arranging the variables in a database, one has to always keep in mind the purpose of the database, i.e. the need for different analyses, which should be possible with least modification efforts.
83
b. data collection (with respective data collection forms), For data collection, one has to differentiate clearly whether respective variables are being collected one time only, or in repetitive regular or irregular intervals. The efforts of data collection should be minimized, for example by integrating data collection into routine activity work. This could be the case for CBO functioning files, where field staff collects the information during their routine visits to CBOs. Data collection can be seen as the transportation of information from the field to the provincial office, where further processing of data takes place. c. data encoding (meaning entering data into the computer), Data encoding could be done by more or less anybody who knows the excel software. For quality reasons, however, it should be ensured that the person entering the data knows what he/she is doing (where figures come from, what they say) and that he/she can work concentrated in order to enter the data in a precise way. d. data validation (error checking), While data encoding could be done by more or less anybody, advanced Excel skills are required for data validation. Using the “Filter” function is very helpful for first data validation, except for certain “number variables”. For the latter, “minimum” and “maximum” checks can identify some obvious encoding errors. e. data storage and back-up system (incl. responsibility for master file?), For CBRDP/RDP quantitative data will be stored in Excel spreadsheets, which are designed as databases. Although the Excel program is not classical data base software, it provides a number of advantages for the purpose within the context of CBRDP/RDP’s monitoring system. They are foremost, that most staff at provincial level has experiences and sometimes advanced skills in operating Excel software, and further trainings facilities are easily available, within and outside the project. Making back-up copies of all data files after a working session with the files is good data management practice. For the responsible person of a master file it is recommended to have two back-up copies (A and B) of the master file on different data storage devices at any one time. Thus, loss of data should not be excusable. For each database file it is important to identify a responsible person for the “master file”. Anybody doing any analysis work should always start the analyses with a new working copy of the master file. Only then it can be avoided, that many different versions exists from the same data file. f. data analyses and extraction for reports or other uses of database, Using the Excel software means very much building the M&E system on existing resources and skills. Basic analyses can be easily performed with this software, like “sums”, “averages”, “percentages”, etc... This will satisfy most of the analysis requirements for standard CBRDP/RDP reporting of quantitative results. The existing data base files allow a huge variety of analyses on different levels, for day-to-day management steering up to basic research questions. More sophisticated analyses require a higher level of data management as well as statistical and analytical skills. The data from the Excel files are easily convertible to other software programms (like ACCESS or SPSS), and therefore do not limit the analyses work because of the Excel software.
84
6.1.2. Overview of CBRDP/RDP data files CBRDP/RDP’s database system presently has three major data files and a great number of components specific data files. The three major data files are the:
a) General village data file b) HRD data file c) Target Group Event data file.
The General village data file is a list of all villages in the province in the first column of the data file. Quantities of physical achievements are entered in respective columns and are always recorded on village level. The data file includes commune and district names, so that respective aggregations can be performed easily. The data file expands with additional columns according to time and needs. Typical column headings would be:
• No. of rice demos in wet season 2005 • No of VAHW • No. of pig raising demos in 2004 • No. of rice banks existing • No of wells constructed in 2005 • Km of roads build in 2004
The data files will be updated by M&E focal point in each component respectively for the columns that refer to a particular component at least twice a year. There is no special data collection form for this information. The M&E unit at the ExCom will compile the updated files from each component into one master general village data base. This village data file provides useful quantitative aggregate information, and a valuable source for sampling of surveys.
The HRD data file contains data on human resource development activities. Although trainings are the most important information, the data file also stores data on all other types of capacity building events, including workshop and meeting participations, study tours, seminars and conferences, etc... All these capacity building measures are recorded for each project staff individually, allowing easy overview of capacity building measures by topic and individual persons. The data file structure is presented in Annex 7.
The information is being collected in each component respectively, not only because the individuals involved are different persons in each component. A special data collection form (see section 5.1.1) is in use for the data collection. The M&E focal point in each component is responsible to systematically collect and update these information. The M&E unit of ExCom will put together the HRD data from all components twice a year.
The HRD data file allows to summarize training inputs, and at the same time provides information to steer capacity building programms and for staff career development.
The Target Group (TG) event file holds similar information as the HRD data file, but for target group events, like farmer classes, village meetings, etc... The main difference to the HRD data file is that target group participants are not monitored individually, but as number of total and female participants, only. The data for this data file include all type of group events with target groups, and the column for the type of event will allow analyzing the data by event type. While many events are with “ordinary villagers”, project staff conduct events often for specific targeted groups (for example commune councilors, Village Animal Health Workers, Village committee members, etc.). The number of participants is then recorded within each of these groups.
There are two types of data collection forms for these TG event files. They have a different format, but collect the same type of variables. While one form is more suitable for a one day event, the other form is more practicable for events which are longer than one day (see
85
section 5.1.1). A target group event of more than one day will be stored in the TG event file with one record line for each day. A separate “counter” column in the TG event allows easy summarizing number of events and event days. The main reason to record participants every day in an event that last longer than one day is that the number of participants at target group level often changes from one day to another.
The TG even file will be maintained in each component. The M&E focal points of each component take responsibility. If data collection is too incomplete, one should link data collection to the accountant, as normally for all events financial aspects are involved to conduct the event.
The TG event file allows many type of analyses, mainly the summary of field activities. It provides gender specific data from field activities, and also can be used for staff performance evaluation, because the data file contains the information on who has conducted the event. A complete list of the variables of the TG event file is in Annex 7. One should be aware for the evaluation, that the TG event file does not provide the data for “No. of people trained or reached”, but only the “number of people participating in events”, which can be the same people for different events. In addition to the three major data files, each component will maintain a number of component or subject specific data files. The following specific data files have been developed so far for different components: Agriculture: data files for each type of field demonstration with respective technical variables; VAHW performance file; Community Development: CBO establishment file (incl. LTC); CBO functioning files for cash associations, rice banks, LTC road, LTC well, etc; Infrastructure: data file for biddings and contractor contracts; separate technical data file for each infrastructure project, especially wells and roads; PID (Seila’s Project Implementation data base); Decentralization: No CBRDP/RDP specific data files have been designed yet for monitoring Commune Council performances, apart from the monitoring through the SEILA process. Data collection for each of these specific data files is expected to be organized by the respective components. Several data collection forms have been developed and are in use already (see Annex 4 for some examples). It is very common, that the data collection forms and the computer data files have a very different design (format), although they contain the same variables. 6.2. Information on quality Another important tool for qualitative and quantitative data collection are “sample surveys”. They will normally analyze using separate data files, which are developed specifically for the particular survey. The tool of “sample surveys” is discussed separately under the Chapter 4. Information collected with qualitative methods does, in most cases, not need computerized data storage, analysis and retrieval but depends to a great extent on written summaries of discussions with the target groups (mostly along interview guidelines with or without the use of PRA tools).
The question of how to store and retrieve the data is less relevant for qualitative surveys, because most information is analysed in different way, and summaries/conclusions are described in a report.
The analysis of information collected with qualitative methods (interviews, BIA etc.) is, unfortunately, more difficult than drawing statistics from a database. It needs strong comprehension capabilities. As CBRDP/RDP increasingly emphasizes qualities of their products/services and impacts, there is no other option than using more qualitative methods. The application of qualitative methods and tools require special skills and capabilities. If those skills and capabilities are
86
not available within the project staff, specially focussed trainings should be included in the CBRDP/RDP plan of operation, as well as the option to outsource some of these monitoring tasks. It is important to remember, that the need for more qualitative data does not replace the requirement for regular quantitative data but rather add “higher value” to the monitoring.
Analysis and documentation of the results of Qualitative Surveys There are a number of qualitative surveys (such as BIA, technical audits ...) to be conducted regularly or irregularly as part of the M&E system. They all have the tasks as follows: • To assess changes induced by CBRDP interventions, and
• To estimate results at all levels of CBRDP interventions…
The main method of data collection in qualitative surveys is interviewing along specific interview guidelines in a number of villages to be determined. The interview guidelines will cover all questions necessary to give satisfactory answers to the objectives of the respective survey. They can be supplemented by quantitative data from the various databases. Using the BIA as an example, the following practical proposals are made for the documentation of the interview results, data analysis and report writing.
Recommendations for documentation of interview results
• Use a notebook A4, hardcover, squares (A4 leaves sufficient space for your notes, hardcover makes writing easier, squares gives you a guide for straight lines in writing)
• Use only the left side of the notebook during interview when covering the respective question. If additional information to the same question comes up later during the interview, add on the right side (see example below)
• Draw a line (2cm) along the edge of the page where you can indicate the question number for easy reference. Also, this space you can use for cross-references to other questions (see example below)
Left page open book right page 1
2
3
see
12
etc
Use this space for your notes during interviews
Leave this space empty for adding information related to the same question which comes up later during the interview
87
Recommendation for analysis of information After the survey you will have information for all questions in the interview guidelines for a number of villages in your notebooks. However, information for each question in the guidelines is scattered in various books and it is extremely difficult to analyse information from the books itself. The following procedures are recommended for data analysis:
• Chose the most capable member from each interview team (3-4: analysis team) for all following work (if you chose more than 4, the additional people are rather confusing than helpful).
• Transfer all information from the books on one wall chart as demonstrated below (this can be done parallel by all members of the analysis team on the wall chart). The wall chart could easily be 1.5 by 6 Meters, depending on the volume of data available. Alternatively, the ‘wall chart’ may also directly be prepared in the computer. Then every participant will receive a print out of all answers from all villages for each question.
• Intensively analyse information concerning one question for all villages. Discuss issues amongst the analysis team and draw your conclusions together.
• One member summarizes information and agreed conclusions on a computer.
Analysing information in this way, the report will be structured along the same or a similar structure as the interview guidelines.
If you have stratified you sample villages (e.g. 3 from a high income/intensive agriculture area, 2 from close to the road with high marketing infrastructure, 1 from remote poor rural areas with high level poverty and little income opportunities), then you have to analyse the different stratas of villages separately (it makes no sense to draw the same conclusions for all villages if their problem situation is completely different and different support needs exist).
Example for a wall chart Ques-
tion 1 Ques-tion 2
Ques-tion 3
Ques-tion 4
Ques-tion 5
Ques-tion 6
Ques-tion 7
Ques-tion 8
Ques-tion 9
Ques-tion 10
…………
Village1
Village2
Village3
Village4
88
………
Annex ____: Data base file structure for Target Group Event File Record Identifier Unit: One target group event day (examples of codes below are used from agriculture component)
No.; Column heading; Type of data field; Data Values (or codes) 1. Running Number; Number; 1 to .....
9. Source of Fund; Text; Codes: “IFAD”, “GTZ”, “AusAID”, etc...
10. Organized by: Text; Codes: “CBRDP”, “CAAEP”,
11. 1st Trainer/Facilitator; Text, Name of one person only!
12. 2nd Trainer/Facilitator; Text; Name of one person only!
13. No of male participants: Number; 1 to ......
14. No of female participants; Number; 1 to ....
15. No of Total participants; Number; 1 to ....
89
Chapter 7
Reporting System
The following formal reporting system is working in CBRDP:
• The CBRDP components report monthly, quarterly, semi-annually and annually to ExCom. The core of the report is a comparison of planned activities with achievements according to the format in the table below.
• The ExCom reports quarterly, semi-annually and annually to PSU basically along the same format, but information summarized to some extent and with a free-format summary of main achievements, problems etc.
• The PSU summarizes the reports from ExCom semi-annually and annually for reporting to UNOPS/IFAD. Basically, again quantities along the activities are reported. In addition, a free-format summary is made on staffing, budgeting, non-achievements/failures and the reasons. This summary reports also on strengths and weaknesses of all components.
• The same report to the UNOPS/IFAD is also sent to PLG/Seila.
Sample format of reporting from components to ExCom and from ExCom to PSU Component/Sub-
Component Targets from the Annual Work Plan
and Budget Achievements
Activity Quantity Unit Quantity % of Plan
The following proposals are made for future reporting:
• Mere physical achievements should be summarized and not reported in detail as at present (see PSU annual progress report 2002)
• The focus of the report should be on the assessment of achievements (physical quantities) and – more importantly – the qualities of achievements and impacts. Focus should be on the questions like: Which of our targets did we not achieve in high quality? What are the causes for non-achievements? What would be most appropriate leverage points for improvements?
90
Background Documents
1. ADESS, 2002: Training Report on Participatory Impact Assessment (Report on Training Conducted by Outreach, Bangalore, India).
2. CAAEP, 2003: Monitoring and Evaluation. Annual Report 2003.
3. IFAD, 2002: Managing for Impact in Rural Development. A Guide for Project M&E.
4. OAE, 2003: M&E Manual.
5. PLG/Seila, 2003: Seila Program Monitoring and Evaluation Manual.
6. PLG/ Seila: 2002: Handbook on Commune/Sangkat Monitoring and Evaluation.
7. PLG/Seila, 2002b: Monthly Reporting Format from PLG Province Advisor to Seila and Example from Kratie-Province (August 2002).
8. PLG/Seila, 2002c: Quarterly Reporting Format from PLG Provincial Advisor to Seila and Examples from Kampot, Kampong Thom and Prey Veng (all 3rd Quarter 2002).
9. PLG/Seila, 2002d: PRDC ExCom Quarterly Progress Report Format to STF and Example from Battambang (for 3rd Quarter 2002).
10. PRASAC, 2002b Monitoring and Evaluation Manual.
11. RGOC, 2002a: Training Manual on Commune/Sangkat Development Project Preparation.
12. Rock, F., 2002: Summary Report on Findings and Recommendations with Regard to German TA.
13. CIDA, 2000: Result-Based Management Handbook on Developing Results Chains.
14. GTZ, 2004: Results-based Monitoring Guidelines for Technical Cooperation Projects and Programmes.
The goal of the project is to reduce the poverty of the targeted households in the project area. (Source: IFAD/CBRD Loan Agreement, RDP offer) Poverty reduction and improvement of living conditions in rural areas (Source: GTZ/ RDP; political development benefit)
1. Poverty incidence reduced by 50% between 1990 and 2015.
2. The number of Most Vulnerable Families in selected project villages is reduced by 30% in 03/08.
3. % of male and female children < 5 years malnourished (weight for age), chronically malnourished (height for age) and wasted (weight for height) compared to the level at programme start-up. (Stunting of children at appraisal < 5 years of age 36% in Kampot and 49% in Kampong Thom)
4. Increase in the number of cattle (or other assets) owned by target group households with data disaggregated by gender of the household head.
Objectives Indicators
Objectives The purpose of the project is to assist approximately 39 150 poor households in the project area to sustain increased food production and farm incomes from intensified and diversified crop and livestock production and increase the capacity of the members of the target group to use the services available from the government and other sources for their social and economic development (Source: IFAD/CBRDP Loan Agreement). The rural population in the provinces of Kampot and Kampong Thom are able to effectively better satisfy their basic needs and have more food and income at their disposal. (Source: GTZ/RDP; indirect
1. No. of target group households with improved food security (months per year) with data disaggregated by gender of the household head. Target number of households 39 150: Months with insufficient food per year reduced: 2004: 6 months; 2005: 5 month; 2006: 4 months; 2007: 3 months.
2. No. of households reporting increased production/yield increases (at least 50% increase). Targets:(accumulated)
3. Households and Commune Councils report satisfaction about frequency and accessibility and quality of services by government and non-government organizations. Targets: Satisfaction: at least 50% of households and at least 75% of CC report satisfaction (qualitative assessment every year through Beneficiary Impact Assessment and quantitatively in 2007). Standards of satisfaction to be defined.
4. 50% of households and CBOs/village networks report satisfaction with the service of CC (Confirmation that the implemented development measures meet their most important needs and are beneficial to their social and economic development and the consideration of people’s priorities in Commune Development Planning, are considered ); qualitative assessment every year through Beneficiary Impact Assessment and quantitatively in
140 Annex 1 CBRDP-RDP LogFrame 0306
benefit, impact) Measures for social and economic development are implemented effectively and efficiently, according to demand, jointly by the rural population, the commune councils and public and private service providers. (Source: GTZ/RDP; direct benefit, outcome))
2007). Standards of satisfaction to be defined.
5. 80 % of target Commune Councils fulfilling their designated tasks (according to respective prakas) to the satisfaction of PLAU. Discussion once a year with PLAU (Guideline for discussion to be designed a long the designated tasks).
6. 50% of commune councils appropriately involve the population and their village organizations in exercising their duties (survey of the population and evaluation of documents, e.g. pertaining to the course of commune-level meeting, to the consideration of the population’s priorities for commune development planning, to the disclosure of the annual financial planning).
7. 60% of commune councilors/commune councils consider the co-operation with the decentralized/ deconcentrated administrative institutions on Province and District level effective, transparent and efficient (survey among commune councils (-lors?).
8. 75% of projects supported infrastructure being regularly maintained through functioning LTCs/ FWUC (farmer water user committees). (780 water supply system and 200 Kms of tertiary rural roads till the end of 2004, from 2005 depending on annual planning of the commune councils; own data collection).
9. No. of farmers adopting technologies recommended by the project (at least 30% are female headed HH); Targets: (accumulated)
10. 50% of all households make use of at least one privately-offered service (e.g. sale of improved seeds and fruit trees, veterinary services) (own data collection).
Output 1 Indicators Management structures and procedures for decentralized management of development activities are in placed on provincial level.
1. No. of PRDC members, ExCom Unit members, DFT/PFT members and LAU management staffs who have successfully increased their capacity. Targets: 70% of trainees reach at least 60% score (according to the test for each training course).
Members PRDC CAU FU LAU PFT DFTKampong Thom 8 6 4 2 8 31 Kampot 8 7 4 2 8 35
2. % of staff who have increased their capacity as shown by staff evaluation results. Target: 80% and 80% of the recommendations from the annual staff assessment is implemented within 6 months after staff assessment.
3. Contracts between ExCom and line departments are established and executed according to contract specification. Target: 90%.
4. PRDC/ExCom implement an annual self-assessment of their internal management (including the results cooperating partners’ satisfaction assessment) and services performance (SWOT) and follow-up appropriately. Target: 50% of weaknesses identified are addressed within 6 months after self-assessment.
Output 2 Indicators Commune Councils are enabled to prepare commune plan with village participation and to manage their implementation and operation.
1. No. of Commune Council members who successfully increase their capacity to perform their tasks, based on defined criteria. Targets:
Members 2004 2005 2006 2007 RemarksKampong Thom 400 450 500 500 Out of 575 councillors.Kampot 260 280 300 320 Out of 542 councillors.
2. CBRDP supported Commune Investment Plans are of high quality and followed up with high quality monthly reports (according to NCSC standards). Target: 50% (increase every year).
3. No. of commune investment plans prepared. Targets: Kampong Thom annually 81. Kampot: 92 annually 4. No. of commune investment plans implemented. Targets: Kampong Thom annually 81. Kampot: 92 annually. 5. % of commune councils raising local resources. Targets: 90% 6. % of commune councils raising external resources from at least one source. Targets: Kampong Thom: 50%
Village population and organizations of civil society are enabled to undertake self-help activities and to play an active role in local self-administration.
1. DCDT/PCDT team members who successfully increased their capacity to perform their tasks, based on defined criteria (per capacity building measure; according to training plan). Targets: 80% of trainees reach at least 65% score (according to the test for each training course).
Members PCDT DCDTKampong Thom 4 22 Kampot 8 33
2. CD staff increase their capacities as proven through staff evaluation results. Targets: 80% and 80% of the recommendations from the annual staff assessment are implemented within 6 months after staff assessment.
3. CBO/Village Network leaders and committee members who receive training successfully increased their capacity to manage their CBO transparently. Targets: Kampong Thom: 94 CBOs (450 persons) annually. Kampot: 162 CBOs (810 persons) annually. 2004: 60%, 2005: 65%, 2006: 70%, and 2007: more than 70% increased capacity; and 35% of committee members are women.
4. All water supply and road projects have a LTC with at least 30% of committee members are women. Targets: Kampong Thom: LTC: 2004: well 360, road 6; Kampot: LTC 2004: well 245, road 105; 2005-2007 "according to RIIF".
5. All community management groups have women in leadership positions; 10% of the groups have women leaders.
6. No. of people (beneficiaries) belonging to community based groups by type of group. Targets: Kampong Thom: 2004: RB: 10,500; CA: 23,500. LTC: 2004 well: 30,000; road: 30,000; 2005-2007 "according to RIIF". Kampot: RB: 40,000, cash association: 2,000, catering club: 7,000. LTC: 2004 well: 30,000, road: 30,000; 2005-2007 "according to RIIF"
7. Community Development Component implement the annual self-assessment (including beneficiary impact assessment) of their internal management and services performance (SWOT) and follow-up the results. Target: 50% of weaknesses identified are addressed within 6 months of self-assessment.
Gender awareness is increased within all line departments and communities and women are enabled to play a more active role in local governance.
8. 50% of women actively participation in local planning process, project management and decision making. 9. No. of gender focal points in Commune Council and Line Departments who successfully increase their
awareness on gender issues. Target: 80% of trainees reach at least 60% score, from 2004 and annually. Focal Points Communes Districts Line Departments NGO Kampong Thom 81 8 27 0 Kampot 92 9 33 5
10. No. of gender training activities completed: (i) gender focal points; (ii) NGOs and commune councils; and (iii) individuals. Targets to be developed in the AWPBs.
143 Annex 1 CBRDP-RDP LogFrame 0306
Output 4 Indicators Support services (technical and CD) for the construction of rural infrastructures (roads, water supply, irrigation schemes, land titling) are delivered and LTC/WUC enabled to manage on-going maintenance needs.
1. No. of TSS members successfully increase their capacity to perform their tasks, based on defined criteria (type of officers per capacity building measure, according to training plan). Target: 20 in Kampot and 20 in Kampong; 80% of trainees reach at least 60% score (according to the test for each training course).
2. No. of staff who increase their capacities as shown by staff evaluation results and recommendations from the evaluations implemented. Targets: 80% and 80% of the recommendations implemented within 6 months after staff assessment.
3. Km of roads constructed according to quality standards as set in the Seila manual. Targets: Kampong Thom: up to 2004: 100 km. Kampot: up to 2004: 100 km. For 2005/2006/2007 figures to be determined once project priorities are determined under the RIIF.
4. No. of water points (wells) drilled/dug for drinking water according to quality standards as set in the Seila manual. Targets: Kampong Thom: Up to 2004: 360; Kampot: Up to 2004: 245. For 2005/2006/2007 figures to be determined once project priorities are determined under the RIIF.
5. 75% of public works activities (infrastructures) and other small projects assisted by service providers satisfy the government-defined quality criteria like Seila Technical Manual (results of regular technical evaluations).
6. No. of households served by wells (water points). Targets: Kampong Thom: accumulated target end 2004 for water points: 5,500 households. Kampot: accumulated target end 2004 for water points: 4,000 households. For 2005/2006/2007 figures to be determined when the commune councils determine their priorities for use of the RIIF and the numbers of households will be increased accordingly.
7. Rural Infrastructure Component implements an annual self-assessment (including the results of beneficiary satisfaction assessment) of their internal management and services performance (SWOT) and follow-up upon the results. Target: 50% of weaknesses identified are addressed within 6 months of self-assessment.
Land Titling
8. No. of households provided with long-term security of tenure of land. Targets: Land titles issued: Kampot 2004: 6,500. Kampong Thom 2004: 3,500; 2005: 5,000.
Irrigation
9. Sufficient PDWRAM staff with suitable skills (technical and community facilitation) to perform defined tasks. Targets: staff evaluations show that 80% of staff increased their capacities and 80% of the recommendations from the annual staff assessment implemented within 6 months after staff assessment.
10. Technical planning and supervision of project irrigation schemes is undertaken on behalf of commune councils
144 Annex 1 CBRDP-RDP LogFrame 0306
on time (including projects under RIIF) and according to professional standards.
11. All irrigation schemes constructed have a functioning WUC dealing with O&M and water distribution. Targets: Kampong Thom: 2004: 7 schemes "7 WUCs". Kampot 2004: 2 schemes "2 WUCs"; 2005 according to RIIF. At least 50% of the members of the all the WUC assisted by the project are women in Kampong Thom and 35% in Kampot.
12. Ha. of newly irrigation schemes rehabilitated/constructed. Target: Kampong Thom: 2004 (2 schemes): wet season 400 (dry season 413). Kampot: 2004 (1 scheme): wet season 750 (dry season 300).
13. No. of water user committees newly formed/ strengthened. Targets: Kampong Thom: 2004: 7; Kampot: 2004: 2.
Output 5 Indicators Demand oriented services to support the crop and livestock production in place. 1. Number of CMTST-, DOT-members who successfully increased their capacity to perform their tasks, based on
defined criteria (type of officers per capacity building measure; according to training plan). Targets: 80% of trainees reach at least 60% score.
Members CMTST DOTKampong Thom 10 56 Kampot 10 49
2. 80% of staff who increased their capacities as proven through staff evaluation results. 80% of the recommendations from the annual staff assessment implemented within 6 months after the assessment.
3. Services supplied are increasingly in response to demands expressed in the Commune Investment Plan (2004: 50%; 2005: 60%; 2006: 70%; 2007: 80%)
4. No. of private agricultural service providers (e.g. VAHWs, FPs) participating in capacity building measures to a level which increases farmer demand for their services. Targets:
Service Providers trained 2004 2005 2006 2007 Kampong Thom 200 250 250 250 Kampot 290 210 250 220
5. No. of extension events attended by target households. Targets:
Kampong Thom 39,000 46,000 62,000 77,000Kampot 9,000 73,000 73,000 73,000
7. Each Agriculture Component (incl. District level) implements an annual self-assessment (including the results of beneficiary satisfaction assessment) of their internal management and services performance (SWOT) and follow-up upon the results. Target: 50% of weaknesses identified are addressed within 6 months after self-assessment.
Output 6 Indicators Farmers are enabled to improve their subsistence production and to expand their market oriented production.
1. No. of households reporting livestock losses. Targets: Kampong Thom & Kampot: based on data collected by VAHWs, all households report decrease of annual livestock losses of at least 3% (cattle and buffalo only).
2. No. of people trained by gender and sector. Targets: (female 40%).
4. No. groups of farmers (associations) that improved their capabilities in selling their products. Targets: Kampong Thom: 6 (2 organic rice asso. 1 VAHWs asso. and 3 seed producer asso.). Kampot: 12 (1 cooperative, 4 seed producer asso., 3 new VAHWs asso. and 4 old VAHWs asso.).
5. No. of demonstrations and associated activities that have been fully implemented according to guidelines and quality standards. Targets: Demonstrations (crop plots, animal husbandry demonstrations) include establishment of demonstration, 3 farmer classes and 1 farmer day with at least 50% female participants.
AWPB endorsed by STF submitted by 15th November each year.
Semi-annual and annual progress reports submitted by 30th September and 31st March each year.
Financial statements submitted to IFAD/UNOPS by 31st March each year.
Audit Report submitted to IFAD/UNOPS by 30 June each year. th
PSU submits the reply to the management st July each y letter by 31 ear.
RIMS
of direct beneficiaries from all projects/components done in the year).
o well x 15 HH x 5 people
Persons receiving project services (direct, total project, m/f). (Calculation will be done by aggregating the no.
ng along the road x 5 people
TC members.
o Km road x No. of HH livi
o No of HH receiving land titling x 5 people
o No. participating in agri. extension event.
o Irrigation HH x 5 people
o Total No. of CC, CBO, L
o Total population in the whole province.
Persons receiving project services (indirect, total project, m/f)
147 Annex 1 CBRDP-RDP LogFrame 0306
Annex 2: Job Descriptions A. Job description for M&E Focal points Each M&E focal points for each component/sub-component/unit, except for LAU/PoLA, has a numbers of similar M&E tasks and responsibilities. The Tasks and Responsibilities of M&E Focal Point for Community Development Component (CD) is as below:
• Define, on the basis of the plan of operation the information needed from all implementing officers (PCD; DCD), the time frame for information delivery and the mode of delivery.
• Organize preparation of annual plan of operation and budget for CD component.
• Organize consolidation and submission of monthly workplan to ExCom (based on the annual plan of operation) including M&E activities.
• Ensure the quality of format filling by PCD; DCD.
• Monitor and evaluate the performance of PCD; DCD by checking the district and individual work result.
• In advance of key management meetings, summarize certain information which is not reported by the implementing officers themselves and present the information either during the management meeting or before to the CD component manager.
• If necessary, report special events immediately to the CD component manager if management decision-making is necessary immediately (e.g. problems at targeted communities, urgent staff issues etc.).
• Organize review and analysis of reports (compare the result achieved to the workplan) and recommend to CD component manager for any actions.
• Maintain documents of management decisions made and secure follow-up of decision-making (Are decisions really implemented?)
• Organize consolidation of monthly, quarterly, semi-annual, annual reports for CD component manager and ensure the submission to ExCom.
• Administer all documents related to community development works prepared by the CD component staff and TA.
• Organize a transparent progress monitoring and reporting system with adequate quantitative and qualitative indicators to allow the project staff (PCD; DCD) to effectively monitor the progress, performance and impact of the component in all targeted villages and districts.(e.g. conduct a monitoring and evaluation of each CD project).
• Take part in annual beneficiary impact assessment and beneficiary monitoring workshop to study about the involvement and satisfactory of CBO committee/LTC and Beneficiaries.
• Maintaining the database of CBO, LTC, Vulnerable families, HRD (for both staff and target group)…and preparing regular database reports for submission to the ExCom.
• Facilitate an annual performance assessment of CD component (SWOT-analysis).
• Initiate in quality of trainers assessment (both staff and target group training).
• Take part in process of staff assessment.
• Prepare and submit end of contract (between ExCom) reports to ExCom.
• Other tasks will appear upon request of the CD component manager.
101 Annex 2 Job description
As the monitoring and evaluation which the LAU support is part of the M&E system which operates within the Ministry of Interior, and which links to DoLA, the Provincial Offices of Local Administration (POLA) and Communes, the LAU supports capacity building and the functioning of the commune councils and the M&E Focal Point for this unit has the following roles and responsibilities:
• Monitoring and evaluating the performance and capacity of the communes to manage the implementation of the regulatory framework, to effectively administer the commune and to promote socio-economic development of the commune.
• Define, on the basis of the plan of operation the information needed from all implementing officers (PFT/DFT), the time frame for information delivery and the mode of delivery.
• Organize preparation of annual plan of operation and budget for Local Administration Unit.
• Organize consolidation and submission of monthly workplan to ExCom (based on the annual plan of operation) including activities to assist the communes to carry out monitoring and evaluation at the local level.
• Oversee the implementation of PFT; DFT in monitoring and evaluation of the communes in accordance with approved guidelines from the MoI/DoLA.
• Ensure that PFTs and DFTs are properly trained in the monitoring, evaluation and reporting system for the communes, and can adequately support and coordinate the delivery of training to the communes.
• Ensure the quality of format filling by PFT; DFT.
• Monitor and evaluate the performance of PFT; DFT by checking the district and individual work result.
• Assist the LAU Chief to organize regular progress review meetings with the Facilitation Teams to identify strengths and weaknesses in performance and to solve problems related to the implementation of decentralization.
• In advance of key management meetings, summarize certain information which is not reported by the implementing officers themselves and present the information either during the management meeting or before to the LAU chief.
• If necessary, report special events immediately to the LAU chief if management decision-making is necessary immediately (e.g. problems at the local level, urgent staff issues etc.).
• Organize review and analysis of reports (compare the result achieved to the workplan) and recommend to LAU chief for any actions.
• Maintain documents of management decisions made and secure follow-up of decision-making (Are decisions really implemented?).
• Organize consolidation of monthly, quarterly, semi-annually, annually reports for LAU chief and ensure the submission to ExCom.
• Take part in annual beneficiary impact assessment and beneficiary monitoring workshop to study about the involvement and satisfactory of CCs and Beneficiaries.
• Maintain HRD database (for both staff and target group) and monitoring reports,… and prepare aggregated database reports for submission through the ExCom to the national authorities in accordance with approved formats.
• Facilitate an annual performance assessment of LAU (SWOT-analysis).
• Initiate in quality of trainers assessment (both staff and target group training).
• Take part in process of staff assessment.
102 Annex 2 Job description
• Prepare and submit end of contract (between ExCom) reports to ExCom.
• Other tasks will appear upon request of the LAU chief.
B. Job description for M&E Officers (ExCom/CAU) Tasks and Responsibilities of ExCom- M&E Officer according PRAKAS 292 STF dated 08 Nov, 2002
Assist the Executive Committee in the monitoring and evaluation of the annual work-plan and budget in accordance with procedures and principles of the CBRDP.
Provide support and training to relevant staff of provincial departments in monitoring and evaluation and the implementation of contracts financed by CBRDP.
Prepare progress reports, quarterly, annual, donor reports and submitting to the Executive committee for review and endorsement in accordance with reporting guidelines and procedures.
Coordinate with the Local Administration Unit to ensure consistency between the commune councils and provincial monitoring and evaluation systems.
Develop a work-plan and schedule for monitoring and evaluation the contract implementation in accordance with criteria approved by the Executive Committee.
Collect and document information on lessons learned within the CBRDP including cases studies and researches.
In collaboration with LAU in developing simple information materials that can be understood by villagers and to promote understanding of decentralization principles and local awareness of commune plans and activities.
Produce and distribute the provincial development newsletters.
Coordinate with the provincial information department on the dissemination of information through radio, TV and print media.
Safe keeping the Executive Committee Resource Library consisting of documents, data and information materials received from national and provincial institutions, IOs/NGOs and other development agencies.
C. Job description for M&E Program Assistant, Kampot Reports to: GTZ-RDP Provincial Coordinator Kampot
Supervises: None
Counterparts: Contract and Administration Unit (CAU) M&E Staff (3), Line Departments M&E Focal Points (5), and the CAU staff (2)
Duty Station: Kampot ExCom Office, occasional travel to project sites within Kampot
1. Goal of the Position Through advice on technical and organizational matters, systematic human resource development measures and intensive coaching of the mandated government institution’s staff, the target population receives effective services to improve their livelihoods.
2. Scope of Responsibility The position M&E Assistant contributes to the achievement of Result 1 of the CBRD/RDP. However, the authority and accountability for implementation is the Cambodian partner’s responsibility. The M&E Assistant’s essential responsibilities are to enable his/her counterparts to
103 Annex 2 Job description
increasingly work independently with the necessary qualifications and quality standards. Occasional assistance in implementation might be necessary but should be kept to a minimum and continually decreased.
3. Tasks of the Position Holder With all counterparts: (1) Clarify your TA role and plan your TA tasks with your counterparts.
(2) Establish and regularly review Capacity Building – TA Agreements with individual counterparts. These agreements shall establish priorities and procedures for essential capacity development during a specified time period.
(3) Qualify and assist your counterparts in fulfilling their M&E tasks.
3.1 With the CAU M&E Staff: The CAU M&E staff have responsibility to ensure the regular monitoring and evaluation of the Annual Work Plan and Budget and Annual Plan of Operation and prepare progress reports.
(1) Coach and qualify and - until sufficient competency is built-up - assist counterparts in setting-up an M&E system that combines CBRD, RDP and SEILA information needs.
(2) Coach and qualify and - until sufficient competency is built-up - assist counterparts, based on the annual and monthly work plans, in conducting M&E activities and compiling meaningful information.
(3) Carry out on-the-job training for essential computer skills.
(4) Coach and qualify and - until sufficient competency is built-up - assist counterparts in data processing and analysis and the regular provision of data for planning and M&E tasks to all units/components and to the national level.
(5) Coach and qualify and - until sufficient competency is built-up - assist counterparts to set-up and maintain the central Commune Data Bank following national standards.
(6) Coach and qualify and - until sufficient competency is built-up - assist counterparts in the collection of Commune Data, their regular updating and their entry into the databank; in this task focus in particular on good coordination with data collection activities of other units/components, use of existing data and data quality.
(7) Carry out on-the-job training for preparation, implementation and evaluation of M&E training sessions, and for a limited period of time, assist counterparts in his/her implementation of training sessions.
(8) Coach and qualify and - until sufficient competency is built-up - assist counterparts to carry out monitoring field visits and provide advice on how to do the collection and reporting of M&E data.
(9) Coach and qualify and - until sufficient competency is built-up - assist counterparts with his/her preparation of monthly status reports, especially in providing timely reports as requested by project internal users (ExCom, PSU, etc.) and external users (IFAD, GTZ, SEILA).
(10) Coach and qualify and - until sufficient competency is built-up - assist counterparts in the preparation of the annual work plan and budget (AWPB), and Plan of Operation.
(11) Assist in the production of the AWPB and coordinate and assure the production, including translation, of the yearly Plan of Operation.
3.2 With the M&E Focal Points: The Line Departments’ M&E Focal Points have responsibility to ensure the regular monitoring and evaluation of their Annual Work Plan and Budget and Annual Plan of Operation and prepare progress reports.
104 Annex 2 Job description
(1) Coach and qualify and - until sufficient competency is built-up - assist counterparts in setting-up an M&E system that combines CBRD, RDP, ADESS, CAAEP, WFP and SEILA information needs.
(2) Coach and qualify and - until sufficient competency is built-up - assist counterparts, based on the annual and monthly work plans, in conducting M&E activities and compiling meaningful information.
(3) Carry out on-the-job training for essential computer skills.
(4) Coach and qualify and - until sufficient competency is built-up - assist counterparts in the collection of Commune Data.
(5) Coach and qualify and - until sufficient competency is built-up - assist counterparts in the preparation of his/her monthly work plan.
3.3 With the CAU staff responsible for contract management: The CAU staff assigned for contract management has responsibility to assist in the preparation of contracts for line agencies, suppliers and other contractual partners of the PRDC-Excom and Commune Councils and, depending on the contractual parties, to support or ensure their effective administration.
(1) Coach and qualify and - until sufficient competency is built-up - assist counterparts in their tasks, especially in contract preparation and documentation and the regular provision of information to all units/components.
3.4 Supportive Technical Assistance Tasks (1) In your field of work, participate, upon request, in the design and piloting of new approaches
and innovations as models for promotion at the national level.
(2) In the field of M&E, participate in linkage to national level institutions to use program experiences for national concept, standards, and policy formulation and to transfer knowledge on reforms and new regulations in order to explore new opportunities in the province.
(3) Promote and follow-up the respect of good governance standards in all fields of work.
(4) Coordinate with the TA staff of other components to ensure the exchange of information and attention to implementation priorities and encountered constraints.
(5) According to the set deadlines, prepare monthly work plans and monthly reports, and contribute to reports on the component work results.
(6) Translate technical documents and perform verbal translations in meetings, workshops and training courses, as assigned.
(7) Independently organize and assure the computerized bilingual documentation of the results of central RDP planning exercises and other central workshops.
(8) Assure that all central data needed by the RDP GTZ-Team is available according to GTZ standards and kept and maintained independently from the CAU-Data Bank, in particular the budget follow up data files and the general RDP data base.
3.5 Perform other tasks related to the assignment, as required by your superior or his representative.
4. Line of Authority Directly subordinate to the gtz Advisor for Decentralized Development Management, Community Development and Agriculture – Provincial Coordinator Kampot, or in case of his absence, to his representative.
105 Annex 2 Job description
D. Job description for M&E Program Assistant, Kampong Thom Reports to: GTZ-RDP Provincial Coordinator, Kampong Thom.
Supervises: None
Counterparts: Contract and Administration Unit (CAU) M&E Staff (3), the CAU staff (2), Local Administration Unit Chief and Vice-Chief, and Line Departments M&E Focal Points (3)
Duty Station: Kampong Thom ExCom Office, occasional travel to project sites within Kampong Thom
1. Goal of the Position Through advice on technical and organizational matters, systematic human resource development measures and intensive coaching of the mandated government institution’s staff, the target population receives effective services to improve their livelihoods.
2. Scope of Responsibility The position M&E Program Assistant contributes to the achievement of Result 1 of the CBRD/RDP. However, the authority and accountability for implementation are the Cambodian partner’s responsibility. The M&E Program Assistant’s essential responsibilities are to enable his/her counterparts to increasingly work independently with the necessary qualifications and quality standards. Occasional assistance in implementation might be necessary but should be kept to a minimum and continually decreased.
3. Tasks of the Position Holder With all counterparts: (4) Clarify your TA role and plan your TA tasks with your counterparts.
(5) Establish and regularly review Capacity Building – TA Agreements with individual counterparts. These agreements shall establish priorities and procedures for essential capacity development during a specified time period.
3.1 With the Contract and Administration Unit (CAU) M&E Staff: The CAU M&E staff have responsibility to ensure the regular monitoring and evaluation of the Annual Work Plan and Budget, and Annual Plan of Operation and prepare progress reports.
(12) Coach and qualify and - until sufficient competency is built-up - assist counterparts in setting-up an M&E system that combines CBRD, RDP and SEILA information needs.
(13) Coach and qualify and - until sufficient competency is built-up - assist counterparts, based on the annual and monthly work plans, in conducting M&E activities and compiling meaningful information.
(14) Provide occasional technical assistance for computer operations.
(15) Coach and qualify and - until sufficient competency is built-up - assist counterparts in data processing and analysis and the regular provision of data for planning and M&E tasks to all units/components and to the national level.
(16) Coach and qualify and - until sufficient competency is built-up - assist counterparts to set-up and maintain the central Commune Data Bank following national standards.
(17) Coach and qualify and - until sufficient competency is built-up - assist counterparts in the collection of Commune Data, their regular updating and their entry into the databank; in this task focus in particular on good coordination with data collection activities of other units/components, use of existing data and data quality.
(18) Carry out on-the-job training for preparation, implementation and evaluation of M&E training sessions, and for a limited period of time, assist counterparts in his/her implementation of training sessions.
106 Annex 2 Job description
(19) Coach and qualify and - until sufficient competency is built-up - assist counterparts to prepare for monitoring field visits, and provide advice on how to do the collection and reporting of M&E data.
(20) Coach and qualify and - until sufficient competency is built-up - assist counterparts with his/her preparation of monthly data summaries, especially in providing timely data summaries as requested by project internal users (Excom, PSU, etc.) and external users (IFAD, GTZ, SEILA).
(21) Coach and qualify and - until sufficient competency is built-up - assist counterparts in the preparation of the annual work plan and budget (AWPB), and Plan of Operation.
(22) Assist in the production of the AWPB and coordinate and assure the production, including translation, of the yearly Plan of Operation.
3.2 With the Contract and Administration Unit (CAU) staff responsible for contract management: The CAU staff assigned for contract management have responsibility to assist in the preparation of contracts for line agencies, suppliers and other contractual partners of the PRDC-Excom and Commune Councils and, depending on the contractual parties, to support or ensure their effective administration.
(2) Coach and qualify and - until sufficient competency is built-up - assist counterparts in their tasks, especially in contract preparation and documentation and the regular provision of information to all units/components.
3.3 With the Local Administration Unit (LAU) Chief and Vice-Chief: The LAU Chief and Vice-Chief have responsibility to ensure data collection on Commune Council activities and on activities for the training of Commune councilors.
(6) Coach and qualify and - until sufficient competency is built-up - assist counterparts, based on the annual and monthly work plans, and compiling meaningful information from M&E activities.
(7) Provide occasional technical assistance for computer operations.
(8) Coach and qualify and - until sufficient competency is built-up - assist counterparts in the compilation of Commune Data.
(9) Coach and qualify and - until sufficient competency is built-up - assist counterparts in their understanding of monthly data collection priorities.
3.4 With the M&E Focal Points (Community Development, Infrastructure and Agriculture): The M&E Focal Points have responsibility to ensure the regular monitoring and evaluation of their Annual Work Plan and Budget and Annual Plan of Operation, and prepare progress reports.
(1) Coach and qualify and - until sufficient competency is built-up - assist counterparts in setting-up an M&E system that combines CBRD, RDP, APIP, CAAEP, WFP and SEILA information needs.
(2) Coach and qualify and - until sufficient competency is built-up - assist counterparts, based on the annual and monthly work plans, and compiling meaningful information from M&E activities.
(3) Provide occasional technical assistance for computer operations.
(4) Coach and qualify and - until sufficient competency is built-up - assist counterparts in the compilation of Commune Data.
(5) Coach and qualify and - until sufficient competency is built-up - assist counterparts in their understanding of monthly data collection priorities.
3.5 Supportive Technical Assistance Tasks
107 Annex 2 Job description
(9) In your field of work, participate, upon request, in the design and piloting of new approaches and innovations as models for promotion at the national level.
(10) In the field of M&E, upon request, participate in linkage to national level institutions to use program experiences for national concept, standards, and policy formulation and to transfer knowledge on reforms and new regulations in order to explore new opportunities in the province.
(11) Promote and follow-up the respect of good governance standards in all fields of work. :
(12) Coordinate with the TA staff of other components to ensure the exchange of information and attention to implementation priorities and encountered constraints.
(13) According to the set deadlines, prepare monthly work plans and monthly reports, and contribute to reports on the component work results.
(14) Translate technical documents and perform verbal translations in meetings, workshops and training courses, as assigned.
(15) Independently organize and assure the computerized bilingual documentation of the results of central RDP planning exercises and other central workshops.
(16) Assure that all central data needed by the RDP GTZ-Team is available according to GTZ standards and kept and maintained independently from the CAU-Data Bank, in particular the budget follow up data files and the general RDP data base.
3.6 Perform other tasks related to the assignment, as required by your superior or his representative.
4. Line of Authority Directly subordinate to the Advisor for Provincial ExCom, or in case of his absence, to his representative.
E. Job Description for Advisor for Training and for M&E Reports to: gtz Team Leader
Supervises:
Counterparts: PSU M&E Officer, M&E staff in the provinces
Workbase: RDP Office Phnom Penh, frequent travel to provinces
1. Goal of the Position Through advice on technical and organizational matters, systematic human resource development measures and intensive coaching of the M&E staff of various institutions on national and provincial level and of TA staff, the monitoring of processes and activities will be improved and as a result the steering of activities in all components and the programme as such. During this time, the TA will support the streamlining of the M&E process in CBRD and RDP at all levels. In addition, he will support the development and implementation of selected training to be conducted in both provinces.
2. Scope of Responsibility The position Advisor for Training and for CBRD/RDP M&E contributes to the achievement of all results CBRD/RDP. The Advisor for Training and M&E essential responsibility is to enable his/her counterparts to increasingly work independently with the necessary qualifications and quality standards. The position holder should work in close cooperation with the M&E officer in PSU, the two M&E officers in ExCom and the two provincial M&E TAs in a team approach. Jointly with the M&E officer in PSU, he should assume the role of a coach for M&E-officers, -focal points – and TA as well as component managers in the field of monitoring and evaluation. In the field of training, he will work through TA staff in the provinces.
108 Annex 2 Job description
3. Tasks of the Position Holder Clarify your TA role and plan your TA tasks with your counterparts. Contribute to establish and regularly review on-the-job training (coaching) agreements with individual counterparts. These agreements shall establish priorities and procedures for essential capacity development during a specified time period. 3.1 With PSU, especially M&E officer: (1) Assist PSU in coordinating CBRD/RDP-wide all necessary steps concerning the harmonization of monitoring methods and formats in use in both provinces • screen methods/formats and advise components on the usefulness of methods and formats;
• document and integrate agreements on formats into the M&E manual;
• Coordinate all further approach development steps necessary for ‘filling the gaps’ in the CBRDP M&E manual and its further development;
(2) Assist PSU in monitoring the implementation of the CBRD/RDP M&E manual • Sensitise PSU staff in their understanding of the role of M&E for successful project
implementation
• Jointly with the M&E officer of PSU develop guidelines and work plan for monitoring tasks of PSU
• Participate in the monthly provincial M&E meetings, check what is being done in comparison with the M&E activities in the annual plan of operation
• Assess why planned M&E activities have not been carried out and discuss the reasons with ExCom M&E staff and M&E focal points.
• Identify shortcomings of and measures for capacity building of PSU staff in the field of M+E
• Support M&E officer of PSU in organizing the M&E meeting between provinces
• Support M&E officer of PSU in leading the M&E team to conduct annually the Beneficiary Impact Assessment and Feedback and to ensure its quality and documentation.
(3) Together with M&E PSU staff, support provincial M&E activities • discuss with M&E focal points the weaknesses identified during the monthly provincial M&E
meetings and inter-provincial meetings and their possible solutions
• identify staff development needs for M&E and facilitate the design and preparation of training courses
• Backstop the M&E activities carried out by provincial M&E staffs in their implementation of the internal survey/ evaluation/ assessment and/or other quantitative data collection in the component.
• Support the provincial M&E staff in developing, maintaining of M&E data files.
(4) Together with PSU M&E, support provincial M&E staff to sensitise component managers for the potential value of monitoring information for management decision making through • observing meetings chaired by component managers and find out in how far M&E is a
recognized topic or not
• making component managers aware of the potential value of monitoring information for their component management and decision making
• sensitising component managers/ TA (national & international) to pay attention on M&E and recognize the important roles of M&E in project implementation.
• sensitising middle level management on their role and responsibilities in providing relevant information to management
3.2 With PSU Project Director, GTZ Team leader and TA staff in the components (1) Advise in the application and further development of the CBRD/RDP M&E approach
109 Annex 2 Job description
• report back to management (PSU and GTZ) on problems, needs for changes and staff development needs in the field of M&E
• propose improvements in M&E procedures, methods and formats and the M&E manual
• assist PSU M&E officer in the development of a concise reporting procedure and in reporting formats
• Ensure the M&E information flow to and from M&E task force, TA (national & international), component managers and PSU.
• give guidance on TA role for capacity building especially coaching and on-the-job training for partners M&E task.
3.3 Other Technical Assistance tasks in the field of training and workshop moderation (1) With the training coordination team members from provinces, contribute to the process of
Poverty Reduction Training
• contribute to the development of training curricula and training manual
• follow-up and ensure the quality of training conducted by local training institution
• contribution to the documentation and experience dissemination of training on poverty reduction strategy for rural areas.
(2) In cooperation with economic promotion and other CBRDP components provide support to the farmer associations training
• contribute to the development of training curricula and training manual
• forming and leading a training team to conduct training for farmer associations and ensure the quality of training follow-up activities
• facilitate the documentation of the process for business training for framer associations and experience of the training
(3) Workshop moderation as required
3.4 Assume any other task as assigned by the GTZ program management (as far the time schedule allows)
4. Line of Authority
The position holder directly reports to the gtz Team Leader, or in case of her absence, to her representative.
110 Annex 2 Job description
Annex 3:
A. Interview Guideline for Beneficiary Impact Assessment 2006
Type of group
interviews
Topics1 to be discussed No of Indicators
Background Information
1. CBO leaders and committees'members
Tasks of CBO leaders and committee's members
Management performance of CBO leaders and committee's members (including transparency of financial management and decision making in the meeting)
Training / capacity building events received from CBRDP (plus from other agencies) and the usefulness of the trainings
Sustainability of CBOs/ ideas for CBO's future
quality of service provided Lesson learnt concerning trainings for CBO leaders and committee members and
implementing) and quality of the participation (ask for examples) Women's participation in the associations (planning, decision making,
Relationship to commune councils
the information flow from CC to villagers, the consideration of people’s priorities in CIP, the transparency of the annual financial plans. Quantify the satisfaction on the 3 criteria respectively (see method attached).
Satisfaction with responses of commune councils to needs of villagers: regarding
OP3.3
OB4
list of performance indicators
Kind of CBOs (all CBOs existing in the village)
List of trainings (If possible, list of related training courses held in the selected villages).
2. CBO s
Tasks of CBO leaders and committee's members
Management performance of CBO leaders and committee's members (including transparency of financial management and decision making in the meeting)
member
other Supports (group/ association meetings) received from CBRDP (plus fromagencies) and the usefulness of the supports
Sustainability of CBOs/ ideas for CBO's future
Lesson learnt concerning support received from CBRDP (plus other agencies) and
OP 3.3 list of performance
Kind of CBO (all CBOs
quality of service provision of committee
indicators
existing in the village
List of support (If possible, list of related support received by the selected villages).
1 If the interview team does not fit enough to any topic, the guideline in detail will be made or referred to the guideline from last year.
111 Annex 3 BIA & BMW guideline
(inside support) Benefit of being the members of the CBO
Women's participation in the association (planning, decisioimplementing) and quality of the participation (ask for exa
n making, mples)
Relationship to commune councils
Satisfaction with responses of comthe information flow from CC to villagers,
mune councils to needs of villagers: regarding the consideration of people’s priorities
in CIP, the transparency of the annual financial plans. Quantify the satisfaction on the 3 criteria respectively (see method attached).
Relationship to the CBRDP
accessibility and quality Satisfaction with the CBRDP technical service: regarding the frequency,
of service. Quantify the satisfaction on the 3 criteria
B4
B3 respectively (see method attached).
O
O
3. LTC Road leaders and
Fir
in first how the roads are used and maintained)
committees members
st, visit the project site and physical achievement and then review technical reports and end-of-project evaluation reports.
1- Functioning of LTC:
(Let committee members expla
LTC establishment and rules creation
events of LTC Support received from CBRDP in establishing and training/ capacity building
Tasks of LTC leaders and committee's members
nagement performance (including transparency of financial management LTC maand decision making in the meeting)
Responsibilities and contribution of villagers
to make LTC function properly Support from outsiders and villagers
Lesson learnt concerning the quality, use and maintenance of roads
2- Technical intervention:
Technical planning for construction
Implementation of road construction (supervision of project staffs and contractors
OP4.3 Technical reports (TSU)
End-of-project evaluation
la
OB7,
(TSU)
Quality standards from Sei(TSU)
LTC performance list
112 Annex 3 BIA & BMW guideline
and involvement of LTC)
Satisfaction with supports of project staffs and contractors and quality of road
councils Relationship to commune
Resource for road maintenance (budget, skill)
Sustainability of roads / ideas for future of the roads
LTC establishment and rules creation (for both roads and wells) includingof establishment (too late or not?)
timing
Support received from CBRDP in establishing and training/ capacity building events of LTC (for both roads and wells)
members Tasks of LTC leaders and committee's
LTC management performance (including transparency of financial management and decision making in the meeting)
Responsibilities and contribution of villagers
Support from outsiders and villagers to make LTC function properly
Lesson learnt concerning the quality, use and
2- Technical intervention:
maintenance of roads and wells
Technical planning for construction (for both roads and wells)
involvement of LTC and villagers) Implementation of construction (supervision of project staffs and contractors and
Satisfaction with supports of project staffs and contractors and quality of products
Relationship of LTC to commune councils
OB7, OP4.3, OP4.4
114 Annex 3 BIA & BMW guideline
Resource for road & well maintenance (budget, skill)
Sustainability of roads & wells / ideas for future of the roads & wells
Opinion about lesson learnt by LTC concerning technsupervision of road & wells construction (Do you thin
ical planning, support and k LTC has improved or
ision of road
3
learnt some lessons concerning technical planning, support and supervor wells construction? should the LTC be reelected?)
- Benefits of the facilities of roads and wells
Direct and indirect benefits
6. FWUC leaders and committees members
hen review technical reports
First, visit the project site and physical achievement and tand end-of- project evaluation reports.
1- Functioning of FWUC:
(Let committee members explain first how the irrigation systems are used and maintained)
FWUC establishment and rules creation
Support received from CBRDP in establishing and training/ capacity buildevent of FWUC
ing
Tasks of FWUC leaders and Committee's members
FWUC management performance (including transparency of financial management and decision making in the meeting)
Responsibilities and contribution of irrigation farmers (including local contribution and exemption for MVF)
ake FWUC function properly Support from outsiders and irrigation farmers to m
Lesson learnt concerning the quality, use and maintenance of irrigation system
2- Technical intervention:
Technical planning for construction
Implementation of irrigation system construction (supervision of project staffs and ent of FWUC) contractors and involvem
Satisfaction with supports of project staffs and contractors and quality of irrigation
OB7, OP4.10, OP4.11
Technical reports (PSWRAM)
End-of-project evaluation (PSWRAM)
Quality standard (PSWRAM)
115 Annex 3 BIA & BMW guideline
system
Relationship to commune councils
ce (budget, skill) Resource for irrigation system maintenan
Sustainability of irrigation system / ideas for future of the irrigation system
Lesson learnt concerning technical planning, support and supervision of irrigation system construction
3- Benefits of the irrigation system facilities
Direct and indirect benefit
7. Irrigation farmers lain first how the irrigation systems are used and
1- Functioning of FWUC:
(Let irrigation farmers expmaintained)
FWUC establishment and ror not?)
ules creation including timing of establishment (too late
Support received from CBRDP in establishing and training/capacity building events of FWUC
Tasks of FWUC leaders and committee's members
FWUC management performance (including transparency of financial management and decision making in the meeting)
Responsibilities and contribution of irrigation farmers (including local contributand exemption for MVF)
ion
Support from outsiders and irrigation farmers to make FWUC function properly
Irrigation farmers training/ capacity building events
improvement Effectiveness of farmers' management of irrigation system, inadequacy and
FWUC/ irrigation farmers capacity building Lesson learnt concerning the quality, use and maintenance of irrigation system and
OB7, OP4.10, OP4.11
Basic data on training courses for irrigation farmers (gender specific)
Tasks and functions of irrigation farmers in
2- Technical intervention:
management
116 Annex 3 BIA & BMW guideline
ning for construction Technical plan
Implementation of irrigation system construction (supervision of project staffs and contractors and involvement of FWUC and irrigation farmers)
Satisfaction with supporsystem
ts of project staffs and contractors and quality of irrigation
Relationship of FWUC to commune councils
Resource for irrigation system maintenance (budget, skill)
Sustainability of irrigation system / ideas for future of the irrigation system
f irrigation system construction (Do you think FWUC has improved
cal planning, support and supervision of ted?)
Opinion about lesson learnt by FWUC concerning technical planning, support and supervision oor learnt some lessons concerning techniirrigation system construction? Should the FWUC be reelec
3- Benefits of the irrigation system facilities
Direct and indirect benefits
8. Private service providers (FPs, VAHWs)
Training and tools/ materials received
Appropriateness of received tools/ materials
Customers demanded services from inside outreach of the service)
and outside village (what is the
Services provided (what kind of service? what is the total income?)
Limitation in service provision
Proposed improvement of service provision (what do you think you need to improve service provision in terms of quantity and quality?)
as private service providers and dependency of external eed a license
OP5.4,
Feeling and understanding support or guidance from government or project (do they feel they nto operate)
9. Villagers (benefit from private
private agricultural services and other services of the government) Availability of agricultural services in the village (highlight the difference between
Difficulty to access agricultural services (government and private; gender specific)
OP5.4OP6.1
117 Annex 3 BIA & BMW guideline
services) Benefits/ satisfaction with private services including willingness to pay for service(gender specific). Do you prefer private or government service provider?
Proposed changes for improvement of service access/ quality (lessons learnt)
Number of people whose animal (cattle) died last year?
10. Demo farmers
Possibility to participate in trainings related to demonstration
Change of practices and behavior after doing demonstration (short term)
Experiences (positive and negative) with recommendations in terms of both new technology and process (process: refer to the support of staff, role and understanding of responsibility of demo farmers)
term) Continuous practice of recommendations (adopting of new technology in long
Satisfaction with extension worker in supporting the demonstration. Quantify the satisfaction (see method attached).
onstrated on of the demonstration (lessons
Proposed changes for improvement (1) actual technology that was demand (2) the process for selection and implementatilearnt)
ning of farmer classes/field days and reaction of participants Happe
Response from other villagers to the demonstration
rs for adopting demo technology Request for advice from other village
Benefits from the new technology and economic aspect (would you be willing to pay to those who give you the inputs?
OP6.5 Basic data on agriculture activities in the selected villages (gender specific)
11. Agric participated farmers
Possibility to participate in trainings (gender specific)
Change of practices and behavior after training (short term)
technology and process Experiences (positive and negative) with recommendations in terms of both new
Continuous practice of recommendations (adopting of new technology in long term)
Satisfaction with extension workers in organizing farmer class. Quantify the
OP6.5 Basic data on agriculture activities in the selected villages (gender specific)
118 Annex 3 BIA & BMW guideline
satisfaction (see method attached).
Proposed changes for improvement of process to make farmers participate in demonstration (lessons learnt)
Success of demo implementation especially farmer classes/ field days
ning of farmer classes/field days and reaction of participants Happe
Benefits from the new technology and economic aspect (would you be willinpay to those who give you the inputs?
g to
12. Agric-non participated farmers
Possibility to participate in trainings (gender specific)
process of demonstrations Reasons for non-participating in
Access to improved seeds and private service
Any experiences with improved technologies
OP6.5
13- Most Vulnerable Villagers
Reason why their names are in the list
Participation of the poorest in development activities (including local contribution for general activities, not specific activities for the poorest)
Accessibility of any service (training/support received) from departments, CC and e) CBO (reasons for being or not being assessabl
Benefit from the development activities (road, well, irrigation scheme, agriculture…)
Benefit from specific project for the poorest if it is existing in the villages.
119 Annex 3 BIA & BMW guideline
B. Program/ Methodology for District Beneficiary Monitoring Workshop 2006
Group work in 4 groups (per commune): Discuss and verify monitoring results along the following guiding questions: In how far do results reflect the situation in your commune? What is different in your commune?
Charts for GW M&E Team members
11:00 Plenary: Presentation of group work results and discussion Charts for GW Facilitat.
12:00 Lunch break
14:00 Plenary: Explain, clarify the purpose of upcoming group discussion and present task for monitoring indicators OB3; OB6; OP2.5; OP2.6; OP3.4; OP3.5; OP3.8 along following questions: OB3: In how do you satisfy with the CBRDP technical service: regarding the frequency, accessibility and quality of service? Quantify the satisfaction on the 3 criteria respectively (see method attached). OB6: In how do you satisfy with the CBRDP administrative set up: regarding the decentralized/ de-concentrated administrative institutions? Quantify the satisfaction and the consideration to cooperate with them (see method attached). OP2.5: Amount of local resources planned/raised last year?
Charts with the guiding questions and a roster for presentation
Facilitat.
120 Annex 3 BIA & BMW guideline
OP2.6: Amount of external resources raised last year? OP3.4: Within LTC and CBO committees in the communes, how many people are women members? OP3.5: Amongst LTC and CBO leaders in the communes, how many people are women leaders? OP3.8: If you recall the planning process, decision making and management: What was the proportion of women involved in these steps? (Please specify for each of the three fields)
14:30 Group work in 4 groups (along above questions) Charts M&E Team members
16:00 Plenary: Presentation of group work results, discussion Facilitat.
17:00 Closing of the workshop PSU/Fac
121 Annex 3 BIA & BMW guideline
Annex 4: Agriculture data collection sheets
A. Data Collection Sheet of Agricultural Extension Activity
Training 1 Training 2 Cross Visit Field Visit No. of Beneficiaries
No.
Farm
er's
na
me
Type
of
Dem
o.
Villa
ge
Com
mun
e
M F M F M F M F poor total
B. Data Collection Sheet of Animal Vaccination Campaign
Total no. Total no. of
animals vaccinated No. Type of Vaccine Village Commune Cow Buffalo Cow Buffalo
C. Data Collection Sheet of Integrated Home Garden
Farmer Selection Farmer Class
No
Nam
e of
Fa
rmer
Dat
e
Mal
e
Fem
ale
Tota
l
Villa
ge
Com
mun
e
Dis
trict
Cro
p
Type
of
Dem
o.
Dat
e
Mal
e
Fem
ale
Tota
l
Transplanting Area m² Remarks Responsible Person
See
d R
ecei
vin
g D
ate
S
owin
g/
Pla
ntin
g D
ate
Ger
m %
Dat
e
Dur
atio
n
No.
of
seed
ling/
se
ed/h
ill
Soil type
Ava
ilabl
e
Act
ual
D. Data Collection Sheet for monitoring of rice demonstration practised by farmers (Type of soil and fertilizer used)
Area [ha] Seed Fertilizer application Kg/ha
No
Nam
e of
fa
rmer
Villa
ge
Com
mun
e
Demo ComparisonSoil type
Variety Kg Urea DAP KclOther
Total
122 Annex 4 Agriculture data collection sheets
E. Data Collection Sheet for monitoring of rice demonstration practised by farmers (seedbed activity)
Seedbed No
Name of farmers
Village Sowing date Area m² Cow manure Green manure Urea
Other
F. Data Collection Sheet for monitoring of rice demonstration practised by farmers (transplantation activity)
No
Nam
e of
fa
rmer
No
of
farm
ers
train
ed
Tran
spla
ntin
g da
te
No
of
seed
lings
pe
r hill
In
row
or
not
dist
ance
be
twee
n hi
llsA
ge o
f se
edlin
gs
Leve
l of
wat
er
Ferti
lizer
us
ed
Re-
trans
plan
tat
ion
Oth
er
F M
Tota
l
Cm
Day
Cm
DA
P
Kcl
Oth
er
Nat
ural
Dat
e N
o of
pe
ople
G. Data Collection Sheet for monitoring of rice demonstration practised by farmers (caring activity 1)
No
Name of
farmers Level of water
1st fertilizer application
No of farmer trained Weeding
Chemical Toxic Other
1st w
eek
Gro
win
g pe
riod
Ure
a
Kcl
Dat
e
Nat
ural
Dat
e
F M
Tota
l
No
of d
ay
No
of
peop
le
Inse
cts
Wee
d
H. Data Collection Sheet for monitoring of rice demonstration practised by farmers (caring activity 2)
No Level of water No of farmer trained 2nd fertilizer application Weeding
Chemical Toxic Other
1st w
eek
Gro
win
g pe
riod
F M
Tota
l
Ferti
lizer
ty
pe
Kg
Dat
e
No
of
day
No
of
peop
le
Inse
cts
Wee
d
123 Annex 4 Agriculture data collection sheets
I. Data Collection Sheet for monitoring of rice demonstration practised by farmers (harvesting activity)
No
Name of
farmers Field day/ Evaluation
No of stems per hill
No of seed heads per hill
No of seeds
per head
Kg/m2
customary plan
Kg/m2
demonstrated plan Other
M
F Tota
l
Max
i.
Min
i.
Max
i.
Min
i.
Max
i.
Min
i.
Max
i.
Min
i.
Max
i.
Min
i.
124 Annex 4 Agriculture data collection sheets
Annex 5: Committee performance list
CD- A4 CBO/LTC M&E Tools Indicator for verification of self-dependence of CBO
Evaluation (Scoring)
No Indicator InformationWeak
(1)
Basic
(2)
Fair
(3)
Good
(4)
Very good
(5)
Recommendation
1 Fund management 1.a Accounting Clear system
Clear debits Clear credits Clearly identified customers Open to public
1.b Expenses Interest spent only on very important and urgent needs
Expense made reasonably Spent on important activities for villagers or
expenses done according to priorities for villagers
Clear rules for expenses Expense made according to regulation only.
1.c Report Reporting regularly, timely and in writing Willingness to write report Understandable reports
1.d Materials Were kept in order Easy to find when needed Clear records Clear reports Clear responsibilities
Written rules Established by all committee members Understood by all committee members There is a copy in the hand of the authorities Regulation have been applied
2.b Committeemeeting
Clear agendas Clear minutes Meeting conducted according to priorities Committee chief acts as facilitator Decision have been made based on the
majority Most questions by members were answered Members have understood the decision and
can spread them to others.
2.c Documents Were kept in order Easy to find when needed Clear records Clear reports Clear responsibilities
2.d Division offunctions and roles in the committee
Clear division Division based on skills Written roles and functions All committee members are active Good cooperation between committee members
2.e Relationshipwith external actors
Regular contact with DCDT Regular contact with other CBO Regular contact with district technical and
authority’s offices Regular contact with provincial departments and
authorities Regular contact with NGOs Regular contact with other relevant institutions.
Subtotal 2 3 Members' participation
126 Annex 5 CBO LTC performance list
3.a Additionalcontributions
Regular contributions Contribution in cash Contribution in kind Contribution in labour Voluntarily contributed There are proofs of the contributions
3.b Participation ofvillagers in meetings and in decision making
Decisions are made by villagers Decisions are made by the committee with
request for acceptance by the villagers. Villagers’ questions were answered Villagers understood the decisions Villagers are able to inform third parties/ others
about the decision taken. 3.c Statute Written statute
Understood by the committee and applied/ implemented.
Established by all members Understood by all members There is a copy in the hand of the authorities Strictly applied
3.d Committeeselection
Elected following the statute At least 2 women are members of the
committee Main positions are held by women Recognition by local authority
Subtotal 3 4 Quality of service provided 4.a Focussing on
women Percentage of women participation:………
At least 30% of member had benefited from service
4.b Ability to help vulnerable people
Willingness to help Ability to help Can change the living condition of the VHH
4.c Objectivessetting
Matched with the human resource Correspond to existing material and fund
provisions targeted for all members.
127 Annex 5 CBO LTC performance list
4.d Accomplishmentagainst objectives
Percent of accomplishment: ………… More than 90% of activities match against the
objectives
Subtotal 4 5 Capacity to
facilitate the village meeting/ contribution
Clear agendas Clear minutes Meeting conducted according to priorities. Committee acts as facilitator Discussion are made by villagers
6 Implementationof decisions
90% of the decisions made at village meetings have been applied (by majority, democratic and consensus)
90% of the decision made at committee meeting have been applied (top down and by authority).
7 Planning Weekly activity plan Monthly activity plan Correspond to reality Flexible Have been strictly applied
8 Future vision Clear vision All committee members are conscious about the
vision All villagers are conscious about the vision.
Subtotal5+6+7+8
Grand Total
128 Annex 5 CBO LTC performance list
To use the CBO/LTC M&E effectively, the district staff members should follow a guideline below: Indicator for verification of self-dependence of CBO
Explanation
No Indicators A Weak
B Basic
C Fair
D Good
E Very Good
1 Fund management 1.a Accounting There is no
accounting system Just simple
remarks in a note book.
There is an accounting system but income, expenses and customers are not clear.
There is a clear accounting system but it is not open to the public.
There are still some little mistakes.
There is an accountable, transparent and clear accounting system.
Income, expenses, balance and customers have been recorded in detail.
Open for public consultation.
1.b Expenses Interests income have been spent up to 100%.
Spent on projects which are not important for villagers.
Sometimes funds are spent without any justification (tell a lie)
There are no rule for regulating expenses.
Interests income have been spent up to 80%.
Spent on projects which are not important for villagers but with a justification.
There are no rules for regulating expenses.
Interests income have been spent up to 60-80%.
Spent on projects which are not important for villagers but with a justification.
There are rules for expense but they are not fully enforced.
Interests income have been spent up 40-60%.
Spent on important projects for villagers.
Expenses were entirely made according to the regulations.
Interests income have been spent up 40% only.
Spent on projects which are important for villagers.
Expense were entirely made according to the regulations.
129 Annex 5 CBO LTC performance list
1.c Report Never reported. Reported
occasionally but without any written records.
Regular verbal reports, but did not write any reports
Did not write any reports until it was requested.
The reports are not clear.
Reports have been regularly written.
Some parts of the reports are not clear.
Reports have been regularly written.
All reports are clear.
1.d Materials Have been kept in disorder. Difficult to find when needed.
No records. No responsible
Have been kept in disorder. Difficult to find when needed.
No records There is someone
in charge but the person does not understand his/her responsibilities.
Have been kept in disorder. Difficult to find when needed.
Sometimes recorded and reported to the relevant people. Those records and reports are sometimes unclear.
There is someone in charge but the person does not understand his/her responsibilities.
Materials have been kept in a designated place and have been regularly recorded and reported to relevant persons. Those records and reports are sometimes unclear.
A reliable responsible person is in charge.
Materials have been kept in a designated place and have been regularly recorded and reported to relevant persons. Those records and reports are very clear.
A reliable responsible person is in charge.
2 Functioning of committee 2.a Committee
internal rules There are no
written rules. The rules were
totally copied from another CBO and which are not understood really fully.
There is a written rule which was established by some members, but without transferring it to the other members.
The rules have been sent to the authorities.
Parts of the rules are implemented.
There are clear rules and some members contributed to establish and amend them.
The rules have been sent to the authorities.
Most articles stipulated in the regulation have been implemented
All members contributed to established the rules and understood them.
The rules have been sent to the authorities.
The rules are implemented correctly.
130 Annex 5 CBO LTC performance list
2.b The committeemeeting
Never set up a committee meeting or only oral information among the members.
There is a meeting when the president of the committee calls for it and letting only a few members know about it.
There are no agendas and minutes of the meeting.
A meeting is conducted when all members decide on the necessity for it.
There are no agendas and minutes of the meeting.
No good results
Monthly meetings are in the work plan or scheduled when needed.
Agenda is raised by members and agreed on.
Chief of committee acts as facilitator
Good results but no minutes of the meeting.
Monthly meetings are in the work plan or scheduled when needed.
Agenda is raised by members and agreed on.
Chief of committee acts as facilitator
Key points are written down in minutes and kept in the records.
2.c Documents Have been kept in disorder. Difficult to find when needed.
No records. Nobody
responsible.
Have been kept in disorder. Difficult to find when needed.
No records. There is someone
in charge but does not understand his/her responsibilities.
Have been kept in disorder. Difficult to find when needed.
Sometimes recorded and reported to relevant persons. Those records and reports are sometimes unclear.
There is someone in charge but does not understand his/her responsibilities.
Documents have been kept in a designated place and have been regularly recorded and reported to the relevant persons. The records and reports are sometimes unclear.
There is a reliable responsible person in charge.
Documents have been kept in a designated place and have been regularly recorded and reported to the relevant persons. The records and reports are very clear.
There is a reliable responsible person in charge
2.d Division offunctions and roles in the committee
Only the president of the committee is actively working.
Only a few members of the committee are actively working but there is unclear division of their functions.
All members are active but there is an unclear division of their functions.
All members are active.
Clear division of function but not according to the skills.
No written division of the functions.
All members are active with a clear written division of the function according to their skills
Good cooperation.
131 Annex 5 CBO LTC performance list
2.e Relationshipwith external actors
Never contacted external actors except DCDT
Used to contact external actors and DCDT but not often.
Used to contact other CBOs, government offices at district level, local authorities and DCDT but not very often.
Regular contact other CBOs, government offices at district level, local authorities, NGOs and DCDT.
Same as D, plus contact to other relevant institutions such as NGOs, government agencies at provincial level and beyond.
3 Members' participation 3.a Additional
contribution There are no
additional contributions from villagers.
Contributed by force.
No records about the contributions.
Contributions are rarely made.
Contributions sometimes follow forced action.
No records about the contributions.
There are regular additional contributions, but only in cash.
Mobilized by collection.
Clear written evidence of the contributions.
There are regular contributions in cash and kind.
There are voluntary contributions from the villagers.
Clear written evidence of contributions.
3.b Participation ofvillagers in meetings and in decision making
No participation of villagers
Committee makes the decisions followed by their announcement to the villagers.
No participation of the villagers
Committee and few elders make the decisions and keep information.
Elders participate in the discussions and make the decision in the pagoda, which are then announced to villagers
Villagers participate but committee makes the decisions and asks for agreement from villagers.
All decisions are made after shared discussions
75% of villagers' questions are resolved and participants can explain the decision to others.
3.c Statute There is no written statute or an unclear statute (e.g. on objectives, procedures, ownership, etc...).
The statute was totally copied from another CBO and is not clear in implementation.
There is a written statute established by the committee, but the villagers are not officially informed.
The statute has been sent to the authorities.
There is a clearly part of written statue. The members contributed to establish and amend it.
The statute has been sent to the authorities.
Most articles
All members contributed to established the statute and understand it.
The statute has been sent to the authorities.
All articles stipulated in the statute have been strictly
132 Annex 5 CBO LTC performance list
stipulated in the statute having been implemented.
implemented.
3.d Committeeselection
the committee was appointed by people in high positions.
No female members.
It was appointed by a few of participants (less than 30%) and influential persons.
No female members.
No recognition by the authorities.
It was appointed by elders on Buddhist holy days and by 30% - 60% of the members.
No female members.
There is only one election held by all the members
Only 1 or 2 women in the committee.
There are regular elections according to the statute.
At least 2 female members in the committee are holding main functions.
4 Quality of service provided 4.a Focus on
women No woman among
the CBO members. Few of women in
the list of CBO members.
30% of members are female but only 20% of female members benefited from CBO services.
30% of members are female all of them benefited from CBO services.
There are more than 30% of members are female all of them benefited from CBO services.
4.b Ability to help vulnerable people
Poor members of the commune have never been taken into consideration.
Poor member of the commune are considered, but there are no means to implement assistance.
There is a will to help the poor but no capacity.
There is a will and capacity to help the poor, but it does not improve their living standard (the causes have never analyzed).
There is a will and strategy to help the poor, which can improve their living standard step by step.
4.c Objectivesetting
Not suited to the existing resources of the association (big resources but small objective, or vice- versa)
Objective is suited to human resources but not to the material resource, or vice-versa.
Serves few members.
Objective is suited to human resource but not to the material resource, or vice-versa.
Serves half of the members.
Objective is suited to both human and material resources
Serves the majority of the members.
Objective is suited to both human and material resources
Serves all members or beyond.
4.d Accomplishmentagainst objectives
Success less than 20% of the objectives
Success 20% - 40% of objectives
Success 40% - 70% of objectives
Success 70% - 90% of the objectives
Success over 90% of the objectives.
133 Annex 5 CBO LTC performance list
5 Capacity tofacilitate the village meeting/ contribution
Committee has never set up a community meeting.
If there is a meeting, it only concerns the pagoda or is a simple public announcement.
If there is a meeting, it only concerns the pagoda or is a simple public announcement.
There are no agendas and minutes of the meeting.
Conduct the meeting when needed
There are no agendas and minutes of the meeting.
No good results
Conduct the meeting when needed
Committee is in charge of discussions and decisions.
No good results or didn't understand the whole content of the meeting.
There is an agenda and minutes of the meeting.
Conduct the meeting when needed
Committee acts as facilitator
Good results and understood by everyone.
There is an agenda and minutes of the meeting.
6 Implementationof decisions
Top down By authority By minority By majority By consensus Through democratic
process
7 Planning No existing plan There is an action plan or weekly / monthly plan but it has not been implemented and is not put into writing.
There a written action plan or weekly / monthly plan and kept every month but it does not meet the real needs.
There a written action plan or weekly / monthly plan and kept every month for document
weekly / monthly plan is flexible and realistic.
There is a written action plan or weekly / monthly plan, which corresponds to reality and it is flexible adapting to the circumstance.
8 Future vision No idea, never thought of a vision
Thought of a vision, but not put into writing.
There is an unclear vision or the vision is not valid
Only 2 or 3 persons have a clear valid vision
There is a clear valid written vision which is understood by all committee members and villagers.
134 Annex 5 CBO LTC performance list
Annex 6: Component specific data files
Abbreviations of different CBOs
Abbreviation Description of CBOs and LTCs
CCA Cash AssociationCCB Cow BankCFA Farmer AssociationCPB Pig BankCRA Rice AssociationCRB Rice BankCRE Religous AssociationCSA Solidarity AssociationLRO LTC road (bridge & culvert)LW6 LTC well Nº 6LWA LTC well AfridefLWC Combined well (also ring well)LWG LTC water groupLWP LTC pondVDC Village Development CommitteeVNW Village Network
135 Annex 6 Component specific data files
A. Example of CBO/LTC establishment data files, Kampot
AutoFilter And CountA (Column A): 1171 AutorFilter Sum By Column: 5,112 1443 216342 113698 114
6.1 Contractor takes laterite from quarries specified in the contract? Yes No
If no, Explain: ......................................................................................................................
6.2 Reshaping and compaction of earth road? Yes No
6.3 Compaction to be compacted by roller with layer thicknesses and number of passes
according to Compaction Methods table? Yes No
6.4 Thickness of laterite specified in the contract:
Before compaction: 200mm; After compaction: 150mm
6.5 Laterite testing in the field:
Thickness of all each tests ≥ 70% of the thickness after compaction (= 105mm) ?
Yes /No
Average thickness of all the tests ≥ 85% of the thickness after compaction(≈ 127 mm) ?
Yes /No
Complete the laterite inspection form as below:
146 Annex 8 Site inspection forms
Laterite Inspection Format Date of inspection:……………………/………………./………………… Name of TSO:…………………………………. Name of TA RDP:……………………………… Name of contracor/Represntative:…………………………………..Tel:………………. Vilage:……………………………….Village code:………………….. Commune:……………………………Dirstrict:…………………………..………. Length of road (Km):………………………Number of test:………………..(+/-10 places per Km) Result of test:
Result in average:………………………..mm.
No. of test
Length (m)
t1(mm) Thickness of laterite before
contract
t2 (mm) Thickness of laterite after contract
t2-t1(mm) Thickness of laterite covered under
contract 1 2 3 4 5 6 7 8 9 10
Thickness of all each tests ≥ 70% of the thickness after compaction (= 105mm): Successful.
Thickness of all each tests < 70% of the thickness after compaction (= 105mm): Failure.
If the result in average ≤ 125mm: Failure; If the result in average > 125 mm: Successful.
Conclusion:……………….……………………….Reason:……………………………………… Date:…………………………….………….. Date:………………………………………… Signature of TSO (PDRD) Sinature of TA RDP Name:…………………………. Name:………………………………….. (Remark: See precedure of inspection and agreement on crieria for latrite road.)
147 Annex 8 Site inspection forms
6.6 Camber of road (camber slope 1:15) is correct? Yes No 6.7 Road surface is smooth? Yes No
6.8 Water can flow off the road into the side drains everywhere? Yes No
7. Instructions passed and actions taken:
1st Instruction: ( Date: …………………………)
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
Action taken: (Date: ………………………….)
…………………………………………………………………………………………………
2nd Instruction: (Date: …………………………)
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
Action taken: (Date: …………………………..)
…………………………………………………………………………………………………
3rd Instruction: (Date: ……………………………..)
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
Action taken: (Date: …………………………..)
…………………………………………………………………………………………………
Date: ……………………………… Date: ………………………… Signature Contractor's representative Signature TSS district (PDRD) Name:................................................ Name:................................................ Date: ………………………… Date: ............................................. Signature TA RDP Signature Sub component manager Rural Road Name:................................................ Name:................................................
148 Annex 8 Site inspection forms
SITE INSPECTION RECORD FOR BUILDING CONSTRUCTION WORKS
Date: ……………………………… Date: ………………………… Signature Contractor's representative Signature TSS district (PDRD) Name:................................................ Name:................................................ Date: ………………………… Date: ............................................. Signature TA RDP Signature Sub component manager Rural Road Name:....................................... Name:................................................
155 Annex 8 Site inspection forms
FIELD INSTRUCTION (FI)
FI No: ………………………..……………………..................... Date: ……………………………………………....................... Contract No: ……………………………………………....................... Description of Contract: ……………………………………………....................... Name of Contractor: ………………………………………………................... Instruction given by: ……………………………………………….....(TSS/TA)
INSTRUCTION: …………………………………………………………………………
…………………………………………………………………………
…………………………………………………………………………
………
SKETCH:
Instructed by: ……………. Instruction Received by contractor: …………………….
(Note: Original copy to be given to the contractor and carbon copy to be kept by TSS/TA for record)
156 Annex 8 Site inspection forms
Annex 9: Feedback sheet PLEASE SEND YOUR COMMENTS & SUGGESTIONS ON HOW THIS BOOK CAN BE IMPROVED
1. How has the book helped you in your work? __________________________________________________________________________________________________________________________________________________________________________________________
2. Is the book well organized? Yes____ No____ 3. Could you find what you wanted easily? Yes____ No____
4. Which parts of the book were most useful and why?
5. Are the concepts clearly explained? Yes____ No ____ If no, what concepts were not clear?_________________________________ ______________________________________________________________
6. Did you find any errors in the book? Yes____ No____
If yes, which and what page numbers?_______________________________ ______________________________________________________________
7. What difficulties did you encounter using the book and why? ____________________________________________________________________________________________________________________________
8. What else should be included in the next modification? ____________________________________________________________________________________________________________________________
9. Do you have any other suggestions/comments for improving the book? __________________________________________________________________________________________________________________________________________________________________________________________
THANK YOU FOR YOUR VALUABLE SUGGESTIONS AND COMMENTS.
Mr. Song Sophal, Planning and M&E officer PSU/CBRDP Ministry of Rural Development Or Mr. Ten Samrach, Advisor for Training and for M&E, CBRDP/RDP-GTZ House 05, Street 310, Boeung Keng Kang, P.O.Box 1226, Phnom Penh, Kingdom of Cambodia.