Final Report Final Evaluation of EU Water Facility WASH ......Final Evaluation of EU Water Facility WASH Programme in Four districts and Four Municipalities – Final Report Page 4

Final Report

Final Evaluation of EU Water Facility WASH Programme in 4 districts and 4 Municipalities

30 July 2010

Index

Confidential –All Rights Reserved Ernst & Young – 2010

Final Evaluation of EU Water Facility WASH Programme in Four districts and Four Municipalities – Final Report

Page 1

Index

Final Report ............................................................................................................................ 1

Index ...................................................................................................................................... 1

List of Abbreviations ................................................................................................................ 2

Executive Summary ................................................................................................................. 4

1. Project and Evaluation Objectives ......................................................................................... 6

1.1. Project Objectives .................................................................................................... 6

1.2. Evaluation Objectives ............................................................................................... 7

2. Evaluation Methodology ....................................................................................................... 8

3. Limitations of the Evaluation ................................................................................................ 9

4. Findings and Discussion of Findings ..................................................................................... 10

4.1. Project Description ................................................................................................. 10

4.2. Origin, context and key assessment ......................................................................... 11

4.3. Summary of project implementation ........................................................................ 12

4.4. Changes in context and in the key assessment areas ................................................. 14

4.5. Progress in achieving the objectives ........................................................................ 14

4.6. Financial Execution ................................................................................................ 16

4.7. Issues and action required ...................................................................................... 17

4.8. Cross-cutting and other issues................................................................................. 17

4.9. Main conclusions and Recommendations .................................................................. 19

Annexes ............................................................................................................................... 22

Annex 1 – Terms of Reference ................................................................................................ 23

Annex 2 – List of people interviewed ....................................................................................... 30

Annex 3 – EU WF Final Evaluation Outline 3 ............................................................................. 33

Annex 4 – List of Documents Reviewed .................................................................................... 35

Annex 5 – Guideline of semi-structured interviews to UNICEF .................................................... 37

Annex 6 – Guideline of semi-structured interviews to DNA ......................................................... 39

Annex 7 – Guideline of semi-structured interviews to DPOPH ..................................................... 41

Annex 8 – Guideline of semi-structured interviews for Municipalities .......................................... 43

Annex 9 – Guideline of semi-structured interviews for Water Committees ................................... 45

Annex 10 – Guideline of semi-structured interviews for Schools ................................................. 47

Annex 11 – Details on Section 4.5 ........................................................................................... 49

Annex 12 – PPT for the Final Evaluation Report ........................................................................ 53

List of Abbreviations



Page 2


ACP African, Caribbean and Pacific

AIDS Acquired Immune Deficiency Syndrome

CF Common Fund

CLTS Community Led Total Sanitation

DAS Department of Water and Sanitation

DC

DNA

Demonstration Centres

National Directorate of Water

DPOPH Provincial Directorate of Public Works and Housing

DRA Demand Responsive Approach

EC European Commission

EUWF European Union Water Facility

FIPAG

GIS

Fundo de Investimento e Património de Agua

Geographic Information System

GoM Government of Mozambique

HH Household

HIV Human Immunodeficiency Virus

IWRM Integrated Water Resource Management

JMP Joint Monitoring Programme

MDG Millennium Development Goals

NGO Non-Governmental Organization

NWP National Water Policy

O&M Operation and Maintenance

ODF Open Defecation Free

PEC Participation and Community Education

PES Plano Económico e Social

PESAR Plano Estratégico de Agua Rural

PESOD Plano Económico e Orçamento Distrital

PHAST Participatory Hygiene Water and Sanitation Transformation

PRONASAR Programa Nacional de Agua e Saneamento Rural

PRSP Poverty Reduction and Strategy Paper

SDPI District Services of Planning and Infrastructures

SINAS Sistema de Informação Nacional de Agua E Saneamento

SWAPs Sector-wide Approaches

UNICEF United Nations Children's Fund




Page 3

UTAS Unidade de Tratamento de Água e Saneamento

WASH Water, Sanitation and Hygiene

WC Water Committee

WSSD Water and Sanitation Sustainable Development

ZAMWAT Zambeze Water

Executive Summary



Page 4

Executive Summary

Under the European Union Water Facility (EUWF), the European Commission (EC) and UNICEF

Mozambique signed a four year programme Contribution Agreement on July 21, 2006 targeting

water, sanitation and hygiene in eight vulnerable areas in Mozambique. The overall objective of the

EUWF was to contribute to poverty reduction and sustainable development through the

achievement of the Millennium Development Goals (MDGs) and Water and Sanitation Sustainable

Development (WSSD) targets in African, Caribbean and Pacific (ACP) countries.

In line with the Government of Mozambique (GoM) and UNICEF Cooperation Programme 2007-

2009, the overall objective of the EUWF WASH Programme was “to contribute to achieving the

Millennium Development Goals (MDG) and Poverty Reduction and Strategy Paper (PRSP) targets for

water and sanitation in vulnerable rural and peri-urban areas of selected provinces and districts of

Mozambique”

The Specific Objectives of the Programme were “to provide increased access to safe and affordable

water, sanitation and hygiene facilities for vulnerable communities in four rural districts and four

peri-urban areas of Sofala and Zambezia Provinces, which have been selected due to high

vulnerability indicators, including diarrhoea prevalence, cholera cases and HIV/AIDS prevalence”.

According to the logical framework the project was expected to achieve the following results: (i)

National Water Policy (NWP) guidelines disseminated and sector planning, monitoring and

evaluation strengthened; (ii) Provincial, municipal and district level capacity to plan, manage,

monitor and evaluate the WASH sector using NWP guidelines and DRA (Demand Responsive

Approach) strengthened; (iii) Participatory and sustainable Water Committees (WC) put in place and

operational, as well as hygiene practices and HIV/AIDS awareness improved in all project districts

and municipalities strengthened; (iv) Access to safe and affordable water supply, sanitation and

hygiene facilities improved at communities levels and schools in all eight district projects; and (v)

Project management and technical assistance timely provided on a yearly basis. In particular, by the

end of the Programme it was expected that:

water points;

dren with safe access to sanitation and

would had benefited from

hygiene promotion.

Executive Summary



Page 5

The main objective of this study was to evaluate the EUWF WASH Programme. UNICEF, the EC and

the GoM were keen on evaluating the performance, impact on target population and partner’s

performance along the Programme implementation.

The evaluation was carried out through desk review of the main project documents. In addition,

personal interviews using interview guides and outline discussed and agreed with UNICEF; direct

observation; informal conversation; and focus group discussions. The evaluation team visited two

out of four districts (Dondo and Nicoadala) and three out of four municipalities (Beira, Dondo and

Quelimane) from Sofala and Zambézia provinces, in the project areas as to assess in depth, the

different aspects of the project activities. Field visits included spot check on various project outputs

i.e. water points, latrines; dish racks; hand washing facilities; etc. and discussion with the WC

members and other key informants.

This evaluation report is based on the format provided by UNICEF included in Section 7: Monitoring,

Review and Reporting - CRIS “Implementation Report“ of the European Commission - Aid Delivery

Methods – Project Cycle Management Guidelines (Volume 1 – March 2004). It provided the main

headings, such as, project description; origin, context and key assessment; summary of project

implementation; changes in context and in the key assessment areas; progress in achieving

objectives; financial execution; issues arising and action required; and cross-cutting and other

issues.

Main findings from the evaluation team were: (i) the capacity building component was rolled out as

per the Programme Plan, with nearly all guideline targets, trainings and workshops achieved and

with strong results in capacity strengthening at national and provincial levels as well as in private

sector. Capacity building at district and municipal level, especially in planning and monitoring

programme activities, had been overall good but with some challenges in district, municipalities and

with NGO (Non-Governmental Organization) staff. In terms of WASH infrastructure provision

component in the target areas, in all cases it exceeded the targets set for infrastructure provision

and the number of people covered by the programme. Especially the construction and rehabilitation

of water sources as well as motivating households to construct their own latrines (without subsidy)

have exceeded expectations in terms of end beneficiaries covered.

At institutional level, the Programme has contributed to the realization of decentralized governance

including planning, implementing and monitoring of interventions. The sense of ownership of the

Programme by (sub) national authorities has been strong.

The total project budget for the four year period 2006-2010 was EUR 5,570,279.20, of which EC

and UNICEF contributed fifty per cent (EUR 2,785,139.60) each. In terms of expenditure, by June

2010 EUR 5,080,676.68 has been spent, which corresponds to 91% of the funds available.

Project and Evaluation Objectives



Page 6

1. Project and Evaluation Objectives

1.1. Project Objectives

Under the European Union Water Facility (EUWF), the European Commission (EC) and UNICEF

Mozambique signed a four year programme Contribution Agreement on July 21, 2006 targeting

water, sanitation and hygiene in eight vulnerable areas in Mozambique. The overall objective of the

EUWF was to contribute to poverty reduction and sustainable development through the

achievement of the Millennium Development Goals (MDGs) and Water and Sanitation Sustainable

Development (WSSD) targets in African, Caribbean and Pacific (ACP) countries.

In this context, and in line with the Government of Mozambique (GoM) and UNICEF Programme of

Cooperation 2007-2009, the overall objective of the EUWF was “to contribute to achieving the

Millennium Development Goals (MDG) and Poverty Reduction and Strategy Paper (PRSP) targets for

water and sanitation in vulnerable rural and peri-urban areas of selected provinces and districts of

Mozambique”

The Specific Objectives of the Programme were “to provide increased access to safe and affordable

water, sanitation and hygiene facilities for vulnerable communities in four rural districts and four

peri-urban areas of Sofala and Zambezia Provinces, which have been selected due to high

vulnerability indicators, including diarrhoea prevalence, cholera cases and HIV/AIDS prevalence”.

According to the logical framework the project was expected to achieve the following results:

► NWP guidelines disseminated and sector planning, monitoring and evaluation strengthened;

► Provincial, municipal and district level capacity to plan, manage, monitor and evaluate the

WASH sector using NWP guidelines and DRA is strengthened;

► Participatory and sustainable community management structures engaged and operational,

and hygiene practices and HIV/AIDS awareness improved in all project districts and

municipalities strengthened;

► Access to safe and affordable water supply, sanitation and hygiene facilities improved at

communities and schools in all eight project districts;

► Project management and technical assistance timely provided on a yearly basis.

Project and Evaluation Objectives



Page 7

In particular, by the end of the Programme it was expected that: (i)

safe and sustainable water points; (ii) 25,000 families (125,000 people) with access to safe

sanitation facilities (Household latrines);

sanitation and hygiene facilities (latrines, hand washing, urinals); and

with hygiene promotion.

1.2. Evaluation Objectives

The main objective of this study is to evaluate the EUWF WASH Programme. UNICEF, the EC and the

GoM were keen on evaluating the performance, impact on target population and partners’

performance in the Programme implementation.

The information gathered in this assignment will contribute to the improvement of the Monitoring

and Evaluation system within the WASH sector. Additionally, lessons learnt in the course of the

evaluation would be very useful for similar ongoing WASH programmes in Mozambique, such as the

One Million Initiative. Moreover, the information collected through the study will help to assess

whether recommendations and conclusions from the previous Programme evaluations of (EU

Monitoring visit and Sustainability Checks) were integrated in this approach.

According to the Terms of Reference provided by UNICEF, specifically the evaluation was:

► To make an assessment of how far has the Programme Monitoring and Evaluation

framework been used by stakeholders;

► To make an assessment of the adequacy and appropriateness of the training provided to the

end-users;

► To make an assessment of the sustainability of water, sanitation and hygiene facilities;

► To make an assessment of the functioning of the WCs put in place; and

► To make an assessment of the involvement and awareness of the end-users in the

management of water points and shared sanitation facilities.

The detailed Terms of Reference for the evaluation are provided in Annex1.

Evaluation Methodology



Page 8

2. Evaluation Methodology

The evaluation was carried out through desk review of the main project documents (please refer to

Annex 2). In addition, personal interviews using interview guides and outline discussed and agreed

with UNICEF (please refer to Annex 3); direct observation; informal conversation; and focus group

discussions (kindly refer to the list of interviewees in Annex 4 and semi-structured interviews guides

to the target group in Annexes 5, 6, 7, 8, 9 and 10).

Key informants consulted included project staff at national level and as well as in target provinces,

districts and municipalities, officers from the Provincial Directorate of Public Works and Housing

(DPOPH) and District Services of Planning and Infrastructures (SDPI).

The evaluation team visited two out of four districts (Dondo and Nicoadala) and three out of four

municipalities from Sofala and Zambézia provinces, in the project areas as to assess in depth, the

different aspects of the project results.

Field visits included spot check on various project outputs i.e. water points, latrines; dish racks;

hand washing facilities; etc. and discussion with the WCs members and other key informants.

This evaluation report is based on the format provided by UNICEF included in Section 7: Monitoring,

Review and Reporting - CRIS “Implementation Report“ of the European Commission - Aid Delivery

Methods – Project Cycle Management Guidelines (Volume 1 – 2004).

Limitations of the Evaluation



Page 9

3. Limitations of the Evaluation

Limitations of the evaluation were related to the scope of the work, timeframe and budget. The

main limitations were the following:

► It was not included in the scope of the work a detailed assessment of the activities

performed by the programme. The evaluation focus mainly on the achievement of the

expected results;

► An in depth assessment of the end-users was also not included in the scope of the work. The

evaluation focuses on the impact of the project but did not perform a survey on the end-

users;

► The report format was agreed after the field work, which poses an enormous challenge for

the evaluation team;

► Limited timeframe and budget did not allow a more in-depth evaluation.

Findings and Discussion of Findings



Page 10

4. Findings and Discussion of Findings

The findings are presented and discussed based on the CRIS “Implementation report” format. The

format provides the main headings, such as project description; origin, context and key assessment;

summary of project implementation; changes in context and in the key assessment areas; progress

in achieving objectives; financial execution; issues arising and action required; and cross-cutting and

other issues.

4.1. Project Description

Overall objective: The Overall Objective of the project was to contribute towards the achievement

of MDG and Poverty Reduction and Strategy Paper (PRSP) targets for water and sanitation in

vulnerable rural and peri-urban areas of selected provinces and districts of Mozambique.

Purpose: The program aimed at providing improved access to safe and affordable drinking water,

sanitation and hygiene facilities for vulnerable communities in 4 rural districts and 4 municipalities.

Results: The project results were (i) National water policy guidelines disseminated and sector

planning, monitoring and evaluation strengthened; (ii) Provincial, municipal and district level

capacity to plan, manage, monitor and evaluate the WASH sector using NWP guidelines and DRA

strengthened; (iii) Participatory and sustainable community management structures engaged and

operational, as well as hygiene practices and HIV/AIDS awareness improved; and (iv) Access to safe

and affordable water supply, sanitation and hygiene facilities improved at communities levels as well

as in schools; and (v) Project management and technical assistance timely provided on a yearly

basis. In particular, the project expected results by 2010 were: (i) 150,000 people with access to

safe and sustainable water points; (ii)

sanitation facilities (Household latrines); (iii) 120 schools and 45,000 schoolchildren with safe

access to sanitation and hygiene facilities (latrines, hand washing, urinals); (iv) 120,000 people had

benefited from hygiene promotion.

Main activities: (i) provide capacity building which included guidelines, trainings and workshops to

strengthen capacity at national and provincial levels as well for private sector; (ii) provide capacity

building at district and municipal level, especially in planning and monitoring programme activities;

(iii) provision of WASH infrastructure in the target areas. Especially the construction and repair of

water sources as well as motivating households to construct their own latrines (without subsidy);




Page 11

(iv) provide assistance at the institutional level, to the realization of decentralized governance

including planning, implementing and monitoring of interventions.

Location and length: Districts of Nicoadala, Milange, Buzi and Nhamatanda and municipalities of

Beira, Dondo, Mocuba and Quelimane in Sofala and Zambézia provinces. The length of the

programme was four years.

Cost and key inputs: Budget for the four year period 2006-2010 was EUR 5,570,279.20, of which

EC and UNICEF contributed fifty per cent each. Key inputs were: (i) staff from WASH, Education and

Health Sections, and (ii) supplies (cars, motorcycle, computers and other equipment).

4.2. Origin, context and key assessment

Rationale/justification for the project: One of the major constraints affecting the water sector is

the high percentage of water points out of order (30 percent). This is due to lack of capacity in

Operation and Maintenance (O&M) linked to the weak supply chain, and to hydro-geological and

climatic factors, such as wells drying out. The capacity development initiative aimed at improving

the overall management of activities, both during and after the external funding period, in line with

the national policies and strategies of decentralisation, integrated water and sanitation and

Integrated Water Resource Management (IWRM) principles, community involvement and the use of

appropriate technologies. This will not only ensure the sustainability of development activities but in

addition the potential for synergistic benefits to this project was also expected to be exploited. The

promotion of appropriate water and sanitation technology options and the establishment of local

centres for spare parts (in direct partnership with the private sector) would contribute to the

sustainability of development activities. This would be complemented with an intensive application

of participatory methodologies for community development, improved hygiene practices and

community based management, which will enhance demand in providing better services and was

expected to promote community ownership.

Main conclusions arising from the assessment of the project context: The assessment was

conducted based on problem analysis. As a result, the four key blocks defined for this project were

identified as crucial to ensure an integrated and coordinated intervention in the sector, such as: (i)

NWP dissemination and overall sector planning, monitoring and evaluation; (ii) Provincial and

district level planning, monitoring and evaluation; (iii) Sustainability of WASH infrastructures and




Page 12

hygiene promotion; and (iv) Access to affordable water and sanitation facilities at community and

school levels.

In that time, the Government has recognized these priorities as crucial to attain the MDGs on WASH

by 2015 and by doing so to contribute to reducing poverty in accordance with the latest PRSP

(2006-2009). The project would strengthen ongoing programmes in the Provinces of Zambezia and

Sofala. This EUWF project would build on the good progress already made in water and consolidate

the actions developed in sanitation and hygiene in schools and communities. The provincial capacity

to manage medium level projects (1 to 2 million USD/year) had been tested with success and a new

project would be crucial to consolidate this capacity and improve the gaps already existing (better

planning and M&E skills, school and household sanitation and hygiene). In addition, this project

would focus on peri-urban areas (Quelimane, Mocuba and Beira) and would consolidate the

experience of integrating WASH improvement in schools with overall community management

capacities.

4.3. Summary of project implementation

Main features of the implementation highlighting main developments: The project aimed at

strengthening the planning, management and monitoring and evaluation (M&E) capacity at national,

provincial, municipal and district levels. It provided training for hundreds of sanitation, health and

education workers at the provincial and district level as well as the training of mechanics in the

maintenance of hand pumps and community activists and teachers in the promotion of good

hygiene practices. The project also supported the creation of sanitation clubs in schools and

awareness programmes on hygiene promotion in the classroom. Another important component of

the project included activities directly linked to the service delivery of water, sanitation and hygiene

facilities.

Problems encountered: in Table below, all major problems encountered and the solutions given are

presented:

Problems encountered Solutions given

(i) weak capacity of the private sector to deliver

timely, creative and high quality services;

(i) provided training to the local construction

companies, a new contract management process for

drilling of borehole was successfully introduced;

(ii) shifted from the inclusion of geophysical surveys in

the contracts of the supervising engineers to include

them in the contract with the drilling contractors,

whose effect was that the many negative boreholes




Page 13

Problems encountered Solutions given

drilled before, were not any longer paid by UNICEF but

accountable to the contractor themselves;

(ii) low sustainability of the constructed and

rehabilitated WASH facilities, particularly in relation

to WC and the O&M of the facilities;

Strong focus on capacity building and strengthening

the Water User Groups in the Annual Work Plan for

2010. A technical manual was developed for WCs to

improve their performance, accountability and

transparency;

(iii) limited human resource capacity both at district

and municipality levels;

hired technicians for WASH in each district and

municipalities to be later integrated in SDPI;

(iv) lack of quantitative indicators related to NGO

contract management to effectively measure the

social mobilisation and hygiene promotion outputs of

NGO partners.

continued strengthening the monitoring and

evaluation systems through the SINAS/JMP

initiatives.

Lessons learned: (i) New results-based contract management process for drilling of boreholes has

been successful in reducing the cost per borehole and ensured the quality of infrastructure;

(ii) The establishment of WC requires more consideration in terms of follow-up training and

monitoring to ensure continued sustainable use of the water supply infrastructure as well as

providing more guidance for management of facilities. The experience has been that sequencing of

PEC with the construction of the water points is crucial, as well as monitoring of important

indicators regarding the organisation of the WCs. The results based contracts for PEC, implemented

from 2008 onwards, are anticipated to significantly improve the monitoring of the WCs, however,

this remains an area of concern. In addition, more guidance has to be given over continued periods

of time to WCs that experience problems with establishing proper systems of tariffs and collection

of revenue, which has a knock-on effect on proper and timely maintenance of water points. One of

the measures taken is the development of a Maintenance Manual to guide WC in management of

their water and sanitation facilities and assure transparency and accountability to improve financial

management as well;

(iii) The need for more flexibility and adjustments during the project implementation, taking into

account the changes in the context and the sector dynamic;

(iv) Regarding to the project impact, it was clear that rural and peri-urban areas have different

characteristics and specifications, and therefore future interventions need to be more geographical

focused for a better definition of intervention models;

(v) The coordination process among the implementation partners at central and provincial level

(UNICEF, DNA and DPOPH) requires time and should be defined and shared in advance to

accommodate potential administrative constrains;




Page 14

(vi) Provinces, municipalities and districts have shown the capacity to manage directly funds

allocated by the project for the local level (around 89% of funds were directly managed by the

provinces, municipalities and districts); and

(vii) The programme established sanitation clubs, trained teachers and students on sanitation and

hygiene promotion. These activities have not only being done in the schools but also in the

surrounded areas of the school. In this context the sanitation clubs mobilized the communities on

sanitation and hygiene promotion, as well. This can be considered as an added value for the

programme, and future interventions may consider this integrated approach.

4.4. Changes in context and in the key assessment areas

From 2007 the rural water and sanitation decentralization process was arranged. The district was

made responsible for water coverage maintenance ensuring the functionality of existing water

points through repair of non-functional water points. It was also introduced the PEC Zonal which is

managed at district level and is being accepted as a way to sustainable water and sanitation. This

approach contributed to reduce water point’s breakdowns and increased the water coverage.

The PEC ZONAL and CLTS approach had contributed to increase the willingness of community and

citizen to have improved water and sanitation facilities, such as improved toilet and tap in house. It

also contributed to trigger the innovation of the communities on latrine construction.

The collaboration between UNICEF and FIPAG on pipe extension had conducted to a situation that

the pressing over the water points with hand pump and with taps in the community reduced because

a considerable number of families decided to have their own taps in their house. These facts

mentioned above were relevant because they contributed to the sustainability of WASH facilities.

4.5. Progress in achieving the objectives1

The programme is expected to have contributed towards the measured increases in coverage;

however, it is not assumed that it is the only attributing factor as several other interventions might

have been contributing to increase coverage as well. Almost all planned outputs have been realized

or exceeded. In both provinces, HH with access to safe water sources increased over the target

defined in the programme, i.e, it was expected an increase of 6%, having been reached an average

1 This section is based on Final Donor Report for European Commission SC/06/450 produced by UNICEF – June 2010.




Page 15

of 12%. In terms of safe sanitation facilities, in average the target has been achieved, although the

increase in Zambézia had been of 3% against 5% defined as target.

An assessment of the progress in achieving each of the expected results is presented below:

Result 1: District Government and Municipality were trained on NWP guidelines which improved

their skills to disseminate and promote it at the community level. The improvement of their skill

leaded to better provision of water and sanitation services as they improved their planning and

monitoring and evaluation skills. The WCs have improved their skills in water and sanitation

infrastructures management, as the WCs members have received training and refreshments on

basic management skills and also have received support from the district government and

municipality in terms of their legitimacy as legal community representative. The sustainability check

conducted by Austral Cowi in 2009, show that the levels of sustainability in Milange and Nicoadala

were highest compared to the other districts (sustainability index of 51,78% in Milange and 49,88%

in Nicoadala). The same situation was found in Dondo (90%), which has also one of the highest levels

of sustainability (please refer to Ernst & Young – Sustainability Check of WASH Activities in 9

Districts).

Regarding to the WASH data bank, it were established in 2007 with indicators for the target

provinces. As part of the process for developing the MDG Road Map, the Strategic Plan for the Rural

Water Supply and Sanitation (PESA-ASR, 2006-2015) was developed with active participation from

WES stakeholders from sub-national and national levels, including the project targeted districts and

municipalities.

Result 2: According to UNICEF, around 89% of the programme funds (cash and goods) were directly

managed by provinces, districts and municipalities, from which 3%; 55% and 31%, respectively,

exceeding the 70% defined by the project. The project has provided training on e-SISTAFE and on

Procurement rules and procedures to strengthen the management capacity of the funds.

In the four districts and four municipalities the planning process took place. The plans are available

and based on the assistance received the staff have developed capacity to plan and implement the

activities.

Result 3: All water points have WC in place. Although there are monthly paying schemes working in

the districts, in urban areas the payments schemes doesn’t work well because there are alternative

sources of water available. The current percentages of households in programme areas with access

to safe water sources is in average 60%, being Nicoadala 48.2%, Dondo 81% and Quelimane (60%).




Page 16

This average represents three out of eight target areas2, which does not allow for a conclusion.

However, the target of 90% set by the programme was too ambitious as the baseline (DNA 2003) in

the two selected provinces (Sofala and Zambézia) was in average 38%.

Result 4: The Programme Specific Objectives had by and large been achieved, namely: (i) over

296,000 people have access to safe drinking water, through construction and rehabilitation of 313

water points; (ii) approximately 215,000 people (nearly 39,000 families) in rural and peri-urban

areas had access to safe sanitation facilities; (iii) around 58,450 learners in 167 schools in poor

rural and peri-urban areas had access to safe school WASH facilities; and approximately 134,000

people in rural and peri-urban areas.

Result 5: Project management and technical assistance was timely provided on a yearly basis for

the implementation of the project.

For more details vide Annex 11.

4.6. Financial Execution

The total project budget for the four year period was EUR 5,570,279.20, of which 50 per cent (EUR

2,785,139.60) was funded by the EC and 50 per cent by UNICEF. The budget allocation was in line

with the need to spend the resources in capacity building and in the provision of WASH facilities. A

large portion of the total budget, around 83%, was allocated to two core project items of

expenditure, namely (i) subcontracting related to construction activities; and (ii) other cost category

(self-help construction of HH latrines, PEC and support training programmes (PEC, DRA, PHAST);

support preparation of MDG progress report; support the establishment of provincial data banks and

GIS. The remaining 17% were allocated to equipment and supplies, human resources, administrative

costs. In terms of expenditure, by June 2010 EUR 5,080,676.68 has been spent, which

corresponds to 91% of the available funds. Some over and under deviations of the budget occurred

but they were all necessary and timely agreed with EC, namely: (i) human resources due to the fact

that the time that additional WASH staff has spent to the EU project has been billed accordingly; (ii)

Latrines (self-help construction of san plats) due to the introduction of CLTS which has costs

significantly low; (iii) M&E (provincial databanks) due to existence of alternative funds and (iv)

services (geophysical studies for boreholes) due to the new contractual arrangement introduced.

2 Data from Beira, Milange, Mocuba, Búzi and Nhamatanda.




Page 17

4.7. Issues and action required

The main issue was the sustainability. In assessing the sustainability it was important to highlight

that the two main project focuses were capacity building and provision of water and sanitation

facilities. A variety of people at provincial, district and municipalities level had been trained, and

they were prepared to continue providing services to the communities. The activities that would be

sustained include the WCs and the maintenance of the water points.

Although limited human resource capacity exists in some selected areas, it was expected that the

trained community members would continue sensitizing their and other communities in sanitation

and hygiene matters. Therefore, the improvements in the living conditions of the target population

would be maintained.

Evidences from the field visits indicated that these activities would be continued. The staff of

DPOPH/DAS, Municipalities and District has acquired knowledge and skills to implement the

activities including in their annual plan. However, high staff rotation within the Government

counterparts may represent a challenge for the sustainability, as a result of the absence of

“institutional memory” that lies on the technician.

Sustainability levels of water, sanitation and hygiene facilities could be considered satisfactory. WCs

put in place were working, but they seemed to require a close monitoring, because in some places

visited the percentage of families paying for the services was very low, which may pose a big

challenge for the future sustainability.

The sustainability relied also on the need to strengthen the capacity of the private sector to deliver

timely and creative services at local level, which also might require additional support.

4.8. Cross-cutting and other issues

Poor women and children are most at risk and frequently made vulnerable from lack of water and

sanitation facilities, that is why the project addressed the gender equality and child issues during its

implementation. Water sector institutions, including WC, are generally dominated by men who often

resist the equal representation of women in decision making bodies. For this reason and the

recognition of the fact that women play a central role in the provision, management and

safeguarding of water, the project addressed the gender equality in its implementation.




Page 18

That was verified through the following aspects: (i) Most of WC have women playing a relevant role

as a president or chief of maintenance group; (ii) In some communities women were trained as

mechanics, which repair the water points. In schools, children play a role in the sanitation clubs

established in the schools. The project worked with communities and schools to provide sanitary and

hygiene education for both boys and girls ensuring that both were responsible for cleaning and

toilet maintenance. In schools, boys and girls play the same roles in the sanitation and hygiene

promotion within established schools sanitation clubs. These sanitation clubs were also mobilizing

the surrounded communities on the hygiene promotion acting as transforming agent.

An environmental assessment has not been carried out in the selected areas. However, the

potential negative environmental implications are considered negligible. It was, therefore, felt that

water supply interventions will not impact negatively on groundwater resources. The major

contribution in the environment issues was through the sanitation component, as follows: (i)

stimulating the HH to construct latrines and innovate new types of latrines; (ii) CLTS have

contributed to create a healthy and sustainable environment; (iii) The implementation of Integrated

Water Resources Management approach which has contributed to develop in each province an

integrated Master Plans, ensuring sustainable use of water resources; (iv) The construction of wells

and boreholes, finished to a standard to protect groundwater supplies was well practised in

Mozambique. The type of hand pump employed, Afridev, usually restricts abstraction rates to less

than one litre per second and their disbursement is widespread; (v) To improve routine water quality

testing and treatment, more affordable and community friendly tools and materials were introduced

(H2S Test Kits), integrated with the participatory community education campaigns for safer hygiene

practices.

The HIV/AIDS has been an important issue addressed by the programme. It is now widely accepted

that availability of safe water and sanitation does not automatically lead to improvement in health if

not combined with hygiene promotion and appropriate hygiene behaviours. This was particularly

important for People Living with HIV/AIDS. In a HIV/AIDS context it was important that water supply

pits and latrines were easily accessible and close to where needed to reduce the burden of long-

distance water collection for care-givers and also to reduce the risk of girls and women being raped

while fetching water, and thus reducing vulnerability of HIV infection. Activists were trained under

the Programme to promote sanitation and hygiene and also trained to promote HIV/AIDS awareness

in the communities, allowing them to change their behaviour.




Page 19

4.9. Main conclusions and Recommendations

The findings presented and discussed under five traditional practice of evaluation of development

aid formalised by the OECD-DAC criteria, namely: Relevance of the project, Efficiency in terms of

implementation, Effectiveness in achieving the results, Impact the project had on the lives of the

beneficiaries and Sustainability of the results, are presented below.

Relevance

The programme was highly relevant. It was implemented in areas selected due to their high

vulnerability indicators, including diarrhoea prevalence, cholera cases and HIV/AIDS prevalence, of

four rural districts and four peri-urban areas of Sofala and Zambézia Provinces. In addition, its

objectives are aligned with the GoM main policy documents regarding to the sector, namely PARPA

and National Water Policy.

Efficiency: The programme set up and implementation procedures were adequate for achieving of

project objectives. The implementation model was adequate and the involvement of provincial,

district, municipality and water committees creates awareness on the project implementation and

results. There was also effective backstopping from UNICEF through monitoring visits that provided

support in relevant areas of the project implementation. The use of both human resources and

funds was efficient. It has spent around 91% of the budget but has exceeded the majority of the

expected results.

Effectiveness

From the review of the project achievements after four years of implementation, the EU Water

Facility WASH Programme has been a success. It has been particularly strong in delivering results in

terms of infrastructure provision in the target areas, where in all cases it exceeded the targets set

for infrastructure provision and the number of people to be covered by the action.

Especially the construction and rehabilitation of water sources as well as motivating households to

construct their own latrines (without subsidy) have exceeded expectations in terms of end

beneficiaries covered.

The Programme Monitoring and Evaluation framework has been used by stakeholders, specifically

the Government at all levels and other implementing partners in their planning and projects

activities.




Page 20

Impact

The project has shown potential to have a great impact on the living conditions of the selected

population, in which almost all planned outputs have been realized or exceeded. At household level,

the main impact of the project will be related to the increase of social status as a result of having

decent improved latrine. In addition to that, HH members, mainly women, have more time available

to participate in other activities as a result of available water supply close to the HH. Based on the

improved access to water and sanitation conditions which leads to a clean environment, less

sickness is expected in the home as well as an increased personal hygiene.

The benefits mentioned can also indirectly impact on the reduction of HH expenditure with health

care, and therefore, in a more disposable income to pay for essentials such as food, soup and

clothing.

However, a specific assessment of the project impact in the selected areas is recommended. The

evaluation team considers too early to make any judgement on the project impact at this stage.

Sustainability

There is a high probability that the benefits provided by the programme may continue after its

termination, taking into account that it has made a great investment in capacity building of local

communities and in the provision of water and sanitation facilities.

However, this is a great challenge for all stakeholders involved, mainly the water management

committees, the local service providers and the local authorities, which may require additional

technical assistance and support.

Taking into account the conclusions above, it is recommended that:

► Better coordination of monitoring and evaluation activities, especially at central level

counterparts;

► Prioritize WASH activities on peri-urban areas, which seem to be neglected in the majority of

interventions. The need for these services has been growing;

► Increase the direct support provided to the counterparts in order to ensure a more closed

implementation of the planned activities;

► Improve information flows about the programme among the stakeholders, specifically regarding

the involvement of central level counterparts;




Page 21

► Share lessons learned with other provinces in order to disseminate best practice being

implemented by the programme;

► Improve the articulation and/or coordination with other complementary programs, in order to

harmonize the interventions at all levels;

Annexes



Page 22

Annexes

Annexes

23

Annex 1 – Terms of Reference

Annexes

24

UNICEF Mozambique

TERMS OF REFERENCE:

Final Evaluation of EU Water Facility WASH Programme in 4 Districts

(Buzi, Nicoadala, Nhatanda, Milange) and 4 Municipalities (Beira, Dondo,

Quelimane, Mocuba)

Submitted by: WASH Section

1. Background and Context

Within the framework of the partnership between the European Commission, UNICEF and the

Government of Mozambique, the EU Water Facility WASH Programme – Improvement of the

Livelihoods of Rural and Peri-urban Poor through Increased Access to Safe and Sustainable Water,

Sanitation and Hygiene Facilities in Sofala and Zambezia Provinces- has been implemented since July

2006. This intervention aims to contribute towards the achievement of the MDG and Poverty

Reduction Strategy Paper (PRSP) targets for water and sanitation in vulnerable rural and peri-urban

areas of underserved provinces in terms of safe water and sanitation coverage. The Programme, which

has duration of 4 years, will end in July 2010 and has the following expected results:

National water policy guidelines disseminated and sector planning, monitoring and evaluation

strengthened.

Provincial, municipal and district level capacity to plan, manage, monitor and evaluate the

WASH sector using NWP guidelines and DRA is strengthened.

Participatory and sustainable community management structures put in place and operational,

and hygiene practices and HIV/AIDS awareness improved in all project districts and

municipalities is strengthened.

Access to safe and affordable water supply, sanitation and hygiene facilities improved at

communities and schools in all eight project districts

Project management and technical assistance timely provided on a yearly basis.

In particular, by the end of the Programme it is expected that the following achievements will have

been made:

150,000 people with access to safe and sustainable water points.

25,000 families (125,000 people) with access to safe sanitation facilities (Household latrines)

120 schools and 45,000 schoolchildren with safe access to sanitation and hygiene facilities

(latrines, hand washing, urinals).

120,000 people benefited with hygiene promotion.

By March 2010, the following results were reported:

132.220 people with access to safe and sustainable water points.

73.664 families with access to safe sanitation facilities (household latrines).

127 schools with safe access to sanitation and hygiene facilities.

134.327 people benefited from hygiene promotion.

Annexes

25

2. Justification

The external final evaluation is a pre-requisite of the European Union cooperation and is part of the

Programme Monitoring and Evaluation framework, which was drawn up jointly by UNICEF, the

European Commission and the Government of Mozambique at the beginning of the Programme. This

assignment is in line with planned activities of UNICEF Mozambique, as it is part of the WASH

Section Programme Annual Work Plan for 2010.

3. Purpose and Objective

The main objective of this study is to evaluate the EU Water Facility WASH Programme, an initiative

that aims to improve access of rural and peri-urban poor people to safe and sustainable water,

sanitation and hygiene facilities in 4 districts and 4 municipalities from Sofala and Zambezia

provinces. The Programme will finish in July 2010, after 4 years of implementation, and UNICEF, the

European Commission and the Government of Mozambique are keen on evaluating its performance,

impact on target population people and performance of partners in the Programme implementation.

The information gathered in this assignment will contribute to the improvement of the Monitoring and

Evaluation system within the WASH sector. Lessons learnt in the course of the evaluation will be very

useful for similar ongoing WASH programmes in Mozambique, such as the One Million Initiative.

Moreover, the information collected through the study will help to assess whether recommendations

and conclusions from previous evaluations of the Programme (EU Monitoring visit and Sustainability

Checks) were integrated in its approach.

4. Methodology and Technical Approach

The proposed methodology for this end of Programme evaluation is a combination of qualitative data

collection methods, such as literature and programme documents review, in-depth individual

interviews with key informants (technical staff involved in the programme implementation,

community leaders, etc.), focus group discussions and direct observations in the field.

The assignment should consider the following key questions:

How far has the Programme Monitoring and Evaluation framework been used by

stakeholders?

Have end-users been adequately trained?

Is sustainability of water, sanitation and hygiene facilities satisfactory?

Are water management committees put in place working?

Are end-users aware of and involved in the management of water points and shared sanitation

facilities?

Due to the time and financial limitations, a process of sampling will be done in which the following

districts and municipalities will be visited. These include Beira and Dondo Municipalities and

Nhamatanda District in Sofala Province and Quelimane Municipalitiy and Nicoadale Distric in

Zambezia Province.

The consultant is expected to develop the questionnaires and sampling protocol and have these pre-

approved by UNICEF.

Annexes

26

5. Activities and Tasks

Desk review of programme documents such as Programme Proposal, Progress Reports,

Sustainability Check, etc. UNICEF will provide all these documents.

In depth interviews and focus group discussion with Programme staff based in Maputo

(UNICEF, DNA-DAR, NGOs which carried out the social mobilization process) and in the

field.

In depth interviews and focus group discussion with other key stakeholders, such as

community leaders, members of Water Management Committees, directors and teachers from

schools were water, sanitation and hygiene facilities were implemented, etc.

Direct observation of Programme outcomes in the field.

Draft of the final report.

Meeting with main stakeholders to present and discuss results

Presentation (Power Point) of main results.

Revise draft report based on stakeholders’ feedback.

6. Deliverables and Timeframe

Deliverables

The consultant or consultancy firm will be expected to produce the following deliverables:

Draft of the final report

A stand-alone Power Point presentation of the evaluation methodology and results with

complete speaking notes that will enable a user to comprehend the assessment without a

presenter needing to speak.

A separate Lessons Learnt document.

Final Report including an executive summary and abstract (based on specific format to be

provided by UNICEF).

All these documents must be presented in English and Portuguese language, ensuring adequate level

of both languages, and in digital format (five CDs) and hard copy (five copies).

Timeframe

The time given for completion of this evaluation is 4 weeks, of which 1 is anticipated to be of

fieldwork in Zambezia and Sofala Provinces. Below there is a detailed timeframe of the Evaluation:

Tasks

Week

1 2 3 4

Issuance of Contract

Literature review

In depth interviews with key

stakeholders (Maputo)

In depth interviews with key

stakeholders (Sofala and

Zambezia)

Data analysis

Draft Final Report

Annexes

27

Meeting with key stakeholders to

discuss preliminary results

Stakeholders analysis of the

Report

Incorporate stakeholders feed

back

Final report

7. Management, Organization and Schedule

This assignment is a critical component of the monitoring and evaluation framework designed at the

beginning of the EU Water Facility WASH Programme by the Directorate of Water of the

Government of Mozambique (DNA), UNICEF and the European Commission. The consulting firm

will be contracted by UNICEF Mozambique, and UNICEF will provide the consulting firm a

stakeholders’ contacts list. The consulting firm will be responsible for scheduling and attending

meetings with relevant authorities and institutions, such as local authorities and institutions at the

district and municipal level, NGOs involved in the Programme implementation, DNA staff, etc.

UNICEF will designate a focal point within the WASH Section, who will liaise with the consultant or

consulting firm during the assignment implementation.

As specified in Section 6 of this document, this assignment has duration of 4 weeks.

7. Budget and Remuneration

The total cost of this evaluation will be no more than $20,000. Timeframe is critical for the

successful completion of this assignment and, therefore, consulting firms must ensure that they

have the right staff to carry out duties, particularly in the fieldwork stage.

9. Qualifications and Specialized Knowledge

The consulting firm should have sound experience in evaluation of development projects/programmes

in Mozambique. In particular, experience within the WASH Sector is highly desired.

10. Conditions of Work

The table below indicates standard UNICEF guidance for conditions of work in which the consultancy

will be carried out:

Condition of work

To be provided by UN

Agencies

Remarks

Yes No

Field Trip Transport

X

Annexes

28

International Travel

X

Visa for internationally

recruited

X

DSA

X

Security Clearance

X

Accidental Death and

Dismemberment and

Accident Medical

coverage

X

Health Insurance

X

Office Space

X

Computer

X

Secretarial Services

X

Photocopier services

X

Other supplies

X

11. Remarks:

Please add any additional information you might found required.

Technical Evaluation Criteria:

Example of technical criteria and relative points:

Technical Criteria Technical Sub-criteria Maximum Points

Overall Response Completeness of response Overall concord between RFP requirements and proposal

Maximum Points 10

Company experience

Range and depth of experience with similar projects Number of customers, size of projects, number of staff per

project

Annexes

29

Client references

Maximum Points 15

Key Personnel Key personnel: relevant experience and qualifications

Maximum Points 20

Proposed

Methodology and

Approach

Technologies used - compatibility with UNICEF Project management, monitoring and quality assurance process Innovation approach

Maximum Points 25

Financial Proposal 30

Total Maximum 100

30

.

Annex 2 – List of people interviewed

31

Nome Posição Intituição Local

Samuel Godfrey WASH Section Head UNICEF Maputo

Angelina Xavier WASH Specialist UNICEF Maputo

Samuel Manhiça WASH Specialist UNICEF Maputo

Carlota Muianga WASH Specialist UNICEF Maputo

Julieta Felicidade DAR Head DNA Maputo

Idalina Alfai Staff DNA Maputo

Renato Solomon Staff DNA Maputo

Abel Florêncio Substituto do Chefe DPOPH/DAS Sofala

Benardete Manga Técnico DPOPH/DAS Sofala

Jerimias Liando Vereador Município da Beira Sofala

Julieta Nhamposse Técnico Município da Beira Sofala

Francisco António Presidente do comité de Água Bairro Nhangau - Município da Beira Sofala

Salomão Pedro Cobrador do fontanario Cemitério do Régulo - Municipio da Beira Sofala

Manual Dimo Gestor de Fontanario EPC de Macaronhe Sofala

Gente António Presidente do comité de Água Bairro Njalane-Municipio da Beira Sofala

Castelo Manuel Director Pedagógico EP1 de Macuti - Beira Sofala

Jorgina Inácio Coordenadora do Núcleo de Saneamento EP1 do Matadouto - Beira Sofala

Gervásio Brito Gestor do Balneário Público bairro maraza - Municipio da Beira Sofala

Maurio Safrao Cobrador do balneário Público bairro Matadouro - Municipio da Beira Sofala

Humberto Lalala Balneário público Bairro Miquejo - Municipio da Beira Sofala

Manuel Augusto Mulima

Técnico Água e Saneamento - representante do Chefe Município de Dondo Sofala

Vilares Patricio Fiscal Município de Dondo Sofala

Araujo Peixe Presidente do comité de Água Bairro Nhamawabwe/ Und C - Municipio de Dondo Sofala

Carlota Augusto Cartoto Presidente do comité de Água

Bairro Macharote - Municipio de Dondo Sofala

Paulo Erbezia Chefe do Núcleo de Saneamento Centro Primário de Acomodação - Dondo Sofala

Celestino João Director da Escola Centro Educacional de Dondo Sofala

Silva Mário Vereador de planificação - substituto do presidente Município de Quelimane Zambézia

Armando Jorge Técnico de Água e Saneamento Municipio de Quelimane Zambézia

Xavier Luis Fagima Director da escola EPC de Nhanhibua -Municipio de Quelimane Zambézia

Maria da Conceição R.J. Tebueira Drectora da escola

EPC de Morrupué - Municipio de Quelimane Zambézia

Amina Suluvai Chefe de Promoção de Higiene Bairro santagua - Municipio de Quelimane Zambézia

Castro conhece Chefe de promoção de Higiene

Presidente do conselho comunitario de promoção de Higiene no bairro Manhaua Zambézia

32

Nome Posição Intituição Local

Carvalho Branquinho Roupa Gestor do centro Centro de demostração Zambézia

Virgílio Augusto Amizade Chefe do SDPI

Governo do distrito de Nicoadala Zambézia

Ailton Mulessiua Técnico do SDPI Governo do distrito de Nicoadala Zambézia

Fernado Alilo Técnico do DAS DPOPH/DAS Zambézia

Guilhardo Perreira Vergonha Director da Escola

EPC de Namitangurinho - distrito de Nicoadala Zambézia

Bonifácio António Presidente do comité de Água Manta mutuela Zambézia

Maria Madalena Chefe do grupo de Manutenção Fitimela A Zambézia

Direscente Armando Mtumula Presidente do comité de Água Mudidi Zambézia

Pilares Laveque Presidente do comité de Água Assumate Zambézia

33

Annex 3 – EU WF Final Evaluation Outline 3

34

Expected Results Indicators What to assess Research tool Information source

1.1.NWP guidelines disseminated in four

districts and four municipalities by 2007

Do districts and municipalities plans for the WASH

Sector follow NWP guidelines? Are Districts and

Municipalities staff familiar w ith NWP guidelines?

Have they attended any training about NWP?

Meetings;

Interview s

DNA (DAR &GPC),

DPOPH, Municipalities

and Districts

1.2.National data bank established and

operational by 2009

What has been done from 2007 to now ? What are

the next steps planned and the timeframe? Meetings DNA (GPC)

1.3. MDG Road Map prepared for the

sector by 2006.

What have been the steps taken so far? Planes

estrategicos??? Meetings DNA (GPC &DAR)

2.1. 70% of project funds managed

directly by provinces and district

authorities by 2009

What percentage of the programme funds (cash and

goods) w as directly managed by by provinces,

municipalities and districts managing?

Desk Review

UNICEF Financial

Reports; Meetings

UNICEF staff (Samuel

godfrey & Teofilo

Nhapulo

2.2. Four districts and four municipal

plans prepared and implemented in all

project areas by 2009

Did the planning process take place? Are there any

plans? Have they develop capacity to plan and

implement? Interview s

Municipalities, Districts

(SDPI), DPOPH

2.3. DRA principles applied in all project

areas by 2009

2.4. Tw o provincial data banks

established & operational by 2009

Are those data banks operational? If so, w hen w ere

they established? Who use them? What service do

they provide? If not, w hy? Were they established at

some point?

On site-observation;

Interview DPOPH

3.1. 90% of w ater points w ith monthly

w ater paying schemes managed by

w ater committees (# of w omen) in all

districts by 2009 ;

Are the w ater committes put in place in all w ater

points? Are the monthly paying schemes w orking? If

not, w ere they put in place at some point during the

project? Why they didn't w ork?

Meetings,

Interview s,

SDPI technical staff,

Municipalities technical

staff, Water

Committees members,

UNICEf staff

3.2. frequency of hand pumps break

dow n in one year reduced to a maximum

of tw o in project areas;

In average, w hat is the frequency of breakdow ns in

hand pumps? Are there records of breakdow ns?

How do w ater committes pay for the hand pumps

reparation? Are there trained artisans available to the

communities?

Water committes

members

3.3. three days as the maximum number

of days that it takes to repair a pump in

project areas;

Are there w ater points not being used for long

periods ? What happen in those cases? Water committes members

3.4. 90% of project areas household

using safe w ater sources by 2009;

What is the current percentage of households in

programme areas w ith access to safe w ater

sources?

UNICEF staff, off icers

from DNA, DPOPH and

SDPI, Municipalities

3.5. 60 % of w omen using HH latrines by

2009 in project areas

3.6. 70 % of school children (girls and

boys) practising hand w ashing after

defecation by 2009 in project areas

4.1. 150,000f people w ith access to safe

and sustainable w ater points in project

areas by 2009;

How many w ater points w ere constructed under the

programme framew ork? How many people do they

serve? % of those w aterpoints that are w orking

now ?


from DNA, DPOPH,


4.2. 25,000 families w ith access to safe

sanitation facilities (HH latrines) by 2009 in

project areas;

Number of latrines constructed under the programme

framew ork? Are these latrines being used?


from DNA, DPOPH,


4.3. 120 schools and respective

schoolchildren w ith safe access to

sanitation and hygiene facilities (latrines,

hand w ashing, urinals) by 2009 in project

areas;

Number of w ater points and latrines constructed in

schools under the programme framew ork? Are these

infraestructures operational? Are students using

them?

UNICEF staff, DPOPH,

Directors of schools,

Municipalities

5.1. At least 90% of project expenditure

level yearly ensured; What w as the yearly expenditure of the programme?

UNICEF staff, f inancial

records of the

programme

5.2. At least 85% of activities are

implemented by 2009;

In 2009, how many activities w ere implemented?

Which ones? And by the end of the programme?

What activities w ere not implemented? Why?

UNICEF staff, DNA,

DPOPH, SDPI and

Municipalities

Result 1. National w ater policy

(NWP)guidelines disseminated and

sector planning, monitoring and

evaluation strengthened by 2009

Result 2. Provincial, municipal and

district level capacity to plan, manage,

monitor and evaluate the WASH sector

using NWP guidelines and DRA

strengthened by 2009

Result 3. Participatory and sustainable

community management structures put

in place and operational, and hygiene

practices and HIV/AIDS aw areness

improved in all project districts and

municipalities by 2009

Result 4. Access to safe and

affordable w ater supply, sanitation and

hygiene facilities improved at

communities and schools in all eight

project districts by 2009

Result 5. Project management and

technical assistance timely provided on

a yearly basis up to 2009

35

Annex 4 – List of Documents Reviewed

36

1. UNICEF Mozambique, June 2010 - Final Report for the European Commission - SC/06/450 -

Improvement of the Livelihoods of Rural and Peri-urban Poor through Increased Access to Safe and Sustainable Water, Sanitation and Hygiene Facilities in Sofala and Zambezia Provinces, Mozambique

2. European Commission, 2006 - Detailed Grant Application Form. ACP-EU WATER FACILITY ACTIONS IN ACP COUNTRIES for Non-state Actors (ACP and EU), Public Bodies (EU) and International Organisations

3. UNICEF Mozambique, June 2010 - EU Water Facility II Final Evaluation Outline 1

4. UNICEF Mozambique, June 2010 - EU Water Facility II Final Financial Report

5. UNICEF Mozambique, June 2010 - Minutes of EU Water Facility II Final Evaluation Meeting

6. UNICEF Mozambique, July 2007 - Progress Report for the European Commission - SC/06/450 - Improvement of the Livelihoods of Rural and Peri-urban Poor through Increased Access to Safe and Sustainable Water, Sanitation and Hygiene Facilities in Sofala and Zambezia Provinces, Mozambique

7. European Commission, July 2008 – Aid Delivery Methods: Project Cycle Management

Guidelines (Volume 1)

8. UNICEF Mozambique, March 2009 - Progress Report for the European Commission - SC/06/450 - Improvement of the Livelihoods of Rural and Peri-urban Poor through Increased Access to Safe and Sustainable Water, Sanitation and Hygiene Facilities in Sofala and Zambezia Provinces, Mozambique

9. UNICEF Mozambique, June 2010 - Final Report for the European Commission - SC/06/450 - Improvement of the Livelihoods of Rural and Peri-urban Poor through Increased Access to Safe and Sustainable Water, Sanitation and Hygiene Facilities in Sofala and Zambezia Provinces, Mozambique

10. UNICEF Mozambique, March 2009 - Progress Report for the European Commission - SC/06/450 - Improvement of the Livelihoods of Rural and Peri-urban Poor through Increased Access to Safe and Sustainable Water, Sanitation and Hygiene Facilities in Sofala and Zambezia Provinces, Mozambique

11. Ernst & Young, November 2009 – Sustainability Check for the WASH

12. Austral Cowi, October 2009 - Sustainability Check for the WASH in Zambézia Province

37

Annex 5 – Guideline of semi-structured interviews to UNICEF

38

Result 1

National WP

1.Describe the involvement of your institution in the dissemination of WP?

2.There was any follow up activity?

Result 2

Funds management

1. What was the role of UNICEF in all this process?

2. What percentage of the programme funds (cash and goods) was directly managed by provinces and municipalities and district managing?

Result 3

Safe water sources

3. What is the current percentage of households in programme areas with access to safe water sources?

4. What are the factors affecting the access to safe water sources?

Result 4

People with access to safe and sustainable water points

5. How many water points were constructed under the programme framework?

6. How many people do they serve?

7. How many waterpoints are working now?

Families with access to safe sanitations facilities

8. Number of latrines constructed under the programme framework?

9. Are these latrines being used? If not why? Is there any change in the approach?

10. What type of latrines were constructed?

Schools with access to safe sanitation and Hygiene facilities

11. Number of water points and latrines constructed in schools under the programme framework?

12. Are these infraestructures operational?

13. Are students using them? If not, why they are not using?

Result 5

Project expenditure

14. What was the yearly expenditure of the programme?

39

Annex 6 – Guideline of semi-structured interviews to DNA

40

Result 1

National WP

1. Describe the involvement of your institution in the dissemination of WP?

2. There was any follow up activity?

3. Do districts and municipalities plans for the WASH sector follow the WP?

4. Are Districts and Municipalities staff familiar with WP guidelines?

5. Who implemented/facilitated the training?

6. How did you get feedback about the dissemination of WP?

Data bank

7. When the data banks were established? How was the process of its establishment?

8. How was the situation before the establishment of data bank?

9. What is the actual situation of data bank implementation?

10. What are the next steps planned and the timeframe?

MDG

11.Are you aware about the MDG for your sector?

12.What have been the steps taken so far by your institution?

1. Please provide evidences for the steps taken?

Result 2

Result 3

Safe water sources



Result 4







20. Are these latrines being used?


Result 5

41

Annex 7 – Guideline of semi-structured interviews to DPOPH

Result 1

National WP





5. Have they attended any training about WP?



Data bank

8. When the data banks were established?

9. How was the situation before the establishment of data bank?

10. What is the actual situation of data bank implementation?

11. What are the next steps planned and the timeframe?

Result 2

Plans and projects implemented by prov and districts

12. Did the planning process take place?

13. Are there any plans?

14. Have they develop any capacity to plan and implement?

15. What type of assistance have you received for this process?

Data bank

16. Are these data banks operational?

17. Who use them?

18. What is the service do they provided?

19. Did you get any training on data bank operation?

Result 3

Safe water sources 20. What is the current percentage of households in programme areas with access to safe water sources?


Result 4




24. How many waterpoints are working now? What has been done to increase the figures?



26. Are these latrines being used?

27. What type of latrines were constructed? What has been done to increase the figures?

Schools with access to safe sanitation and Hygiene facities


29. Are these infraestructures operational?


Result 5

Annex 8 – Guideline of semi-structured interviews for Municipalities

Result 1

National WP








Result 2

Plans and projects implemented by prov and districts

8. Did the planning process take place?

9. Are there any plans?

10. Have they develop any capacity to plan and implement?

11. What type of assistance have you received for this process?

Result 3

Water points

12. Are the water committes put in place in all water points?

13. Are the monthly paying schemes working? If not, were they put in place at some point during the project? Why they didn't work? 14. How many women are involved in management? Sustainability ..

Safe water sources


16. What are the factors afecting the access to safe water sources?

Result 4



18. How many people do they serve? 19. How many water points are working now?



21. Are these latrines being used? 22. What type of latrines were constructed? Sustainability .. Latrinas familiares ...Balneários ... Drenagem Como o programa ajudou a aumentar a capacidade de saneamento … conhecer a rede?


24. Are these infraestructures operational? 3. Are students using them? If not, why they are not using?

Result 5

Annex 9 – Guideline of semi-structured interviews for Water Committees

Result 1

National WP



Result 2

Result 3

Water points

3. Are the water committes put in place in all water points?

4. Are the monthly paying schemes working? If not, were they put in place at some point during the project? Why they didn't work?

5. How many women are involved in management?

Hand pumps break down

6. In average, what is the frequency of breakdowns in hand pumps? Are there records of breakdowns?

7. How do water committes pay for the hand pumps reparation?

8. Are there trained artisans available to the communities?

9. If not, who fix the pumps?

Days to repair

10. Are there water points not being used for long periods ? What happen in those cases?

11. Why there are not being used?

12. Were do you get the spair parts?

Result 4







Result 5

Annex 10 – Guideline of semi-structured interviews for Schools

Result 1

Result 2

Result 3

Result 4



2. Are these infraestructures operational? 3. Are students using them? If not, why they are not using? 4. What has been done to ensure the operations and maintenance of water points and latrines? (O que tem feito para garantir a operacionalidade das escolas?) 5. Como é que feita a manutenção e limpeza das latrinas nas escolas e a capacitação dos professores e crianças?

Result 5

Annex 11 – Details on Section 4.5

Result 1: District Government and Municipality were trained on NWP guidelines which improved

their skills to disseminate and promote it at the community level. The District Government and

Municipality promote regular visits to the communities in order to hear their view points, inform

them about the opportunities that the demand responsive approach offers and establish, in

collaboration with their authorities, the channels for requesting funds. The improvement of their

skill leaded to better provision of water and sanitation services as they improved their planning and

monitoring and evaluation skills.

At the community level the managing body is the water and sanitation committee. This body was

responsible for the following up the operation and maintenance of the water source; it is also

responsible for the stocks of spare-parts; controlling user’s contributions, promotion and

dissemination of sanitation and hygiene messages in the communities. The WCs are increasingly

improving their skills in water and sanitation infrastructures management, as they receive training

and refreshments on basic management skills and also support from the district government and

municipality in terms of their legitimacy as legal community representative. The WC also plays an

important role in the improvement of community participation.

Regarding to the WASH data bank, it were established in 2007 with indicators for the target

provinces. In order to gather, analyze and disseminate the information, were undertaken the

following activities: (i) conducted the GIS/Mapping in all districts; (ii) were established the National

Information System for Water and Sanitation (SINAS), implemented under DNA leadership. This

system promotes synergies between Government and sector partners in terms of defining strategic

options and sector priorities as well as further enhancing sector harmonisation; (iii) A training

workshop on WES data analysis and trends towards the achievement of country MDG targets had

been held in 2008; (iv) The training provided basis for review and refinement of sector performance

indicators and definitions aligned with the international M&E guidelines for WASH; and would also

support the sector to strengthen monitoring to track sector progress.

As part of the process for developing the MDG Road Map, the Strategic Plan for the Rural Water

Supply and Sanitation (PESA-ASR, 2006-2015) was developed with active participation from WES

stakeholders from sub-national and national levels, including the project targeted districts and

municipalities. This strategic document leads to the establishment of the Sector Wide-Approach

(SWAps) for the rural water and sanitation – the National Rural Water Supply and Sanitation

Programme (PRONASAR) – and the Common Fund (CF) for the PRONASAR. Both PRONASAR and CF

form the basis for harmonized sector programming, funding, monitoring and evaluation.

Result 2: Around 89% of the programme funds (cash and goods) were directly managed by

provinces (3%), municipalities (31%) and districts (55%), exceeding the 70% defined by the project.

In the four districts and four municipalities the planning process took place. The plans are available

and based on the assistance received their staff have developed capacity to plan and implement the

activities.

In Zambézia province the first data bank was established in 1998 but it didn’t work for a long time.

In 2002 was established a second data bank which was improved by DFID/UNICEF in 2006. To

operate the data bank was indicated a staff to operate it and received training in Maputo carried out

by DNA. Between 2006 and 2008 the staff received training on job training type through WE

Consults agency which helped on data’s interpretation. In time being the data bank is operational

and the information is provided to the provincial government, ONGs, DPOPHs to plan their activities.

The Sofala data bank was established in 2008 by ZAMWATER programme funded by EU. This data

bank never worked because the computers memories were stolen. In 2009, with the support of

UNICEF/EUWF the new memories were bought, but they still need to be installed and the staff

trained.

Result 3: All water points have WC in place. The situation is good in the districts but improvements

are required in the peri-urban areas. Today, in peri-urban areas, there are a variety of alternative

sources of water available, such as water taps and network supply taps provided by FIPAG. When

the project was designed, there were not expected that the network would cover such areas.

Currently, in the visited locations, breakdowns in hand pumps are almost inexistent. Therefore, no

records of breakdowns were found. However, the WC still collecting monthly payments for future

hand pumps reparation. There are trained artisans available to the communities. All water points

visited by the evaluation team are being used.

The current percentages of households in programme areas with access to safe water sources is in

average 60%, being Nicoadala 48.2%, Dondo 81% and Quelimane (60%). This average represents

three out of eight target areas3, which does not allow for a conclusion. However, the target of 90%

set by the programme was too ambitious as the baseline (DNA 2003) in the two selected provinces

(Sofala and Zambézia) was in average 38%.

3 Data from Beira, Milange, Mocuba, Búzi and Nhamatanda.

Result 4: The Programme Specific Objectives had by and large been achieved.

The programme result of 150,000 people with access to safe and sustainable water points was

exceeded thus over 296,000 people have access to safe drinking water, through construction and

rehabilitation of 313 water points. All the water points visited were operational.

The objective of providing access to safe sanitation facilities (Household latrines) for 25,000

families (125,000 people) has been exceeded thus approximately 215,000 people (nearly 39,000

families) in rural and peri-urban areas had access to safe sanitation facilities. All six latrines visited

in Dondo by the evaluation team were operational, and the same number in Nicoadala was also

operational.

The expected result of 120 schools and 45,000 school children with safe access to sanitation and

ited with hygiene

promotion, have been also exceeded. Around 58,450 learners in 167 schools in poor rural and peri-

urban areas had access to safe school WASH facilities. The visited facilities in eight schools were all

operational and being used.

The programme result of 120,000 people benefiting from hygiene promotion had been also

exceeded; reaching approximately 134,000 people in rural and peri-urban areas. These people have

increased knowledge on proper hygiene behaviour through hygiene promotion activities.

Result 5: Project management and technical assistance was timely provided on a yearly basis for

the implementation of the project. It was also provided support for the implementation by assisting

responsible for managing the financial aspects of the programme at the provinces as well

supporting the procurement of equipment in line with UNICEF rules and regulations. Monthly field

visits have been carried out in each project area in coordination with national, provincial, municipal

and district teams.

Annex 12 – PPT for the Final Evaluation Report