Republic of South Africa Department Water Affairs and Forestry M&E Unit Water Services Support Directorate Department of Water Affairs & Forestry Private.

Republic of South AfricaDepartment Water Affairs and Forestry

M&E UnitWater Services Support DirectorateDepartment of Water Affairs & ForestryPrivate Bag X313 PRETORIA

MONITORING, EVALUATION AND REPORTING STRATEGY AND THEORETICAL FRAMEWORK:

• The South African government was mandated in 2004 to halve poverty and unemployment by 2014 (The Presidency, 2006). Government recognises that for economic growth and sustained development delivery of basic household service is critical. Provision of infrastructure and services will not directly alleviate poverty or stimulate economic growth but will be necessary to achieve this goal. Accountability, transparency and improvement of intervention performance should be central to any democratically elected government. To be more efficient, transparent and accountable the functions and role of state must be clear and the conditions under which government interventions take place need to be defined (DLA, 1998). Any government policy lead intervention, such as provision of basic water services, needs to systematically assess the impact of this intervention against the goals of such a policy or programme. Monitoring, evaluation and reporting (M, E & R) is the systematic examination of the effectiveness and efficiency of the design and implementation of policies, programme and projects. The idea is to learn from experience and apply the lessons learned in the designing of new policies, programmes and projects to make them more effective

BACKGROUND

• According to The Presidencies proposed GWM&ES, to operationalise the GWM&ES all public service entities need to undertake their own credible M&E processes that meet clearly defined standards and deliver information on their progress and performance. Government will, therefore, need to ensure that adequate monitoring and evaluation processes and procedures are in place to allow for reporting on progress and impact of policy targets and goals.

• DWAF as the sector leader and regulator of water supply and sanitation service is under increased pressure to reach national and international goals. Funding and human resource capacity within the water service sector will need to be substantially increased to meet these targets. At a minimum, increased funding and human capacity need to be augmented by a robust M&E system for assessment of planning, progress, quality and finance of projects and programmes (Van Zyl, 2006). Ideally, M&E systems should be extended to strategic, planning, operational and impact evaluations of service delivery interventions. The result of these M&E initiatives provides evidence-based decision-making to support adjustment of policy, strategy and implementation processes.

• To support the above statement by Mr van Zyl (2006) the Joint Free Basic Service and Municipal Infrastructure Technical Task Teams and Section 78 Committee took a decision in the meeting of 21 September 2006 that “DWAF is required to re-establish a monitoring framework after the M&E (monitoring and evaluation) function was moved from DPLG to DWAF. DWAF needs to mobilize the regions to monitor again and the DDG requests a plan from Regional to this effect”.

• The National Water Service Act (Act No 107 of 1997): It includes monitoring, evaluation and reporting for the objective of the right of access to basic water supply and the right to basic sanitation necessary to secure sufficient water and an environment not harmful to human health or well-being (Section 2 (a)). The Act makes mention of the monitoring of water services and intervention by the Minister or by the Provinces (Section 2 (e)). According to the Act, the Minister (Department) and any relevant Province must monitor the performance of water services institutions to ensure compliance with every applicable development plan and policy statement (Section 63 (c)). A national information system must be established to enable the monitoring of the performance of these water service institutions (Section 68 (b)).

ACTS AND POLICY

• Strategic Framework for Water Services (SFWS) (2003) denotes that monitoring take place at all levels of government (i.e. local, regional and national) and by all the role-players in this sector (i.e. water service providers, water service authorities, catchment management agencies, provincial government, national departments and the water services regulator). DWAF as the regulator is responsible for monitoring sector performance and managing information to be used for support, monitoring, regulation and planning.

KEY CONCEPTS

• Monitoring is a continuing function that aims to provide the DWAF management and other key stakeholders with early indications of progress, or lack thereof, of ongoing interventions achievement of results, goals, objectives or targets.

• Standards, Norms & quality

KEY CONCEPTS

• Evaluation is a time-bound exercise that systematically and objectively assesses the relevance, efficiency, effectiveness, impact and sustainability of a policy, programme or project. Evaluation of outcomes and impacts provide evidence-based decision-making in support of policy, strategy and implementation adjustments.

– REPORTING • For the purposes of this document reporting is defined as a method in which

the information for monitoring and evaluation is reflected. It is a systematic and timely provision of essential information at periodic intervals in order to track the progress of projects or programmes.

– QUALITY OF LIFE– A key to evaluation of impact is conducting a quality of life survey. This is the

most significant tool in addressing the question whether the direct and indirect beneficiaries’ life have improved by receiving improved or new services.

• Quality of life is an individual’s satisfaction or happiness with life in domains s/he considers important. Known as "life satisfaction" or "subjective well-being", it is sometimes referred to as "overall quality of life" or "global quality of life" to distinguish it from "health-related quality of life". It is the broadest of all concepts, being influenced by all of the dimensions of life that contribute to its richness and reward, pleasure and pain (Testa & Simonson, 1996).

• The term “quality of life” can have several meanings. It may be used to refer to outward material circumstances, such that good quality of life is represented by good physical health, material security, supportive family and friends, etc. Alternatively, it can refer to subjective well-being, by this being meant an individual's sense of happiness or satisfaction, typically reflecting a global assessment of all aspects of their life.

KEY CONCEPTS

KEY CONCEPTS

• Effort: quantity and quality of a project.• Performance: effect criteria, i.e. the results of

effort, intended or unintended.• Adequacy: the degree to which effective

performance is adequate to the total amount of need.• Efficiency: whether there are any better ways of

achieving the same results/capacity to produce results in proportion to effort.

• Process: how and why a project works or does not work, ie making sense of the evaluative findings.

Strategy Policy

Strategy

Monitoring

Planning

Implementation

Evaluation

Multiple layers of each activity

combine to form sector

Phases overlap to varying degrees

Feedback Decision support

feedback

Good Practice Project Management Principles: M&E Process (DWAF)

DPLG Key Performance Indicator Reporting

Impact Evaluation by DWAF

Quarterly or Monthly Water Sector reports

Regional Spot checks report

1.TRIANGULATION AS A VERIFICATION APPROACH2.This approach will make use of three or more sources or types of information to verify and substantiate the water sector research/survey findings. This approach seeks to overcome the bias from a single informant source, a single method or a single observation. For the purpose of this project triangulation means utilising multiple sources of data/information to strengthen findings. Some of the proposed points of data sources are illustrated in Figure 2 below:

Figure 3: Triangulation approach recommended by DWAF for the verification and substantiates water service information and reporting.Verification of the implementation of the water and sanitation projects will be done through on-site “spot-checks” of both current (quality, standards and norms) and completed (sustainability) projects. In agreement with the country-wide monitoring and evaluation system recommended by the Presidency, monitoring reports will need to be supplemented by periodic evaluations that assess the impact of government programmes and propose changes to policy and implementation strategies (The Presidency, 2005). The one data source in the triangulation approach will be gathered through impact studies such as Quality of Life surveys; observations and case studies.

• The GWM&ES is based on the monitoring and evaluating process shown in Figure 4. The DWAF triangular framework includes verification of input data and evaluation (quality of life) of outcomes and impacts. These two approaches compliment each other. The DWAF ‘spot-check’ relates to verification of outputs from water sector project and programmes, while the impact evaluation (i.e. Quality of Life surveys) provides information of the outcomes and impacts of sector initiatives. Information generated from these DWAF monitoring and evaluation initiatives, can act as sources of information for the GWM&E system.

Input Monitoring

Output Monitoring

OutcomesEvaluation

ImpactEvaluation

Verification by DWAFImpact Evaluations

by DWAF

Input Monitoring

Output Monitoring

OutcomesEvaluation

ImpactEvaluation

Verification by DWAFImpact Evaluations

by DWAF

Monitoring, Evaluation and Reporting Theoretical Strategy

DWAF Spot-

Checks

ProgrammeEvaluations

Quality of Life/

Impact Assess-ments

DPLG KPI Reports

Monthly, quarterly, reports from Regions

PWSP

PSSIP

WSDP

Process/programmeefficiency, effectiveness & sustainability

Impact on beneficiariesQuarterly MIG

basic services reports

Monitoring Evaluation

Internal consultations with DWAF Regions, P&I, Sanitation

External consultations through sector collaboration

External evaluations for objective views to counter the “player also becoming the referee”.

Development of DWAF M,E&R implementation / consultation Strategy The DWAF strategy for M,E&R will be developed to provide a framework for the assessment of water and sanitation projects, through conducting of spot-checks, programme evaluations and quality of life surveys – refer to Figure 6. Some information will also be collected from the sector through monthly and quarterly KPI reports from sector partners. The consultation actions is also clearly included so that it becomes clear where the Regional inter phase will take place to provide the input for the various developmental models (A,B,C) for conducting the spot checks.

Consultation & development

Consultation & development

• Development of M,E&R Training

• Develop training strategy and materials with input of regions to empower and enable regional staff to conduct spot check assessments and data verification.

Way forward for 2006/7

• Further Consultation with regions• Regional M & E coordinators 2 year contract

posts• Implementation of independent spot check

methodology – priority of DDG:R• Incremental development and implementation

of framework and regional M & E actions

Thank you