The Provincial-wide Monitoring and Evaluation System ... · 1. PROVINCIAL-WIDE MONITORING AND EVALUATION (M&E) FRAMEWORK 1.1 The purpose of the Framework for a Provincial-wide Monitoring

PROVINCIAL-WIDE MONITORING

AND

EVALUATION FRAMEWORK

Department of the Premier

2009


CCOONNTTAACCTT DDEE TTAA II LL SS ::

Chief Directorate: Monitoring, Evaluation, Review and Reporting Chief Director: Ms Zeenat Ishmail Department of the Premier Tel.: 021-483-8709


AAAAAAAA CCCCCCCC RRRRRRRR OOOOOOOO NNNNNNNN YYYYYYYY MMMMMMMM SSSSSSSS BI Business Intelligence BPO Business Processing and Outsourcing BRS Business Requirement Specification CCRS Climate Change Response Strategy CeI Centre for e-Innovation CD: MER Chief Directorate: Monitoring, Evaluation and Reporting DCF District Coordinating Forum DG Director-general GDS Growth and Development Strategy (Western Cape) GWMES Government-wide Monitoring and Evaluation System HCDS Human Capital Development Strategy HOD Head of Department ICT Information and Communication Technology IGR Inter-governmental Relations ILRP Integrated Law Reform Programme IMA International Monitoring Association M and E Monitoring and Evaluation MEDS Micro Economic Development Strategy MERR Monitoring, Evaluation, Review and Reporting MTEF Medium Term Expenditure Framework MTSF Medium Term Strategic Framework NSS National Statistics System PCF Premier’s Coordinating Forum PMCF Premier’s Metro Coordinating Forum PRS Poverty Reduction Strategy PSDF Provincial Spatial Development Framework PSA Provincial Strategic Agenda PSP Provincial Strategic Plan PTM Provincial Top Management PWMES Provincial-wide Monitoring and Evaluation System RBM and E Results-based Monitoring and Evaluation SALDRU: South African Labour Development Research Unit SAMEA South African Monitoring and Evaluation Association SASQAF South African Statistical Qualification Assessment Framework SCFS Social Capital Formation Strategy SDIP Sustainable Development Implementation Plan SIP Strategic Infrastructure Plan SOPA State-of-the-Province Address SSS Scarce Skills Strategy Stats SA Statistics South Africa WCSHSS Western Cape Sustainable Human Settlements Strategy


CCCCCCCC OOOOOOOO NNNNNNNN TTTTTTTT EEEEEEEE NNNNNNNN TTTTTTTT SSSSSSSS ACRONYMS ............................................................................................................................................................................................... 3 CONTENTS ................................................................................................................................................................................................ 4 DIAGRAMS................................................................................................................................................................................................. 5 A. PURPOSE AND AIM OF THE FRAMEWORK FOR THE PWMES................................................................................................... 6 1. PROVINCIAL-WIDE MONITORING AND EVALUATION (M&E) FRAMEWORK............................................................................. 6

1.1 The purpose of the Framework for a PWMES .............................................................................................................................. 6 1.2 Aims of the Provincial-wide M & E Framework ............................................................................................................................. 6 1.3 Applicability of the Framework........................................................................................................................................................ 6

B. PWMES .................................................................................................................................................................................................. 7 2. OVERVIEW............................................................................................................................................................................................. 7

2.1 Background and Rationale for a PWMES ..................................................................................................................................... 7 2.2 Definition of Monitoring, Evaluation, Review and Reporting ........................................................................................................ 7 2.3 Results-based Monitoring and Evaluation (RBM & E) approach ................................................................................................. 8

3. REGULATORY FRAMEWORK............................................................................................................................................................. 8

3.1 Government-wide Monitoring and Evaluation System.................................................................................................................. 8 3.2 Statistical Mandate and GWMES................................................................................................................................................... 8 3.3 The PWMES within the PGWC ...................................................................................................................................................... 9

C. PWMES: SEVEN PHASE MODEL ................................................................................................................................................... 11 4. THE PWMES AND ITS ELEMENTS .................................................................................................................................................. 11

4.1 Provincial-wide Monitoring and Evaluation System (PWMES): Western Cape Model....................................................... 12 4.1.1 Phase 1: Readiness Assessment and Stakeholder Engagements ........................................................................................ 12 4.1.2 Phase 2: Overarching Frameworks for the PWMES ............................................................................................................... 13 4.1.3 Phase 3: Indicator Definition Process and Indicator Frameworks.......................................................................................... 14 4.1.4 Phase 4: Monitoring and Results Frameworks ........................................................................................................................ 14 4.1.5 Phase 5: Data Management and Data Assessment................................................................................................................ 15 4.1.6 Phase 6: Information Architecture............................................................................................................................................. 15 4.1.7 Phase 7: PWMES – Planning to Implement and Sustain the PWMES.................................................................................. 17

5. COMMON STANDARDS AND CRITERIA FOR THE PROVINCIAL-WIDE MONITORING AND EVALUATION SYSTEM ....... 17

5.1 Overview of Common Standards and Criteria............................................................................................................................. 17 5.2 Guidelines and Principles ............................................................................................................................................................. 18 5.3 Standards and Concepts .............................................................................................................................................................. 18 5.4 Criteria and Quality Dimensions................................................................................................................................................... 19 5.4.1 Relevance ................................................................................................................................................................................... 19 5.4.2 Accuracy ..................................................................................................................................................................................... 19 5.4.3 Timeliness ................................................................................................................................................................................... 20 5.4.4 Accessibility ................................................................................................................................................................................ 20 5.4.5 Interpretability ............................................................................................................................................................................. 20 5.4.6 Coherence................................................................................................................................................................................... 20 5.4.7 Methodological soundness ........................................................................................................................................................ 20 5.4.8 Integrity........................................................................................................................................................................................ 20

D. INSTITUTIONALISATION OF THE PROVINCIAL-WIDE MONITORING AND EVALUATION SYSTEM.................................... 22 6. PWMES REPORTING FRAMEWORK............................................................................................................................................... 22

6.1 M and E within the Government Planning Framework ............................................................................................................... 22 6.2 Role of the Clusters and Inter-Governmental Relations............................................................................................................. 24

7. CONCEPTUAL APPLICATION OF THE MONITORING AND EVALUATION FRAMEWORK...................................................... 26

7.1 Application of Monitoring and Evaluation Framework ................................................................................................................ 26


DDDDDDDD IIIIIIII AAAAAAAA GGGGGGGG RRRRRRRR AAAAAAAA MMMMMMMM SSSSSSSS

Diagram 1: PWMES MANDATE WITHIN PGWC .................................................................................................................................. 10

Diagram 2: PWMES PROCESSES......................................................................................................................................................... 11

Diagram 3: KEY MONITORING AND EVALUATION ELEMENTS IN PWMES: SEVEN-PHASE MODEL....................................... 12

Diagram 4: BUSINESS REQUIREMENT SPECIFICATION AND PWMES......................................................................................... 17

Diagram 5: RELATIONSHIP BETWEEN PERFORMANCE MANAGEMENT AND RBM AND E...................................................... 19

Diagram 6: CRITERIA AND QUALITY DIMENSIONS .......................................................................................................................... 21

Diagram 7: MONITORING AND REPORTING FRAMEWORK............................................................................................................ 23

Diagram 8: PWMES AND THE INSTITUTIONS OF THE PGWC ........................................................................................................ 25

Diagram 9: MONITORING AND EVALUATION FRAMEWORK........................................................................................................... 26


AAAAAAAA........ PPPPPPPPUUUUUUUURRRRRRRRPPPPPPPPOOOOOOOOSSSSSSSSEEEEEEEE AAAAAAAANNNNNNNNDDDDDDDD AAAAAAAAIIIIIIIIMMMMMMMM OOOOOOOOFFFFFFFF TTTTTTTTHHHHHHHHEEEEEEEE FFFFFFFFRRRRRRRRAAAAAAAAMMMMMMMMEEEEEEEEWWWWWWWWOOOOOOOORRRRRRRRKKKKKKKK FFFFFFFFOOOOOOOORRRRRRRR TTTTTTTTHHHHHHHHEEEEEEEE PPPPPPPPWWWWWWWWMMMMMMMMEEEEEEEESSSSSSSS

11.. PPRROOVVIINNCCIIAALL--WWIIDDEE MMOONNIITTOORRIINNGG AANNDD EEVVAALLUUAATTIIOONN ((MM&&EE)) FFRRAAMMEEWWOORRKK

11..11 TThhee ppuurrppoossee ooff tthhee FFrraammeewwoorrkk ffoorr aa PPrroovviinncciiaall--wwiiddee MMoonniittoorriinngg aanndd EEvvaalluuaattiioonn SSyysstteemm

[[PPWWMMEESS] This document proposes the development of an M&E Framework for the]. It outlines the following:

• The conceptual understanding of the PWMES

• The conceptual understanding of Results-based Monitoring and Evaluation (RBM & E)

• The context of the PWMES1

• The application of national guidelines and common standards to ensure RBM & E

• The application of statistical criteria to ensure integrated and coherent monitoring and evaluation processes as well as to improve the data-quality management.

• The Institutional Framework of the PWMES.

11..22 AAiimmss ooff tthhee PPrroovviinncciiaall--wwiiddee MM && EE FFrraammeewwoorrkk With this framework it is envisaged that those responsible for the Monitoring & Evaluation [M & E] of the Provincial Strategic Plan (PSP), Provincial Government of the Western Cape (PGWC) Strategies and provincial policy areas will have a greater understanding of the following:

• Identification of the PWMES, alignment with the Government-wide Monitoring and Evaluation System (GWMES) and the Regulatory Framework

• Improvement of the understanding of the processes and systems required for RBM&E for the PWMES

• Improvement of the understanding of the processes and systems required for the PWMES

• Identification of national guidelines and their application within the context of the PWMES

• Clarification of compliance-driven M&E in relation to achieving RBM&E

• Clarification of common standards and criteria for monitoring, evaluation, review and reporting in support of the M&E data-quality management process

• Promotion of accountability and transparency from provincial and local government departments to provide reliable data and information timeously for efficient reporting

• Identification of institutional and reporting requirements for managing M&E and supporting systems within the Western Cape.

11..33 AApppplliiccaabbiilliittyy ooff tthhee FFrraammeewwoorrkk

The framework is applicable to those responsible for M&E in all spheres of government and more specifically to those who are responsible for reporting on the overarching objectives of the PSP and collating statistical evidence on policy areas or statistical themes. The framework sets the scene for how RBM&E requirements are integrated into the provincial policy areas of the Provincial Government of the Western Cape (PGWC). This framework demonstrates how the M&E elements are required for achieving RBM&E within the PWMES should be applied to the strategic developmental agenda of PGWC. It also demonstrates how components of compliance-driven M&E should be integrated into the RBM&E approach in order to achieve success in measuring the PSP.

1 This refers to whether the PWMES is applied within an international (MDGs), Provincial Policy Areas, Provincial Strategies,

and the PSP of the PGWC context. The PSP is the 5 year strategic plan for the PGWC and its represents a blueprint for co-ordinated action by provincial departments to realise the vision of ‘an open opportunity society for all’.


BB.. OOVVEERRVVIIEEWW AANNDD RREEGGUULLAATTOORRYY FFRRAAMMEEWWOORRKK OOFF TTHHEE PPWWMMEESS

22.. OOVVEERRVVIIEEWW

22..11 BBaacckkggrroouunndd aanndd RRaattiioonnaallee ffoorr aa PPWWMMEESS The GWMES is a system developed by government clusters for tracking the performance of government programmes. It is a mechanism for assembling and reporting information to stakeholders [especially to national, provincial and local governments] on the performance of programmes of government departments and other public bodies with the aim to improve governance. The Proposal and Implementation Plan for a GWMES provides the framework for the PWMES and it therefore takes an integrative approach. [The Presidency of the Republic of South Africa, The Presidency Policy Unit, Proposal and Implementation Plan for a GWMES, 2005, p.1] The objective of the Western Cape Government is to develop a PWMES that is able to collect, interpret, analyse and disseminate data and information to key stakeholders that adds value to the performance management and decision-making processes of the Provincial Government. [Department of the Premier: Chief Directorate: Monitoring, Evaluation and Review, Conceptual Framework, 1st draft, 2006-2007, p.47]

22..22 DDeeffiinniittiioonn ooff MMoonniittoorriinngg,, EEvvaalluuaattiioonn,, RReevviieeww aanndd RReeppoorrttiinngg For the purpose of the Provincial-wide M&E Framework, the definition of monitoring, evaluation, review and reporting is informed by the proposal and implementation plan of the GWMES and the Conceptual Framework of the PWMES.

MMoonniittoorriinngg is seen as ongoing collection, processing and analysis of data on implementation processes, strategies and results for the purposes of evaluation according to agreed-upon strategic objectives, outcomes and measurable indicators to use the findings to recommend corrective measures to improve decision-making. [Source: Department of the Premier: Chief Directorate: Monitoring, Evaluation and Review. Conceptual Framework for the PWMES Western Cape. 2007. pp.37-39]

EEvvaalluuaattiioonn commonly aims to determine relevance, efficiency, effectiveness, impact and sustainability. Evaluation is a time-bound exercise that systematically analyses and assesses performance against the agreed objectives for the purpose of review. [Source: Department of the Premier: Chief Directorate: Monitoring, Evaluation and Review. Conceptual Framework for the PWMES Western Cape. 2007. pp.37-39]

RReevviieeww aanndd RReeppoorrttiinngg is seen as the hub of reporting. Essentially it is coordinating and facilitating the reporting of various provincial entities and reviewing existing strategies and programmes based on the tracking of progress within the implementation plans and feedback obtained through the action plans.


22..33 RReessuullttss--bbaasseedd MMoonniittoorriinngg aanndd EEvvaalluuaattiioonn ((RRBBMM && EE)) aapppprrooaacchh The RBM&E2 approach of the PWMES for the Western Cape aims to provide feedback on the progress and outcomes of the actual higher-level overarching objectives of the PSP as well as providing statistical data on key policy areas and statistical themes. The RBM& E approach further serves to evaluate over a longer-term period whether the outcomes and impacts of the PSP have been achieved and if so, to ask if they would be sustainable.

The Conceptual Framework of the PWMES attempts to construct the platform for a common understanding of M&E and does so by explaining why the PGWC has chosen the RBM& E approach. It further aims to coordinate, guide and align efforts amongst government departments to ensure that results on implementation can be demonstrated on an outcomes level. [Department of the Premier, Conceptual Framework. 1st draft 2006-2007 p.137]

33.. RREEGGUULLAATTOORRYY FFRRAAMMEEWWOORRKK

33..11 GGoovveerrnnmmeenntt--wwiiddee MMoonniittoorriinngg aanndd EEvvaalluuaattiioonn SSyysstteemm ((GGWWMMEESS))

In 2004 the Cabinet initiated plans for M&E to be conducted by the Government, and the Presidency subsequently developed the GWMES. Although there are various systems gathering valuable information within the Government, there are a number of gaps in the information needed for the planning, delivery of services, reviewing and analysing the success of policies. The GWMES seeks to enhance these various existing systems within the Government by describing them and explaining how well they relate to each other within the components of the GWMES. These components of the GWMES are identified within the Framework for Managing Programme Performance Information, 2007, as follow:

• Programme performance information, Framework for Managing Programme Performance Information

• Social, economic and demographic statistics, Framework for Statistics and Surveys Framework

• Evaluations, Evaluations Framework The PWMES is using the RBM &E approach in relating to the Framework for Statistics and Surveys and the Evaluation framework as indicated above.

33..22 SSttaattiissttiiccaall MMaannddaattee aanndd GGWWMMEESS

Statistics South Africa (Stats SA) has the legal mandate in terms of the Statistics Act (Act No.6 of 1999) to produce and coordinate official statistics in the country. In the absence of acceptable mandatory norms and standards in M&E data, the collection and dissemination of data is the responsibility of government departments in their efforts to track and measure performance of government programmes and strategies. These include administrative data records and sector-specific M & E data.

2 The RBM & E approach of an M & E system focuses on measuring outcomes as it serves to help find the answer to the

fundamental questions of “what results have been achieved?” Do the results indicate a movement towards achieving the desired goals? And if so, what was the impact? (Kusek, J.Z. and Rist, C. 2004. Ten Steps to a Results-based Monitoring and Evaluation System.)


The National Statistical System (NSS), a function of Statistics South Africa (Stats SA), is the coordinating framework within which information in the form of indicators is generated (Annual Performance Plan: Statistics South Africa 2007/08). Within this context, the Memorandum of Understanding between Stats SA and the Department of the Premier of the Western Cape is an attempt to streamline the process on a provincial level.

33..33 TThhee PPWWMMEESS wwiitthhiinn tthhee PPGGWWCC The PWMES for this province focuses on the following three key mandates:

• Develop and implement provincial-wide M&E policies, strategies and programmes for M&E on implementation and results-based level.

• Ensure compliance with the GWMES.

• Ensure continuous provincial-wide M&E of the PSP and Provincial Strategies by focusing on measuring the results on implementation and results-based levels.

The core processes, supporting processes and outcomes of the PWMES for this province, are illustrated in Diagram 1 below:

10


DDDDDDDDiiiiiiiiaaaaaaaaggggggggrrrrrrrraaaaaaaammmmmmmm 11111111:::::::: PPPPPPPPWWWWWWWWMMMMMMMMEEEEEEEESSSSSSSS MMMMMMMMAAAAAAAANNNNNNNNDDDDDDDDAAAAAAAATTTTTTTTEEEEEEEE WWWWWWWWIIIIIIIITTTTTTTTHHHHHHHHIIIIIIIINNNNNNNN PPPPPPPPGGGGGGGGWWWWWWWWCCCCCCCC

M o n ito r in g an d Ev a lu atio n

D e v e lo p a n d im p le m e n t Pr o v in c ia l- w id e p o lic ie s, s tr a t e g ie s a n d p r o g r a m s f o r M &E o n im p le m e n ta t io n a n d r e s u lts - b a s e d le v e lE n su re c o m p lia n c e w ith t h e G o ve rn m e n t – wid e M o n ito r in g a n d E va l u a tio n S ys t e m ( G W M E S)E n su re c o n tin u o u s p r o v in c ia l- wi d e M & E o f t h e PSP a n d p r o vi n cia l st r a te g ie s o f P G W C

Su p p o rt Pr oc e ss es

De t er m in e da ta a n d in fo rm a ti on r eq u ire m e nt s fo r t h e p u rp o se s of PW M E S ; po lic y a r ea s a nd s ta ti st ica l t he m esDir e ct t he m a n ag e m en t o f th e PW M E S i n l ine wit h PS P, po lic y a r ea s a nd s ta ti st ica l t he m esEn s ur e pr o vi nc ia l a lig nm e n t w ith t he n at io na lly p re s cr ib ed M a nd E fra m e wo rk

D e v e l o p c o m p e n d i u m o f i n d ic a t o rs a n d c o re d i re c to ry o f c o m m o n

d a t a s o u r c e s

In di ca to r s f or PW M E S a lig ne d to p ol icy a re a s, s ta tis t ica l t he m es an d PS PEf fec t ive i nd ic at o r a n d d a ta m a na g em en t sy s tem fo r e ac h co r e i nd ic at o rM o ni to rin g a nd R es u lts F r am e wo r k f or th e PW M ES; PSPCo r e d ir ec t or y of co m m o n d at a so u rc es fo r th e PW M ES a nd P SP

P o l ic y M o n it o r in gP o l ic y M o n it o rin g , Ev a lu a t io n ( M & E)

D e te rm i ne M & E r e qu ir em en ts fo r p u rp o se o f M & E o n i m pl em e n ta t ion le ve l D e te rm i ne M & E r e qu ir em en ts fo r p u rp o se o f M & E o n r e su lt s- b as e d le ve lE n su re p ro vin ci al a lig n m e nt wi th th e n a tio n al ly pr es c rib e d M a n d E fr a m ew or k

Su s ta in i n g t h e Pro v i n c i a l- w i d e M & E

S ys t e m

D at a an d In f o rm a tio n o n in d ica t o rs f o r M DG s ; PSP ; Pr ov in c ial S t ra te g ies ; p o lic y a rea s ; s ta t is t ica l t h em es . In f o rm im p le m e n ta tio n o f Pro v in cia l S t ra te g ic P la n n in g p ro c es se s ; Pr ov in c ial p o lic y p rio rit ies

O u tp u ts / O u tc o m e s

I nfo r m at io n Ar ch it ec tu r e t o su p po r t t he P W M ESD e pa rt m e nt a l Sy s te m sE x te rn a l Sy st em s

In t er na l M &E for u m r ep r es e nt in g p r ovin c ial g o ve r nm e n tC o- or d ina t ion o f d a ta p ro d uc e rs a nd d a ta u s er s N et wo rk o f m o n ito r ing a n d e va lua t ion p ra c tit io ne r s a n d s p ec ia l in te re s t gr ou p s

T o fa ci lita t e t h e i m pl em e n ta t ion o f m on it or in g a n d e va lu at io n m et h od o log ie s to a c hi eve RB M a n d EF ac ilit at e wo rk sh o ps on m on it or in g a n d e va lu at io n

Pr ovid e an e ffe c tive co m m u n ic at io n s er vic e an d co lla b or a tio n wit h st a ke h old e rs r eg a rd in g M , E a nd R m a tt er s

St ra t e g i c L e a d e r s h i p a n d Su p p o r t o n M a n d E p ro c e s s e s

Pro v i n c i a l w id e M o n i t o r i n g a n d

e v a lu a t io n s y s t e m

M o n i t o r in g a n d Ev a l u a t i o n F o ru m

Ca p a c it y Bu il d i n g C o m m u n i c a t io nO t h e r Pro v i n c i a l

S t ra t e g ie s

A lig n m en t of a m on it or in g a n d e val ua ti on fu n ct io n w it h o t he r pr ovi nc ia l s e co n d ar y st ra te g ie s

Co re P ro ce ss es

E n su re p ro vin ci al a lig n m e nt to t he G W M E S f ra m e wo rkD e ve lo p pr o vinc ia l M a n d E fr a m ew or k s, s y st e m s a n d p r oc e du r esP ro vid e lea d er s hip t o M a nd E o r ga n isa t ion a l s t ru ct u re s

D es ig n an d m a int a in t he p ro vin ci al- wi de M & E p o lic es , s ys te m s, p ro c ed u re s an d m e c ha n is m s E ns u re p ro vin cia l a lig nm en t wit h th e n a tio n all y p r es c rib e d M a n d E fr am e w or kD ire c t t h e m an a ge m e nt o f t he P W M ES in lin e w ith t he G W M E S

11 Department of the Premier

CCCCCCCC........ TTTTTTTTHHHHHHHHEEEEEEEE PPPPPPPPWWWWWWWWMMMMMMMMEEEEEEEESSSSSSSS -------- SSSSSSSSEEEEEEEEVVVVVVVVEEEEEEEENNNNNNNN PPPPPPPPHHHHHHHHAAAAAAAASSSSSSSSEEEEEEEE MMMMMMMMOOOOOOOODDDDDDDDEEEEEEEELLLLLLLL

44.. TTHHEE PPWWMMEESS AANNDD IITTSS EELLEEMMEENNTTSS

The PWMES as mentioned is a RBM&E system and operates through an effective indicator and data management system. The PWMES has all the required frameworks, M&E elements and mechanisms in place to conduct integrated and coherent monitoring, evaluation and reporting. The strategic approach of the PWMES is documented in the Provincial-wide M&E Strategy.

The design of the PWMES is informed by international results-based models. The implementation of the PWMES is in line with the PSP and provincial strategies for the PGWC. The overarching processes of the PWMES are illustrated in Diagram 2. These processes include the conceptual, strategic and operating M&E frameworks. It also includes the mechanisms required to ensure effective M&E systems at a departmental level. The key M&E elements relates to the indicator frameworks and data source systems to measure outcomes of the PSP and provide statistical evidence on key policy areas and statistical themes. The management of the data and information source systems is critical to ensure sound M&E. DDDDDDDDiiiiiiiiaaaaaaaaggggggggrrrrrrrraaaaaaaammmmmmmm 22222222:::::::: PPPPPPPPWWWWWWWWMMMMMMMMEEEEEEEESSSSSSSS PPPPPPPPRRRRRRRROOOOOOOOCCCCCCCCEEEEEEEESSSSSSSSSSSSSSSSEEEEEEEESSSSSSSS

P W M E S

F ra m e w o rk s

C o n c e p tu a l F r a m e w o rk

P r o v i n c ia l -w i d e M & E F ra m e w o r k

P r o v i n c ia l -w i d e M & E S t ra te g y

M e c h a n i s m s

D e p a r t m e n ta l M a n d E S y s t e m s

D a t a A s s e s s m e n t P r o c e s s

M a n d E P e r fo rm a n c e N o r m s a n d S t a n d a rd s

C a p a c i t y b u i l d i n g

M a n d E F o r u m

M A N D E E L E M E N T S

I n d i c a t o r d e v e l o p m e n t

P W M E S ; M D G ’ s

P S P

M o n i t o ri n g a n d R e s u l t s F ra m e w o r k

D a t a M a n a g e m e n t

M a i n s t re a m

P r o v i n c ia l A d m i n i s t r a t iv e

I n d i c a t o r a n d D a t a R e l e a s e / B a s e l i n e s

M o n i t o r i n g

E v a l u a t i o n

R e p o r t i n g

I n fo rm a t i o n A rc h i t e c t u re

The key interdependent M&E elements that form the basis from which the PWMES seven-phase model operates is illustrated in Diagram 3 as described below:


DDDDDDDDiiiiiiiiaaaaaaaaggggggggrrrrrrrraaaaaaaammmmmmmm 33333333:::::::: KKKKKKKKEEEEEEEEYYYYYYYY MMMMMMMM&&&&&&&&EEEEEEEE EEEEEEEELLLLLLLLEEEEEEEEMMMMMMMMEEEEEEEENNNNNNNNTTTTTTTTSSSSSSSS IIIIIIIINNNNNNNN PPPPPPPPWWWWWWWWMMMMMMMMEEEEEEEESSSSSSSS SSSSSSSSEEEEEEEEVVVVVVVVEEEEEEEENNNNNNNN--------PPPPPPPPHHHHHHHHAAAAAAAASSSSSSSSEEEEEEEE MMMMMMMMOOOOOOOODDDDDDDDEEEEEEEELLLLLLLL

44..11 PPrroovviinncciiaall--wwiiddee MMoonniittoorriinngg aanndd EEvvaalluuaattiioonn SSyysstteemm ((PPWWMMEESS)):: WWeesstteerrnn CCaappee MMooddeell Seven (7) phases have been developed for the PWMES. The status of the processes completed within the seven different phases of the PWMES model is outlined below. Each of these phases is continuously updated and reviewed to address the strategic agenda of Government. These elements within the seven (7) phases are critical as it forms the platform from which to conduct integrated and coherent M&E on implementation and results-based levels.

44..11..11 PPhhaassee 11:: RReeaaddiinneessss AAsssseessssmmeenntt aanndd SSttaakkeehhoollddeerr EEnnggaaggeemmeennttss

RReeaaddiinneessss AAsssseessssmmeennttss:: Three Provincial Audits have been conducted to provide an overview of the readiness of the Provincial Government of the Western Cape to implement RBM&E.

• The Provincial Audit Phase One served as a detailed scoping exercise for establishing a comprehensive M&E study across all M&E units in the province. [Source: Department of the Premier: Chief Directorate: MER: Current reality of M and E in the Western Cape Province. 2005]

• The Provincial Audit Phase Two served as a readiness assessment. It identified a number of critical challenges facing the establishment of a PWMES, detailing capacity constraints, strategic alignment

Phase 1

Readiness Assessment and Stakeholder Engagements

Phase 2 Overarching Frameworks for the PWMES

Phase 3

Indicator Development Process

Phase 5

Data Management and Data Assessment

Phase 6 Information Architecture

Phase 7

PWMES Process – Planning to Implement and Sustain the PWMES

Phase 4 Monitoring and Results Frameworks


problems and reporting inconsistencies. [Source: Department of the Premier: Chief Directorate: MER: Provincial M and E Audit: Phase Two. 2007]

• The Provincial Audit Phase 3 determines to what extent PGWC departments can implement the PWMES seven-phase model to conduct RBM&E at departmental level. [Department of the Premier: Chief Directorate: MER: Data Assessment Report. 2009]

TThhee SSttaakkeehhoollddeerr EEnnggaaggeemmeenntt PPrroocceessss:: The stakeholder engagement process has been institutionalised through the establishment and maintenance of a provincial M&E Forum consisting of M&E staff in the respective Western Cape provincial government departments. In collaboration with the provincial M&E Forum, annual work plans related to the PWMES have been established for the Provincial M&E Forum. The partnership has been invaluable in terms of building critical technical capacity. This Forum facilitates the management of the M&E requirements on provincial departmental level and serves as the strategic link to sustain the departmental alignment of the sector specific indicators to the PSP. The role of partnership is strengthened by the structured stakeholder engagements and broader collaboration. Within the PWMES strong networking and collaboration also exists on an international level and with national and local government departments. On a national level, the Department of the Premier, through a signed memorandum of understanding with Statistics South Africa and the National Statistics System [NSS] Division, is striving towards the coordination and grouping of data producers and users to strengthen research capacity and align data systems in the Western Cape. In this regard, the PWMES conducts data assessments of provincial administrative data and supports local municipalities with data-quality management and indicator development. On an international and provincial level, the International Monitoring Association (IMA) and the University of Cape Town (UCT): SALDRU have provided support to ensure that the mechanism related to M&E capacity building for the PWMES is strengthened. An external M&E reference group will be established and formalised to facilitate continuous learning through the M&E Forum technical capacity working sessions and seminars. The M&E reference group will consist of delegates from key institutions such as Statistics South Africa, the National Technical Assistance Unit, the Southern Africa Labour and Development Research Unit, universities in the Western Cape, the South African Monitoring and Evaluation Association (SAMEA) and the United Nations Population Development Fund and independent specialist.

44..11..22 PPhhaassee 22:: OOvveerraarrcchhiinngg FFrraammeewwoorrkkss ffoorr tthhee PPWWMMEESS The overarching frameworks provide the conceptual understanding of the PWMES, the strategic and operational context of the PWMES towards RBM&E. The following frameworks have been developed:

• The Conceptual Framework (2006/07) establishes the basic framework for an integrated, transversal team-based approach for all stakeholders and key players to shape an efficient M& E process and system in the Province.

• An abridged version of the Conceptual Framework initiates the discourse on the integrated, transversal and team-based approach to a PWMES.


• The Provincial-wide M& E Framework provides the conceptual understanding of the PWMES and introduces the key elements required for the PWMES to ensure integrated and coherent M&E for the PSP and provincial strategies. It outlines the application of national guidelines, common standards and statistical criteria for the data and information requirements for such a system.

• The Provincial-wide M& E Strategy sets out the strategic approach to implement M&E over the period 2009/10 - 2014 to ensure that the PSP and provincial strategies meet its intended outcomes.

44..11..33 PPhhaassee 33:: IInnddiiccaattoorr DDeeffiinniittiioonn PPrroocceessss aanndd IInnddiiccaattoorr FFrraammeewwoorrkkss

The compendium of indicators for the GWMES provides statistical evidence to assess the performance of development programmes of the Government, and guides planning and decision-making. The Indicator Framework for the PWMES provides a compendium of outcome indicators as a suggested measuring instrument to monitor and report on the outcomes of the PSP, PGWC strategies, policy areas and statistical themes. The M&E system for each core indicator aims to drive all subsequent data collection, analyses and data interpretation. It also provides the base for the selection of appropriate monitoring methods, evaluation techniques and reporting requirements. The indicator development phase entailed the following:

• The document “Compendium of Indicators for the Growth and Development Strategy [GDS]” (Green Paper) outlines the indicator development process that has been followed to develop the compendium of indicators for the GDS. The indicator definition process accompanied by the conceptual clarification process provides clarity for the strategic goals and derived outcomes of the GDS.

• The document “Constructing a compendium of indicators for the PWMES to monitor outcomes of the GDS” provides a brief outline for the reasons as to why outcome indicators were developed.

• The Indicator Framework provides a compendium of outcome indicators as a suggested measuring instrument for the GDS. The compendium of outcome indicators is contained in the “Provincial Growth and Development Strategy”, [White Paper dated 15 February 2008]

• An Indicator Framework for the Lead Interventions has been aligned with the GDS indicators, goals and objectives.

• A compendium of indicators for the State-of-the-Province Address (SOPA) 2009.

• A compendium of indicators for the PWMES clusters and aligned to policy areas and statistical themes.

• A compendium of indicators for the PSP (2009/10 -2014)

44..11..44 PPhhaassee 44:: MMoonniittoorriinngg aanndd RReessuullttss FFrraammeewwoorrkkss

The Monitoring and Results Frameworks serves as a basis to identify and match appropriate data sources on national, provincial and local level including provincial administrative data for the compendium of indicators. It also provides sufficient attributes to align the compendium of indicators with baselines, frequency of data collection, responsible data producers and level of disaggregation to measure results of the overarching strategic goals and objectives. The Monitoring and Results Framework phase entailed the following:

• The Monitoring and Results Framework for the GDS and the Strategic Priorities provides the attributes of the outcome indicators, the appropriate data sources and its responsible data producers for the compendium of indicators.


• The indicator development process was further refined into two (2) sets of Indicator and Results Frameworks for the GDS namely Lead Interventions and Legacy Projects.

• The Indicator Analysis Framework provides an overview of the comments which were received from provincial government departments through the technical engagement process. This draft remains a work in progress and will be the baseline for the review of the output and outcome indicators for the financial years 2009/10-2014.

• The Monitoring and Results Framework for the State of the Province Address (SOPA) 2009 provides the first instance of measurement required for provincial policy priority areas of the new Government.

• The most recent Monitoring and Results Framework for the PWMES provides a cluster and alignment of indicators according to policy areas and statistical themes.

• The intensive technical engagements around the indicator development process for the PSP delivered the most updated Monitoring and Results Framework of the PSP (2009/10-2014), which takes into account the various iterations of the PSP.

44..11..55 PPhhaassee 55:: DDaattaa MMaannaaggeemmeenntt aanndd DDaattaa AAsssseessssmmeenntt The data-management process includes the compilation of the Core Directory of Common Data Sets and the Indicator and Data Releases for the PWMES. The Core Directory is a composite system that draws its data from contributory systems and provides mainstream and administrative data sources for the indicators. Drawing data from these sources, the PWMES indicators provide statistical baselines, benchmarks and trends to measure the results for the PSP, policy areas and statistical themes. The need for data quality is well documented including the most recent State of the Province Address (SOPA) (May 2009) by the Premier. In response to the quest to improve data quality and performance of routine information systems, the PWMES used the South African Statistical Quality Assessment Framework (SASQAF) tool to conduct data assessments on the M& E data within PGWC. The self-evaluation assessment by PGWC departments provided the platform to assess their M&E data against the eight (8) dimensions of quality. The data assessment process has delivered the following:

• A selection of common data sources for the outcome indicators which places the Province in a position to monitor and evaluate using the same data sources for key thematic focal areas

• A Core Directory of Common Data Sources Part 1: Mainstream data sources

• A Core Directory of Common Data Sources Part 2: Provincial administrative data

• A Core Directory of Common Data Sources Part 3: Research studies conducted in PGWC

• Data assessment reports of PGWC M&E systems

44..11..66 PPhhaassee 66:: IInnffoorrmmaattiioonn AArrcchhiitteeccttuurree

The use of Information Communication Technology (ICT) plays a vital role in advancing data compilation and the integration or harmonisation of the provincial administrative data, official statistics and other M&E data into an electronic M&E system. The Information Architecture is also of vital importance for the proper functioning of a RBM&E system. It is devised from a range of sources such as departments, external


sources (Stats SA) and civil organisations. This information forms part of the wider Business Intelligence (BI) for the Province. The Business Requirement Specification (BRS) for the PWMES is a conceptual design for the Information Architecture and provides a detailed outline of the technological requirements for a computer-based ICT system for provincial-wide M&E. The proposed system is based on promoting and managing the required data for the overarching objectives and strategic issues collected from various sources. The system will assist in analysing and generating M&E reports that measure progress on implementation and results-based levels. It will also receive different evaluation design reports and make these available to system users. Diagram 4 shows the data integration related to the provincial-wide M&E electronic system with the e-platform Dashboard. The Source Systems are all of the 12 departments, including Local Government, feeding data into the e-platform Dashboard, which in turn feeds relevant data into the provincial-wide M&E. Since the systems used by the Source Systems might be different, there is a need for a Data Interchange to convert data from sources and the e-platform Dashboard into the provincial-wide M&E electronic system. External interfaces such as the National Government, Stats SA and other agencies will also feed data and information into the system. The information relevant to the PWMES will then be stored on the M&E Repository to form part of the evidence base: Provincial M&E body of knowledge repository.


DDDDDDDDiiiiiiiiaaaaaaaaggggggggrrrrrrrraaaaaaaammmmmmmm 44444444:::::::: DDDDDDDDAAAAAAAATTTTTTTTAAAAAAAA IIIIIIIINNNNNNNNTTTTTTTTEEEEEEEEGGGGGGGGRRRRRRRRAAAAAAAATTTTTTTTIIIIIIIIOOOOOOOONNNNNNNN RRRRRRRREEEEEEEELLLLLLLLAAAAAAAATTTTTTTTEEEEEEEEDDDDDDDD TTTTTTTTOOOOOOOO TTTTTTTTHHHHHHHHEEEEEEEE PPPPPPPPRRRRRRRROOOOOOOOVVVVVVVVIIIIIIIINNNNNNNNCCCCCCCCIIIIIIIIAAAAAAAALLLLLLLL--------WWWWWWWWIIIIIIIIDDDDDDDDEEEEEEEE AAAAAAAANNNNNNNNDDDDDDDD MMMMMMMM&&&&&&&&EEEEEEEE EEEEEEEELLLLLLLLEEEEEEEECCCCCCCCTTTTTTTTRRRRRRRROOOOOOOONNNNNNNNIIIIIIIICCCCCCCC SSSSSSSSYYYYYYYYSSSSSSSSTTTTTTTTEEEEEEEEMMMMMMMM

44..11..77 PPhhaassee 77:: PPWWMMEESS –– PPllaannnniinngg ttoo IImmpplleemmeenntt aanndd SSuussttaaiinn tthhee PPWWMMEESS

The Provincial-wide M&E Strategy provides the basis for the roll-out of the first phase of an integrated M&E planning and reporting framework. This framework serves as a guide to ensure that the processes completed to date will act as the first phase during which the information architecture of the PWMES will be built. To sustain the PWMES, the elements of the system are to be updated and reviewed annually. The facilitation of the implementation of the elements of the PWMES ensures a sustainable and effective indicator system.

55.. CCOOMMMMOONN SSTTAANNDDAARRDDSS AANNDD CCRRIITTEERRIIAA FFOORR TTHHEE PPWWMMEESS

55..11 OOvveerrvviieeww ooff CCoommmmoonn SSttaannddaarrddss aanndd CCrriitteerriiaa In 1994, at the time of changeover from apartheid to a democratic state, there was very little statistical information to effect the desired transformation. There was an absence of reliable statistical information for tracking the performance of development programmes, projects and policies. There was also fragmentation of data and information to inform planning and routine decision-making; and to evaluate policies. Today, the state of national statistics is still characterised by:

• An information gap in terms of relevant statistics, data and information to meet the needs of users

• A quality gap in terms of guidelines, common standards and criteria

• A capacity-building gap in terms of both human resources and infrastructure.

Data Interchange

Source Systems: Departmental M and E Systems

M and E Information

Systems

M and E Repository [Indicators and Data]

External interfaces: National Departments; Stats SA; Universities, Research Institutions; etc.

Internal interfaces: Dashboard, OPMIS, PERMIS, etc.


The limitations that were emerging because of the absence and fragmentation of data and information prompted the Government to initiate practices that were geared towards establishing the required guidelines, principles and standards for data collection and collation and to establish practices that would assist with integration and coherence in monitoring and evaluation. .

55..22 GGuuiiddeelliinneess aanndd PPrriinncciipplleess The draft national guidelines, guiding principles and standards for monitoring and evaluation of public policies and programmes in South Africa in terms of the Constitution set out the national M&E principles and practices, including minimum norms and standards for monitoring and evaluating government policies, programmes and projects. It also focuses on procedures to be adopted by national, provincial and local governments in the M&E of the implementation of government policies, programmes and projects. The GWMES advocates principles to ensure that M&E contributes to good governance, is rights-based and development- and utility-oriented. Further, M&E should be methodologically sound, be operationally effective and undertaken ethically and with integrity. The PWMES is premised on the guidelines and principles of the Constitution and the GWMES to achieve development and utility-orientated M&E by measuring the progress of the developmental agenda for the PGWC. The PWMES would apply the minimum norms and standards to be operationally effective in tracking the progress using the RBM and E approach. The PWMES would apply the guiding principles of M&E to ensure an integrated, coherent and methodologically sound management of data and information for the PSP, provincial policy areas and the MDGs. The PWMES would apply the evaluation guidelines according to the scope of the evaluation exercise but ensures that the evaluation purpose always remains in the interest of the provincial development agenda. The PWMES would integrate these M&E principles into the Government’s planning cycle to improve governance.

55..33 SSttaannddaarrddss aanndd CCoonncceeppttss The Framework for Managing Programme Performance Information, 2007, from the National Treasury outlines key concepts of the design and implementation of management systems to define, collect, report and use performance-management information and M&E data to achieve RBM&E. The framework stipulates that key performance indicators to measure performance are input, activities, output, outcomes and impacts. It also stipulates that the criteria for good performance indicators are that performance indicators should be reliable, well-defined, verifiable, cost-effective, appropriate and relevant. The level of performance is specified in terms of suitable performance targets and standards relative to current baselines. This approach to planning performance information is called the “logic model”. The relationship of managing performance information to the RBM&E approach of the PWMES is that the latter focuses its function on managing data towards achieving results. The PWMES requires the availability of the performance information of PGWC to pursue RBM & E approaches. There is an interface between the performance information that uses output indicators to measure progress towards the direct outcomes and the RBM&E approach that uses outcome indicators to measure outcomes at intermediate and results-based levels. Within this context, Diagram 5 below illustrates the relationship between the role of performance information in planning, budgeting, reporting and the management of results.


DDDDDDDDiiiiiiiiaaaaaaaaggggggggrrrrrrrraaaaaaaammmmmmmm 55555555:::::::: RRRRRRRREEEEEEEELLLLLLLLAAAAAAAATTTTTTTTIIIIIIIIOOOOOOOONNNNNNNNSSSSSSSSHHHHHHHHIIIIIIIIPPPPPPPP BBBBBBBBEEEEEEEETTTTTTTTWWWWWWWWEEEEEEEEEEEEEEEENNNNNNNN PPPPPPPPEEEEEEEERRRRRRRRFFFFFFFFOOOOOOOORRRRRRRRMMMMMMMMAAAAAAAANNNNNNNNCCCCCCCCEEEEEEEE MMMMMMMMAAAAAAAANNNNNNNNAAAAAAAAGGGGGGGGEEEEEEEEMMMMMMMMEEEEEEEENNNNNNNNTTTTTTTT AAAAAAAANNNNNNNNDDDDDDDD RRRRRRRRBBBBBBBBMMMMMMMM AAAAAAAANNNNNNNNDDDDDDDD EEEEEEEE

Outputs

Activities

Inputs

OutcomesResults

Intermediate

Direct

ImpactsIndicators

Indicators

Indicators

Indicators

Indicators

Results-

based

Monitoring

and

Evaluation

Manage

towards

achieving

results

Equity

Effectiveness

Efficiency

Economy

Planning

Budget

Implement

Implementati

on

Monitoring

and

Evaluation

55..44 CCrriitteerriiaa aanndd QQuuaalliittyy DDiimmeennssiioonnss Stats SA uses its mandate for statistical leadership to implement the NSS to address the gaps in data and as part of the standardisation process, to put a framework for certification, namely SASQAF, in place. In the context of the above, the PWMES ensures that the data produced for M&E is reliable through building quality standards into the provincial administrative data records. In Diagram 5 below, it shows these quality standards, which are in accordance with the prerequisites and the eight dimensions of quality as defined in SASQAF. These dimensions of quality have been identified within the concept of “fitness for use” and they are described as follow:

55..44..11 RReelleevvaannccee The relevance of the M&E data and information reflects the degree to which it meets the real needs of the PSP, the policy areas, statistical themes and the MDGs. It is concerned with whether the available data and information shed light on the provincial policy areas.

55..44..22 AAccccuurraaccyy The accuracy of the M& E data and information is the degree to which the output correctly describes the phenomena it was designed to measure. Data sources within the Core Directory of Common Data Sources should provide an adequate basis to compile data for the PWMES indicators.


55..44..33 TTiimmeelliinneessss Timeliness of the M& E data and information refers to the delay between the reference point to which the information pertains and the date on which the information becomes available. It also considers the punctuality of release. The timeliness of reporting information as pertaining to the government planning cycle will influence its relevance.

55..44..44 AAcccceessssiibbiilliittyy The accessibility of M&E data and information refers to the ease with which it can be obtained from the data producer. This also includes the ease with which the existence of information can be ascertained from departments, as well as the suitability of the format or medium through which the information can be accessed.

55..44..55 IInntteerrpprreettaabbiilliittyy The interpretability of M&E data and information refers to the ease with which the users understand the M&E data through the provision of supporting information, concepts used, methodology of monitoring, evaluation designs, data collection and processing, and indicators or measures of the accuracy of the statistical information.

55..44..66 CCoohheerreennccee The coherence of the M& E data reflects the degree to which it can be successfully brought together with other statistical information within a broad analytical framework and over a period of time. The use of standard concepts, classifications and target populations promotes coherence, as does the use of common methodology across surveys and provincial administrative data records.

55..44..77 MMeetthhooddoollooggiiccaall ssoouunnddnneessss Methodological soundness refers to the application of international, national, or peer-agreed standards, guidelines, and practices to produce the M & E data and information.

55..44..88 IInntteeggrriittyy The integrity of the M&E data refers to the values and related practices that maintain users’ confidence in the data producer and ultimately in the data source.


DDDDDDDDiiiiiiiiaaaaaaaaggggggggrrrrrrrraaaaaaaammmmmmmm 66666666:::::::: CCCCCCCCRRRRRRRRIIIIIIIITTTTTTTTEEEEEEEERRRRRRRRIIIIIIIIAAAAAAAA AAAAAAAANNNNNNNNDDDDDDDD QQQQQQQQUUUUUUUUAAAAAAAALLLLLLLLIIIIIIIITTTTTTTTYYYYYYYY DDDDDDDDIIIIIIIIMMMMMMMMEEEEEEEENNNNNNNNSSSSSSSSIIIIIIIIOOOOOOOONNNNNNNNSSSSSSSS


DDDDDDDD........ IIIIIIIINNNNNNNNSSSSSSSSTTTTTTTTIIIIIIIITTTTTTTTUUUUUUUUTTTTTTTTIIIIIIIIOOOOOOOONNNNNNNNAAAAAAAALLLLLLLLIIIIIIIISSSSSSSSAAAAAAAATTTTTTTTIIIIIIIIOOOOOOOONNNNNNNN OOOOOOOOFFFFFFFF TTTTTTTTHHHHHHHHEEEEEEEE PPPPPPPPWWWWWWWWMMMMMMMMEEEEEEEESSSSSSSS

66.. PPWWMMEESS RREEPPOORRTTIINNGG FFRRAAMMEEWWOORRKK

66..11 MM aanndd EE wwiitthhiinn tthhee GGoovveerrnnmmeenntt PPllaannnniinngg FFrraammeewwoorrkk

The Government Planning Framework defines the cycle of strategising, programme development, budgeting, monitoring, evaluation and public communication. This cycle represents a continuous process of planning, implementation and review and is characterised by a calendar of events within the different phases of the cycle. The cycle is time bound and pronounces appropriate M&E methods to coordinate and integrate the Government’s responses to its performance. The framework, which is a multi-year one covering the Medium Term Strategic Framework [MTSF] which is linked to the Medium Term Expenditure Framework [MTEF], would be used as the reference point to which the M&E data and information pertains and becomes available. The timeliness of producing M&E data and generating reporting information will influence its relevance to the specific products required within the different phases. In order to align the PWMES to the planning and budgetary cycles of government, it is essential that reliable M&E data and evaluation findings feed timeously into the e-platform Dashboard and before entering the Information Architecture of the PWMES. The tracking of performance against the compendium of indicators for the PSP, policy areas and statistical themes are to be proved as reliable data and statistical evidence to support the decision-making process of the provincial strategic agenda for each cycle. At any point in time historical M& E data and information would be required to inform strategic planning processes for the period 2010 to 2014. The annual planning, budgeting and reporting cycle of the Annual Performance Plans [APPs] and hence the Annual Plan of the PGWC, should be used as a tool to assess the gaps in relation to the desired outcomes of the PSP. This assessment should influence the annual review process of the PSP towards achieving the provincial policy areas. The Planning Framework thus provides the base for conducting M&E in an integrated and coherent way. The e-platform Dashboard and web-based Information Architecture of the PWMES would support the accessibility of data and information produced during the monitoring, evaluation and review cycle. The data collection, data interpretation and data analysis process of the monitoring cycle would feed into the web-based Information Architecture in a timeous manner to ensure that relevant M&E data is accessible for the specific requirements of the planning cycle. The findings of the evaluation assessments conducted during the evaluation cycle and the integrative reports produced during the reporting cycle would also feed into the web-based Information Architecture. Diagram 7 below illustrates the monitoring, evaluation, review and reporting cycle within the government’s planning cycle and its calendar of events. It further illustrates the human resources required to process information using the Web-Based System, i.e. the RBM& E Portal, to produce documents and deliver reports at any given time as required within the MTEF

23


DDDDDDDDiiiiiiiiaaaaaaaaggggggggrrrrrrrraaaaaaaammmmmmmm 77777777:::::::: MMMMMMMMOOOOOOOONNNNNNNNIIIIIIIITTTTTTTTOOOOOOOORRRRRRRRIIIIIIIINNNNNNNNGGGGGGGG AAAAAAAANNNNNNNNDDDDDDDD RRRRRRRREEEEEEEEPPPPPPPPOOOOOOOORRRRRRRRTTTTTTTTIIIIIIIINNNNNNNNGGGGGGGG FFFFFFFFRRRRRRRRAAAAAAAAMMMMMMMMEEEEEEEEWWWWWWWWOOOOOOOORRRRRRRRKKKKKKKK

Government Planning

Cycle

Review and Reporting Cycle

Evaluation Cycle

Monitoring Cycle

Calendar of Events

Web Based System RBM and E Portal:

Portal and Online Services to support Knowledge Workers.

Tool with Content and Document Management Capability.

Event Driven Transactions and Workflow for Results Based

Monitoring and Evaluation within Government Planning Cycle.

View Event

Attach Documents

Compose Report

Search Compendium

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Five Year Electoral Cycle | Five Year Medium-Term Strategic Framework (MTSF)

Three Year Medium-Term Expenditure Framework (MTEF)

Annual Budgetary Cycle | Annual Parliament Cycle

Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun

National Government

Provincial Government

Local GovernmentAcross 3 Spheres of Government

Human Organisation Process and Information Documents and Work ProductsInformation Systems


66..22 RRoollee ooff tthhee CClluusstteerrss aanndd IInntteerr--GGoovveerrnnmmeennttaall RReellaattiioonnss

In order to streamline planning processes and to bring the administrative and political planning environments closer together within the framework of Inter-governmental Relations (IGR), the Cluster System presently caters primarily for Cabinet Committees where the Ministers and their departmental Heads of Office are clustered to plan collectively according to sectors. Within this system the purpose, functions, lines of communication and inter-relationships of every stakeholder are defined within a clear institutional, IGR and methodological framework. This approach provides for expedited decision-making, more efficient and effective implementation. In order to ensure operationalisation within the programme-management context, the Cabinet resolved that the Cluster System be viewed as the focal point to operationalising the PSP and provincial strategies. The consequence of this is that all transversal interventions defined within the Programme context, be filtered through the Cabinet System as custodians of provincial transversal programme interventions. Within this context, the PWMES is a mechanism for assembling M&E data for the indicators and reporting information to the Clusters on the performance of programmes embedded within the PGWC. Diagram 8 illustrates the institutional flow of M& E data and information from provincial departments, consolidated in a cluster level through the M & E mechanism to report to administrative and political stakeholders. The M&E data collated through various monitoring methodologies and evaluated through various evaluation designs is used to review the PSP and it is then reported against to inform strategic planning processes. This in turn influences the National and Provincial Programme of Action and hence the policy environment and the annual Provincial-wide M&E Plans for 2009/10-2014 of this Province.

25


DDDDDDDDiiiiiiiiaaaaaaaaggggggggrrrrrrrraaaaaaaammmmmmmm 88888888:::::::: PPPPPPPPWWWWWWWWMMMMMMMMEEEEEEEESSSSSSSS AAAAAAAANNNNNNNNDDDDDDDD TTTTTTTTHHHHHHHHEEEEEEEE IIIIIIIINNNNNNNNSSSSSSSSTTTTTTTTIIIIIIIITTTTTTTTUUUUUUUUTTTTTTTTIIIIIIIIOOOOOOOONNNNNNNNSSSSSSSS OOOOOOOOFFFFFFFF TTTTTTTTHHHHHHHHEEEEEEEE PPPPPPPPGGGGGGGGWWWWWWWWCCCCCCCC

Cluster level

Departments

Line Departments

Results based M and E:

Projects

Policy

Development

iKapa GDS

Policy

Implementation Support

Monitoring,

Evaluation and Review

Communication

DGPTM

M and E of projects

Output and outcome

indicators

Implementation M and E

Political

Stakeholders


77.. CCOONNCCEEPPTTUUAALL AAPPPPLLIICCAATTIIOONN OOFF TTHHEE MMOONNIITTOORRIINNGG AANNDD EEVVAALLUUAATTIIOONN FFRRAAMMEEWWOORRKK

77..11 AApppplliiccaattiioonn ooff MMoonniittoorriinngg aanndd EEvvaalluuaattiioonn FFrraammeewwoorrkk The Department of the Premier will produce an independent annual review of the PSP and provincial strategies that will indicate how successfully the outcomes are being implemented in relation to:

• The impact on the regional spatial, economic, environmental and social development trajectory

• Progress being made to achieve the strategic objectives and the strategic issues institutionally

• Identified areas where further action is needed. Diagram 9, The Provincial-wide M&E Framework sets the parameters and identifies the links between (1) Measuring Performance, (2) Measuring Impact, (3) Overarching Policy Frameworks, (4) PSP (5) Indicators and Annual Targets. DDDDDDDDiiiiiiiiaaaaaaaaggggggggrrrrrrrraaaaaaaammmmmmmm 99999999:::::::: PPPPPPPPRRRRRRRROOOOOOOOVVVVVVVVIIIIIIIINNNNNNNNCCCCCCCCIIIIIIIIAAAAAAAALLLLLLLL--------WWWWWWWWIIIIIIIIDDDDDDDDEEEEEEEE MMMMMMMM&&&&&&&&EEEEEEEE FFFFFFFFRRRRRRRRAAAAAAAAMMMMMMMMEEEEEEEEWWWWWWWWOOOOOOOORRRRRRRRKKKKKKKK

(3)

Overarching Policy

Frameworks

(4)

•PSP goals and objectives

•MDGs

(1) Measuring Performance

• Ensure that targets are achieved

• Provide regular information on progress

• Ensure continuous review and interventions

(2) Measuring Impact

• Ascertain the extent to which strategic objectives have been achieved

• Review long term policy interventions and make informed policy choices

(5)

Annual Targets

Indicators


REFERENCE

1. Kusek, J.Z. & Rist, R.C. 2004. Ten Steps to a Results-based Monitoring and Evaluation System.

Washington.

2. The Constitution of Republic of South Africa 1996, Act 108 of 1996. The Draft National Guiding

Principles and Standards for Monitoring and Evaluation of Public Policies and Programmes in South

Africa. http://www.info.gov.za/documents/index.htm. 2008.

3. The Department of the Premier: Chief Directorate: MER: Current reality of M and E in the Western

Cape Province. 2005.

4. The Department of the Premier. 2007. Conceptual Framework 1st draft. Western Cape.

5. The Department of the Premier. 2007. Indicator Development Framework. Western Cape.

6. The Department of the Premier. 2007. The iKapa Growth and Development Strategy. Western Cape.

7. The Department of the Premier: Chief Directorate: MER: Provincial M and E Audit, Phase Two. 2007.

8. The Department of the Premier: CD: MER: Data Assessment Report. 2009.

9. The Presidency of the Republic of South Africa. 2005. Proposal and Implementation Plan for a

Government-wide Monitoring and Evaluation System. The Presidency Policy Unit, Pretoria.

10. National Treasury. 2007. Framework for Managing Programme Performance Information. Pretoria.

11. Statistics South Africa. 1999. Statistics Act (Act 6 of 1999).

12. Statistics South Africa. 2005. Strategic Plan 2005-2008. Pretoria.

13. Statistics South Africa. 2006. South African Statistical Quality Assessment Framework. Pretoria.

14. Statistics South Africa. 2007. Annual Performance Plan 2007/2008. Pretoria.

15. Statistics South Africa. 2007. The Memorandum of Understanding on Partnership in the National

Statistics System between Statistics South Africa and the Department of the Premier of the Western

Cape.


ISBN: 978-0621-39153-4