Top Banner
GGGI Peru Country Program Evaluation Approach Paper Impact and Evaluation Unit – April 2018
37

Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

Jul 22, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

GGGI Peru Country Program

Evaluation Approach Paper

Impact and Evaluation Unit – April 2018

Page 2: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

2

Page 3: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

3

Contents

1. Introduction ...................................................................................................................... 4

1.1 Purpose of this paper ............................................................................................................................................................... 4

1.2 About GGGI ................................................................................................................................................................................... 4

2. Country Program overview ............................................................................................. 6

3. Forestry ............................................................................................................................. 8

3.1 Opportunity or problem addressed .................................................................................................................................. 8

3.2 Relevant policies and partners ............................................................................................................................................ 8

3.3 Project description .................................................................................................................................................................... 9

4. Water ............................................................................................................................... 13

4.1 Opportunity or problem being addressed .................................................................................................................. 13

4.2 Relevant policies and partners .......................................................................................................................................... 13

4.3 Project description .................................................................................................................................................................. 14

5. Green growth.................................................................................................................. 17

5.1 Opportunity or problem being addressed .................................................................................................................. 17

5.2 Relevant policies and partners .......................................................................................................................................... 17

5.3 Project description .................................................................................................................................................................. 18

6. Public sector eco-efficiency .......................................................................................... 22

6.1 Opportunity or problem being addressed, and relevant policies and partners ...................................... 22

6.2 Project description .................................................................................................................................................................. 22

7. Evaluation design ........................................................................................................... 24

7.1 Intended audience and use of the evaluation ........................................................................................................... 24

7.2 Key evaluation questions and scope .............................................................................................................................. 24

7.3 Timelines and deliverables .................................................................................................................................................. 28

7.4 Management arrangements ............................................................................................................................................... 29

Annex 1 – Hierarchy of evaluation questions ....................................................................... 30

Annex 2 – Data collection methodology (draft).................................................................. 31

Annex 3 – Table of contents for evaluation report (draft) ................................................. 36

Page 4: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

4

1. Introduction

1.1 Purpose of this paper

The Impact and Evaluation Unit (IEU) was established to commission independent evaluations of GGGI’s work to

support accountability needs and inform program improvements. Under IEU’s Annual Evaluation Workplan for

2018, the Peru country program was selected for evaluation.

In line with GGGI’s Evaluation Policy, IEU has developed this Evaluation Approach Paper in close consultation

with key stakeholders. The main purpose of this document is to guide the independent evaluator in conducting

the evaluation of the Peru country program. To this end, the paper provides:

A description of the country program activities to be evaluated;

A description of the proposed evaluation approach, including:

- Key evaluations questions to be addressed;

- Evaluation approach, methodology and data sources; and

- Implementation arrangements and timeframes for the evaluation.

1.2 About GGGI

The Global Green Growth Institute (GGGI) is an intergovernmental organization established in 2012 to support

countries to pursue a model of economic development known as ‘green growth’, which aims to simultaneously

promote economic growth, social inclusion and environmental sustainability.

GGGI is overseen by an Assembly, comprising 28 member countries, a Council which serves as the executive

organ of GGGI and the Management and Program Sub-Committee (MPSC) which advises the Council on

programmatic and financial matters. GGGI is headquartered in Seoul, Republic of Korea, with operations in over

27 countries.

GGGI’s strategic priorities and directions are laid out in its Strategic Plan 2015-20, initially approved by the

Council in 2014 and most recently updated in 2017. This strategy outlines 6 strategic outcomes that the Institute

ultimately seeks to contribute to in member countries:

1. GHG Emission Reductions

2. Creation of Green Jobs

3. Increased Access to Sustainable Services

3.1. Increased Access to Clean Affordable Energy

3.2. Increased Access to Improved Sanitation

3.3. Increased Access to Sustainable Waste Management

Page 5: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

5

3.4. Increased Access to Sustainable Public Transport

4. Improved Air Quality

5. Adequate Supply of Ecosystem Services Ensured

6. Enhanced adaptation to climate change

To contribute to these strategic outcomes, GGGI supports its members to achieve the following intermediate

outcomes:

• Strengthen national, sub-national, local green growth policy planning, financing and institutional

frameworks

• Increase green investment flows

• Improve knowledge sharing and learning between countries on green growth

The Strategic Plan also prioritized 4 thematic areas in which GGGI’s work would focus:

• Sustainable Energy

• Water and Sanitation

• Sustainable Landscapes

• Green Cities

To implement the Strategic Plan, a biennial Work Program and Budget (WPB) is developed by GGGI and

approved by the Council every two years. The WPB outlines a portfolio of country and global programs, and

supporting corporate operations and reforms, to be implemented over a 2-year period. The first WPB period

covered 2015-16 and GGGI is currently implementing its second WPB during 2017-18.

Implementation of GGGI’s country and global programs is largely undertaken by two divisions who work in a

complementary and integrated way:

• Green Growth Planning and Implementation division (GGPI), which leads the development and

implementation of GGGI’s in-country green growth programs; and

• Investment and Policy Solutions division (IPSD), which designs and delivers specialist policy and financial

products and services to support the delivery of in-country programs.

In addition, GGGI’s Office of the Director-General (ODG) and the Operations Enabling Division (OED) manage a

range of functions and initiatives in non-programmatic and corporate areas.1

For more information on GGGI, visit the website at: http://www.gggi.org

1 GGGI’s organizational chart is available at: http://gggi.org/about/people

Page 6: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

6

2. Country Program overview

GGGI has been supporting Peru since 2013, making it the Peru country program one of the longest-running

programs in the Institute. Since this time, GGGI’s relationship with Peru has steadily broadened and deepened.

Formal MOUs have been established with 5 ministries and GGGI is in the process of putting a Host Country

Agreement in place to formalize its legal status within the country. Peru formally joined as a member of GGGI in

October 2016.

Since 2013, the country program has comprised projects in the following areas:

Program Areas2 Period Funding source

Forestry 2013-2015 Earmarked – BMU

Water 2013-2016 Earmarked – SDC

Green growth 2016-present Core

Public sector eco-efficiency 2015-present Core

Further details on each of these areas is provided in Sections 3-6 of this paper.

Between 2013-18, the Peru Country Program has increasingly evolved and matured. This process can be briefly

summarized in 3 stages:

2013-15: With the program mainly comprised of earmarked funded projects, GGGI’s focus was driven by

delivering on agreed commitments in its funding agreements with donors. There was less focus on

expansion of the country program and integration with the broader organization during this period.

2015-16: A shift in program funding from earmarked to core introduced greater programming flexibility and

stimulated a wider search for programming opportunities. During this period, the team in Peru have sought

to engage a broader range of counterparts and topics, strengthen integration with HQ and other country

programs3, and invest more in building GGGI’s brand in country.

2017-Present: GGGI has begun to focus its programming on a more focused set of issues, based on

identified government priorities and resource mobilization potential. This focusing has coincided with the

development of a Country Planning Framework (CPF), scheduled for release in 2018.

2 Some of these core-funded projects have used different titles over time, since GGGI plans and budgets programs on 2-

yearly cycles. However, for this evaluation they are treated as different stages of the same project, with the same or related

outcomes.

3 As per key directions outlined in GGGI’s Strategic Plan 2015-20, initially released in late 2014 and ‘refreshed’ in late 2017.

Page 7: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

7

Moving forward, the CPF will guide GGGI’s programming in Peru. At the time of writing, the draft CPF – which

was developed in close consultation with the Government of Peru – proposes to focus GGGI on contributing to 3

key outcomes between 2017-21:

Outcome 1: Contribute to Peru’s NDC target of reducing GHG emissions by 30% by 2030 compared to

BAU levels and achieving zero net deforestation by 2020.

Outcome 2: Enhancing water security through greater investment in green infrastructure, particularly in

the water-scarce coastal regions of Peru.

Outcome 3: Increase energy efficiency and diversification of energy supply, to ensure economic

development is more resilient to potential climate change impacts.

At present, the Peru country team comprised 3 staff - a Country Representative, supported by a Senior Green

Growth Officer and a Public Policy Specialist. The Country Representative reports to the Seoul-based Head of

Program for Latin America, Caribbean and the Middle East, within the GGPI division. In addition, staff from other

parts of GGGI4 have also supported delivery of specific program activities at various points.

4 Mainly the Investment and Policy Solutions Division (IPSD) and Office of Thought Leadership (TL), and their former

incarnations.

Page 8: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

8

3. Forestry

3.1 Opportunity or problem addressed

Forests cover more than half of Peru’s land area. Most of this is Amazonian rainforest, whose ecosystem services

are of significant economic, social, cultural and environmental importance. Peru is ranked among the 17 most

biodiverse countries in the world. Between 2003-12, forested area declined by almost 2%, mostly in Amazonian

regions.

Deforestation and land use changes contributed to around 50% of Peru’s GHG emissions in 2012, up 60% over

the previous decade. Although Peru’s contribution to global GHG emissions is low (0.34% of global emissions in

2012), the country is highly vulnerable to the effects of climate change.

Around 95% of deforestation is driven by small holders clearing land for farming and livestock. Though

agriculture made up only 7% of GDP in 2013, it accounted for 65% of the economically active population. The

sector thus plays a significant role in social and economic inclusion, particularly in terms of the livelihoods of

rural and indigenous communities.

Forestry contributes 1-4% of Peru’s GDP but has potential to contribute much more. To illustrate, Peru has 4 times

more forest area than Chile, but official exports are worth $150m per annum compared to $6b for Chile. Peru is a

net importer of forest products, due to low levels of industrialization and value added. In addition, Peru suffers

economic losses from illegal timber extraction estimated at around $200m per year.

3.2 Relevant policies and partners

Given the above, tackling deforestation and moving to more sustainable forestry management practices

presents a key green growth challenge and opportunity for Peru. This is well recognized by the Government,

who have responded in a several key ways.

In 2009, at COP15 in Copenhagen, Peru pledged to reduce net tropical deforestation to zero by 2020. To this

end, it has developed a National Program of Forest Conservation for Mitigating Climate Change and signed a

Joint Declaration of Intent to work with Norway and Germany on this issue. Leading up to COP21 in Paris, 2015,

Peru’s INDC further committed to reduce GHG emissions by 30% from base levels by 2030, with the forestry

sector expected to contribute 60% of this reduction.

Domestically, a new Forestry and Wildlife Law was passed in 2011 and took effect in 2015 when its associated

regulations were passed. Though originated by a need to comply with aspects of the US-Peru Free Trade

Agreement of 2009, it nevertheless provided a critical opportunity for further reform of the forestry sector in

Peru.

The law updates and clarifies the rights and responsibilities for access to, and use of, Peru’s forests and wildlife,

in accordance with procedures set by national/regional authorities and key planning/management instruments.

Page 9: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

9

Whilst substantively similar to the previous 2001 forestry law with respect to management of logging

concessions, the new 2011 law does include stronger recognition and provisions for:

• Consultation and social inclusion, particularly for indigenous peoples;

• More sustainable, ecosystem approach to forest management that acknowledges the value of

environmental goods/services provided;

• Recognition of small holders and their livelihoods, and pathways to formalization; and

• Reinforcing government decentralization in Peru, with more authority given to subnational governments

for awarding of rights and supervision of forest management compliance.

Institutionally, the 2011 law and 2015 regulations established a new forestry institution, the National Forestry

and Wildlife Service (SERFOR). Formerly a directorate within the Ministry of Agriculture (MINAGRI), SERFOR is

now an autonomous entity with higher level responsibilities to lead the realization of sustainable and

participatory management of Peru’s forestry and wildlife resources.

SERFOR (and its predecessor within MINAGRI) has been GGGI’s main counterpart in this project. GGGI has also

worked closely with others including the Ministries of Environment, Agriculture and Irrigation, Economy and

Finance, Production, and Foreign Trade and Tourism.

3.3 Project description

Following consultations with the Government of Peru in 2012, GGGI was invited to support SERFOR with

development the National Forestry & Wildlife Plan (NFWP). Mandated by the 2011 law, the NFWP aims to help

operationalize the law and regulations, by guiding and aligning priorities, planning and budgeting efforts

amongst relevant institutions across multiple sectors. SERFOR is responsible for leading the development and

implementation of the NFWP.

The project was funded by BMUB5 and implemented by GGGI from 2013-2016. The total budget for the project

was approximately $1,293,000, of which $1,268,160 was spent. A simplified theory of change6 for the program is

summarized as follows.

In terms of impact, the project sought to contribute to the GoP’s policy goals relating to halting deforestation

and reducing GHG emissions, as well as improving the economic value of the sustainable forest-related activities

and the welfare of communities living in forested areas.

The project sought to achieve two main outcomes in the medium-term to contribute to the desired longer-term

impacts.

5 This project was one of several components in a larger multi-country program funded by BMUB.

6 This theory of change was not contained in the original project design document but has been developed retrospectively

based on the project logframe, results reporting and discussions with the GGGI Peru team.

Page 10: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

10

Figure 1: Theory of Change for the Forestry Project

Outcome 1: Ensure the design of the NFWP embedded objectives, principles and approaches consistent

with the paradigm of green growth.

At the time of writing, the NFWP expected to be finalized and released in 2018. Timelines for the preparation

and adoption of the NFWP were delayed by slower-than-expected passage of the Forestry and Wildlife

regulations, which were adopted in 2015 (versus the original target of 2012), and the establishment of SERFOR

as an autonomous agency in 2015.

Outcome 2: Ensure effective implementation of the NFWP, particularly on selected priority issues

including smallholder land tenure and payment for ecosystem services.

To realize these outcomes, the project sought to deliver 4 key outputs:7

Output 1: Technical green growth inputs to NFWP (completed in 2015)

GGGI delivered various technical inputs that collectively sought to ensure the NFWP realized the full economic

potential of forest assets (beyond timber production), whilst doing so in an environmentally sustainable and

7 The descriptions below reflect what was delivered, which in some cases varied from the original funding agreement in

response to government requests and changed circumstances.

Page 11: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

11

socially inclusive way. In doing so, this output also aimed to ensure the NFWP was informed by a rigorous

evidence base. These inputs delivered by GGGI included:

- Proposed green growth objectives for the NFWP

- Assessment of the actual and potential contribution of forests to economic growth, jobs and social

inclusion

- Analysis of existing value chains for timber and non-timber products and opportunities for improvement

- Policy options to address the role of smallholders in deforestation and sustainable growth of the forest

economy

- Estimated economic value of environmental services provided by forests, and possible instruments to

realize this value (e: payment for ecosystem services)

- Analysis of social inclusion issues relating to forest populations

- Analysis of ecotourism as a component of the forest economy

TORs for each input were developed collaboratively with SERFOR. In addition, GGGI collaborated with a range of

partners in developing these inputs, including the German Development Institute (DIE) – the project’s

implementing partner, the World Centre for Research on Agroforestry (ICRAF) and The Nature Conservancy.

Output 2: Implementation plans for selected priorities in the NFWP (completed in 2015)

At the request of SERFOR, GGGI undertook further work to scope possible approaches to implementing selected

priority issues under the NFWP, namely:

- Formalization of land tenure for smallholder farmers

- Establishment of a national payment for ecosystem services fund.

The first product was prepared by GGGI in collaboration with ICRAF and the second was developed internally.

Output 3: Institutional arrangements for NFWP (completed in 2015)

SERFOR is responsible for preparing the NFWP but does so in consultation with other ministries via an

Intergovernmental Group.

GGGI provided inputs on proposed institutional arrangements for developing and implementing the NFWP,

including the structure, roles and procedures for such an arrangement.

The Institute also supported SERFOR in the planning the development of the NFWP. This included inputs on a

proposed structure for the NFWP document, planning of steps and timelines, and convening and facilitating

meetings of the Intergovernmental Group.

Page 12: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

12

Output 4: Knowledge sharing to inform the NFWP (completed by 2015)

GGGI conducted the following events to help inform SERFOR’s development of the NFWP:

- Training course for SERFOR staff on policy making for ecosystem services, jointly delivered with The Nature

Conservancy

- Workshop to share international experiences in payment for ecosystem services, delivered in Costa Rica

with participation of officials from Peru, Colombia, Costa Rica, Guyana, Republic of Korea, Mexico,

Myanmar and Nepal

- Workshop between Colombian and Peruvian officials to exchange experiences on payment for ecosystem

services, in collaboration with donors

- Workshop with members of the NFWP Intergovernmental Group on the applicability of green growth to

the NFWP

- Roundtable on tropical forests and green growth at COP20 in Lima, with participation from representatives

of Peru, Colombia, the Inter-American Development Bank and BMUB

While this project officially ended in 2015 when GGGI’s funding agreement with BMUB concluded, the Institute

has continued to support the GoP to progress towards the outcomes and impacts above as part of the National

Green Growth Strategy project - see Section 5 for further details.

Page 13: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

13

4. Water

4.1 Opportunity or problem being addressed

Although Peru is a water-abundant country overall, water is very unevenly distributed. Around 98% of available

surface water is in the Amazonian watershed. Only 1.5% is in the Pacific watershed, where 63% of the population

lives and substantial export-oriented agriculture (which makes up 87% of water consumption) also takes place.

Consequently, the water balance in the Pacific watershed is in deficit and not currently on a sustainable

trajectory. Growing demand for water has been met by increasing groundwater extraction. Around 22% of water

supply came from groundwater in 2012, an increase of 21% between 2003-12. The risk of future water scarcity in

the Pacific watershed is thus of growing concern, a problem compounded by Peru’s high vulnerability to the

impacts of climate change.

Gaps remain in access to water-related services. On average 86% of the population had access to improved

water sources in 2013 and 75% had access to improved sanitation facilities. However, these figures mask urban-

rural disparities in access: 91% urban vs 67% rural access to improved water sources, and 81% urban vs 50%

rural access to improved sanitation as of 2013. In many locations, water supply is not yet reliable and around

40% of water supply went unbilled in 2008 due to leaks and losses.

4.2 Relevant policies and partners

In recognition of the above challenges, the government enacted a new Water Resources Law in 2009, with the

goal of ensuring sustainable and efficient management of water to meet the needs of all. The law introduced a

new system for water resource management in Peru, based on modern principles of integrated water resource

management (IWRM)8.

The National Water Authority (ANA), an autonomous national agency under the Ministry of Agriculture, was

created to regulate and oversight the implementation of the new system. Its functions include:

• Water resource planning at national and basin levels (the latter via watershed-level boards)

• Setting economic incentives to increase water use efficiency and decrease pollution

• Issuing and controlling water rights and discharge permits

• Setting and monitoring water quality standards and environmental flows, in coordination with the Ministry

of Environment

• Ensuring water resources are managed to facilitate adaptation to climate change

• ANA’s mandate does not include implementation of infrastructure projects.

8 Key principles of IWRM include: stakeholder participation; integration of sectoral policies; decentralized management at

basin levels; recognition of water as a social, economic and environmental good.

Page 14: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

14

To support the 2009 law, the ANA developed a National Water Resource Strategy and Policy (PERH), which

elaborates a high-level management framework of principles, guidelines, strategies and public instruments that

guide the interactions of key public and private actors in implementing IWRM.

In addition, ANA also developed a National Water Resources Plan 2015-2035 (PNRH) which was adopted via

Supreme Decree in 2015. This plan is more implementation-oriented and sets out the programming priorities,

costs, sources of financing, investment recovery criteria, responsible entities and other relevant information to

achieve the goals of the Water Resource Law and the PERH.

Among other things, the National Water Resources Plan calls for the following by 2035:

• Increasing the crop area under mechanized irrigation from 2% (current - 2015) to 25%

• Installation of water meters in homes

• Improving water distribution canals

• Reforesting upstream parts of watersheds to prevent over-sedimentation of reservoirs

• More than doubling the reuse of treated urban wastewater for irrigation

• Proper treatment of 99% of wastewater generated by the target population9

• Increasing by 50% inter-basin water transfers from the Amazonian to Pacific basin

To achieve these priorities, the National Water Resources Plan identified an investment need of USD45 billion

between 2015-35, which would need to be met through a combination of public and private financing sources.

4.3 Project description

Following a scoping mission conducted in 2012, GGGI identified an opportunity to support ANA to address the

investment need above and in doing so address a critical green growth-related priority in Peru’s water sector.

This project commenced in 2013 and concluded in 2016. It was supported by earmarked funding from the Swiss

Agency for Development and Cooperation (SDC). As part of its implementation agreement with SDC, GGGI

implemented this project jointly with the 2030 Water Resources Group. The total project budget was $400,000,

of which $393,857 was expended. The theory of change for this project is shown below.10

The eventual impact the project sought to contribute to aligns with GoP’s water policies, which is to ensure

sustainable and efficient management of Peru’s water resources that meets the needs of all by 2035.

To this end, the project aimed to achieve the following outcome:

9 Urban and rural populations of the Pacific basin, and urban populations of Amazonas & Titicaca basins.

10 This theory of change was not contained in the original project design document but has been developed retrospectively

based on the project logframe, results reporting and discussions with the GGGI Peru team.

Page 15: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

15

Outcome 1: To mobilize additional investments aligned to the goals of the NWRP.

As noted above, the NWRP identified a financing requirement of around USD 45 billion in achieving the goals of

the plan between 2015-35. The development, approval and financing of public projects is complicated by the

complex interactions between different public policy, investment planning and budgeting systems within Peru,

through which potential projects must navigate. Specifically, closing infrastructure gaps in the water sector

requires alignment between the National Water Resource Management System, the National Public Investment

System and the National Budget System.

Figure 2: Theory of change for the Water Project

To achieve these outcomes, the project aimed to deliver 2 key outputs.

Output 1: Priority projects for investment identified (completed in 2016)

This output was delivered by WRG 2030. It involved reviewing projects contained in Peru’s national public

investment system (SNIP, by its Spanish acronym) and developing a methodology to prioritize those that would

best deliver on the aims of the NWRP.

A multi-criteria analysis methodology was developed that combined hydro-economic analysis with an

assessment of the political, environmental, social impacts of potential projects in selected river basins along the

Pacific coast of Peru. Based on this methodology, an initial list of 2,303 projects was reduced to 204 projects. The

projects assessed were drawn from different sectors, including agriculture, irrigation, climate change and

household use. Collectively, these projects were estimated to require around PEN 22 billion in investment, whilst

reducing the water supply-demand gap by around 4.9 billion m3/year.11

Consultation workshops were conducted with key stakeholders to review and validate the analysis before the

output was finalized.

11 http://www.2030wrg.org/wp-content/uploads/2014/07/AMEC-2030WRG_Peru-Final-Report-Spanish_English.pdf

Page 16: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

16

Output 2: Implementation and funding plans for priority projects developed (completed in 2016)

Building on the above output, GGGI undertook a second stage of filtering to reduce the list of +200 projects

down to around 50 projects. For this, a simpler multi-criteria analysis methodology was developed that ANA

would be able to understand and utilize on their own. The analysis also added information about where each

project was in the SNIP stages of development, and the remaining cost and time required to complete the

designs for these shortlisted projects, to assist the ANA in advancing these projects to financing.

As with the above output, consultations with key stakeholders were also undertaken to refine the methodology

and the priority projects identified.

At ANA’s request, GGGI also undertook additional work to advise ANA on how private financing could be

mobilized for these projects, given the available public financing was insufficient. For this, a review of

international practice was undertaken on different types of financing mechanisms and how they are used to fund

a variety of water projects. The analysis also included a description of the risks and mitigation measures involved

with different financial instruments.

The output was finalized and submitted to ANA shortly after the election of a new government in mid-2016.

Page 17: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

17

5. Green growth

5.1 Opportunity or problem being addressed

Peru has enjoyed strong economic growth in recent times, driven by macroeconomic stability, open trade and

strong inflows of FDI. Between 2003-13, GDP grew at 6.4% pa, per capital income rose by 60%, net FDI rose by

over 600% and the poverty rate fell from 52% to 24%.12

However, much of this growth has been driven by exploitation of renewable and non-renewable natural

resources. In addition, widespread inequality and labor informality also pose challenges to the inclusiveness of

economic growth. The rural poverty rate in 2016 was 48% compared to 16% for urban.

The Peruvian Government has taken steps to strengthen its laws, policies and institutions with respect to

environmental protection and management and promote social equity.

Notable developments include the creation of the Ministry of Environment (MINAM) in 2008 in which

environmental responsibilities have been consolidated, along with a suite of environmental laws and specialized

agencies such as SERFOR and ANA. In addition, decentralization reforms have seen decision making

responsibilities transferred to subnational levels, closer to constituents, along with increasing opportunities for

civic engagement in policy processes.

Yet despite these positive advances, Peru has not yet succeeded in decoupling economic growth and

environmental degradation. The cost of environmental damage is estimated at some 4% of GDP between 2003-

13. Peru’s high vulnerability to climate change is likely to exacerbate this, with additional future economic losses

due to climate change predicted to be around 15% of GDP for the period 2010-2100.

5.2 Relevant policies and partners

Recognizing the need to pursue a more sustainable pathway of growth, and as part of its effort to seek

membership of the OECD, Peru has committed to adhere the OECD’s Declaration on Green Growth13 and

develop a green growth strategy. In 2014 at COP20, the Government formalized its intention to do just this with

MINAM presenting a roadmap to develop a National Green Growth Strategy (NGGS).

Additionally, as part of the OECD’s country program for Peru, the OECD conducted an Environmental

Performance Assessment in 2015, which resulted in 66 recommendations. One of these was the development of

a National Green Growth Strategy. Peru responded with an action plan to address the recommendations. In

2016, Peru passed a Supreme Decree which made the action plan legally binding, thereby creating a legal

requirement for Peru to develop a National Green Growth Strategy.

12 OECD Environmental Performance Review 2016

13 http://www.oecd.org/greengrowth/greengrowthincountriesandterritories.htm

Page 18: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

18

In seeking to develop this strategy, Peru is not starting from scratch. A range of policies already exist that align

with the concept of green growth, providing a solid foundation upon which to build. However, there remains in

general a lack of consistency and coherency between policies, strategies, plans and targets, with unresolved

tensions between investment promotion and environmental protection imperatives. This reflects institutional

fragmentation and lack of coordination, both vertically and horizontally, among relevant government

institutions.

Recognizing the need for a coordinated effort towards developing the NGGS, the government has established a

cross-ministerial Working Group. The group composed of the Ministry of Economy and Finance (MEF), and

includes strong collaboration with MINAM.

5.3 Project description

Following an initial request for support from MINAM in 2015, GGGI initiated a new project which commenced in

2016 and is presently ongoing until the end of 2018.14,15 The project is financed from core funds. A total of

$1,392,598 has been budgeted to date, and $768,746 had been expended up until the end of 2017.

A theory of change for the project is shown in Figure 3 below.16

The final impact that the project seeks to contribute to is for Peru to successfully transition to a green growth

pathway, as reflected by achievement of objectives and targets contained in the NGGS by 2030, as well as Peru’s

NDCs and SDGs which align to the NGGS.

To contribute to this impact, the project aims to support a series of linked outcomes. These are described below,

along with the associated GGGI outputs designed to help realize these outcomes.

Outcome 1: NGGS and implementation arrangements are adopted and implemented

As noted above, the GoP is seeking to adopt a NGGS as part of its domestic commitment to pursue green

growth and in line with international commitments made in seeking membership of the OECD. In addition to

developing the substantive content of the strategy, it is important for the NGGS’s implementation arrangements

to be well designed and clearly articulated, given the challenges of implementing complex policies such as this

that cut across multiple sectors and levels of government.

14 GGGI plans and budgets for core-funded projects in biennial (2-yearly) cycles. For this reason, GGGI’s internal

documentation/systems reflects this work as 2 projects, because the implementation period covered two biennial cycles

(2015-16 and 2017-18). However, for this evaluation, they are treated as different phases of the same project given

similarities in the logframes for each biennium.

15 Further work may be undertaken beyond 2018, but at the time of writing details were unknown as GGGI was still in the

process of preparing its Work Program and Budget for 2019-20.

16 This theory of change was not contained in the original project design document but has been developed retrospectively

based on the project logframe, results reporting and discussions with the GGGI Peru team.

Page 19: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

19

Figure 3: Theory of change for the NGGS Project

Assuming the NGGS is successfully adopted, GGGI would then transition to supporting the GoP to implement

selected high priority areas under the NGGS. (Two such areas have been discussed and agreed in principle with

the GoP, as discussed further under outcomes 2-5 below.) To achieve this outcome, the following outputs were

implemented in 2016.

Output 1: Technical inputs to develop the NGGS delivered (Completed in 2016)

Output 2: Technical inputs to implement the NGGS delivered (Completed in 2016)

Under Output 1, GGGI delivered a series of inputs to help MEF, MINAM and other participating ministries to

develop the NGGS. These included:

• Diagnostic exercise using GGGI’s Green Growth Potential Assessment (GGPA) tool, which helped identify

areas of relative ‘strengths and weaknesses’, along with suggested priority sectors and issues to focus

on.

• Logical framework for the including objectives, indicators and targets.

• Detailed analysis to identify existing activities in the 5-year and 3-year forward planning documents of

12 government agencies that aligned with the NGGS, and map these to the major themes of the

strategy.

• Standard analysis required of all new policies, including justification for the NGGS and diagnosis of the

Page 20: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

20

overall problem(s) to be addressed

• Support the drafting of the NGGS document.

• Workshop co-organized by GGGI and KOICA to share lessons with other countries involved in

developing green growth strategies.

To enhance the likelihood of the NGGS being implementable, GGGI also delivered the following under Output 2:

• 50 stakeholder interviews across 12 sectors to identify priority measures that the NGGS should focus on.

• Detailed preparation of costs, benefits and 2-year implementation plans for 12 priority measures for the

NGGS.

• Assessment of options for financing implementation of the NGGS, including from within public budgets,

international sources of financing and competitive fund options.

Outcomes 2 & 3: Priority actions to address land tenure are adopted and implemented, leading to

reduced deforestation and improved livelihoods in target areas.

One priority area the GoP has identified is the need for support to implement existing policies regarding

formalizing land ownership for smallholders in forested areas, which is if successful, should help reduce

deforestation and improve small holder livelihoods.

Lack of formal land ownership is closely linked to deforestation, as it encourages exploitative land use practices

(slash and burn) aimed at short-term gain while discouraging longer-term investments that boost productivity

and maintain natural capital. In 2012, around 1 million farmers (45% of total), 1000 indigenous communities and

800 rural communities were without land titles.

In addition to greater recognition being given to smallholder rights under Forestry and Wildlife Law of 2009,

OECD has also recommended17 in one of its pre-membership assessments that Peru strengthen institutions

responsible for land titles and speed up land titling process, particularly in indigenous territories, to promote

more sustainable approach to agriculture and forestry/biodiversity management.

To work towards this outcome, GGGI undertook the following outputs.

Output 3: NGGS action plan for land use priorities developed (completed in 2017)

One of the major tools for land use actions in the context of the NGGS is the implementation of the National

Forest and Wildlife Plan, led by SERFOR. As noted above, while BMUB financing ended in 2015, during 2017

GGGI worked closely with SERFOR to ensure that its inputs were directly incorporated into the design of the

NFWP. The NFWP draft was completed in 2017 and salient aspects of its implementation and approval will be

shared with civil society actors for consultation in the first half of 2018. At the time of writing, approval was

expected in June 2018.

17 OECD Environmental Performance Review

Page 21: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

21

Outcomes 4 & 5: Priority actions to apply payment for ecosystem service models are adopted and

implemented, leading to reduced deforestation and improved livelihoods in target areas.

A second priority area under the NGGS identified with the GoP is to operationalize the concept of payment for

ecosystem services (PES) in critical watersheds in the coastal region of Peru. By doing so, the assumption is that

the quality of water supply will be protected and/or enhanced, and thus contribute to improved water security in

these water-stressed regions.

Constraints in public finance means that funding for conservation of ecosystems that provide economically-

important services is generally insufficient. Payment for ecosystem services is recognized as a promising

instrument that can help address this financing gap. The government has enacted legislation and policies to

promote the use of PES18, but more needs to be done to operationalize the concept and evaluate its

effectiveness as a policy solution.

To contribute to this outcome, the project includes the following outputs.

Output 4: NGGS action plan for water management priorities developed (in progress)

GGGI is in the process of scoping this output in more detail. At the time of writing, an opportunity had been

identified to work with the National Superintendent for Sanitation Services (SUNASS by its Spanish acronym), the

national agency responsible for regulating water utilities in Peru. SUNASS currently oversees the use of tariffs

collected from water users in Lima for watershed conservation, which is expected to finance around USD 10m of

green infrastructure projects in the Rimac, Chillon and Lurin basins where Lima is situated. GGGI has established

an MOU with SUNASS to support them in developing arrangements for how best to invest these funds. This will

be done in collaboration with other key agencies, including ANA, MINAM and SEDAPAL19.

Other project outputs

Two other outputs were originally contained in the project logframe (for 2017-18). These include:

• Output 5: Water pricing analysis to support the development of the NGGS action plan for water

management. Following consultations with government stakeholders, this output has been discontinued due

to lack of demand.

• Output 6: Regional knowledge sharing on land use and water issues of mutual interest to regional

countries. Implementation of this output remained ongoing at time of writing.

18 For example, a law was enacted in 2013 on mechanisms for payment of ecosystem services.

19 The state-owned utility providing water and sewerage services to Lima and Callao.

Page 22: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

22

6. Public sector eco-efficiency

6.1 Opportunity or problem being addressed, and relevant policies and

partners

To continue to attract investment and sustain strong economic growth and improve Peru’s global

competitiveness (as measured by tools including the World Bank’s Doing Business Index), the GoP has been

undertaking reforms to improve the competitiveness of the Peruvian economy.

These flagship of these reforms is the multi-sectoral Competitiveness Agenda (2014-18), led by the National

Competitiveness and Formalization Council within the MEF. The reforms within the Agenda have been organized

according to several priority themes, including more efficient use of natural resources.

Within this theme, one specific action area is to improve the environmental performance (“eco-efficiency”) of

30% of public agencies by 2018. The public sector is a key economic actor in Peru and a significant consumer of

natural resources, procuring around USD 7-9 billion in goods and services each year.

6.2 Project description

GGGI has been supporting the GoP to develop and implement the Competitiveness Agenda since 2013. The

overall intended impact of this support has been to contribute to achievement of the goals of the NCA. To this

end, the specific outcomes GGGI has sought to deliver include:

• Ensure that green growth elements are integrated into the formulation of the Competitiveness Agenda

2014-18

• Selected green growth-related priorities within the Agenda are implemented and their associated

objectives are achieved

Since 2013, GGGI has thus undertaken the following outputs:

2013

Provided technical inputs into the preparation of the Natural Resources and Energy chapter of the

Competitiveness Agenda

2014:

Diagnostic analysis in selected areas under the Agenda relating to water and natural resource

efficiency.

2015-16:

Supporting MINAM to develop a National Ecoefficiency Strategy

Supporting MINAGRI to develop a National Irrigation Plan

Page 23: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

23

2017-18:

Supporting MINAM to implement its “EcoIP” program, which is piloting support to an initial 16

national ministries to improve public report of their environmental performance in 5 areas: water use;

waste production; electricity use; fuel use; paper use.

Subject to the outcomes of this pilot:

- MINAM, MEF and other ministries may consider reforms to ensure budget savings from

reduced or more efficient natural resource consumption can be retained by line ministries

and directed to other activities, thus strengthening public sector incentives to improve

environmental performance.

- The EcoIP program will be extended to additional public sector agencies at national and

regional levels.

The project is financed from core funds. A total of $975,301 has been budgeted to date, and $614,106 had been

expended up until the end of 2017.

Page 24: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

24

7. Evaluation design

7.1 Intended audience and use of the evaluation

The purpose of this evaluation is to provide lessons and recommendations to strengthen the outcomes and

impacts of GGGI’s current and future work in Peru (and potentially elsewhere).

The target audience for this evaluation includes GGGI’s Peru country team, as well as relevant units in GGGI HQ

who manage policies and systems that influence country programming, operations and effectiveness.

7.2 Key evaluation questions and scope

The design and conduct of the evaluation will be guided by the Key Evaluation Questions (KEQs) summarized in

Table 1 and further elaborated on below. The KEQs here have been developed based on the ‘hierarchy of

evaluations questions’ framework developed by Rossi, Lipsey and Freeman (2004) and widely used in evaluation

practice – see Annex 1 for details.

In terms of the evaluation scope, the KEQs below will be applied to the following projects20 covering the period

2014-18:

• BMUB-funded forestry project

• SDC-funded water project

• National Green Growth Strategy (including the expected focus on land informality and payment for

ecosystem services)

• Public sector eco-efficiency project

KEQ 1: Project Outcomes

How has GGGI’s work made a difference to the development or implementation of selected policies?

Ultimately, the attainment of outcomes is the fundamental reason for GGGI’s presence in Peru. The design of this

evaluation is thus organized around this central focus. The rationale for KEQ 1 is to begin by understanding the

extent to which GGGI’s work to date has led to outcomes.

Specifically, given the nature and status GGGI’s projects, the main outcomes to be focused on are:

• Forestry: Whether GGGI has influenced the National Forestry and Wildlife Plan in a direction that moves

20 The remaining project on eco-efficiency has been excluded from the scope of this evaluation. The main reasons for this are

(1) the need to focus GGGI’s limited budget for this evaluation, and (2) the likely discontinuation of this project moving

forward as the GoP’s National Competitiveness Agenda 2014-18 ends.

Page 25: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

25

it closer to concept of green growth.

• Water: Whether GGGI has influenced improvements in ANA’s approach to prioritizing and mobilizing

finance for investments that address Peru’s water priorities.

• Green growth: Whether GGGI has influenced improvements in the quality of the National Green Growth

Strategy.

• Public sector eco-efficiency: Whether GGGI has influenced improvements in the development and

implementation of relevant parts of the Competitiveness Agenda.

Table 1: Summary of evaluation questions to be addressed

Areas of opportunity for

improvement

Key evaluation questions (KEQ) Sub evaluation questions (EQ)

Stronger outcomes due to

improvements in the design

and implementation of

current / future projects in

Peru

KEQ 1: PROJECT OUTCOMES

How has GGGI’s work made a difference on

the development or implementation of

selected policies?

KEQ 2: FACTORS AFFECTING OUTCOMES

What have been the key factors that have

supported or hindered the achievement of

project outcomes?

(Informed by sub evaluation questions 2.1,

2.2 and 2.3.)

EQ 2.1: Project Need

To what extent was there a well targeted

need (problem or opportunity) identified

for these projects to address?

EQ 2.2: Project Design

To what extent did the project designs

ensure that the targeted need would be

effectively addressed?

EQ 2.3: Project Implementation

To what extent did implementation of these

projects ensure that the identified need was

effectively addressed?

Stronger engagement with

the GoP and delivery of

value adding services

KEQ3: CLIENT ENGAGEMENT & VALUE

How does GoP view GGGI’s performance in

terms of engagement and value add? How

could this be improved?

In addressing this question, the evaluation may consider outcomes relating to various dimensions, for example:

• Individual (changes in knowledge, attitudes, practices)

• Policy (changes in content of policy and/or the way policy is being implemented)

• Institutional (changes in interagency mandates, relationships or cooperation)

KEQ 2: Factors affecting outcomes

What key factors that have supported or hindered the achievement of project outcomes?

Page 26: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

26

In addition to assessing whether outcomes have been achieved, it is important to understand the key factors

that have supported or hindered the achievement (or lack thereof) of outcomes. This understanding provides the

basis for identifying and recommending improvement actions and is the rationale for KEQ 2.

Analysis and conclusions on this KEQ will be organized into 3 areas: (1) project need; (2) project design; and (3)

project implementation. These areas are based on the widely-used evaluative framework by Rossi, Lipsey &

Freeman (2004).21 Implicit in this framework is the idea that the successful achievement of outcomes in any

project depends on getting certain critical things right in each of these 3 stages of a project life. Therefore, three

sub-evaluation questions have been set to support and inform KEQ 2.

EQ 2.1: Project need

Was the need (problem or opportunity) to be addressed by the project well targeted?

Project need relates to the selected problem or opportunity which a project is designed to address. In

exploring this question, the evaluation may be guided the following considerations:

Was the targeted need clearly defined?

Was the targeted need realistic to address within the available resources and timeframe?

Was the targeted need aligned to the priorities of both GoP and GGGI?

Was there clear demand from the GoP to address the targeted need?

Was the targeted need aligned to the funding priorities and/or activities of other development

agencies in Peru?

EQ 2.2 – Project design

Did the design of the projects ensure that the targeted need would be effectively addressed?

In addressing this question, the evaluation may consider the following issues:

Was there a sound technical and institutional analysis underpinning the project design?

Were there clear, achievable outcomes identified for the project, along with a clear theory of change

for how to achieve these?

Was the implementation, monitoring and resourcing arrangements for the project realistic and

adequate?

Were the key risks to achieving the project outcomes and mitigation strategies identified?

Were relevant and important cross-cutting issues relating to safeguards, poverty reduction and

social inclusion identified and addressed in the design?

21 Evaluation: A Systematic Approach, Rossi, P., Lipsey, M., Freeman, H., 2004

Page 27: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

27

EQ 2.3 – Project implementation

Did implementation of these projects ensure that the identified need was effectively addressed?

A key assumption is that project outcomes are more likely to result if the solution (the project) to address

the need has been effectively implemented. In assessing this issue, the evaluation may consider evidence

relating to the following aspects.

Were the outputs delivered understood by the target audience, and was the quality perceived to be

high?

Did the project engage stakeholders in a way that would facilitate achievement of the desired

outcomes?

Were the internal arrangements for management and delivery of the project outputs efficient and

effective?

Did the project identify and adapt appropriately to significant changes in the context that could

have affected the achievement of outcomes?

KEQ 3: CLIENT ENGAGEMENT & VALUE

How does GoP view GGGI’s performance in terms of engagement and value add? How could this be improved?

As a member-based organization, a crucial and fundamental aspect of GGGI’s work is to engage our clients

(partner governments) effectively and ensure we provide advice and programs that add value. Specific issues

that may be explored under this evaluation question include:

• Has GGGI made adequate efforts to ensure its visibility and accessibility to existing and potential future

GoP partners? How could this be improved?

• How effectively is GGGI engaging with government partners in Peru? What makes GGGI’s engagement

approach effective or not effective? What could be improved?

• What is GGGI’s perceived value add to GoP partners relative to other development partners? How can

this be strengthened?

a) Evaluation approach and data collection methodology

The design of this evaluation adopts and integrates elements of the following approaches:

• Goal based evaluation: evaluation of progress towards the stated outcomes of an intervention.

• Theory based evaluation: evaluation that employs use of an explicit theory about how a project

intervention is expected to bring about its desired change.

• Utilization focused evaluation: evaluation aimed at producing knowledge for specific audiences, with the

intention that the knowledge be acted on.

• Outcome mapping: an evaluation approach that focuses on outcomes in the form of behavioral change

in targeted ‘boundary partners’, in lieu of difficulties in attributing downstream impacts back to a

Page 28: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

28

specific project intervention.

In line with good practice, the evaluation design seeks to use and triangulate multiple sources of data/evidence

to answer the identified evaluation questions. Data collection will primarily be through analysis of key

documents and structured interviews with key stakeholders.

Annex 2 provides a detailed summary of the proposed data collection methodology for this evaluation.

7.3 Timelines and deliverables

Consistent with GGGI’s evaluation policy, the evaluation will be conducted by an Independent Evaluator (IE) to

be engaged and managed by GGGI’s Impact & Evaluation Unit.

The expected timing of the evaluation is between August and December of 2018. An indicative workplan,

including major steps and key deliverables to be undertaken by the IE are summarised below.

TIMING STEPS & DELIVERABLES

April 2018 • Publish Evaluation Approach Paper

• Establish Evaluation Advisory Group (EAG)

May - June 2018 • Procure Independent Evaluator (IE)

August 2018 • Inception meeting with IE

• Deliverable 1: Inception report from IE

• Review of Inception report by key stakeholders including EAG

September 2018 • Data collection and analysis, including in-country mission by IE to Peru to

interview key stakeholders

October 2018 • Deliverable 2: Draft evaluation report by IE

• Review of Draft evaluation report by key stakeholders including EAG

• IE presentation of draft findings & recommendations to key GGGI stakeholders

November 2018 • Deliverable 3: Final evaluation report by IE

• Prepare GGGI management response to evaluation report

December 2018 • Publish final evaluation report and GGGI management response

The final output of the evaluation will be the Evaluation Report, prepared by the IE (deliverable 3). An indicative

table of contents for this report is outlined in Annex 3 and finalized in consultation with the selected IE as part

of their Inception Report to GGGI (deliverable 1).

The budget for the implementation of this evaluation is approximately $45,000, including costs associated with

engaging an IE and the in-country mission to gather data through stakeholder interviews.

Page 29: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

29

7.4 Management arrangements

In line with GGGI’s evaluation policy, the management arrangements for this evaluation are as follows:

• Impact & Evaluation Unit: Responsible for planning and designing the evaluation, contracting and

managing an independent evaluator to conduct the evaluation. IEU will work closely and collaboratively

with the Peru country team in doing so.

• GGGI Management: Responsible for endorsing this Evaluation Approach Paper and providing GGGI’s

management response to the report of the Independent Evaluator.

• Evaluation Advisory Group: An Evaluation Advisory Group will be established to provide advice to IEU

and oversight the quality of key evaluation deliverables. The group will be comprised of representatives

from Denmark and Papua New Guinea who have confirmed their participation.

• Independent Evaluator: Responsible for conducting the evaluation in accordance with the design

outlined in this Approach Paper. The independent evaluator will report to IEU but will retain

independence in expressing their views on key evaluation findings and recommendations.

Page 30: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

30

Annex 1 – Hierarchy of evaluation questions

Rossi, Lipsey and Freeman (2004)22 developed the 5-level hierarchy of evaluation questions outlined below,

based on the observation that most evaluation questions typically deal with one of five general types of

program issues.

The 5 categories of questions are related, in that each level the hierarchy is generally informed by answers to the

questions below it. It can be difficult, for example, to assess program outcomes without first understanding: (1)

the need or opportunity to be addressed by the program; (2) what the program design was intended to achieve;

and (3) whether the program was implemented as envisaged in the design.

The framework is helpful in supporting evaluators to determine the stages in a program lifecycle where factors

relating to success and/or failure have arisen – eg: distinguishing between design issues versus implementation

issues.

22 Evaluation: A Systematic Approach, Rossi, P., Lipsey, M., Freeman, H., 2004

Page 31: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

31

Annex 2 – Data collection methodology (draft)

This data collection methodology is a draft and will be reviewed and finalized with Independent Evaluator as part of their inception report

(deliverable 1).

Key and Sub Evaluation Questions Evidence sought Data collection method and source

KEQ 1: PROJECT OUTCOMES

How has GGGI’s work made a

difference on the development or

implementation of selected policies?

Forestry

• Evidence that the project influenced key actors

(SERFOR) in preparing the NWFP

Document analysis

• NFWP

Interviews

• Name, position, SERFOR

• Aaron Drayer, Peru Country Representative, GGGI

Water

• Evidence that the project influenced key actors (ANA)

in the way they prioritize and/or mobilize water

investments

Document analysis

• Directoral norm

Interviews

• Name, position, ANA

• Aaron Drayer, Peru Country Representative, GGGI

NGGS

• Evidence that the project influenced key actors (MEF,

MINAM) in preparing the NGGS

Document analysis

• NGGS

Interviews

• Name, position, MEF

• Name, position, MINAM

• Aaron Drayer, Peru Country Representative, GGGI

Public sector eco-efficiency

• Evidence that the project influenced key actors (MEF,

MINAM, others tbc) in preparing and implementing the

Competitiveness Agenda

Document analysis

• tbc

Interviews

• Name, position, MEF

• Name, position, MINAM

• Others tbc

• Aaron Drayer, Peru Country Representative, GGGI

KEQ 2: FACTORS AFFECTING

OUTCOMES

This will be a synthesis of findings from EQ 2.1, 2.2 and 2.3.

See below for more details.

This will be a synthesis of findings from EQ 2.1, 2.2 and 2.3.

See below for more details.

Page 32: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

32

Key and Sub Evaluation Questions Evidence sought Data collection method and source

What have been the key factors that

have supported or hindered the

achievement of project outcomes?

EQ 2.1: Project Need

To what extent was there a well

targeted need (problem or

opportunity) identified for these

projects to address?

Forestry

• Evidence that the selected need (supporting NFWP)

was a well targeted choice

Document analysis

• Project design document

Interviews

• Name, position, SERFOR

• Name, position, BMU

• Aaron Drayer, Peru Country Representative, GGGI

Water

• Evidence that the selected need (prioritizing and

accelerating financing of critical water investments)

was a well targeted choice

Document analysis

• Project design document

Interviews

• Name, position, ANA

• Name, position, SDC

• Aaron Drayer, Peru Country Representative, GGGI

• Name, position, WRG 2030

NGGS

• Evidence that the selected need (supporting NGGS)

was a well targeted choice

Document analysis

• Project design document

• GGGI Strategic Plan 2015-20

Interviews

• Name, position, MINAM

• Name, position, MEF

• Aaron Drayer, Peru Country Representative, GGGI

Public sector eco-efficiency

• Evidence that the selected need (supporting selected

aspects of the Competitiveness Agenda) was a well

targeted choice

Document analysis

• Project design documents

Interviews

• Name, position, MEF

• Name, position, MINAM

• Aaron Drayer, Peru Country Representative, GGGI

Page 33: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

33

Key and Sub Evaluation Questions Evidence sought Data collection method and source

EQ 2.2: Project Design

To what extent did the project

designs ensure that the targeted

need would be effectively

addressed?

Forestry

• Evidence that the project was effectively designed to

address the identified need

Document analysis

• Project design document

Interviews

• Name, position, SERFOR

• Name, position, BMU

• Aaron Drayer, Peru Country Representative, GGGI

Water

• Evidence that the project was effectively designed to

address the identified need

Document analysis

• Project design document

Interviews

• Name, position, ANA

• Name, position, SDC

• Aaron Drayer, Peru Country Representative, GGGI

• Name, position, WRG 2030

NGGS

• Evidence that the project was effectively designed to

address the identified need

Document analysis

• Project design document

Interviews

• Name, position, MINAM

• Name, position, MEF

• Aaron Drayer, Peru Country Representative, GGGI

Public sector eco-efficiency

• Evidence that the project was effectively designed to

address the identified need

Document analysis

• Project design document

Interviews

• Name, position, MINAM

• Name, position, MEF

• Aaron Drayer, Peru Country Representative, GGGI

EQ 2.3: Project Implementation

To what extent did implementation

of these projects ensure that the

Forestry

• Evidence that the project was effectively implemented

Document analysis

• Output documents

• Project EOY results reports 2015, 2016

• Budget vs expenditure summaries for 2014, 2015,

Page 34: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

34

Key and Sub Evaluation Questions Evidence sought Data collection method and source

identified need was effectively

addressed?

2016

Interviews

• Name, position, SERFOR

• Name, position, BMU

• Aaron Drayer, Peru Country Representative, GGGI

Water

• Evidence that the project was effectively implemented

Document analysis

• Output documents

• Project EOY results reports 2015, 2016

• Budget vs expenditure summaries for 2014, 2015,

2016

Interviews

• Name, position, ANA

• Name, position, SDC

• Aaron Drayer, Peru Country Representative, GGGI

• Name, position, WRG 2030

NGGS

• Evidence that the project was effectively implemented

Document analysis

• Output documents

• Project EOY results reports 2016, 2017 for NGGS &

GGPA

• Budget vs expenditure summaries for 2016, 2017,

2018

Interviews

• Name, position, MINAM

• Name, position, MEF

• Aaron Drayer, Peru Country Representative, GGGI

• Pablo Pena / Paula Aparcana, Peru Program Officers,

GGGI

Public sector eco-efficiency

• Evidence that the project was effectively

implemented

Document analysis

• GoP Competitiveness Agenda

• Output documents

Page 35: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

35

Key and Sub Evaluation Questions Evidence sought Data collection method and source

• Project EOY results reports 2015, 2016, 2017 for Public

Sector Eco-efficiency Project

• Budget vs expenditure summaries for 2015, 2016,

2017, 2018

Interviews

• Name, position, MINAM

• Name, position, MEF

• Aaron Drayer, Peru Country Representative, GGGI

• Pablo Pena / Paula Aparcana, Peru Program Officers,

GGGI

KEQ 3: CLIENT ENGAGEMENT &

VALUE

How does GoP view GGGI’s

performance in terms of

engagement and value add? How

could this be improved?

• Feedback from GoP stakeholders that GGGI engages

them effectively and is adding value

Document analysis

• Letters from GoP stakeholders regarding the quality

of GGGI’s work on various projects to date

Interviews

• Name, position, MEF

• Name, position, MINAM

• Name, position, SERFOR

• Name, position, ANA

• Others (tbc)

Page 36: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

36

Annex 3 – Table of contents for evaluation report (draft)

This table of contents is a draft and will be reviewed and confirmed with Independent Evaluator as part of their

inception report (deliverable 1).

SECTION CONTENT

1. Executive summary

2. Introduction

• Purpose of evaluation

• Key evaluation questions

• Evaluation scope and approach

• Overview of Peru country program

3. Forestry project – key findings

• Project outcomes

• Factors affecting outcomes

- Project need

- Project design

- Project implementation

4. Water project – key findings

• Project outcomes

• Factors affecting outcomes

- Project need

- Project design

- Project implementation

5. NGGS project – key findings

• Project outcomes

• Factors affecting outcomes

- Project need

- Project design

- Project implementation

6. Public sector eco-efficiency

project – key findings

• Project outcomes

• Factors affecting outcomes

- Project need

- Project design

• Project implementation

7. Client engagement and value

add – key findings

• Areas of relative strength / value add

• Areas for improvement

8. Conclusion – overall findings

and recommendations

• Findings

• Recommendations

Page 37: Evaluation Approach Paper - Global Green Growth Institutegggi.org/.../2019/03/Peru-Evaluation-Approach-Paper... · - Evaluation approach, methodology and data sources; and - Implementation

37

Follow our activities on

Facebook and Twitter

www.gggi.org