INTER-AGENCY HUMANITARIAN EVALUATION OF THE RESPONSE TO CYCLONE IDAI IN MOZAMBIQUE Inception Report September 2019 Evaluation Team: Jock Baker (Team Leader) Tristi Nichols Felisberto Afonso Pity Estajo With contributions from: Dr. Luis Artur (Team Leader – Household Survey) Emanuel Malai Rogério Sithole Evaluation Management: Ali Buzurukov (OCHA) Jane Mwangi (UNICEF) Robert Mccouch (WHO) Sergio Lenci (WFP) Elma Balic (IOM) Inter-Agency Humanitarian Evaluation Steering Group
59
Embed
INTER-AGENCY HUMANITARIAN EVALUATION OF …...2019/10/23 · • the proposed outline of the evaluation report. Purpose, Scope, Use, Timeframe & Target Audience 2.1. Purpose & Use
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
INTER-AGENCY HUMANITARIAN EVALUATION OF THE RESPONSE TO CYCLONE IDAI IN MOZAMBIQUE
Inception Report
September 2019 Evaluation Team: Jock Baker (Team Leader)
Tristi Nichols
Felisberto Afonso
Pity Estajo
With contributions from: Dr. Luis Artur (Team Leader – Household Survey)
Emanuel Malai
Rogério Sithole
Evaluation Management: Ali Buzurukov (OCHA)
Jane Mwangi (UNICEF)
Robert Mccouch (WHO)
Sergio Lenci (WFP)
Elma Balic (IOM)
Inter-Agency Humanitarian Evaluation Steering Group
IAHE Mozambique – Inception Report | i
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Contents
Contents ................................................................................................................................................ i
Acronyms ............................................................................................................................................. ii
6 Measures of Success for this evaluation ............................................................................. 25
Annex A – Aerial Maps of Affected Areas ...................................................................................... 26
Annex B – Evaluation Matrix ............................................................................................................ 28
Annex C - Instruments for the Review ............................................................................................ 35
Annex D – Community Focus Group Discussion Guide ............................................................... 39
Annex E – Evaluation Workplan ...................................................................................................... 52
Annex F – Letter of Introduction ...................................................................................................... 53
Annex G – Informed Consent Form................................................................................................. 54
Annex H –Data Analysis Plan .......................................................................................................... 55
IAHE Mozambique – Inception Report | ii
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Acronyms AAP Accountability to Affected People
COSACA emergency and humanitarian consortium
DDR Demobilization, Disarmament and Reintegration
DPOs Disabled Peoples Organizations
ERC Emergency Relief Coordinator
ERIC Ethical Research Involving Children
FGDs Focus Group Discussions
GBV Gender-Based Violence
GoM Government of Mozambique
HR Human Resources
HQ Headquarters
HC Humanitarian Coordinator
HCT Humanitarian Country Team
HH Household
HPC Humanitarian Program Cycle
HRP Humanitarian Response Plan
IAHE Inter-Agency Humanitarian Evaluation
IAHE AG Inter-Agency Humanitarian Evaluation Advisory Group
IAHE MG Inter-Agency Humanitarian Evaluation Management Group
IAHE SG Inter-Agency Humanitarian Evaluation Steering Group
IASC Inter-Agency Standing Committee
IASC EDG Inter-Agency Standing Committee Emergency Directors Group
IASC WG Inter-Agency Standing Committee Working Group IT Information Technology
INGC Instituto Nacional de Gestão de Calamidades
KIIs Key Informant Interviews
M&E Monitoring and Evaluation
MIRA Multi-sector Initial Rapid Assessment
MRP Management Response Plan
NFI Non-Food Item
NGO Non-government Organization
OPR Operational Peer Review
OCHA United Nations Office for the Coordination of Humanitarian Affairs
PREPOC Post-Cyclone IDAI Recovery Program
RC Resident Coordinator
SMART Specific, Measurable, Attainable, Relevant and Timely.
TOR Terms of Reference
UN United Nations
UNEG United Nations Evaluation Group
WASH Water, sanitation, and hygiene
IAHE Mozambique – Inception Report | 1
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Introduction 1 This inception report outlines the understanding of the Evaluation Team regarding the international humanitarian
response in 2019 the Republique of Mozambique based on the Terms of Reference (TOR). During the inception
phase the Evaluation Team undertook document reviews and preliminary discussions with key stakeholders.
This report presents:
• the purpose and scope;
• target audience;
• objectives of the evaluation;
• analytical framework;
• methodology (including instruments to be used for the community household survey);
• timeline; and
• the proposed outline of the evaluation report.
Purpose, Scope, Use, Timeframe & Target Audience
2.1. Purpose & Use
2 Based on the TOR, it is understood that the specific purpose of the Inter-agency Humanitarian Evaluation (IAHE)
is to:
• provide an independent assessment of the extent to which planned collective objectives set out in the
Humanitarian Response Plan (HRP)1 to respond to the needs and concerns of affected people in
Mozambique have been met; and
• assess the extent to which Inter-Agency Standing Committee (IASC) mechanisms, including the scale-up
activation and Humanitarian Programme Cycle (HPC), have successfully supported the response.
3 Under both objectives, the evaluation will identify lessons from the response and system-wide scale-up activation
and will generate recommendations for future responses and preparedness. This IAHE will emphasize both
learning and accountability. Given that it will assess the first inter-agency response involving a scale-up
activation, it will thus provide valuable lessons for future scale-up activations, in particular in response to the
sudden-onset of natural disasters.
2.2. Scope
4 Defining the substantive areas to be covered in the evaluation
The IAHE will primarily focus on the inter-agency response to Cyclone Idai, with Cyclone Kenneth considered as
a factor that significantly influenced the course of the system-wide scale-up response due to various
considerations including the relative scale of Kenneth, time/resource constraints and security conditions that
would make it difficult to obtain a representative perspective from affected communities. Cyclone Idai put a strain
on the capacity of the humanitarian system and Cyclone Kenneth’s added to the demands. This situation offers
a potential learning opportunity about how agencies could respond to two simultaneous emergencies. This part
of the scope would draw from secondary data, since time constraints and access to the affected communities
probably would not yield a representative sample. This evaluation’s recommendations will also be carefully
1 Along with other plans developed and used by the HCT during the course of the response.
IAHE Mozambique – Inception Report | 2
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
framed – taking in consideration the complexity and multiple demands. This approach has been validated with
key stakeholders during inception interviews, the Advisory Group in Maputo, and with senior staff who had been
directly involved in both responses.
5 The IAHE will examine the effectiveness of the scale-up activation and the global support provided to the
humanitarian response of the Humanitarian Country Team (HCT) and GoM. The following references will be
used as the main references against which to ascertain achievements:
• Revised version of the HRP version of March 2019;
• The IASC Scale-Up Response Benchmarks and Transition Strategy for Mozambique – April 2019 (dated 16
April 2019);
• Cyclone Idai – Response Strategy – Updated Discussion Paper (dated 24 May 2019); and
• Review of the Humanitarian System-Wide Scale-Up by the Mozambique HCT Benchmarks & Transition Plan
– Two Month Update (dated 24 May 2019).2,3
6 Geographic Areas
The evaluation will cover the flood-affected areas of Mozambique, or Sofála, Maníca, Quelimane, Tête, and to a
limited extent, Cabo Delgado (see figure 1), and this forms the basis of the overall selection criteria.
Figure 1: Map of Affected Areas4
2 While these inter-agency frameworks and strategies were recently secured, there may be additional frameworks which will be provided to the Evaluation Team in the field. 3 The IAHE will not evaluate the impact of humanitarian response covered by the original HRP launched in November 2018, which focused primarily on drought. 4 Source: Humanitarian Response Plan (Revised following Cyclones Idai and Kenneth, May 2019): Nov 2018 – Jun 2019
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
7 Project Selection
• The selection of projects, partners and sites for visits will be determined on the basis of the following factors
to obtain a representative view of the program:
• Volume, budget and scale of project activities;
• Possibility to observe both ongoing and completed projects (to provide an independent assessment of
activities, outputs and outcomes);
• Locations with a concentration of various activities for different affected populations;
• Examples of both successful and less successful interventions;
• Direct access to male and female representatives of beneficiary communities;
• Communities which have been assisted by multiple agencies; and
• Logistically reachable within the limited timeframe.
8 Period Covered
While the temporal scope of the evaluation starts on 14 March, when Cyclone Idai made landfall in Mozambique
until the time that the Evaluation Team visits the region (1 September). The IAHE will be carried out from August
through December 2019. The field visit for the evaluation will take place during the transition phase after the
emergency phase came to an end at the end of June and before large scale recovery programs have launched.
9 Target Groups
In line with the principle of accountability to affected people (AAP), the IAHE will assess how the targeting of
beneficiaries focused on the most vulnerable groups, as well the extent of the coverage in relation to the total
population in need.
2.3. Stakeholder Analysis
10 The primary intended users of the results of this evaluation include stakeholders from the international
humanitarian agencies (HQ, regional, and country, levels), GoM, donors (including the private sector), peer
agencies (e.g., civil society), and affected communities, including direct beneficiaries. The table below outlines
the roles in the response and assumed interests in the IAHE.
IAHE Mozambique – Inception Report | 4
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Table 1: Key Stakeholders of the IAHE, Their Role(s), and Assumed Interests in the Evaluation
Stakeholder Group Role(s) in the Response Assumed interest in/Benefits from the IAHE
International
humanitarian agencies
and their local partners
operating in country
Implementers and supporters of
GoM’s efforts
As the primary stakeholders, these agencies are
assumed to be interested in emerging lessons at
both the strategic and operational levels, as well
as mutual accountability and AAP. Will
participate in Advisory Group meetings as well.
Headquarters (HQ) of
international
humanitarian agencies
Source of policy guidance,
administrative and regulatory
frameworks, guidance and
support, surge/standby capacity,
accountability frameworks and
oversight (varies by organization)
As another primary stakeholder group, this
group is assumed to be interested in learning so
as to better implement its role in this response
and in future responses, as well as in
accountability.
Regional offices of
international
humanitarian agencies
Government5 Coordination and support to
national priorities
As another primary stakeholder, the GoM’s
interest is assumed to be focused on holding the
humanitarian agencies to account for their
results in the response – and in learning how it
might better exercise its role and execute its
response in the future. Will participate in
Advisory Group meetings as well.
Donors
Financial and in-kind support to
the response itself, and to these
same organizations’ humanitarian
functions at the regional and HQ
levels
Donors’ interest is assumed to be in ensuring
that funded agencies have exercised fiduciary
responsibility over funds received, achieved
results, and critically self-reflected on their
performance through this evaluation and other
exercises. Private Sector
In-kind support to the response
itself at the local level from
foundations.
Peer agencies (e.g.,
civil society)
Lesson learned in the
coordination, partnerships
established, among other areas
Dissemination of lessons learned that could
potentially be used to better establish its own
contributions to the response.
Affected communities,
including direct
beneficiaries
Ultimate beneficiaries targeted by
humanitarian operations
This group can be categorised into village-level
leadership and residents, and both are
assumed to be interested in better
understanding how the humanitarian system
works (e.g., criteria for receiving assistance and
needs met), what is expected from them during
and after the experience (e.g., returning to
normalcy as soon as possible), and how they
ought to be treated (with respect, dignity, no
sexual harassment).
5 This includes: INGC, Instituto Nacional de Estatística (INÉ) [Statistics Institute]; Instituto Nacional de Acçao Social (INAS); Ministério da Saúde (Min. Health); Governo Da Província De Sofála; Governo Da Província de Maníca; Governo Da Província de Tête; Governo Da Província de Quelimane; Governo Da Província de Cabo Delgado.
IAHE Mozambique – Inception Report | 5
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
2.4. IAHE Background & Strategy
11 The two tropical cyclones resulted in loss of life, injury and illness, destruction of livelihoods and infrastructure.
Following the cyclones and flooding caused, an estimated 2.2 million people required urgent assistance (1.85
million people due to Cyclone Idai and 374,000 people due to Cyclone Kenneth), on top of 815,000 people
already in need, because of an ongoing drought. Cyclone Idai hit the central region of Mozambique (Sofála,
including its capital Beira; Maníca; Zambézia, and Tête), leaving an estimated 3,000 km2 of land submerged and
damaging more than 240,000 houses. Subsequently, Cyclone Kenneth struck the northern region, including
Cabo Delgado, destroying more than 45,300 houses.
12 On 22 March 2019, following consultations with the Inter-Agency Standing Committee (IASC) Principals and the
GoM, the Emergency Relief Coordinator (ERC) formally declared a Humanitarian System-Wide Scale-Up
Activation (henceforth referred to as ‘scale-up activation’) for a period of three months, or until 22 June. This was
the first time such a designation had been made following the IASC decision in November 2018 to replace
previous “L3” system-wide activation definitions and procedures.6
13 The scale-up activation is an inter-agency mobilization mechanism in response to a sudden onset and/or rapidly
deteriorating humanitarian situation in a given country, including at the subnational level, where capacity to lead,
coordinate and deliver humanitarian assistance does not match the scale, complexity and urgency of the crisis.
14 The procedure activates mechanisms and tools to: (a) ensure that the IASC system delivers effectively in support
of national authorities and existing capacities and monitors its own performance, (b) ensure that adequate
capacities and tools for empowered leadership and coordination of the humanitarian system are in place, and
(c) engage IASC member organizations and Global Cluster Lead Agencies to put in place the required systems
and to mobilize the required resources to contribute to the response as per their respective mandates.
15 In line with the new protocols, a scale-up activation requires that an Operational Peer Review (OPR) of the
response be undertaken within five months of the crisis, and that an IAHE be conducted 9-12 months after the
declaration. OPRs, designed to be brief and using a collaborative process, are undertaken by peers. They are
not intended to measure results or the impact of the response. IAHEs, conducted at a later stage of the
humanitarian response, are independent and have the purpose of not only promoting accountability to donors
and affected populations but also encouraging strategic learning for the humanitarian system. The OPR in
Mozambique was conducted from 26-31 May 2019 and the report released on 24 June 2019.7 For this IAHE, it
is anticipated that the final report will be made available approximately seven months after the scale-up
declaration.
6 For further information, the scale-up protocols are located here. (see link) 7 IASC (2019) Operational Peer Review, Mozambique Cyclone Idai Response.
10. Particip (2018) “Evaluation of DG ECHO's disaster preparedness and DRR actions in Southern Afric& Indian Ocean”, Contract n°: ECHO/ADM/BUD/2011/01205;
11. International Federation of Red Cross and Red Crescent Societies (IFRC) (2019) Real-Time aluation, Mozambique: Tropical Cyclones Idai and Kenneth;11
12. Mozambique Red Cross Society and the International Federation of Red Cross and Red Crescent Societies (IFRC) (2015) The Final Evaluation of MDRMZ011;
13. Independent Review of COSACA Management of cyclone response in Mozambique (2019) Forthcoming.12
14. Other relevant references.13
Oth
er
15. IASC Operational Peer Review of the Mozambique Cyclone Idai Response. June 2019
16. Active Learning Network for Accountability and Performance (ALNAP) (2006)Evaluatinghumanitarian action using the OECD-DAC criteria An ALNAP guide for humanitarian agencies;
17. Organisation for Economic Co-operation and Development’s (OECD) Development Assistance Committee (DAC) criteria; and
18. The SPHERE Handbook (2018 edition).
8 INGC (2019) Post disaster needs assessment. 9 República de Moçambique Conselho de Minístros (2019) Plano Anual de Contingência 2019; The IASC Scale-Up Response Benchmarks and Transition Strategy for Mozambique – April 2019 (dated 16 April 2019); Cyclone Idai – Response Strategy – Updated Discussion Paper (dated 24 May 2019); Review of the Humanitarian System-Wide Scale-Up by the Mozambique HCT Benchmarks & Transition Plan – Two Month Update (dated 24 May 2019). 10 UN OCHA Situation Report No. 14 as of 15 April 2019; UN OCHA, Mozambique: Cyclone Idai & Floods Situation Report No. 22 As of 20 May 2019 UN OCHA Flash Appeals #1, #2, #3, #4, among other documents. 11 The authors of this report are: Juergen Hoegl, team leader (external) Sammy Fwaga, IFRC Africa Regional Office, Marie Manrique, IFRC Country Cluster Support Team, and Lima Victoria Stodart, IFRC Secretariat. 12 COSACA consists of the international aid organizations CARE International, Oxfam, and Save the Children (link here). 13 OCHA has been compiling a list of other planned Idai-related evaluations.
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
17 Based on an initial scoping exercise and after reviewing these key resources, the Evaluation Team has been
able to: (i) identify the areas that can realistically be examined during this study; and (ii) develop a detailed
methodology, taking into account what has already been proposed in the TOR.
2.6 Country Context
18 Elements of a Complex Emergency (2018–2019)
The pre-existing socio-economic conditions in Mozambique show that Cyclone Idai took place in a context of
high vulnerability, creating conditions that have exacerbated poverty levels among affected populations.
Mozambique ranks as a country with one of the lowest human development index (HDI) in the world (0.437), at
180th among 189 countries.14 The agricultural sector accounts for 25 percent of GDP and employs 71 percent
of the labor force. Almost 94 percent of the poor are primarily engaged in agriculture.15 Before Tropical Cyclone
Idai, the country was already facing high levels of food insecurity. In 2017 and 2018, Mozambique’s Technical
Secretariat for Food Security and Nutrition (SETSAN) had conducted several studies documenting severe food
insecurity throughout the country due to the onset of drought conditions.16
19 Political dynamics were also operating in the background, compounding the effects of Cyclone Idai. For example,
preparations for the general elections to be held on 15 October 201917, a peace process between the
Government and Renamo, and the process of Demobilization, Disarmament and Reintegration (DDR) are
currently taking place. The nature and national scope of these processes, as well as the time limits for achieving
them, make them especially sensitive in the post-disaster context left by Idai. These are all vital for the
consolidation of peace and democracy in Mozambique.
20 GoM-Led Immediate Response to Cyclones (2019)
In the immediate aftermath of the storm, rapid aerial assessments were carried out to gather a general overview
of the scale and severity of the crisis, information from which was used to inform immediate operational decision-
making regarding priority areas for response.18 The GoM immediately implemented a series of actions in
response to the unfolding disaster, including the:
• declaration of a National State of Emergency on 19th March 2019;
• implementation of immediate search-and-rescue operations, and provision of humanitarian aid;
• appeal for international assistance;
• establishment of a post-Cyclone Idai Post-Disaster Recovery Programme (PREPOC) on 26 March;
• approval the TOR of the PREPOC on April 2, 2019;
• creation of the Post-Cyclone Reconstruction Office on April 09, 2019;
• extension of the Scope Assessment for Cabo Delgado and Nampula Cyclone following Cyclone Kenneth, on
April 30, 2019;
• training of staff of ministries / sectors central, provincial, district and municipal levels on the post-disaster
needs assessment (PDNA) methodology and calculation of damage, loss and recovery needs;
• conduct of the PDNA, approved by the Council of Ministers on May 7th; and
• development of the Final Report of the PDNA, which was approved by the Council of Ministers.19
14 UNDP. Human Development Indices and Indicators 2018 Statistical Update. 2018. 15 World Bank. Mozambique Poverty Assessment. 2018, page 25. 16 Ministério da Agricultura e Segurança Alimentar: Relatório Final da Avaliação Sazonal De Nutrição De Março-Abríl de 2018; Acute Food Insecurity CPI Analysis Report April 2018: Resultados das análises de IPC conduzidas em 36 distritos no período de Abril à Maio de 2018; Relatório da Monitoria da Insegurança Alimentar Aguda de Outubro e Novembro de 2017. 17 The general elections in October will elect a new President, Members of Parliament and for the first time, and the governors of 10 provinces. 18 INGC, ACAPS, IFRC, MapAction, OCHA, REACH Initiative, United Nations Disaster Assessment and Coordination. Multi-Sectoral Rapid Assessment Post-Cyclone Idai: 14 Districts in Sofála and Maníca Provinces, Mozambique 1-17 April 2019. 19 Government of Mozambique (2019) Post-Disaster Needs Assessment (PDNA), page 41. It is not yet known the exact date when the Council of Ministers approved the Final PDNA report, but the Evaluation Team will seek this information in the field.
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
An international appeal for assistance in the amount of USD$281.7 million and revised the country’s humanitarian
response plan. The HCT also commissioned a multi-sectoral rapid assessment to inform the humanitarian
response.
Methodology
3.1. IAHE Design
21 The methodology for this IAHE has been designed with the aim of answering three main questions within the
limits imposed by time, resources and other constraints:
1) To what extent were priority humanitarian needs of affected communities met?
2) Given this was a nationally led response, what was the contribution of the international humanitarian system?
3) How effectively did the international humanitarian system work together to support the response and meet
priority needs of affected populations?
22 The results of this evaluation should thus provide a useful overall perspective but should not be expected to
provide detailed guidance on operational performance of individual agencies,20 apart from where agencies have
significant lead coordination roles. The aim is to examine how the international humanitarian system delivered
priority needs to affected populations as one system. Looking at how to better harness the support that is
provided from all partners is one issue that has been raised as very important in inception interviews.
3.2. Overall Approach
23 As shown in figure 2 below, the quantitative and qualitative information sources of the IAHE include: (1) structured
interviews with village level leadership; (2) a community HH survey; (3) community focus group discussions
(FGDs); (4) semi-structured individual and group interviews with humanitarian actors; and (5) document and data
analysis. The green colored icons reflect the work of the Core Evaluation Team, and the other data collection
activities are to be undertaken by the Survey Team.21 Section 3.3 discusses team composition.
24 While the qualitative data from individual and group interviews with humanitarian actors22/village level leadership
will be purposively gathered, often times using a snowball sampling method,23 the quantitative sampling process
for the community HH survey is based on a methodological approach defined by Glenn Israel (2000).24
Specifically, probabilistic methods will ensure that findings are at the 95 percent of confidence level and five
percent of precision. A total of 400 HH will be sampled in the three regions affected by Idai Cyclone (Maníca,
Sofála, and Tête) during the evaluation.25
20 During the Mozambique HCT meeting on 02 July 2019, the Team Leader outlined the limitations of the IAHE noting that the IAHE will not a suitable instrument for capturing operational lessons learned for individual agencies. Agencies were encouraged to undertake After Action Reviews (AAR) or similar reflective exercises, so that they could use the IAHE results to understand their individual contributions and at the same time provide the IAHE team with useful secondary data. 21 The Survey Team will be led by Dr. Luis Artur from Universidade de Eduardo Mondlane. 22 These interviews would be in Maníca, Sofála, Tête, and Zambezia. To the extent possible, the Evaluation Team will conduct skype/telephone interviews with humanitarian actors based in Cabo Delgado. 23 Noy, C. (2008). Sampling knowledge: the hermeneutics of snowball sampling in qualitative research. International Journal of Social Research Methodology, 11(4), 327-344. The definition on page 330 is: “A sampling procedure may be defined as snowball sampling when the researcher accesses informants through contact information that is provided by other informants. This process is, by necessity, repetitive: informants refer the researcher to other informants, who are contacted by the researcher and then refer her or him to yet other informants, and so on.” 24 Israel, Glenn D. (2000). Determining Simple Size. University of Florida, IFAS Extension.
25 This should give a confidence level of 95% for an affected population in excess of 1 million
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Figure 2: IAHE Methods
Quantitative and Qualitative Data Sources
Community Level Interviews Community HH Survey
Service delivery stakeholders at community levels. Interview take place where survey is conducted in Maníca, Sofála and Tête
A total of 400 face-to-face surveys from a random sample of HHs in Maníca, Sofála and Tête
Community Focus Group Discussions Agency & Partner Interviews
Multiple FDGs in each province which satisfy specific criteria. Take place in the same villages where HH survey is conducted
Service delivery stakeholders at provincial, district and regional levels (Government, UN, Donors, NGOs) in Maníca, Sofála and Tête, Zambezua and Cabo Delgado
Document Analysis Secondary Data Analysis
• Analysis of existing & other relevant evaluations
• Humanitarian requirements & standards
• Meeting minutes
• Cluster-Level Strategy Plans
• Data for needs assessments
• Monitoring data
25 Each team member will be assigned focal point responsibilities for specific themes and issues to ensure an
adequate coverage of documentation, analysis, documentation on key issues emerging from interviews and
FDGs, while also periodically taking advantage of opportunities for validation by key stakeholders throughout the
entire process. Although evaluative methods will be employed to ensure an appropriate level of rigor and
credibility, specific emphasis will be given to maximizing learning, sharing, and emphasizing the need to utilize
results. The final part of the approach undertaken by the Evaluation Team is the systematic comparison,
triangulation, and analysis of data collected, which ensures data integrity and factual accuracy.
26 Contextual and Cross-Cutting Issues to be Taken into Account
The 2030 Agenda of “Leaving no one behind” draws attention to review factors that contribute to inequality (e.g.,
gender, equity, human rights, inclusion, and disempowerment linked to climate change-related risks). 26 Part of
the analysis in this evaluation will be dedicated to better understanding the implementing mechanisms designed
(and used) to empower and actively encourage the participation of all vulnerable groups in relevant decision-
making processes.
27 The National Context
Although this was a government-led response, emphasis on capacity building and national ownership are still
prevailing issues which will also be assessed.
26 A/RES/70/1 of 21 October 2015. Transforming our world: the 2030 Agenda for Sustainable Development.
IAHE Mozambique – Inception Report | 10
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
28 Donors
As Mozambique is heavily affected by climate related risks, the extent to which there are devoted resources to
address this permeating risk will be further explored.
3.3. Evaluation Questions27
29 According to IAHE guidelines (page 15), the evaluations draw upon evaluation criteria from:
• the UNEG Norms and Standards;
• OECD / DAC criteria for development program, including: (1) relevance/appropriateness; (2) efficiency;
(3) effectiveness; (4) impact; (5) sustainability; and
• the ALNAP criteria for the evaluation of humanitarian action, which are: (1)
coherence/coordination/partnerships; (2) coverage; and (3) connectedness.
30 Understandably, not all of these criteria are applicable in this IAHE. The following seven evaluation criteria will
To what extent have the objectives set out in the HRP, other strategic documents, and
other joint planning documents been based on identified needs, including those of the
most vulnerable groups affected by the crisis?
1.1. To what extent were efforts undertaken to support disaster preparedness before
the event?29
1.2. After the event, how were the needs assessments undertaken, and to what
extent was the information used for response planning?
1.3. To what extent were assessment processes coordinated?
1.4. To what extent do the planning documents reflect identified needs and priorities
of affected people, including those of vulnerable groups?
1.5. To what extent did the mechanisms actively encourage the participation of
vulnerable groups in decision‐making processes, including in environmental
matters?30
27 The assessment criteria is based on definitions from: ALNAP (2006) Evaluating humanitarian action using the OECD-DAC criteria An ALNAP guide for humanitarian agencies; and the IAHE Guidelines from the following website (see link). 28 These sub-questions were designed to fall in line with the 2030 Agenda “Leave no one behind”. 29 In the discussion various members of the Advisory Group in Maputo highlighted the importance of reviewing and recognizing early actions (prior to landfall), as well as preparedness issues (pre-position of relief items), which played an important role in the response. As the evaluation team will review program participant participation into the design of response, it is important to understand that design of prepositioned kits, including verifications of which were undertaken during the immediate response. Moreover, was there targeting vs. blanket distributions needs considered? 30 Appropriateness of humanitarian assistance is linked to engaging affected populations and other vulnerable stakeholders in decision-making. Two official records were consulted from the Economic and Social Council (the Committee for Development Policy): (1) E/2018/33 (2018) Supplement No. 13; and (2) E/2019/33 (2019) Supplement No. 13
2.1. To what extent were the targets articulated in the HRP achieved (in terms of
assistance delivery)?
2.2. To what extent were the targets effective in meeting the needs of the most
vulnerable? [protection question]
2.3. To what extent has the scale-up activation supported the response (as stated
under ‘Purpose’ above)?
2.4. Did the response have any unintended, positive or negative effects on affected
communities?
2.5. To what extent were strategies, approaches or methodologies executed to
protect vulnerable groups, particularly at the community level? Are any new
approaches needed?
3. Connectedness
How was the IASC humanitarian system’s emergency assistance for people affected by
the crisis linked to longer-term recovery, resilience and development efforts? What, if
any, were the challenges in implementing this linkage?
3.1. What are the interventions designed to support the transition from recovery to development?
3.2. To what extent has the response contributed to restoring livelihoods (and returning to normalcy) at the community level, and particularly for vulnerable groups? [protection question]
3.3. Is there any evidence that the response helped affected people cope better with subsequent or future cyclones?
4. Coverage
To what extent were different groups of affected people in all locations affected by
Cyclone Idai reached with humanitarian emergency aid?
4.1. To what extent was protection integrated throughout the response efforts for
vulnerable groups in all locations?
4.2. How did the funding and donor priorities influence the overall coverage?
5. Partnerships To what extent have adequate partnerships been established (with international,
national and local stakeholders) to deliver assistance to affected people?
6. Localization
To what extent and in what ways have national and local stakeholders (GoM, IINGOs,
NNGOs, private sector, military) been involved in international coordination
mechanisms, and have their capacities and systems to respond in the future been
strengthened through the response?
6.1. How did the response strengthen the capacities of partners to contribute in
cross-cutting areas?
7. Coordination
How well-coordinated was the assistance, avoiding duplication of assistance and gaps?
7.1. How effective were coordination mechanisms at the strategic, inter-cluster, and
cluster/sector levels?
7.2. Is there any evidence that coordination helped to avoid duplications?31
7.3. How effective was the resource mobilization effort in raising sufficient, timely
and long-term funding? How did the timing of the funding and donor priorities
influence the overall response, and in what form of assistance (financial and in-
kind assistance) did donors provide assistance? 32
7.4. What major gaps in human and financial resources to carry out activities in
cross-cutting areas were identified? [protection question]
31 What are the challenges in the surge management among UN agencies, including the occurrence of “double-hatting”? 32 How (well) did the CERF mechanism function and is this the most effective way to release funds only via UN agencies?
IAHE Mozambique – Inception Report | 12
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
3.4. Assessing Achievements and Gaps
31 Assessing the UN’s Contribution to the Government-Led Response
Reflecting on assumptions that underlie the inter-agency response in the Mozambique and taking into account
the IAHE Impact Pathway from the Inter-Agency Humanitarian Evaluations Process Guidelines, the Evaluation
Team has reconstructed a Humanitarian Response Pathway which better reflects the elements of this IAHE.
(See Table 4.)
Table 4: Humanitarian Response Pathway for the Inter-Agency Response in Mozambique
OUTPUTS33 SUPPORT CORE
RESPONSIBILITIES
INTENDED RESULTS OF
SCALE-UP ACTIVATION
DESIRED LONGER-
TERM IMPACT
International inter agency humanitarian system functions effectively.
International response complements and supports the government response.
Ten IASC Clusters in 10 hardest-hit districts.34
Humanitarian access secured for all, including the most vulnerable groups within the population in need.
Affected people live in enhanced safety and dignity with better prospects of thriving as agents of their own destinies.
Coordination
The lead agency is the National Disasters Management Institute (Portuguese acronym is INGC)
CCCM-Camp Coordination & Camp Management
Livelihoods protected and restored
Logistics Appropriate
response
Education Connectedness and coordination between humanitarian actors
Accurate Assessment
ECT-Emergency Tele-communications
Planning Food Security
Relevant and effective coverage
Partnerships Health
Funding & Fundraising
Nutrition
Learning Protection/ Child Protection/ GBV
Adequate capacity strengthening partnership with INGC.
Cross-cutting issues
Shelter/NFI
WASH
32 International support to the GoM is realized through activities and outputs. These are the key components of the
contribution during an emergency. The core responsibilities are from the ten IASC Clusters in ten of the hardest-
hit districts. Assuming that IF the efforts of these clusters are appropriate, effectively delivered (effectiveness,
coordination, and partnerships) met longer term needs (connectedness), had adequate coverage, and
33 This Response Pathway was adapted from UN OCHA. 2018. Inter-Agency Humanitarian Evaluations Process Guidelines (page 16). 34 Source: Review of the Humanitarian System-Wide Scale-up By the Mozambique HCT Benchmarks & Transition Plan – Three Month Update.
IAHE Mozambique – Inception Report | 13
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
strengthened capacity (localization), THEN the specific intended results and desired longer-term impact are likely
to have occurred. These assumptions are linear, which may not be realistic for a humanitarian effort. However,
this Humanitarian Response Pathway (model) serves to promote a shared understanding of the response logic
model and guide the Evaluation Team’s thinking throughout the process.
33 Assessing Appropriateness
According to the TOR, appropriateness focuses on better understanding how the contributions of the international
humanitarian system supported the national humanitarian response, and in particular, the extent to which
activities were adapted to needs.35 The Evaluation Team will, therefore, review the objectives set out in the HRP
and determine the extent to which (and how) objectives were based on identified needs. The findings
emerging from this exercise will depend on a degree of coherence and accuracy between the data collected from
official sources of assessed needs, planning documents, and monitoring data.36 Although all of the needs
assessments are not yet available, an attempt will be made to secure additional official reports, such as
estimations of those who were exposed to multiple hazards (i.e., loss of life, flooding and loss of food security
and livelihood, forced resettlement, etc.). Through a HH Community Survey, satisfaction levels of the assistance
received as well as perspectives about the usefulness of the support received will also be explored.
34 Assessing Effectiveness
A review of effectiveness entails the examination of the extent to which the overall international response
achieved its objectives. Implicit within the criteria of effectiveness is the timeliness of the delivery of assistance.
While IASC documents will provide the Evaluation Team with the general scope of the prioritized and most time-
critical life-saving activities undertaken, including the efforts of the ten IASC Clusters,37 learning about whether
(and which) targeted results were (and were not) met in the different sectors – and why – will also shed
light on the precise and tangible benefits that were made available
immediately after Cyclone Idai. The study’s community HH community
survey results, also used for triangulation, will give further insights about
what was and was not achieved at the community level. The data from
FGDs will illuminate the overall experience of receiving assistance,
particularly the extent to which the beneficiaries with disabilities or specific
vulnerabilities were given sufficient attention. Such groups include the
elderly with mobility challenges, small children and children with disabilities,
pregnant women, girls who may have been subject to sexual abuse,
isolated communities, among others.38
35 Addressing the Prevention of Sexual Exploitation and Abuse (PSEA) was a
top priority in this evaluation effort. In ensuring that effective systems were
in place for preventing and responding to acts of sexual exploitation and
abuse, the Evaluation Team will proactively pursue beneficiary experiences
and examine if strict adherence to Standard Operating Procedures took
place with identified cases, while respecting “do no harm” protection
principles. The cases from the Linha Verde hotline (Figure 3) will be an
initial data source to examine.
35 UN OCHA, MOZAMBIQUE: Cyclone Idai and Floods Situation Report No. 22 As of 20 May 2019. 36 The key planning documents to be used as a reference are: (1) INGC (2019) Post disaster needs assessment; (2) HCT plans; (3) Cluster-level strategies (not yet available); (3) HRP version of March 2019; and (4) Monitoring data (not yet available) for the re)assessment of needs (i.e., through on-going monitoring activities, sitrep). This also includes the efforts which took place for the Scale up which took place before the scale down (of the drought). The alignment of the PDM align with the PDR will also be taken into account. 37 Information sourced from: Review of the Humanitarian System-Wide Scale-up By the Mozambique HCT Benchmarks & Transition Plan – Three Month Update. The ten clusters are: 1. CCCM-Camp Coordination & Camp Management; 2. Logistics; 3. Education; 4. ECT-Emergency Tele-communications; 5. Food Security; 6. Health; 7. Nutrition; 8. Protection/ Child Protection/ GBV; 9. Shelter/NFI; and 10. WASH. 38 UN OCHA, Mozambique: Cyclone Idai & Floods Situation Report No. 22 As of 20 May 2019.
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
36 Assessing Connectedness
Connectedness refers to the need to ensure that activities of a short-term emergency nature are carried out in a
context that takes longer-term problems into account.39 Although the general scope of activities undertaken by
the ten IASC Clusters will provide an overview of the UN’s contribution to the government-led response, the
Evaluation Team will also learn about how the response met longer-term needs (e.g., material changes in their
lives from assistance), if the support was perceived as timely, and what type of assistance aided the recovery
and reconstruction process.
37 Assessing Coverage
Coverage refers to the need to reach major populations facing life-threatening suffering wherever they are. This
IAHE is unique that it will examine an emergency response that involved two different tropical cyclones (above
Category 2) which made landfall in two different time periods and in two different regions of the country. As both
of the cyclones made landfall during the critical harvest period, one of the compounded effects was that families
who lost their livelihood (houses and crops) were left without the prospect of another full harvest until March
2020.40 The ensuing lessons will show when decisions to scale up were made and how the international
humanitarian system supported the GoM in mobilizing and extending its reach to deliver assistance in multiple
locations. Further, the community HH community survey results about the extent to which the assistance
received was timely in different sectors and by region will provide additional insight. Survey results will also
indicate the extent to which assistance provision was tailored to the needs of the most vulnerable (also termed
social coverage).
38 Assessing Partnerships and Localization
This crux of this criteria will be to understand how the contribution of international humanitarian agencies
supported the government to alleviate the suffering of affected populations. The partnership component will be
examined at multiple levels, including with international, national and local stakeholders. There is a need to
better understand the strength of these partnerships, including whether they were pre-existing with mutual trust
and respect, newly established, strategic, operational, and what the stakeholder perceptions are regarding their
effectiveness. At the community level, FGDs will examine: (a) how communities were consulted; (b) how families
who received assistance were selected and what role they had in this decision-making process; and (c) how
support affected different members of the family, including those with family members who are disabled or
vulnerable. All of this information is intended to inform whether the partnerships could adequately deliver
assistance. Similarly, understanding the capacity needs to effectively partner with the GoM is another area that
will be explored. The share of resources that civil society received (if any at all) to engage in the response,
disaggregating support to international and national NGOs, is another area that will be examined.
39 Assessing Coordination
The degree to which the coordination mechanisms work coherently and efficiently will be examined, notably
between the HCT and the GoM at country and provincial levels, in order to understand the extent to which
the international humanitarian system complemented and supported this government-led humanitarian
response. In addition, coordination will be assessed in the following other areas:
1. among the HCT members at country level;
2. between and among the HCT and non-HCT and non-GoM partners (e.g., national and International NGOs
operating within the clusters, representatives from the private sector with in kind donations and
individuals/groups who functioned within clusters);
39 The assessment criteria is based on definitions from: ALNAP (2006) Evaluating humanitarian action using the OECD-DAC criteria An ALNAP guide for humanitarian agencies; and the IAHE Guidelines. 40 UN OCHA (2019) Humanitarian Response Plan: Mozambique 2018-2019 (Revised following Cyclones Idai and Kenneth, May 2019), page 5.
Gender, equity, human rights, inclusion, and environmental issues are integrated throughout the evaluation
criteria of appropriateness, effectiveness, coverage, localization, and coordination (see table 2). Nonetheless,
it is recognized that the impact of the cyclone has a differential impact on women and girls. One report noted
that there was an elevated risk of Gender-Based Violence (GBV) during the humanitarian emergency, as there
is “greater exposure of women and girls to distant and unsafe locations, such as water collection points, sanitation
facilities, and health centers.”41 Moreover, households with women and girls, and particularly women-headed
households, are being subjected to sexual exploitation by men in power or with access to relief products and
resources. The Evaluation Team plans to assess the extent to which communities perceived that the support
and assistance provided contributed to the medium- and longer-term needs of vulnerable groups (i.e., elderly,
orphaned children). INGC drafted and adopted a Gender Strategic Plan (2016-2020) which aims to reduce the
vulnerability of women in relation to disasters and mitigate impacts on sociocultural, economic and political rights
arising from gender inequalities. This plan provided for the establishment of a gender unit to ensure the
implementation and monitoring of the actions undertaken by the INGC. The team will consult with the gender
unit as well as the Ministry of Women and Social Action.
41 It cannot be emphasized more that protection issues will be thoroughly examined. Due to the movement of
populations to safer areas, many children were separated from their families. The number of orphaned children
is currently unknown.42
The Evaluation Team plans to analyze data by gender, inclusion, and other relevant drivers of inequality, in line
with: (1) the UNEG Guidance on Integrating Human Rights and Gender Equality in Evaluation; (2) the UN
System-Wide Action Plan (UN-SWAP) on gender equality; (3) the IASC Gender Equality Policy Statement; and
(4) the IASC Guidelines on the Inclusion of Persons with Disabilities in Humanitarian Action.43
Please see annex B for the Evaluation Matrix which summarizes the evaluation questions, indicators, data
sources, and reliability of information to be gathered.
42 Stakeholder Analysis
Interviews in the field (including Maputo), the region (Kenya) and at a HQ level will allow the Evaluation Team to
gather different perspectives from various stakeholders. (See the table 1 for a description of key stakeholders.)
Due to time constraints, there will be a need to carefully select key informants for this evaluation (both
stakeholders and non-stakeholders), so as to secure a balanced sample. Sampling techniques are described in
section 3.2.
43 Observations
It is anticipated that the Evaluation Team will engage in field observations when traveling to the provinces and
districts. While observations are an important part of any evaluation process, a structured observation
process/plan (with an established methodology) has not been integrated into this evaluation due to resource
constraints.
41 INGC (2019) Post disaster needs assessment, page 20. The need to collect water more frequently and from greater distances and the use of public toilets and shared latrines increases the risk for women and girls. With the destruction of health facilities, pregnant women have limited access to safe deliveries. 42 INGC (2019) Post disaster needs assessment, page 170. 43 The Convention on the Rights of Persons with Disabilities (CRPD) will be consulted.
communications, Shelter/NFI (including seed distributions), Food Security and
livelihoods, among others.
• Notetaking and data analysis; and
• Data analysis and report drafting to complement sections on coordination,
partnerships, coverage, [part of] effectiveness, and connectedness.
SU
RV
EY
TE
AM
Survey Team Leader Dr. Luis Artur
• Lead all the necessary communication and coordination with local government and
relevant entities for the smooth realization of the work;
• Controls the quality of the work;
• Assures the utilization of an adequate methodology;
• Assures the quality and delivery of deliverables within target deadlines;
• Interact and coordinate with the Core Team members during all the process;
• Co-design methodology and research tools;
• Responsible for submitting survey deliverables, including the survey report, to the
Management Group for their review; and
• Leads the conduct of FGDs.
Emanuel Malai
• Co-Design methodology and research tools;
• Design of data collection instruments;
• Train enumerators;
• Monitoring and quality control of data collection;
• Implements the FGDs and manages the survey team in Tête;
• Supports the drafting/reviewing of the reports.
Rogério Sithole
• Co-Design methodology and research tools;
• Design of data collection instruments;
• Train enumerators;
• Monitoring and quality control of data collection;
• Implements the FGDs and manages the survey team in Beira and Sofála; and
• Supports the drafting/reviewing of all sections of the reports.
As the elections delayed the planned work of the survey team until after October 15 th, the Senior Evaluator will
return to Mozambique to participate in pretesting all tools. These tools include the survey and the FGDs.
IAHE Mozambique – Inception Report | 21
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
3.6. Key Milestones for the Study46
Table 6 Key milestones for the study
Milestone Date(s) Responsibility Participants
Final draft version of Inception Report returned from management group
Thu, Aug 27th Team Leader Evaluation Team and management group
Final report including all inputs from the Management and Advisory groups
Fri, Sep 12th Team Leader Evaluation Team
Core Team Field Travel Sep 1-25 Evaluation Team
Presentation of preliminary findings and emerging conclusions to to the IAHE Advisory Group
Sep 20th Team Leader Evaluation Team
Senior Evaluator travel to provide technical assistance to the Survey Team for instrument pretesting and data collection
End of October Senior Evaluator Senior Evaluator and Survey Team
Household survey team fieldwork Oct 28 – Nov 10
Team Leader, HH survey
Evaluation Team
First draft of survey report from Survey Team Nov 26th Team Leader, HH survey
Evaluation Team
Final version of survey report from Survey Team
Nov 29th Team Leader, HH survey
Evaluation Team
Draft evaluation report for wide circulation December 6th Team Leader Evaluation Team and management group
After completion of draft report, workshop in Maputo takes place with IAHE Advisory Group, HCT, and the IAHE Management Group
December 12th Team Leader and OCHA
Team Leader
Deadline for written feedback December 20th Team Leader and OCHA
All levels
Final version of report Jan 22nd Team Leader Evaluation Team
Presentation for IASC Operations, Policy and Advocacy Group and to the IASC Emergency Directors Group in Geneva and/or New York.
TBA Team Leader and OCHA
Evaluation Team
The Evaluation Workplan (annex E) includes specific activities.
3.7. Potential risks for the study
53 The presidential and parliamentarian elections are the biggest risk identified thus far, since most if not all of the
GoM stakeholders will be heavily preoccupied with campaigning activities. The implication is that there will be
very little time dedicated for the IAHE consultations, and this very important stakeholder group could risk being
inadequately represented in the process. This would be an unfortunate limitation, as the response was
government-led. Mitigating this risk requires quick deployment of the Evaluation Team without delay, so that all
data queries and consultations are undertaken at least one month before the election (i.e. before 15 September).
46 See also the attached workplan in the Annex.
IAHE Mozambique – Inception Report | 22
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
3.8. Limitations
54 The Evaluation Team foresees the following limitations that could affect the achievement of the TOR:
• Given the relatively broad scope of the activities and the remoteness of many intervention sites and time and
capacity constraints, the coverage of the evaluation will be limited. This includes the response to cyclone
Kenneth. Due to various constraints (time, resources, security) it will not be possible for either the evaluation
team or the survey team to visit the area and data will be collected exclusively via secondary data and remote
interviews.
• Similarly, after considering resource constraints, logistics challenges and the relatively small affected
population in Zambezia the preferred approach is to carry out the household survey remotely in this province.
• The national election process in Mozambique has already affected the evaluation timeline so that the
household survey will only take place after the election on October 15th. Although post-election violence is
not seen as likely, it is nevertheless a risk that could potentially cause further delays.
• Assessment of effectiveness and coverage will also be dependent on the availability of suitable data,
including the availability of comparative data from the GoM and peer agencies.
• The high turnover of senior staff, since the beginning of the current strategy, may limit the amount of relevant
information available to the team.
• As Cyclone Kenneth also influenced the response, depleting already limited human and financial resources,
the recommendations that must ensue from this evaluation ought to have this narrow scope in mind.
IAHE Mozambique – Inception Report | 23
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Quality Assurance
4.1. Quality Assurance
55 The Evaluation Team will aim to ensure that the evaluation is conducted according to principles are inspired by
the United Nations Evaluation Group (UNEG) ethical principles, guidelines, standards (specifically Norm 6 and
standard 3.2.) and the OCHA Quality Assurance System for Evaluations. The evaluation’s design, planning and
implementation processes are inherently quality oriented, covering appropriate methodologies for data collection,
analysis and interpretation. The Evaluation Team includes national evaluators and will seek the participation of
the GoM throughout the evaluation process, as appropriate and possible. National and sub-national level disaster
risk management institutions and local actors have been identified and will be interviewed. In addition, the core
team will observe some of the surveys and FDGs conducted by the Survey Team. This would have the dual
advantage of: (1) fulfilling the quality assurance role; and (2) giving the core team some direct insights from a
community level.
56 To avoid any misinterpretations of the evaluation objective, the team will obtain a letter from OCHA explaining
the evaluation (see annex G). This letter will be presented to provincial and district leaders and will also be used
by both the Core and Survey Teams. The Management Group and Advisory Group also add layers of quality
assurance.
4.2. Ethical Considerations
57 Compensating for potential biases
During the orientation phase at the start of the field work, any potential biases of Evaluation Team members will
be raised so that they could be compensated for when planning interviews, conducting analysis, developing
conclusions and recommendations to help ensure impartiality.
58 Special Considerations
Protection: In line with the ALNAP Guide: Evaluation of Protection in Humanitarian Action47 and the IAHE
Guidelines, the evaluation will consider the extent to which the inter-agency humanitarian response has
mainstreamed protection issues and has considered the protection risks particularly affecting the most vulnerable
people. In the event of interviews or discussions take place with children, the Ethical Research Involving Children
(ERIC) approach48 will for followed.
47 https://www.alnap.org/help-library/alnap-guide-evaluation-of-protection-in-humanitarian-action 48 From Ethical Research Involving Children (ERIC) online compendium, p. 18. Graham, A., Powell, M. Taylor, N., Anderson, D. and Fitzgerald,
R. (2013). Florence: UNICEF Office of Research - Innocenti.
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Annex B – Evaluation Matrix
Evaluation Questions (ToR) Indicators Data Sources Availability and
Reliability of Data
1. Appropriateness
To what extent have the objectives set out in the HRP been based on identified needs, including those of the most vulnerable groups affected by the crisis?50
• Qualitative indicator: degree of coherence and accuracy between the data collected from official sources and planning documents51;
• Number of affected people stating that the response met their most urgent needs; and
• Number of persons from vulnerable groups surveyed stating that the response met their most urgent needs.
Secondary data collected: Official and relief actors’ databases of population in need assessments in three Provinces.52 Document reviews of UN humanitarian actor’s internal information.53 Household (HH) survey results regarding the satisfaction of support received and the usefulness of the support received.54, 55
Unknown.
1.1. To what extent were efforts undertaken to support disaster preparedness before the event? 1.2. After the event, how were the needs assessments undertaken, and to what extent was the information used for response planning? 1.3. To what extent were assessment processes coordinated? 1.4. To what extent do the planning documents reflect identified needs and priorities of affected people, including those of vulnerable groups? [protection question] 1.5. To what extent did the mechanisms actively encourage the
UNOCHA data is reliable.
High. Survey data is reliable.
50 The three strategic objectives from the HRP version of March 2019 are: (1) Provide immediate life-saving and life-sustaining assistance to the population affected by severe food insecurity; (2) Provide immediate life-saving assistance to the population affected by the impact and flooding caused by cyclones Idai and Kenneth; and (3) Support the restoration of livelihoods and strengthen resilience of climate-affected population. 51 The key planning documents to be used as a reference are: (1) the GoM plan; (2) HCT plans; (3) Cluster-based plans; (3) HRP version of March 2019; and (4) other relevant plans. 52 The three provinces are Manica, Nampula, and Sofala. Preliminary reviews suggest that needs assessment data may be limited or insufficient. 53 UNOCHA Mozambique: Cyclone Idai Flash Update Numbers 1, 2, 3, 4 among other documents. 54 The survey is undertaken by Quantitative Team led by Dr. Luis Artur from Universidade de Eduardo Mondlane. 55 This information will likely be disaggregated by gender, region/province, and type of vulnerability.
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Evaluation Questions (ToR) Indicators Data Sources Availability and
Reliability of Data
participation of vulnerable groups in decision‐making processes, including in environmental matters?
2. Effectiveness56
2.1 To what extent were the targets articulated in the HRP achieved (in terms of assistance delivery)? 2.2 To what extent were the targets effective in meeting the needs of the most vulnerable? [protection question]
• Qualitative indicator: degree of coherence and accuracy between the data collected from official sources and planning documents;
Document review and analysis of past evaluations, monitoring reports, internal information, & needs assessments; Community HH survey results used for triangulation; and Interview data from KIIs in Maputo and in the field also used for triangulation.
Moderate. This information will not likely be consistent across all of the agencies. It is for this reason that other data sources will be used for triangulation purposes.
56 This evaluation question has three parts, and so it was split up into three sections to illustrate exactly how each part of the question would be addressed.
IAHE Mozambique – Inception Report | 30
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Evaluation Questions (ToR) Indicators Data Sources Availability and
Reliability of Data
• Number of documents emphasizing the need to consider cross-cutting issues related to the most vulnerable;
• Number of vulnerable persons supported, and cases from the Linha Verde hotline, reached at “1458”;
• Number of respondents stating positive satisfaction levels and positive perceptions of impartiality (includes treatment with respect);
• Qualitative indicator from FDGs about if and how needs of family members with disabilities (including the elderly with mobility problems and children with disabilities) were given sufficient attention.
• Qualitative indicator: stakeholder perceptions about utility and effectiveness of targets.
Document analysis; Community HH survey results; FGD data; and Data from KIIs in Maputo and in the field.
High.
2.3 To what extent has the scale-up activation supported the response (as stated under ‘Purpose’)?57 2.4 Did the response have any unintended, positive or negative effects on affected communities? 2.5 To what extent were strategies, approaches or methodologies executed to protect vulnerable groups, particularly at
• Proportion of respondents stating positive satisfaction with:
o the support received; and o the usefulness of the support received.
• Qualitative indicator: stakeholder perceptions about the effectiveness of the scale-up activation and Humanitarian Programme Cycle (HPC).
Community HH survey data; Document review & analysis; and KIIs in Maputo and in the field.
High.
57 Recalling that one of the purposes of the evaluation will be to assess the extent to which IASC mechanisms, including the scale-up activation and Humanitarian Programme Cycle (HPC), have successfully supported the response.
IAHE Mozambique – Inception Report | 31
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Evaluation Questions (ToR) Indicators Data Sources Availability and
Reliability of Data
the community level? Are any new approaches needed? [protection question]
3. Connectedness
How was the IASC humanitarian system’s emergency assistance for people affected by the crisis linked to longer-term recovery, resilience and development efforts? What, if any, were the challenges in implementing this linkage?
• Proportion of surveyed respondents with positive statements for:
o The response met longer-term needs (e.g., material changes in their lives from assistance).
o The support received was timely;
• Qualitative indicator from FGDs: Perceptions of the process and challenges of returning to normalcy. 58
Community HH survey data; Data from FDGs; and KIIs in Maputo and in the field for triangulation.
High.
3.1 What are the interventions designed to support the transition from recovery to development?
3.2 To what extent has the response contributed to restoring livelihoods (and returning to normalcy) at the community level, and particularly for vulnerable groups? [protection question]
3.3 Is there any evidence that the response helped affected people cope better with subsequent or future cyclones?
4. Coverage
To what extent were different groups of affected people in all locations affected by Cyclone Idai reached with humanitarian emergency aid?
• Qualitative indicator: degree of coherence and accuracy between the data collected from official sources and planning documents in different regions;
• Proportion of surveyed respondents with positive results that:
Document review & analysis of meeting minutes;
High.
58 The focus group question is: To what extent were you assisted in the recovery and reconstruction process? Discuss the response based on the 5 W (What? Where? When? Why? Who does?) And how?
IAHE Mozambique – Inception Report | 32
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Evaluation Questions (ToR) Indicators Data Sources Availability and
Reliability of Data
1.1 To what extent was protection integrated throughout the response efforts for vulnerable groups in all locations? [protection question]
1.2 How did the funding and donor priorities influence the overall coverage?
o The support received was timely; o The response met their longer-term
needs (e.g., material changes in their lives from assistance).
Community HH survey data: cross-tabulations to generate region-based findings;59 and KIIs in Maputo and in the field for triangulation.
5. Partnerships
To what extent have adequate partnerships been established (with international, national and local stakeholders) to deliver assistance to affected people?
• Qualitative indicator: Stakeholder reflections on: o the type of partnerships that were
established; o how relevant the partnership was; o what could be improved to solidify
partnerships in the future.
• Qualitative indicator: Positive perceptions about involvement in the response.
• Proportion of resources received from civil society to engage in the response.
Interview data from KIIs in Maputo and in the field regarding the strength of partnerships at all levels. Budget information from civil society (as compared to the HRP), disaggregating support to international and national NGOs.
Moderate. Perceptions are likely to differ by stakeholder group.
Qualitative Indicator: Positive responses about the level of participation experienced with aid delivery system, examining the process of.
• How community groups were consulted;
• How criteria for receiving assistance was explained;
• How the assistance was delivered and how often?
• How support affected different members of the family.
Data from FGDs. Moderate to high.
6. Localization
59 A crosstabulation of satisfaction levels in different sectors and by region will demonstrate what different groups received emergency assistance.
IAHE Mozambique – Inception Report | 33
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Evaluation Questions (ToR) Indicators Data Sources Availability and
Reliability of Data
To what extent and in what ways have national and local stakeholders (GoM, IINGOs, NNGOs, private sector, military) been involved in international coordination mechanisms, and have their capacities and systems to respond in the future been strengthened through the response?
• Qualitative indicator: Stakeholder reflections on: o Type of training received; o How relevant their contributions were to
support the response; o What could be improved to strengthen
their capacities for future responses.
• See qualitative indicator above about the positive responses about the level of participation that the community may have experienced with aid delivery system.60
• Amount of resources received from civil society to engage in the response.
Interview data from KIIs in the field Community HH survey; and Budget information from civil society (as compared to the HRP).
High.
6.1 How did the response strengthen the capacities of partners to contribute in cross-cutting areas? [protection question]
7. Coordination
How well-coordinated was the assistance, avoiding duplication of assistance and gaps?
o the effectiveness of coordination (including examples where duplications were realized or and avoided);
o timeliness of funding by aid organizations;
o their experiences of coordination at all levels; and
o challenges faced and how they were mitigated (the negative impact of challenges reduced).
o Amount of resources dedicated to cross-cutting issues in the response.
KIIs in Maputo and in the field; Document review [meeting minutes] & analysis for triangulation purposes; and Budget information from civil society (as compared to the HRP).
Moderate. Perceptions are likely to differ by stakeholder group.
7.1 How effective were coordination mechanisms at the strategic, inter-cluster, and cluster/sector levels?
7.2 Is there any evidence that coordination helped to avoid duplications?61
7.3 How effective was the resource mobilization effort in raising sufficient, timely and long-term funding? How did the timing of the funding and
60 It is recognized that there may be some overlap examining partnerships and localization at the community level. This could be further examined during the data collection. 61 What are the challenges in the surge management among UN agencies, including the occurrence of “double-hatting”?
IAHE Mozambique – Inception Report | 34
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Evaluation Questions (ToR) Indicators Data Sources Availability and
Reliability of Data
donor priorities influence the overall response, and in what form of assistance (financial and in-kind assistance) did donors provide assistance? 62
7.4 What major gaps in human and financial resources to carry out activities in cross-cutting areas were identified? [protection question]
62 How (well) did the CERF mechanism function and is this the most effective way to release funds only via UN agencies?
IAHE Mozambique – Inception Report | 35
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Annex C - Instruments for the Review A set of questions and sub-questions have been developed based on the TOR for the evaluation that will be used to
organize results from interviews, focus group discussions and document research. The questions for each instrument
below will provide the framework for the evidence matrix the team will use to collate and analyze data.
Interview Guide (Provincial and District-level INGC)
INFORMED CONSENT SOUGHT
Questions Sub-Questions
Appropriateness In what way have you participated in the response? How were the needs of the most vulnerable identified? What were the biggest vulnerabilities? (please tell us an example) Probe assessments undertaken,
Effectiveness
In your opinion, how effective was the scale up activation and Humanitarian Program Cycle? What were the challenges overall with delivering assistance in this region? How timely do you think the support was? In your view, what is the most important change brought about by the project/response?
Coverage In your opinion, please give your thoughts about whether longer-term needs were met in this province/district (in first 6 months)?
Coordination
Were you in contact with international agencies and/or other partners (civil society, NGOs, communities)? If yes, which ones? [Probe: Coordination mechanism] Was comparative advantage maximized? Were there any instances that you can remember where efforts were duplicated? What aspects of the coordination could be improved?
Partnerships
Describe the different partnerships; What were the top three/five most relevant partnerships? What was the biggest success of your organization and your partners? Why? Give examples. What could be improved to solidify partnerships in the future? How could the international community assist INGC?
Localization
Did you receive any training from the UN? If so, list. [Probe training in the areas of protection – GBV in particular, working with the disabled or older persons with mobility issues) To what extent was this training relevant and useful for your ability to contribute to the response?
Securing additional information
Are there any relevant documents that we should review?
Recommendations for the UN
Do you have any suggestions for improvement for international humanitarian agencies?
Misc. Is there anyone else that you think we should try and speak to?
Misc. Any other comments?
Interview Guide (Provincial and District-level NGOs/Civil Society Service Providers) INFORMED CONSENT SOUGHT
IAHE Mozambique – Inception Report | 36
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Questions Sub-Questions
Appropriateness In what way have you participated in the response?
Effectiveness
In your opinion, how effective was the scale up activation and Humanitarian Program Cycle? What were the challenges overall with delivering assistance in this region? How timely do you think the support was? In your view, what is the most important change brought about by the project that you worked on?
Coverage In your opinion, please give your thoughts about whether longer-term needs were met in this province/district?
Coordination Were you in contact with international humanitarian agencies and/or other partners (civil society, NGOs, communities)? If yes, which ones? [Probe: Coordination mechanism] Was comparative advantage maximized? Were there any instances that you can remember where efforts were duplicated? What aspects of the coordination could be improved?
Partnerships Describe the different partnerships; What were the top three most relevant partnerships? What was the biggest success of your organization and your partners? Why? What could be improved to solidify partnerships in the future? How can the UN assist your organization?
Localization Did you receive any training from the international humanitarian agencies? If so, list. [Probe training in the areas of protection – GBV in particular, working with the disabled or older persons with mobility issues) To what extent was this training relevant and useful for your ability to contribute to the response?
Securing additional information Are there any relevant documents that we should review?
Recommendations for the UN
Do you have any suggestions for improvement for the international humanitarian agencies?
Misc. Is there anyone else that you think we should try and speak to?
Misc. Any other comments?
Interview Guides UN RC/HC and the Mozambique HCT INFORMED CONSENT SOUGHT
Questions Sub-Questions
Appropriateness In what way have you participated in the response?
Effectiveness
In your opinion, how effective was the scale up activation and Humanitarian Program Cycle? What were the challenges overall with delivering assistance in this region? How timely do you think that the support was? In your view, what is the most important change brought about by the project you worked on?
Coverage In your opinion, please give your thoughts about whether longer-term needs were met?
IAHE Mozambique – Inception Report | 37
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Questions Sub-Questions
Coordination
Were you in contact with international humanitarian agencies and/or other partners (civil society, NGOs, communities)? If yes, which ones? [Probe: Coordination mechanism] Was comparative advantage maximized? Were there any instances that you can remember where efforts were duplicated? In what way do you think that the coordination mechanism was strengthened after this humanitarian response? What aspects of the coordination could be improved? Specific areas for probing include:
1. among the HCT members at country level;
1. between and among the HCT and non-HCT and non-GoM partners (e.g.,
national and International NGOs operating within the clusters, representatives
from the private sector with in kind donations and individuals/groups who
functioned within clusters);
2. regional level coordination for HCT members and INGOs which have a regional
presence (CARE, Save the Children, IFRC) , particularly as it relates to surge
management;
3. coordination at the HQ level; and
4. between the IASC and Emergency Management Group and the HCT,
particularly as it relates to raising funds and reporting updates and results to
Describe the different partnerships; What were the top three most relevant partnerships for [agency]? What was the biggest success of your organization and your partners? Why? What could be improved to solidify partnerships in the future? How can international humanitarian agencies be of more assistance to the GoM?
Localization
Did you provide any training? If so, list. [Probe training in the areas of protection – GBV in particular, working with the disabled or older persons with mobility issues] What are the key elements to making training more relevant and useful so that partners may effectively contribute to the response? (Probe other issues besides timeliness)
Securing additional information
Are there any relevant documents that we should review?
Recommendations for the UN
Do you have any suggestions for improvement for international humanitarian agencies? [Engage in some self-reflection here…]
Misc. Is there anyone else that you think we should try and speak to?
Misc. Any other comments?
Interview Guides (Regional Level ) The above interview guide will be used for Regional offices. However, the focus will be about the role that the regional offices ought to play in supporting the HCT and HC/RC to be more empowered decision-makers within the institutional context of different international humanitarian agencies. The surge management will be the main focus of interviews.
Interview Guides (Donors) INFORMED CONSENT SOUGHT
IAHE Mozambique – Inception Report | 38
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Questions Sub-Questions
Appropriateness In what way have you participated in the response?
Effectiveness
In your opinion, how effective was the scale up activation and Humanitarian Program Cycle? What were the challenges overall with delivering financial assistance in this region? How timely do you think that the support that your office provided was? In your view, what is the most important change brought about as a result of funding from this office?
Coverage In your opinion, please give your thoughts about whether longer-term needs were met with the financial assistance provided?
Coordination
Were you in contact with international humanitarian agencies and/or other partners (civil society, NGOs, communities)? If yes, which ones? [Probe: Coordination mechanism] What aspects of the coordination could be improved to support the efficient use of resources?
Partnerships
Describe the different partnerships; What were the top three most relevant partnerships for [your office]? What could be improved to solidify partnerships in the future? How can international humanitarian agencies be of more assistance to the GoM?
Localization
Did you provide any resources for training activities? If so, list. [Probe training in the areas of protection – GBV in particular, working with the disabled or older persons with mobility issues] In your view, what are the key elements to making training more relevant and useful so that partners may effectively contribute to the response? (Probe other issues besides timeliness)
Securing additional information
Are there any relevant documents that we should review?
Recommendations for the UN
Do you have any suggestions for improvement for international humanitarian agencies? [Engage in some reflection here…]
Misc. Is there anyone else that you think we should try and speak to?
Misc. Any other comments?
IAHE Mozambique – Inception Report | 39
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Annex D – Community Focus Group Discussion Guide
HOUSEHOLD SURVEY QUESTIONNAIRE Good morning/Good afternoon. My name is________ and I am here as part of a survey research team from the Universidade Eduardo Mondlane that is gathering information on behalf of the United Nations and its partners to better understand the experience of people affected by Cyclone Idai in March of this year. Your inputs into this survey will be used to help the UN and its partners to improve their work here in Mozambique and in future humanitarian responses elsewhere. If you agree, I would like to ask of you for anywhere from a few minutes to 40 minutes of your time, depending on your experience after the cyclone. Your participation is this survey is voluntary: whether you choose to participate or not participate will not affect any future assistance to you or anyone else, and if there are any of the questions in the survey that you cannot or do not want to answer, you can choose not to answer them. You can also choose to finish the survey at any time. In addition, the information you provide will be strictly confidential: only our research team will have access to the survey data, and the final report on the survey results will not present information on individual survey participants but rather on the feedback provided by the community as a whole. Please feel free to speak to us openly. If you find something wrong with this interview, you can call this number… Give permission to continue [seek informed consent.]
CONFIDENTIALITY AND STATISTICAL AUTHORITY (Law 7/96 July)
ARTICLE 6 STATISTICAL AUTHORITY- The principle of statistical authority consists of the power conferred to the National Institute of Statistics to carry out, in the conduct of statistical activities, obligatory surveys within the time limits set, as well as to undertake steps to produce statistics. Article 14 STATISTICAL CONFIDENTIALITY - All individual statistical information collected by the official statistics production organs bodies are of a strictly confidential nature.
SECTION A: HOUSEHOLD IDENTIFICATION
A000 Questionnaire code
A001 Province
A002. District
A003 Administrative post
A004 Locale
A005 Community
A006 Household Head (AF) name
1 Sofala
2 Manica
3 Tete
4 Zamézia
5 Cabo Delgado
IAHE Mozambique – Inception Report | 40
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
A007 AF Sex 1 Male 2 Female
Disability? 1. Yes 2. No 3. If yes, what kind?
A008 AF Head Contact
A009 Name of the respondent
A010 Name of the Ennumerator
A011 Supervisor Name
A012 Date of interview
A013 Start time
A014 End time A015 Geographic coordinates of the interviewee.
IAHE Mozambique – Inception Report | 41
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
SECTION B: SOCIO-DEMOGRAPHIC CHARACTERISTICS OF THE HOUSEHOLD Before we get started, we would like to get some information about the people who usually live here. Once again, this information is not being collected to identify you; we are collecting this information only to ensure that we are reaching different types of households in our survey, and to enable us to compare the different types of experiences after the cyclone.
B100. AF member name
(Caution: Start filling out the
questionnaire by the head of the AF)
B110. AF member Sex
1 Male 2 Female
3 Other
B111. Relationship with the head of the
AF
1 Self (Head) 2 Spouse/Partner
3 Child 4 Brother/Sister
5 Father/Mother 6 Nephew/Niece 7 Grandson/daughter 8 Other Relative 9 No relation
B112. Age (full years)
B113. Marital status
1 1. Single 2 2. Married 3 4 The Union:
2a. marital 2b. Polygamous
4 5 3. Divorced
/ Separated 4. Widower
B. 114 Can you read and
write?
1 1 Yes
2 No
If no,
skip to
B117
B115. Have You
ever attended school?
1 Yes 2 No
B116. Highest level of schooling
you've reached?
0 No formal school
1 1 ª Classe 2 2 ª Classe 3 3 ª Classe 4 4 ª Classe 5 5 ª Classe 6 6 ª Classe 7 7 ª Classe 8 8 ª Classe 9 9 ª Classe 10 10 ª Classe 11 11 ª Classe 12 12 ª Classe 13 Upper level
19th Literacy 20. Other (Specify)
Questions intended for people aged 10 years and over
1 B.117 What are the sources of income?
2 1 trade
3 2 agriculture
4 3 cattle raising
5 4 other
6
7
8
9
IAHE Mozambique – Inception Report | 42
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
SECTION C. HOUSING CHARACTERISTICS, ACCESS TO WATER, ENERGY AND TELECOMMUNICATIONS Enumerator will make observation of the respondent’s living situation for C100 through
C100. Ownership
status of dwelling
C111. What kind of material was used in the cover of the main
house?
1 Slab/Concrete 2 Tile 3 Zinc Plate
4th Lusalite 5 Grass/ plastic/canvas 6. Other (Specify)
C112. What kind of material was used on the walls of the main
house?
C113. What kind of material was used in building the floor of
the main house?
1 Beaten down earth 2 Adobe 3 The Cement 4 Parquet
5. Other (Specify)
C114. Is the residence of the AF connected to the public/private
electricity network?
1 Yes 2 No
C115. What is the main source of water for the consumption
used by the AF?
1 Public Network 2 Fountain/bore 3 Well 4 River/Lake/Lagoon
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
SECTION D. Early Warning In this section, we have a few questions for you about early warning -- that is, whether and when you were notified in advance of the Cyclone's arrival so that you could act to minimize its effects on your household."
D100. Were you
directly affected by
Cyclone Idai?
1 Yes 2 No
If not, complete inquiry
D101. In the days or hours prior to the cyclone, did you
receive any kind of
advance warning that
it was coming?
1 Yes 2 No
If not, pass to E100
D102. Through what information sources did you receive the warning or notice?
[Mark all that apply]
1 Radio 2 Television 3 Phone number 4 Local Risk 5 Management 6 Committee 7 Local leader 8 Local 9 Government 10 Family /friends
neighbors 11 SMS 12 No, Other
(Specify)
D103. How many days in advance
did you receive
the notice?
(Register the number of days, if it was on the same day place zero)
D104.
Was the warning or notice clear to you?
1 Yes 2 Yes but with some difficulties
3 No
➔ If yes pass to
D106
D105. If the warning was not clear, what
was the reason was?
[Mark all that apply]
1. Very technical language
2. Very vague/not clear
3. Other (Explain )
D106. In general, did you receive the warning or notice in time to minimize the
cyclone’s harm to your household?
1. Yes
2. No
➔ If yes, skip to
E100
D107. Explain what happened because you did not receive the early warning
on time.
IAHE Mozambique – Inception Report | 44
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
SECTION E. Information about Assistance Received/response readiness In this section, we have a few questions for you about they typeof assistance that you have received. --
E100. Did you
need rescuing
after the
cyclone ?
1 Yes 2 No
E101. After approximately
how long were you eventually rescued? (Enter the number of days and if you do not
know to put the Code
98)
(Register the number of days, if it
was on the same day
place zero)
E102. Do you think
the rescue
operation was:
1 Early
2 On
time
3 Late
4 Other
Explain
E103. After the
cyclone's occurrence, what if any
of the following forms of
assistance did your
family need right away?
[Mark all that apply]
1 To be
rescued 2 Shelter 3 Food 4 Clothing 5 Medicine 6 Other
(specify)
E104. Did you
receive any
assistance after the cyclone's
occurrence (that is, within the first 72 hours?)
1 Yes 2 No
E115. What kind of assistance did you receive? [Mark all
5. Education (ex: school material) 6. Agricultural inputs
(ex: seeds) 7. Money (voucher) 8. Building Material
9. Land for construction in a safe area
10. Other (specify)
E116. Who provided assistance/support? [Mark all that apply]
1 Government INGC 2. United Nations institution 3. Red Cross
4. NGOs 5. Religious institution 6. Family/friends/neighbors
7. My community besides
family 8.Do not know
9. Other (specify)
E117. Do you think
the assistance
you received
was 1 Early
2 On
time
3 Late
4 Other
[Each
response from E116
will be asked this question]
E118 Was the
assistance received
according to your
expectations ?
1 Yes 2 No
If yes, skip to F120
E119. IF
E118=No, insert “not met” here
E120. In what specific ways were your
needs met?
[Open Ended question]
IAHE Mozambique – Inception Report | 45
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
SECTION F. Consultation Process In this section, we have a few questions for you about how it was explained to you about receiving the assistance and making any complaint about the overall process.
F100 Before receiving assistance , did anyone ever approach you to try to understand what your needs were?
1 Yes 2 No
F101.a Were the criteria, or means
of determining households' eligibility for assistance,
clearly explained to you?
1 Yes 2 No 3 Don’t recall
F102. Do you think
the criteria used to receive assistance were fair? →If so, go to F104.
1 Yes 2 No
F103. What aspects
of the eligibility
determination process do you think
could have been done
better?
OPEN ENDED
F104.a Was it very clear to you
about the assistance you would receive regarding food
aid? 1.yes 2.no 3. More or less
F105.a If no or more or less, please explain:
OPEN ENDED
F106. At the time, did those
who provided
Assistance to you
treat you with respect
and dignity?
1 Yes 2 No
F107. Did anyone at
any time explain to you
the process by which you
could complain about the
assistance?
1 Yes 2 No 3 Don’t know
F108. In thinking about the overall
process, do you think it was easy for you to get the assistance?
1 Yes 2 No
F109. (ONLY IF “no”) What specifically
was difficult about
accessing the
support?
F104.b When you would receive it?
1.yes 2.no 3. More or less
F105.b If no or more or less, please explain:
OPEN ENDED
F101.b If no, please explain:
OPEN ENDED
F104.c The quantity
You would receive?
1.yes 2.no 3. More or less
F105.c If no or more or less, please explain:
OPEN ENDED
F104.d How often would you receive? 1.sim 2.não 3. More or less
F105.d If no or more or less, please explain:
OPEN ENDED
a) a)
b) b)
IAHE Mozambique – Inception Report | 46
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
c) c)
SECTION G. Impartiality In this section, we have a few questions for you about factors that may have influenced the assistance that you received.
G100. Do you think the assistance you received was distributed fairly?
1 Yes
2 No
If yes, go to G102
G101. (ONLY IF “no”) please tell
me why?
G102. Do you think your gender influenced the help you received? 1. Yes 2. No 3. More or
less
If not, go to G104
G103. If yes or more or less please tell me why?
G104. Do you think your age influenced the help you received? 1. Yes 2. No 3. More or less
If not, go to G106
G105. If yes or more or less please tell me why?
G106. Do you think your ethnicity influenced the help you received? 1. Yes 2. No 3. More or less
If not, go to G108
G107. If yes or more or less please tell me why?
IAHE Mozambique – Inception Report | 47
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
G108. Do you think your level of schooling influenced the help you received? 1. Yes 2. No 3. More or less
If not, go to G110
G109. If yes or more or less please tell me why?
G110. Do you think your physical status (with special needs or not) influenced the help you received? 1. Yes 2. No 3. More or less
If not, go G112
G111. If yes or more or less please tell me why?
G112. Are there political influences that may have helped you to receive the assistance? 1. Yes 2. No 3. More or less
If not, go to G114
G113. If yes or more or less please tell me why?
G114. Do you think your shelter area influenced the help you received? 1. Yes 2. No 3. More or less
If not, go to G116
G115. If yes or more or less please tell me why?
G118. Do you think that the assistance benefited the people who needed it most (most vulnerable people, seniors, disabled people, widows, orphaned children, etc.)?
1 Yes 2 No
G119. In your opinion, do you think there were people who needed the most, but who were left out for the benefit of the less needy?
1 Yes 2 No
G120. If so, who were these people that were left out?
1 Elderly people 2 Orphaned Children 3 Women/Widows 4 People who have lost everything 5 5. Very poor people 6. Other (Specify)
G121. For the next storm, what do you think should be done so that people who need it the most do not go back out?
OPEN ENDED
G122. Should a cyclone like this happen again, what do you think humanitarian agencies should do differently?
OPEN ENDED
IAHE Mozambique – Inception Report | 48
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
SECTION H. Relevance of assistance/support received In this section, we have a few questions for you the appropriateness of the help that you received.
H100. What kind of support did you receive (based on what has already been said)? (Tick all valid options for them)
1. Rescue 2. Power 3. Accommodation 4. Water
5. Medical assistance
6. Household utensils 7. Construction
Material
8. Psychosocial
Apoio 9. Agriculture
Material (e.g. seeds, hoes, etc.)
10. Others (Specify)
H101. Do you think that assistance you received included what you most needed most at that time?
1 Yes 2 No
If yes, go to H103
H102. If not, what kind of assistance would you like to have received?
(Listen to the answers and tick according to
the categories below)
1 Agriculture 2 Shelter 3. Health care 4.Education 5. Food 6. Clothing
7. Water 8. Protection
against threats (ex-infringement, etc.
9. Other (Specify)
H103. Can you tell us
after how long after the cyclone
did you start to receive
the assistance?
(Register
the number of days you received the
first assistance)
H104. Were you satisfied with the quality of
the assistance you
received?[ [Multiple replies] With respect to:
a) Food aid b) Rescue c) Education d) Accommodation e) Water f) Medical care g) Household utensils h) Construction Material i) Psychosocial Apoio j) Agriculture Material (e.g. seeds, hoes, etc.)
1 Yes 2 No 3 more or
less
Food aid 1.yes 2.no 3. More or less
105. If not or more
or less
can you tell us why?
106. Were you satisfied with
the quantities received?[multi ple
replies]
a) feeding b) Accommodation c) Water D) Medical assistance e) Household utensils f) Construction Material g) Psychosocial Apoio (h) Material of agriculture (e.g. seeds, hoes, etc.)
1 Yes 2 No 3 more or less
107. How useful was
the assistance/
support received in helping to
recover in the short term
(immediate)? [Multiple replies]
1. Very useful 2 More or less 3 Not useful
108. How useful was the assistance/
support received in helping to
recover in the long term?
1. Very useful 2 More or less 3 Not useful
Rescue 1.yes 2.no 3. More or less
Education 1.yes 2.no 3. More or less
Water 1.yes 2.no 3. More or less Medical assistance 1.yes 2.no 3. More or less Household utensils 1.yes 2.no 3. More or less Construction Material 1.yes 2.no 3. More or less
IAHE Mozambique – Inception Report | 49
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Psychosocial Apoio 1.yes 2.no 3. More or less
Materials for agriculture 1.yes 2.no 3. More or less
a) a) a)
b) b) b)
c) c) c)
d) d) d)
e) e) e)
f) f) f)
g) …. g) g)
IAHE Mozambique – Inception Report | 50
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
SECTION I. Effectiveness of assistance In this section, we have a few questions for you about factors that may have influenced the impact of the assistance that you received.
I100. How have you been
affected by the cyclone?
1 Death of at least one
family member
2 Injuries to at least one
family member 3. Illness to at least one
family member 4 Destruction of housing 5. Crop loss 6. Loss of animals 7.
Loss of
materials/production
equipment 8 Family displacement 9.Other (specify)
I101. If you have moved, to what
location?
1 Temporary
shelters 2 Family/friends'
home 3. Resettlement Zone 4 Other (Specify)
I102. If you compare your current level of
living with the previous one
before the cyclone, would you say your situation
has…?
1. Worsened 2 Been normal 3. Improved
4. Stayed the
same
I103. If it got worse or better,
why.
I104. What would have happened to your family if you had not benefited from any assistance? [Multiple responses]
1. We would not be alive 2. We would be hungry /
suffering malnutrition 3. We would be sleeping in
the bush
4. Diseases 5. Children would not be
studying 6. It would be difficult to
resume life 7. Other (Specify)
I105. Do you think the assistance received had any positive effect on your family?
1st Yes 2nd No
If not, go to I107
I106. If Yes, please tell me what
were the positive effects?
I107. Do you think the
assistance received had any negative effect on your
family?
1st Yes 2nd No
If not, go to I109
IAHE Mozambique – Inception Report | 51
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
I108. If so, please tell me what the effects were?
I109. Do you think Assistance has helped to
create some kind of conflict(s) in your
community?
1 Yes 2 No
If not, go to I112
I110. If so, what kind of
conflicts?
I111. What do you think should have
been done to avoid this kind of conflict?
I112. If the cyclone were to happen again, do you think your family is better prepared to deal with the
phenomenon?
I1113. Why?
I 1114 – do you have any recommendations for future assistance – what would you do differently?
I 1115 – would you like to be informed on the results of this survey?
We appreciate your time. End
IAHE Mozambique – Inception Report | 52
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Annex E – Evaluation Workplan
IAHE Ethiopia – Inception Report | 53
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Annex F – Letter of Introduction
31 August, 2019
Dear Sir/Madam;
Humanitarian assistance is of the utmost importance for the victims of natural disasters and
other emergencies. As per General Assembly Resolution 46/182, the Inter-Agency Standing
Committee recognizes that Mozambique has the primary responsibility to care for victims
of disasters and other emergencies, including the coordination and implementation of
humanitarian assistance.
The United Nations Humanitarian Team is in the process of conducting an independent
assessment of the extent to which planned collective objectives set out in the Humanitarian
Response Plan responded to the needs and concerns of affected people during cyclones Idai
and Kenneth.
In my capacity as Humanitarian Coordinator, I have the pleasure of informing you that our
office has engaged an Evaluation Team to conduct this evaluation from 1 – 31 September
2019.
The Evaluation Team will have questions related to the relevance, effectiveness, efficiency,
sustainability, scalability, and lessons learned about the humanitarian response.
As we value greatly your contributions that you may provide, we appreciate your
cooperation with these individuals,
Should you have any questions about the evaluation or the Evaluation Team, please feel
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations
Annex G – Informed Consent Form Good morning/Good afternoon. My name is________ and I am here as part of the team that is gathering information to enable us to improve the humanitarian response in the future. You are being asked to take part in a research study. Before you decide to participate in this study, it is important that you understand why the research is being done and what it will involve. PURPOSE OF STUDY: The purpose of this study is to understand if your priority humanitarian needs of were met during the cyclone Idai. We want to know if you were respected; treated with dignity, no sexual harassment, or discrimination. CONFIDENTIALITY: According to the law of Confidentiality and Statistical Authority (Law 7/96 July), your comments will be anonymous [unless you participate in a focus group]. Every effort will be made by the researchers to preserve your confidentiality. Our final report that we will produce will not present any information of one person but of the community as a whole. If you agree, I would like to ask of you about a half an hour (30 min) to 45 minutes of your time to discuss the humanitarian aid that has been provided to you during cyclone Idai. Note that participant data will be kept confidential except in cases where the researcher is legally obligated to report specific incidents. These incidents include, but may not be limited to, incidents of abuse, violence, or emotional risk [the protection protocol must be followed]. NO BENEFITS FOR PARTICIPATION: Your participation is completely voluntary. Our study is not linked to any of these election. We are impartial and independent. We want to understand your opinion and your situation. If you partcipate in this study, it will not influence any opportunity to receive any future assistance. VOLUNTARY PARTICIPATION: It is up to you to decide whether or not to take part in this study. If you decide to take part in this study, you will be asked to sign a consent form. I have read and I understand the provided information and have had the opportunity to ask questions. I understand that my participation is voluntary, and I agree to take part in this study. Participant's signature ______________________________ Date __________ Investigator's signature _____________________________ Date __________
190902_Informed_C
onsent IAHE.docx
INTER-AGENCY HUMANITARIAN EVALUATION OF THE RESPONSE TO CYCLONES IDAI AND KENNETH IN MOZAMBIQUE
IAHE Ethiopia – Inception Report | 55
Inter-Agency Humanitarian Evaluations Steering Group interagencystandingcommittee.org/evaluations