Evaluation of the Winter Cash Programme for Lebanese Poor Children and their Families Evaluation Report Submitted by Awny Amer Morsy Independent Monitoring, Research and Evaluation Consultant Submitted to UNICEF Lebanon- MENA Region Jan, 2017 Disclaimer: The opinions expressed in this report are those of the evaluation consultant who conducted and led the evaluation process that was captured from the views of the project stakeholders interviewed. The evaluation consultant takes responsibility for any errors reported herein that are based on its own independent data collection. [email protected]Egypt Mobile: 002-01020525500 Skype: awny2210
84
Embed
Evaluation of the Winter Cash Programme for Lebanese Poor ... · 1.2 Programme overview- The expected results ... ATM Automated Teller Machine ... Despite the fact that the Winter
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Evaluation of the Winter Cash Programme for Lebanese Poor
Children and their Families
Evaluation Report
Submitted by
Awny Amer Morsy
Independent Monitoring, Research and Evaluation Consultant
Submitted to
UNICEF Lebanon- MENA Region
Jan, 2017
Disclaimer: The opinions expressed in this report are those of the evaluation consultant who
conducted and led the evaluation process that was captured from the views of the project
stakeholders interviewed. The evaluation consultant takes responsibility for any errors reported
herein that are based on its own independent data collection.
Annex 9: Summary overview about the Post Distribution Monitoring (PDM) process ............................ 84
4
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 4
List of Acronyms
ALNAP Active Learning Network for Accountability and Performance
ATM Automated Teller Machine
CMU Central Management Unit
CPD Country Programme Document
CRC Convention on the Rights of the Child
CTP Cash Transfer Programming
DAC Development Assistance Criteria
FGD Focus Group Discussion
GDP Gross Domestic Product
HACT Harmonised Approach to Cash Transfer
HH Household
HRBA Human Rights Based Approach
HRRP Humanitarian Response and Resilience Plan
IT Information Technology
M&E Monitoring and Evaluation
MDG Millennium Development Goal
MEHE Ministry of Education and Higher Education
MoJ Ministry of Justice
MoSA Ministry of Social Affairs
MOU Memorandum of Understanding
NPTP National Poverty Targeting Programme
OECD Organisation for Economic Cooperation and Development
OMR Optical Marketing Recognition
PCM Presidency of the Council of Ministries
PDM Post Distribution Monitoring
PIN Personal Identification Number
PMT Proxy Means Test
SDC Social Development Centre
SDG Sustainable Development Goal
SMS Short Message Service
TOR Terms of Reference
UNEG United Nations Evaluation Group
5
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 5
UNHCR United Nations High Commissioner for Refugees
UNICEF UN Children’s Fund
USD United States Dollar
WFP World Food Programme
6
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 6
0. Executive summary Addressing the seasonal needs of families in Lebanon requires a multi-sectorial approach. During the
harsh winter months, vulnerable families rely on winter cash assistance to address their basic needs, often
at the expense of purchasing other key items. Poor Lebanese and refugee households are highly affected
by inconsistent flows of income, which shift season to season and year to year. Employment is largely
concentrated in the informal jobs market and remains unreliable. In particular, during the winter months,
the availability of work is less which directly impacts the earning ability of vulnerable families (reduced
by 40 per cent as reported by Lebanese poor households1).
Cash Transfer Programming (CTP) has rapidly become an important and often preferred mode of
assistance for humanitarian actors around the world. Organisations operating in areas with conflict-
affected or displaced persons and with functioning markets are increasingly using cash-based assistance
to meet the needs and improve livelihood outcomes of targeted populations.
The Winter Cash programme supports economically vulnerable Lebanese children and their families. It
was designed through a collaborative process between UNICEF, National Poverty Targeting Programme
(NPTP), Ministry of Social Affairs (MoSA), and World Food Programme (WFP), the Office of the Prime
Minister and the Lebanese Presidency of the Council of Ministers (PCM). The objective of the
programme was to introduce cash-based assistance as a pilot to further strengthen the existing social
safety nets and social protection mechanisms in Lebanon. Implementation is the responsibility of the
NPTP, Central Management Unit (CMU) at the presidency of the Council of Ministries and MoSA, with
technical support provided by UNICEF and WFP.
The Winter Cash programme relied on the Proxy Means Test (PMT) score to target the most socio-
economically vulnerable Lebanese households with children. MoSA social workers updated data in the
NPTP database which identifies the most vulnerable households. They visited these households to gather
updated information (specifically the number of children in the household and telephone numbers) which
was then entered into an Optical Marking Recognition (OMR) form, which ranked and selected the most
vulnerable families corresponding to the target of 75,000 children. At the end of this process, 26,052
families were selected as beneficiaries of the Winter Cash programme. A cash grant of USD 40 per child
was transferred to beneficiary households. This value is based on the estimated cost of the winter
clothing kit which was distributed during the 2014/2015 UNICEF winter campaign. The payment was
transferred through ATM cards issued by WFP and Banque Libano-Francaise. Distribution of cards took
place between February 15th and March 10th 2016 by MoSA social workers in 22 selected SDCs, out of a
total 220 existing in the country.
The specific objectives/expected outcomes of the Winter Cash programme were focused on two key objectives - (1) Poor Lebanese children and families provided with the means to cover a portion of their winter expenses to help their children cope with the winter cold in Lebanon exposed to seasonal hazards and unexpected winter emergencies are able to maintain safe access to goods and services; (2) Strengthened delivery mechanisms of the NPTP through the incorporation of new systems such as a Cash-based programme. The overall purpose of the programme
evaluation was to assess (or measure) the outcome and effectiveness of the winter cash programme in
order to strengthen the joint planning, design and implementation capacity of the NPTP, MoSA, PCM and
UNICEF as well as meet the needs of the children by providing appropriate quality assistance in a
dignified manner.
1Poverty, Inequality and Social Protection in Lebanon, January 2016, Oxfam, AUB
7
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 7
The key objectives of the evaluation were guided by OECD/DAC criteria with special focus on assessing
the overall effectiveness of the winter cash modality in Lebanon; and the appropriateness and
acceptance of the programme from the perspective of the target population. Additionally, the evaluation
aimed to assess the efficiency of joint collaboration among the different actors in addition to providing
concrete recommendations for the improvement of planning and implementation of joint programme
targeting for vulnerable Lebanese children.
The key programme beneficiaries (vulnerable Lebanese poor children and their families) were selected
from the following targeted working areas: Bekaa, Mount Lebanon, Nabatieh, Akkar, Tripoli and South.
The key target audience of the evaluation was the NPTP and MoSA as well as other stakeholders
including the PCM, CMU, SDC directors, social workers & field coordinators. WFP in addition to the
households under “Hala” programme having benefited from the programme; children boys & girls and
their families.
Evaluation Methodology:
In order to address the key questions outlined in the TOR, the consultant used formative evaluation
practices combined with participatory approaches, depending on the type and category of the different
stakeholders (including programme beneficiaries). The evaluation process was initiated by carrying out a
desk review of ten of the key documents outlined in the TOR. Thirteen semi-structured individual
interviews were conducted with UNICEF staff and key partners (Five males and eight females). Seven
semi-structured group interviews were conducted with a total of 24 participants. The participatory
approach was applied by conducting Focus Group Discussions (FGDs) to capture the outcomes and key
changes impacting the quality of life of the targeted groups. A total of 90 stakeholders were interviewed
during the evaluation process in addition to 101 who were interviewed during the Post Distribution
Monitoring (PDM) process which was conducted in 2015. Therefore, a total of 191 individuals were
interviewed in order to assess the winter cash programme.
Additionally, the survey that was conducted by a third party during the PDM, using Computer Assisted
Telephone and face to face interviews, was also used in this evaluation as part of the triangulation process
to validate the data collected and the key findings captured. A total of 457 out of 500 beneficiaries were
interviewed in Bekaa, Mount Lebanon, North and South Lebanon.
Key Findings and Conclusions
Relevance
The Winter Cash programme focused mainly on addressing the rights and the needs of Lebanese poor
children and their families in targeted areas across Lebanon. Overall, the evaluation found that the
programme intervention reached the most socio-economically vulnerable Lebanese households and met
most of their basic winter needs.
The evaluation revealed the absence of the capacity building component from programme intervention
strategies. Despite the fact that the Winter Cash programme is a pilot project, this discovery led to the
recommendation for the inclusion of capacity development in future programmes. Capacity needs to be
strengthened to ensure fairness in the provision of cash assistance.
The cash grant of USD 40 per child was found to be insufficient to meet the basic needs of the most
socio-economically vulnerable Lebanese households. Moreover, the programme excluded a large number
of vulnerable families who were also in need of financial assistance due to their harsh economic
conditions owing to the pilot nature of the Winter Cash programme in addition to the available financial
resources.
8
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 8
The selection process and criteria were developed jointly by the NPTP, UNICEF, and CMU/PCM.
However, four major pitfalls were reported – firstly, there lacked clear documentation and understanding
of how the selection process worked. Secondly social workers and field coordinators did not participate in
setting the selection criteria or other relevant questionnaires. Thirdly, the data used in the selection
process relied on outdated sources (as far back as 2011) in addition to omitting some of the important
variables and criteria in the selection process. Lastly, the age of selected children (aged up to 15 years)
was the key selection criterion. The programme did not consider gender as one of the selection variables
and therefore a gender perspective was not incorporated in the targeting process. Although the payment
list of beneficiaries includes the sex of the ATM cardholder, there exists no disaggregated data for the
children supported per family. As a result, no sex-disaggregated data for children who benefitted from the
programme is available.
The programme was successful in identifying the most relevant partners for the project, with each of the
five partners (UNICEF, MoSA, CMU, NPTP and WFP) playing a significant role in bringing an added
value to the implementation of the programme.
Effectiveness
The evaluation revealed that the majority of beneficiaries who took part in the FGDs expressed a high
degree of satisfaction with their participation in and benefits from the Winter Cash programme.
Dissatisfaction regarding the value of the cash transfer was reported amongst a number of beneficiaries
who believed it to be insufficient for meeting their needs. Others suggested that the assistance should be
recurrent on a monthly or quarterly basis throughout the year. The evaluation also found that the
programme did not take into account the special needs of families who have children with disabilities and
did not adequately consider other vulnerable groups such as working women, orphans and families who
do not have children.
Due to a delay in the implementation of the project, ATM cards were not distributed until March instead
of during the previous winter. This hindered the achievement of the main planned objective of the
programme which was “to provide the aid in the winter season” in order to reduce the vulnerability of
Lebanese poor families. UNICEF, with the support of other key partners, developed a well-structured
system to track the distribution cohorts, delivery system process, hotline utilisation, and question and
answer mechanism, and complaint mechanism, which are guided and led by WFP. However, there is a
lack of monitoring and post follow-up activities by social workers and field coordinators to assess the
changes in the lives of the beneficiaries as a result of their participation in the programme and collection
of information on the spending patterns of the cash amounts.
Some beneficiary families (particularly in the Akkar and Mount Lebanon governorates) received their
ATM cards without a pin code. To respond to this issue, a complaint mechanism was introduced.
However, it was weak and the programme ended before the issues were resolved.
The distribution mechanism was considered as one of the key challenges in the Winter Cash programme
especially for those living far from SDC selected distribution points and who incurred additional
transportation fees.
Communication issues relating to the SMS notification which alerted beneficiaries to the distribution of
ATM cards were reported. Many selected families could not understand the SMS for three primary
reasons: (1) They were illiterate, (2) They presumed the message to be an advertisement, and (3) The
model of their phone could not read Arabic messages.
Although the grant assisted families in securing some basic needs for their children, it cannot be
substantiated that the cash grant of USD40 per child resulted in significant changes, or directly impacted
on the lives of the beneficiaries.
9
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 9
Based on these findings, no clear conclusion can be drawn on the success of unrestricted cash transfer as
opposed to restricted cash transfer. Some stated their stance based on their points of view, insights and
previous experiences from other similar programmes with poor vulnerable groups. All partners including
WFP, NPTP, MoSA staff, six SDC directors, five field coordinators as well as 14 out of the 20 social
workers interviewed during the evaluation process expressed their preference for restricted cash transfer
in e-voucher form in exchange for food, clothes or fuel. They indicated that if it was not possible for the
unrestricted cash transfer assistance to be recurrent, it would be better to have the restricted e-vouchers
for food, fuel, clothes or other essentials.
On the other hand, the majority of direct beneficiaries and few of the social workers and field
coordinators interviewed during the evaluation process expressed their preference for
unconditional/unrestricted cash transfers. This corresponds with the results of the PDM survey showing
that the majority of beneficiaries were satisfied with the cash assistance modality (63 per cent)2 compared
to those who were very satisfied (32 per cent) and those who were not satisfied (5 per cent). MoSA staff
and the direct beneficiaries interviewed during the PDM FGDs indicated that the cash modality was
perceived as a better tool, allowing beneficiaries to purchase the essentials most applicable for their
personal needs. Spending patterns of the priority five spending areas3 among the participants interviewed
are highlighted as follows:were: Winter Household expenses (56 per cent), food (55 per cent), education
fees-including the transporation fees to school (30 per cent), repaying of debts (20 per cent), health
expenses (20 per cent), electercity bills (12 per cent), heating- fuel or others (11 per cent) and rent (11 per
cent).
Capacity building activities are crucial for programme development and investment in enhancing the
skills and abilities of social workers in the field is crucial. Not only will this lead to improvement of
programme delivery, but it will also promote ownership of the NPTP.
Efficiency
The successful collaboration between implementing partners demonstrated an excellent example of the
partnerships model. Each of the key stakeholders were well-informed, roles and responsibilities were
agreed upon in advance, and specific measures were set in order to maintain cost-efficiency during each
phase of the programme. Most of the participants stated that they consider the unrestricted cash transfer as
the preferred modality of assistance due to its cost-efficiency, in comparison to the restricted cash transfer
(e-vouchers). The prevailing opinion of beneficiaries, captured during the FGDs, was that the cash value
meant that they felt more respected and were treated “like human beings”.
As the beneficiaries could withdraw the cash grant directly, using ATM cards, the need to engage a third
party was null, thus reducing costs and risk of wasted resources. The programme was less efficient
during the verification phase, which took much longer than planned leading to the delayed delivery of the
cash grant.
Sustainability The absence of a sustainable strategy to address the graduation approach in some of the targeted areas
was unveiled during the evaluation. Lack of skills training and activities aimed at improving
beneficiaries’ ability to secure employment opportunities were not evident. Although beneficiaries
learned how to use the ATMs, illiteracy impacted negatively on the results of this training.
2Statistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 72 3Statistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 69
10
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 10
Coordination
Coordination during each implementation phase was aided by communication between all partners.
Weekly meetings were organised and attending by representatives from each of the five partners to ensure
follow up on the implementation of the programme, resolution of problems in a timely manner, and
maintain effective communication between staff. This mechanism will be referred hereafter as the
“steering committee”. However, they lacked a well-designed management and coordination system with
clearly defined roles and responsibilities of each partner.
Although the SMS as a communication mechanism was very well received by the respondents, MoSA
staff did not approve this mode of communication due to the belief that beneficiaries did not read or
understand the messages. The MoSA staff did not provide full approval of the brochures and flyers being
distributed to beneficiaries for reasons including the fact that social workers had arrived late in this
process and sometimes during the phase of distribution of the ATM cards which means that it was not
helpful and came at the wrong time.
The complaint mechanism was identified as one of the gaps in the programme. Issues such as the absence
of ATM pin codes, deposit of incorrect cash amounts and other technical issues, were not resolved
through this structure. Another key challenge in the coordination of the programme was the poor selection
of distribution centres which resulted in lengthy and difficult travel conditions for many beneficiaries.
Key Recommendations
Below are the key recommendations following the evaluation:
Programme approach, design and interventions Adopting a multi-dimensional approach when developing cash-based programme interventions
within NPTP is highly recommended. This approach comprises an integrated and comprehensive
package of activities and does not focus only on the cash transfer amount as a single activity. This
means that programme developers during the design phase should consider the “Cash for
what?” question, as the cash modality should not be seen as an end in itself or as a stand-alone
intervention.
Special attention should be given to developing a separate, well-structured programme proposal
or concept note for the Winter Cash programme, considering that it is part of the Country
Programme Document (CPD).
A gender sensitive strategy should be devised for the programme. The strategy should state how
the programme will be guided by equality and equity measures during each phase and should
outline the importance of disaggregated data for all the beneficiaries.
The target age group of children who are eligible to benefit from cash transfer assistance should
be revised and extended to include children aged up to 18 years. (In compliance with the
definition of the child adopted by UNICEF).
The cash transfer amount should be revised, ensuring that it is sufficient to secure winter basic
needs and also considers the transportation fees of those who must travel to selected distribution
centres.
Capacity building Investment in capacity building activities for field coordinators and social workers is highly
recommended. Enhancing their abilities and skills in the areas of communication, field work,
monitoring, mentoring, follow-up, data collection, data integrity, data validity, child rights, and
research ethics will lead to greater institutional and human resource sustainability. This could be
organised in the form of a four to five days training workshop to cover basic information and
skills which they can also apply to other projects managed by the NPTP.
Awareness raising sessions should be provided for parents to educate them on how to spend the
cash grant appropriately, focusing on the essential needs of their children.
11
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 11
Selection process In order to gather accurate and updated baseline data, it is recommended to conduct a
vulnerability assessment to inform on specific vulnerability and welfare indictors of families and
communities registered in the NPTP. The information collected will provide important data
which is vital for implementation of similar programmes in addition to the winter cash
programme.
Promoting greater involvement of field coordinators, social workers and other stakeholders in the
selection process and development of selection criteria is recommended in order to strengthen
their sense of cooperation - not only being seen as implementers but as implementing partners.
The questionnaire currently does not include the minimum assessment requirements and
variables. Revision of the questionnaire tool will enable the appropriate identification of
vulnerable poor families referring to the vulnerability variables.
Coordination A well designed communication and coordination plan is recommended. The plan should include
specific milestones and measures outlining the specific roles and responsibilities to guide each
phase of the programme: planning, distribution, monitoring and post-delivery activities.
Establishing unified guidelines and principles before the implementation phase will help to avoid
addressing issues which arise in an ad hoc manner.
Cash modality Joint partnership with other organisations, such as WFP, where each partner complements the
other to address the needs of the NPTP beneficiaries is recommended. This means that UNICEF,
for example, can provide cash assistance while WFP can provide conditioned e-vouchers under
the same programme in a systematic way throughout the year, combining both restricted and
unrestricted cash transfer modalities.
It is highly recommended that NPTP adopts the unrestricted cash transfer, which is considered to
be the most favourable cash modality by a majority of beneficiaries. Certain conditions are
encouraged to be incorporated including a revision of the cash grant amount; adoption of a
recurrent policy throughout the year and replacement of cash grants with an e-food voucher on
quarterly basis.
Monitoring & Evaluation A well-structured M&E system should be developed comprising various methodologies and tools,
as well as community reflection tools which are linked to a combination of qualitative and
quantitative indicators.
Best practices should be supported, documented and shared regularly for enhanced learning
purposes.
Special attention should be given to increasing the involvement of field coordinators and social
workers in the different phases of programme implementation.
Joint partnerships
Developing joint partnerships with other organisations working with the same vulnerable families
registered in the targeted areas is strongly recommended. Through collaboration and utilisation of
data collected during the vulnerability assessment, integrated, joint interventions can be
developed reducing duplication of efforts.
The evaluation suggests investigating the possibility of targeting the winter needs of groups
whose profiles do not meet the current selection criteria or the mandate of partner organisations.
12
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 12
Scaling up opportunities
In order to promote the institutional sustainability of the NPTP and strengthen its ownership of other
similar programmes, special attention should be given to adopt a well-structured strategy that includes
the following sub-strategies:
Develop a comprehensive, well-designed capacity building training in advance of project
implementation.
Provide awareness sessions for targeted parents and/or caregivers to ensure the correct use of cash
transfer assistance to secure the basic needs of their families during the winter season.
Conduct a vulnerability assessment to collect accurate and updated data on vulnerability and
welfare variables.
Promote collaboration with the other organisations working with the same targeted groups to
maximise the use of resources.
Set clear and structured communication and coordination mechanisms for all stakeholders
including key partners and the participating bank. This will increase the efficiency of the
programme; improve timely delivery of activities and help to identify and address challenges
promptly.
Develop a well-structured monitoring and quality assurance framework with clear mechanisms,
methodologies and tools to track the progress during the project lifecycle.
13
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 13
1. Objective of the Evaluation
1.1 Introduction/Background
Today, Lebanon’s four million citizens struggle to withstand the social, economic and political
impact of the Syrian conflict. A stagnant economy, coupled with the threat of insecurity, increases
fears that children and families living in poverty will sink further into deprivation. As a consequence
of the on-going Syrian crisis, population augmentation, of both host and displaced communities has
resulted in gradually shrinking spaces for livelihood and income-generation which significantly
impacts the ability for vulnerable children and their families to secure their basic needs.
In 2016, President Michel Aoun was elected as Head of State after a 29-month political deadlock and
presidential vacuum. Despite a magnitude of challenges, the Government with UNICEF Lebanon’s
financial and technical investments in strengthening public institutions and their delivery systems
contributed to over one million vulnerable children being reached.
Nearly half of those affected by the Syrian crisis are children and adolescents and are currently
growing up at risk, deprived, and with an acute need for basic services and protection4. Vulnerable
families are subject to seasonal hazards and a substantial proportion of these have had to increase
their spending patterns to cover winter needs such as heating fuel, shelter repairs and winter clothing.
Addressing the seasonal needs of families in Lebanon requires a multi-sectorial approach. During the
harsh winter months, vulnerable families rely on the winter cash assistance to address their basic
needs, often at the expense of purchasing other key items. Poor Lebanese and refugee households are
highly affected by inconsistent flows of income, which shift season to season and year to year.
Employment is largely concentrated in the informal jobs market and remains unreliable. In particular,
during the winter months, the availability of work is less which directly impacts the earning ability of
vulnerable families (reduced by 40 per cent as reported by Lebanese poor households5).
Cash Transfer Programming (CTP) has rapidly become an important and often preferred mode of
assistance for humanitarian actors around the world. Organisations operating in areas with conflict-
affected or displaced persons and with functioning markets are increasingly using cash-based
assistance to meet the needs and improve livelihood outcomes of targeted populations.
1.2 Programme overview
The Winter Cash programme supports economically vulnerable Lebanese children and their families.
It was designed through a collaborative process between UNICEF, National Poverty Targeting
Programme (NPTP), Ministry of Social Affairs (MoSA), and World Food Programme (WFP), the
Office of the Prime Minister and the Lebanese Presidency of the Council of Ministers (PCM). The
objective of the programme was to introduce cash-based assistance as a pilot to further strengthen the
existing social safety nets and social protection mechanisms in Lebanon. A cash-based intervention
was considered to be the most relevant and effective mechanism for reaching 75,000 vulnerable
children across Lebanon. Implementation is the responsibility of the NPTP, Central Management
Unit (CMU) at the presidency of the Council of Ministries and MoSA, with technical support
provided by UNICEF and WFP. The programme was designed to be implemented by the NPTP, the
4LCRP 2015-2016 p.9 5Poverty, Inequality and Social Protection in Lebanon, January 2016, Oxfam, AUB
14
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 14
Central Management Unit (CMU) at the presidency of the Council of Ministries and Ministry of
Social Affairs (MoSA) with technical support from UNICEF and WFP. The NPTP established in
2011, aims to support vulnerable Lebanese families in meeting their most urgent needs. To date,
86,000 Lebanese households have been identified as living below the poverty line (USD3.84/day), of
which nearly one-third (25,000 households) live in absolute poverty (below USD2/day)6. The NPTP
provides health and education subsidies to eligible beneficiaries, and food assistance through e-
vouchers is provided to 10,000 (27,000 extremely poor individuals).
The NPTP, PCM and the World Bank developed a Proxy Means Test (PMT) formula to target NPTP
beneficiaries. Inclusion of applicant households was based on a proxy score that calculates the
welfare of individuals by estimating their consumption per day as a proxy measure of income per
day. The test included questions to assess the applicant’s standard of living, such as the employment
status, level of education, marital status, physical ability, housing conditions, assets owned and their
geographical location. It is agreed that if the score is under a certain determined threshold that
corresponds to the Lower Poverty Line, then the household was included as a beneficiary of the
NPTP. This was followed by providing the head of the household or the applicant with a “Halla”
card, which can be used to access the education and health services at any of the 220 SDCs, and local
clinics.
The Winter Cash programme relied on the Proxy Means Test (PMT) score to target the most socio-
economically vulnerable Lebanese households with children. MoSA social workers updated data in
the NPTP database which identifies the most vulnerable households. They visited these households
to gather updated information (specifically the number of children in the household and telephone
numbers) which was then entered into an Optical Marking Recognition (OMR) form, which ranked
and selected the most vulnerable families corresponding to the target of 75,000 children. At the end
of this process, 26,052 families were selected as beneficiaries of the Winter Cash Programme. A cash
grant of USD 407 per child was transferred to beneficiary households. This value is based on the
estimated cost of the winter clothing kit which was distributed during the 2014/2015 UNICEF winter
campaign. The payment was transferred through ATM cards issued by WFP and Banque Libano-
Francaise. Distribution of cards took place between February 15th and March 10th 2016 by MoSA
social workers in 22 selected SDCs, out of a total 220 existing in the country.
Expected outcomes/results
Subsequent to the Winter Cash programme project proposal, the Humanitarian Response and
Resilience Plan 2016 (HRRP) and other project documents, the following key expected outcomes
were developed: which targets
Key Objectives/ Expected Results (medium and short-term results)
Outcome 1:
Lebanese children and the families vulnerable to seasonal hazards and unexpected winter
emergencies are able to maintain safe access to goods and services.
Expected Outputs:
1.1: Households in the targeted areas at risk of seasonal and unexpected hazards survive without
6 Poverty, Inequality and Social Protection in Lebanon, January 2016, Oxfam, AUB 7 The US$40 was the monetized amount of the children’s winter clothing kits and e-vouchers which UNICEF had provided to children of NPTP
support families in previous years.
15
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 15
adverse effects.
Planned activities
1.1.1: Provide cash grants for seasonal hazards affected households (USD40/child in the targeted
families)
Outcome 2:
Strengthened delivery mechanisms for the NPTP via the incorporation of new systems: Cash-
based programme
Expected Outputs:
2.1: Strengthened child-focused delivery mechanisms for NPTP via the incorporation of new child-
sensitive systems, including cash-based programmes
Planned activities
2.1.2: Conduct a joint study with MoSA/NPTP on outcomes and impacts of winter cash-based
programme for socio-economically vulnerable children and their households.
1.3 Key programme stakeholders – including key partners
Table 1 below shows a brief mapping of the stakeholders involved in the programme. It lists the
different key actors/stakeholders – including the key partners who were targeted during the data
collection phase.
# Key stakeholder Actors involved
1 UNICEF Lebanon
staff
UNICEF Programme Specialist Cash and Vouchers
UNICEF Winter Focal Points
UNICEF Chief of Field Operations
2 NPTP Director of the NPTP
NPTP Business IT
SDC directors
3 MoSA MoSA Winter Cash Transfer Programme Focal Point
MoSA Field work coordinators
MoSA social workers
Adviser of the Minister of Social Affairs
4 PCM/CMU CMU Director
PCM/CMU Statistician
5 WFP WFP Cash and Vouchers Unit Programme Associate(s)
6 Direct beneficiaries Sampled groups from the poor vulnerable families (parents and
caregivers)
Table1. Key Stakeholders
16
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 16
2. Evaluation Purpose, Objectives and Scope of Work
Referring to the agreed scope of work with UNICEF & other stakeholders, UNICEF commissioned
the consultant to conduct “The final evaluation of the winter cash transfer programme for
Lebanese poor children & their families”.
2.1 Overall Goal of Evaluation
The overall goal of the programme evaluation is to assess (or measure) the outcome and
effectiveness of the Winter Cash programme in order to strengthen the joint planning, design and
implementation capacity of the NPTP, MoSA, PCM and UNICEF as well as meet the needs of the
children by providing appropriate quality assistance in a dignified manner.
2.2 Specific Evaluation Objectives
In order to provide a robust evaluation of the joint NPTP / UNICEF winter cash response, the
specific objectives of the programme evaluation are as follows:
Assess the overall effectiveness of the Winter Cash programme modality in Lebanon.
Assess the appropriateness and acceptance of the Winter Cash programme modality from
the perspective of the target population.
Assess the joint collaboration of the overall implementation of the Winter Cash programme.
Provide concrete recommendations to improve planning and implementation of joint programmes
and deliver optimal results for the most economically vulnerable Lebanese children.
2.3 Scope of Work
In order to achieve the specific objectives, the evaluation covered the entirety of the programme
including the following components (in reference to the project cycle process):
The design & planning phase: This includes the selection process, payment modality and
repartition of the duties and responsibilities among the different actors as well as the planning
phase.
The Implementation phase: This includes all training conducted, distribution of ATM
cards, feedback mechanisms and monitoring.
It is important to note that the scope of work in the current evaluation focused mainly on poor
vulnerable Lebanese families only and did not consider Syrian and Palestine refugees addressed
under the winter HRRP-2016.
17
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 17
The geographic scope of the evaluation:
The evaluation focused primarily on the targeted areas of programme implementation shown in Table
2 below.
The key programme beneficiaries (vulnerable Lebanese poor children & their families) were selected
from the following targeted working areas8:
S Governorate Cazas Targeted # Of SDCs involved
1 Beirut Beirut 4
Sub-Total 4
2 Bekaa Baalbek 9
Hermel 1
Rashaya 4
West Bekaa 4
Zahle 3
Sub-Total 21
3 Mount-Lebanon Aley 6
Baabda 6
Chouf 10
El Metn 5
Jbeil 4
Kesrouan 5
Sub-Total 36
4 Nabatieh Hasbaya 1
Marjayoun 3
Nabatiyeh 3
Sub-Total 7
5 Akkar Akkar 10
Batroun 4
Bchare 1
Koura 2
Sub-Total 17
6 Tripoli Minieh-Dannieh 3
Tripoli 3
Zgharta 2
Sub-Total 8
6 South Bint Hbeil 4
Jezzine 1
Saida 5
Sour 5
Sub-Total 15
Total 26 108
Table2. Geographic scope of the evaluation
8List of Distribution of UNICEF_NPTP beneficiaries by Governorate_Gaza_SDC.
18
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 18
Time period covered by the evaluation: The evaluation covered the period between August 2015
and May 2016, starting from the first meetings held between PCM, MoSA, WFP and UNICEF
pertaining to the design of the programme and ending with the last Post Distribution Monitoring
(PDM) process.
2.4 Evaluation Criteria & Questions
With reference to the TOR and the approved inception report, the evaluation is primarily based on
the DAC criteria with a special focus on Relevance (including appropriateness), Effectiveness
(including acceptance), Efficiency, Sustainability and Coordination reflected in the key questions
outlined in Table3 below.
Evaluation
criteria Key questions / areas of discussion
Relevance
(including
Appropriateness)
1. To what extent were the project intervention strategies, project design and
approaches appropriate and relevant to promoting the intended objectives of
the programme? And to what extent were the needs of the most socio-
economically vulnerable Lebanese households met?
2. To what extent were the criteria of the selection process developed to ensure
that the most vulnerable and poor children were reached and benefited
properly from the programme?
3. To what extent has the programme been successful in identifying the most
relevant partner/actors for this project and what was the added value of these
partnerships?
4. How gender sensitive was the programme when reaching the most
vulnerable children?
5. Do the current Log frame and/or results framework and relevant indicators
in the Winter Cash programme document need improvement to better serve
the programme objectives?
Effectiveness
6. To what extent were the vulnerable children and their families targeted by
the Winter Cash programme reached (as planned)? How satisfied were they
about the programme?
7. To what extent did the programme have an impact on beneficiaries in
improving access to essential goods and services of their choice in a safe,
dignified, and empowered manner? (Key changes affecting the lives of
beneficiaries as result of the programme)?
8. Are there other unintended or unforeseen changes evident in the lives of the
targeted groups of poor children & their families? Are there any unintended
beneficiaries of the programme?
19
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 19
9. What was the level of acceptance of the cash modality as a tool to support
poor Lebanese households to cover their winter needs (winterisation) from
the point of view of all the different actors: MoSA, NPTP, SDCs, field staff
and the beneficiaries?
10. To what extent were the capacities of the different actors involved -
particularly the NPTP and MoSA - adequate and efficient to ensure the
quality of the programme implementation? What can be improved in this
regard?
11. To what extent does the programme have an effective monitoring and
evaluation mechanism (including reporting, quality assurance and
reflection)? Are there proper processes on different levels (UNICEF, NPTP,
and CMU/PMC) in place? How has the programme used information
generated to inform and make programmatic adjustments & corrective
actions during the life of the project?
12. To what extent did the project consider gender equality in the programme
implementation – including the selection process (e.g. is USD40 an
equitable amount for both boys and girls? On what basis was the amount
calculated and does it take into account gender-based requirements?)
Efficiency 13. To what extent was the Winter Cash programme cost-efficient – including
optimal usage of programme resources & time efficiency as opposed to
other modalities/programmes that could be designed to cover the same
needs of the Lebanese vulnerable poor children and their families?
Sustainability 14. To what extent did the programme equip service providers (NPTP/MoSA)
and beneficiaries with the required capacities (skills, knowledge and
sustainability), resources (connections) and confidence (institutional and
financial sustainability) to sustain the programme implementation?
Coordination
15. How effective was the management and coordination system - as part of
joint programme partnership between UNICEF, NPTP, MoSA, CMU/PCM
and WFP - in achieving results for the target population?
Table3. Evaluation Criteria and Key questions / areas of discussion
2.5. HRBA, Gender Equality and Equity Principles in the Current Evaluation The current evaluation took into consideration the following key Human Rights Based Approach
(HRBA), gender equality and equity principles to ensure that they were mainstreamed in the
evaluation process as follows9:
9 United Nations Evaluation Group (2014). Integrating Human Rights and Gender Equality in Evaluations. New York: UNEG- page 32.
20
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 20
Inclusion: The evaluation investigated which groups benefitted from the programme and
targeted them during the assessment process. The evaluation developed disaggregated data
of all the groups interviewed as shown in Annex 2 of this report. Participation: The evaluation adopted and followed a participatory evaluation approach,
applying methodologies and data collection tools engaging the programme stakeholders in a
consultative manner. The evaluation measured the level of their participation and
involvement in the project design, planning, and monitoring activities. Fair power relations: The evaluation focused specifically on assessing the degree to which
power relations among the different programme stakeholders and implementers supported or
undermined programme results. Additionally, the evaluation considered the status of groups
in the targeted population within the programme.
3. Evaluation Methodology
3.1 Overview
With reference to the TOR, the evaluation process was guided by the following four internationally
recognised standards:
Utility – Evaluation findings and recommendations will serve the different stakeholders.
Feasibility – Realistic, prudent, diplomatic and frugal.
Propriety – Conducted legally, ethically and with due regard for the welfare of those involved in the
evaluation process, as well as those affected by its results.
Accuracy – Reveal and convey technically adequate information about the features that determine
the worth or merit of the programme being analysed in the evaluation process.
3.2 UNEG Norms and Ethical Standards
On the other hand, the evaluation was guided in all its processes with the key UNEG norms and
ethical standards10 as follows:
Intentionality of evaluation - by ensuring the utility and necessity of the information and
results captured from the current evaluation for the different stakeholders and target
audiences.
Key obligations11 to the evaluator:
Independence - both behavioural and organisational independence.
Impartiality- objectivity, professional integrity and absence of bias.
Credibility – transparent evaluation processes and inclusive approaches involving relevant
stakeholders.
Conflict of interest- Ensure neutrality in all activities related to the programme design,
planning and implementation.
Honesty and integrity
Accountability
10Linda G. MorraImas&Ry C. Rist: The Road to Results- The world bank – Washington- 2009- Page 509-510 11United Nations Evaluation Group (2016). Norms and Standards for Evaluation. New York: UNEG- page 11-12.
21
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 21
Key obligations to participants:
Respect for dignity & diversity
Rights
Confidentiality
Avoidance of harm
In order to address the key questions outlined in the TOR, the consultant used formative evaluation
practices combined with participatory approaches when engaging the different stakeholders and
beneficiaries.
As the programme was a pilot project, the scope of the evaluation was mainly focused on the
learning purpose rather than applying it for accountability processes.
The evaluation process was initiated by conducting a desk review of ten key documents
including: the TOR; basic assistance/winter HRRP 2016; implementation plan; PDM final
products; distribution documents; forms and sheets; lessons learnt document as part of the
PDM process; complaint mechanism document; and NPTP card management in addition to
the respective questionnaires. (See Annex 3).
Thirteen semi-structured individual interviews were conducted with relevant UNICEF staff
and key partners. (Five males and eight females).
Seven semi-structured group interviews were conducted with a total of 24 participants (Five
males and nineteen females) with relevant MoSA staff in the five targeted working areas
located in four governorates.
One group interview took place with relevant WFP staff.
The participatory approach was applied by conducting FGDs to capture the outcomes of
programme activities and any significant impact on the quality of life of the beneficiaries.
Five FGDs were facilitated with five sample groups of beneficiaries from five working areas
affiliated to five SDCs located in four governorates with a total of 53 parents & caregivers.
The current evaluation used eight FGDs (men and women) and four key informant interviews
(with SDC directors, social workers and field coordinators) which were conducted last year
during the Post Distribution Monitoring process.
Ninety stakeholders were interviewed in the current evaluation process in addition to the 101
who were interviewed during the Post Distribution Monitoring process last year. A total of
191 were therefore interviewed to assess the Winter Cash assistance programme. Those
interviewed were involved in the programme either as partners or as direct programme
beneficiaries (Refer to Annex 2 for more details).
Additionally, the survey that was conducted by a third party during the Post Distribution
Monitoring - using Computer Assisted Telephone and face to face interviews - was also used
in this evaluation. A total of 457 out of 500 beneficiaries were interviewed in Bekaa, Mount
Lebanon, North and South. Triangulating the data to draw comparisons from the information
was essential in order to validate the findings.
Table4 below shows the number of stakeholders interviewed and consulted during the current
evaluation process disaggregated by sex and type of evaluation methodology/tool. The details of this
list are presented in Annex 2.
22
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 22
S Type of evaluation
methodology/tool
# of
interviews/FGDs
Classification of participants by
sex
Male Female Total
1 Individual semi-structured interviews 13 5 8 13
2 Group semi-structured interviews 7 5 19 24
3 Focus Group Discussions 5 16 37 53
Total 25 26 64 90
Table4. Type of evaluation methodology/tool
3.3 Data Collection Methodologies and Tools
3.3.1. SOURCES OF DATA COLLECTION
The following are the key sources of information referred to in the current evaluation process:
- Information collected and captured from FGDs with direct beneficiaries.
- Information collected from the individual and group interviews conducted with the key
stakeholders including the main partners of the Winter Cash Programme.
- Post Distribution Monitoring (PDM) findings.
The following key criteria were applied in order to select the sample of SDCs that were to
facilitate the interviews and FGDs:
The SDCs which have the highest number of the beneficiaries referring to the above actual
distribution schedule.
Ensure that the sample of SDCs, representing each of the rural/semi-rural/urban/semi-urban
and/or slums, is useful for analysing the spending patterns and sufficiency of the cash transfer
amount to secure their winter basic needs.
This sample was developed after the initial recommendation from UNICEF o and the adviser of the
NPTP and MoSA staff.
The following criteria were applied for the selection of participants in FGDs: The
participants were selected from the beneficiaries of the Winter Cash programme in each
SDC, ensuring that all targeted working areas were represented under the SDC.
The number of children per household who benefitted from the Winter Cash programme was
considered during the selection process to ensure a diverse group of beneficiaries from those
with only one child to those with several children aged up to 15 years. This proved to be
helpful in capturing different views during the evaluation process.
Access to targeted working areas was a factor due to security issues in some areas.
Other basic criteria are outlined in the FGD guide included in Annex 4.
23
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 23
3.3.2. DATA COLLECTION TOOLS
A. SEMI-STRUCTURED INTERVIEWS (GROUP AND INDIVIDUAL INTERVIEWS)
The evaluation included a series of semi-structured interviews with key partners and stakeholders
involved in the programme implementation. These interviews were conducted with NPTP, MoSA,
WFP, SDCs, field coordinators, Social Workers, and CMU/PMC in addition to UNICEF. Participants
were selected based on TOR requirements and consultations with UNICEF and the desk review
findings. The selected individuals were interviewed for their first-hand knowledge about the
programme and their involvement in the programme implementation itself.
The interviews were roughly structured with a list of topics for discussion, and an interview guide to
allow a free flow of ideas and information. The consultant framed questions spontaneously, probed
for information and took notes accordingly. (Details of the interviews are available in the tools’
guides & protocol section in Annex 4).
Twenty interviews (13 individual interviews and 7 group interviews) were conducted with a total of
37 stakeholders (Refer to Annex 2 for more details).
B. FOCUS-GROUP DISCUSSIONS (FGDS)
The consultant facilitated FGDs in four selected governorates across Lebanon. The purpose of the
FGDs was to capture the most significant impacts, if any, of participation in the programme and to
gain a better understanding of their opinions, insights, ideas, and challenges encountered, and
recommendations based on their level of satisfaction.
The advantage of FGDs is that they allow for open dialogue and discussions that helps to gather
insightful information for the evaluation. .
Table 5 below shows the number of beneficiaries/households (HHs) per governorate, Caza and SDC
and was used as a guiding reference to identify the targeted working areas and the beneficiaries who
took part in the FGDs.
S Governorate Caza Targeted # Of SDCs
involved
# of HHs
benefitted
1 Beirut Beirut 4 46
Sub-Total 4 46
2 Bekaa Baalbek 9 2982
Hermel 1 534
Rashaya 4 122
West Beqaa 4 552
Zahle 3 572
Sub-Total 21 4762
3 Mount-Lebanon Aley 6 737
Baabda 6 921
Chouf 10 306
El Meten 5 33
Jebeil 4 69
24
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 24
Kesrouan 5 105
Sub-Total 36 2171
4 Nabatieh Hasbaya 1 8
Marjayoun 3 3
Nabatiyeh 3 149
Sub-Total 7 160
5 North Akkar 10 7568
Batroun 4 124
Bchare 1 59
Koura 2 182
Minieh-Dannieh 3 3085
Tripoli 3 4959
Zgharta 2 679
Sub-Total 25 16656
6 South Bint Hbeil 4 322
Jezzine 1 4
Saida 5 1069
Sour 5 862
Sub-Total 15 2257
Total 26 108 26052
Table 5. Number of beneficiaries/households per governorate, Caza and SDC
The following table (Table 6) shows the results of the selection process and number of beneficiaries
per target area. The consultant selected four governorates with the highest number of HHs that
benefitted from the programme. Referring to Table 5 above, the key specific areas in the targeted
governorates with the highest administrative level and highest number of beneficiaries were: Baalbek
Having a complete form of the theory of change that combines between vertical & horizontal
logic template and the traditional logical framework matrix is helpful in humanitarian aid
programmes in order to track whether the outputs are achieved and to identify the means of
verification through which data and evidence are communicated to confirm the indicators. In
addition, risks and assumptions that might be faced or encountered as a result of external
factors during the implementation of the program should be stated.
A gender component should be devised for the programme, and managed and led by NPTP.
This should state how the programme will be guided by equality and equity measures relating
to gender during the different phases and should include the need for disaggregated data. (Relevance & appropriateness findings#4-page 33 + Effectiveness findings # 12-page46)
The age of eligibility of children should be revisited, with the possibility to extend assistance
to children aged up to 18 years (in compliance with the definition of a child adopted by
UNICEF) and/or agree jointly on the maximum number of children who will be supported
per family considering the available financial resources. (Relevance & appropriateness findings#1-
pages 28-29)
The cash transfer amount should be revised to ensure that it is adequate to secure basic winter
needs of vulnerable children. Consideration should also be given to the supplementation of
transportation fees of those who travel to collect the ATM card from their identified SDC.
)Effectiveness findings # 6-page 36)
Capacity building
It is highly recommended that the NPTP provide capacity building activities for field
coordinators and social workers and invest in raising their capacities and skills in the areas of
communication, field work, monitoring, mentoring, follow-up, data collection, data integrity,
data validity, child rights, and research ethics. With these skills, they can play a vital role in
updating the existing data regarding vulnerable families in Lebanon and work as a permanent
community M&E team not only for the Winter Cash programme but also for other future
programmes. This could be organised in the form of a 4-5 day training workshop to cover the
57
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 57
basic information and skills under each of these areas. Investing in capacity development
will also promote enhance institutional and human resources sustainability. (Relevance &
Immediate resources should be allocated by the NPTP to develop a well-structured
vulnerability assessment for the communities and families registered in the NPTP programme
to gather accurate baseline data that can be referred to when needed in other similar
programmes.. This requires the compilation of accurate, complete and updated data that
addresses all the vulnerability and welfare indicators (employment and income status,
number of family members, illness status, disability, gender variables, education status,
orphans’ status, house conditions and others).The NPTP has already started this process. (Relevance & appropriateness findings#2-pages31- 33 + Effectiveness findings pages 35-37)
It is highly advised to promote and increase the involvement of the field coordinators, social
workers and other actors in the SDCs in the selection process starting from jointly developing
the selection criteria and including the SDCs in the implementation phase as partners and not
only as implementing mechanisms. (Relevance & appropriateness findings#2-page 32)
The questionnaire must be revised as it currently does not include the minimum assessment
requirements and variables; this will enable the accurate identification of vulnerable poor
families referring to specific vulnerability variables. (Relevance & appropriateness findings#2-page
31 + Effectiveness findings #1- page 37)
Coordination
It is recommended to continue working with the key partners following the format of the
steering committee - which represents all the partners and different actors. - A
communication and coordination plan outlining specific roles and responsibilities to guide
them during the different phases of the programme should be developed. The steering
committee should meet regularly and on a timely basis to immediately deal with the
challenges and problems encountered. UNICEF has announced that these meetings will
If in the future the restricted cash transfer (e-vouchers) - based on the recommendations of
some groups interviewed in the current evaluation process – is adopted, it is highly
recommended to engage in a joint partnership with other organisations such as WFP and
UNHCR where each partner can complement the other to address the needs of beneficiaries. (Effectiveness findings # 9- pages 41-42 + Annex 8 attached to this report)
It is highly recommended that NPTP continues to implement the unrestricted cash transfer,
which is the preferred cash modality of the majority of beneficiaries, but with some
conditions/or alternatives included:
58
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 58
Ensure the selection process reaches the poorest vulnerable families fairly.
Provide the cash transfer on a monthly or quarterly basis to secure basic needs.
Increase the cash transfer amount significantly in order to secure basic needs especially in the
winter season when employment opportunities are scarce. (Effectiveness findings # 9- page 42 +
Annex 8 attached to this report)
The majority of the participants indicated that they prefer the e-food voucher if they can
receive it regularly and not once per year, as it represents the best e-voucher modality
compared to the e-voucher for fuel or clothes as mentioned in the report. This preference is
explained by the fact that they need to have control in the malls to ensure that they deliver
food items not other non-food items in a corrupt manner. (Effectiveness findings # 9- pages 41-42 +
Annex 8 attached to this report)
Monitoring & Evaluation
Special attention should be given at the NPTP level to developing a well-structured M&E
system that comprises methodologies and tools, including some of the community reflection
tools linked to a combination of qualitative and quantitative indicators.. Accordingly, the
selected field coordinators and social workers in each community/area per SDC will work as
a community M&E team by incorporating a reflection and reporting component when
meeting with the direct beneficiaries and make corrective action when needed. (Effectiveness
findings # 11- pages 43-44)
Best practices should be supported and documented. These examples or success stories can
be shared with others for learning purposes. (Effectiveness findings # 11- pages 43-44)
Special attention should be given to increasing the involvement of the field coordinators and
social workers in the different phases of programme implementation. (Effectiveness findings #
11- page 45)
Joint partnerships It is recommended to develop joint partnerships with organisations who work with the same
vulnerable families registered in the NPTP in the targeted areas. Through this partnership, the
results of the vulnerability assessment can be used and the areas of programme interventions
can be jointly developed referring to the scope of work of each organisation. Based on such
practice, duplication of services can be avoided and organisations can complement each other
through an integrated approach. (Relevance & appropriateness findings#2-pages 31- 33)
The evaluation suggests investigating the possibility of targeting the winter needs of some
groups whose profile might not be consistent with the scope of work or the mandate of some
of these organisations. (Effectiveness findings # 6- page 36)
Scaling up opportunities In order to promote the institutional sustainability of the NPTP and strengthen its ownership in other
similar programmes, special attention should be given to adopt a well-structured strategy that
includes the following sub-strategies (Sustainability findings # 14-page 50):
Implement a comprehensive well-designed capacity building training in advance, before the
beginning of the project implementation.
Provide awareness sessions for targeted parents and/or caregivers to ensure that they will use
the cash transfer assistance appropriately and secure the basic needs of their children during
the winter season.
Conduct a vulnerability assessment to collect accurate and updated data on vulnerability and
welfare variables.
59
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 59
Promote integration with the other organisations working with the same targeted groups to
maximise the optimal use of financial resources.
Set clear, structured communication and coordination mechanisms among the different actors
including the key partners and the bank. This will be helpful in effective implementation of
programme activities and in a timely manner and identify and manage gaps.
Develop a well-structured monitoring and quality assurance framework with clear
mechanisms, methodologies and tools to track the progress of the programme.
60
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 60
Annexes
Annex 1: The Terms of Reference (TOR)
61
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 61
62
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 62
63
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 63
64
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 64
65
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 65
66
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 66
67
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 67
68
The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017 Page 68
Annex 2: List of stakeholders interviewed
S Name Sex Position Affiliation /
Governorate
Date of
interview/F
GD
M F Total
(1) Key partners ( Individual & group interviews) 1 Mrs. Marie Camaghia 0 1 1 Winter Cash programme Focal