WB Targeting assessment of cash transfer program in Gaza and West Bank

Document of the World Bank

Report No: ACS890 .

West Bank and Gaza Targeting Assessment of the Cash Transfer Program

. June 25, 2012

. MNSSP MIDDLE EAST AND NORTH AFRICA .

2

Standard Disclaimer: .

This volume is a product of the staff of the International Bank for Reconstruction and Development/ The World Bank. The findings, interpretations, and conclusions expressed in this paper do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. . Copyright Statement: . The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development/ The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, http://www.copyright.com/. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail [email protected].

3

ACKNOWLEDGEMENTS

This technical assistance was prepared under the overall guidance of Mariam Sherman (Country Director, MNC04), by a multi-disciplinary team led by Task Team Leaders Samira Hillis (Human Development Coordinator for West Bank and Gaza (WBG), MNSSP) and Phillippe Leite (Economist, HDNSP) and including Diane Steele (Household Survey Coordinator, DECPI), Suha Rabah (Program Assistant, MNC04) and Laura McDonald (Consultant, MNSSP). We would also like to thank peer reviewers Pablo Ariel Acosta (Economist, LCSHS), Nandini Krishnan (Economist, MNSED) and Cem Mete (Senior Economist, SASSP) for their helpful comments and guidance. The study was conducted by two technical firms: Al Markaz for Development and Marketing Consultancies in the West Bank and Engineering & Management Consulting Center (EMCC) in Gaza. The report was written by Phillippe Leite. The report was further revised, modified and edited by Samira Hillis and Laura McDonald. The team wishes to thank H.E. Majida Al Masri, Minister of Social Affairs and the Ministry of Social Affairs (MoSA) staff, including the social workers, who played an important part in facilitating data collection in the field. We would also like to thank the Palestinian Central Bureau of Statistics (PCBS) for their support of this effort. The team is also extremely grateful to the support of various important partners who have supported this effort from the very beginning and with whom we look forward to continued collaboration including: the European Union (the largest donor of the Cash Transfer Program), United Nations Relief and Works Agency for Near East Refugees (UNRWA) and Food and Agriculture Organization (FAO). In addition, we would like to express our gratitude to the World Food Programme (WFP), who provided important continual feedback throughout the study process and contributed greatly to the design of the survey instrument and the development of the training manual. Further, we are very grateful to the consulting firms who spent countless hours collecting data in the field: Al Markaz for Development and Marketing Consultancies in West Bank and EMCC in Gaza. We are also very appreciative to households who participated in this survey and provided the data for this report.

4

ACRONYMS

CTP Cash Transfer Program EU European Union FAO Food and Agriculture Organization MoSA Ministry of Social Affairs PA Palestinian Authority PCBS Palestinian Central Bureau of Statistics PECS Palestinian Household Expenditure and

Consumption Survey PMTF Proxy Means Test Formula SHC Special Hardship Case Program SSN Social Safety Net SSNRP Social Safety Net Reform Project UN United Nations UNRWA United Nations Relief and Works Agency for

Near East Refugees WBG West Bank and Gaza WFP World Food Programme

Vice President Inger Andersen Country Director Mariam Sherman Sector Director Steen Jorgensen Sector Manager Yasser El-Gammal Task Team Leader Samira Ahmed Hillis

5

TABLE OF CONTENTS

I. Introduction ............................................................................................................................................... 7

II. The Cash Transfer Program (CTP) .......................................................................................................... 9

III. Data ...................................................................................................................................................... 11

IV. Assessment tools .................................................................................................................................. 13

V. Findings ................................................................................................................................................. 15

V.a. Impact on poverty ........................................................................................................................... 15

V.b. Sensitivity-specific analysis ........................................................................................................... 17

V.c. Beneficiary (Benefit) Incidence Analysis ....................................................................................... 19

VI. Conclusion ............................................................................................................................................ 20

VII. Recommendations ............................................................................................................................... 22

6

Abstract This assessment evaluated the Palestinian Cash Transfer Program (CTP) which is the flagship program managed and implemented by the Ministry of Social Affairs (MoSA) to mitigate poverty in the West Bank and Gaza (WBG). The CTP was launched in mid-2010 as a result of the merger of the two main cash assistance programs in place at that time: the Special Hardship Case program (SHC) financed by the European Union (EU) and the Social Safety Net Reform Project (SSNRP) financed by the World Bank. The CTP selects as beneficiaries extremely poor households according to a proxy means test formula (PMTF) model that predicts the welfare of the household for each applicant. Using a sample of applicants for the CTP, this study compared the actual consumption level of applicants and the PMTF score to measure how accurate the PMTF model was in identifying extremely poor and poor households. This study found that a large majority of applicants classified as extremely poor by the PMTF are indeed extremely poor. Results differ across the two regions, Gaza and West Bank, but in general the PMTF model targeting accuracy is among the best in the world. Moreover, it was found that the CTP outreach is largely effective in reaching the poor, the majority of beneficiaries are satisfied with the CTP approach and the application process, payments are received in a timely manner, and most beneficiaries use the cash transfer primarily to purchase food items.

7

I. Introduction 1. According to the recent World Bank Poverty Assessment (2011)1 a little more than one-fifth of the Palestinian population lived in poverty in 2009. Compared to the last five years, poverty rates in the region have declined, an encouraging 4 percentage point reduction, but trends differed across West Bank and Gaza. The overall poverty headcount rate is estimated at 22.1 percent, but the rate in West Bank is almost 50 percent lower than in Gaza with West Bank rate being an estimated 15.8 percent and Gaza’s being 33.2 percent. In addition, while poverty steadily decreased over time in the West Bank, Gaza has seen a large increase in poverty in 2007 and as of 2009, poverty rates are still above 2004 levels. Regarding deep poverty2, which is defined by the concepts of household expenditure on food, clothing and housing, the report estimates shows that the deep poverty rate is 12.2 percent in West Bank and Gaza (WBG) but the rate is driven by the high percentage in Gaza, 19.9 percent compared to 7.7 percent in the West Bank. 2. The Ministry of Social Affairs (MoSA) is responsible for operating and administrating the social protection sector, formulating the social protection policies in WBG. It is responsible for providing assistance to poor families and marginalized groups through the muderia and MoSA’s branch offices in each governorate. There are 17 muderias in WBG that provide special services for targeted families and groups (Social Protection Sector Strategy, 2010). The social protection sector includes a large group of governmental and non-governmental institutions (including charitable societies and Zakaat committees), the private sector, the United Nations (UN) and other international organizations and partners including the European Union (EU), United Nations Relief and Works Agency for Near East Refugees (UNRWA), World Food Programme (WFP), Food and Agriculture Organization (FAO) and the World Bank. To provide coordinated and adequate assistance to the poor, MoSA, worked with different international agencies including the World Bank to develop its Social Protection Sector Strategy (April 2010). The driving vision of the social protection sector according to the Strategy is: “A decent life for the Palestinian citizens on the path to sustainable human development in the independent Palestinian state”. The strategy’s four main goals are to: 1) alleviate poverty amongst Palestinians; 2) care for and enable weak and marginalized groups (including persons with disabilities, the elderly and children, amongst others); 3) form and reinforce social security in an effort to maintain an integrated social security system; and 4) develop the legislative and institutional environments and the cooperation to achieve objectives of the social protection sector.

3. The newly designed strategy seeks to increase and optimize support to the poor population, aligning its policies to the concept of deep (extreme) poverty and poverty as defined by the Palestinian Central Bureau of Statistics (PCBS). The Strategy also aims for the standardization of both methods and targeting criteria across the Government, donors and other parties implementing social assistance programs in WBG. The overall goal is to protect Palestinians from severe shocks such as the economic shock faced by Gaza in 2007 and to support the neediest families – those that have limited access to productive activities and who face high levels of unemployment and a lack of investments as a result of the closure in Gaza and global economic contraction that equally affected Gaza and West Bank. As the most recent West Bank and Gaza Poverty Assessment (2011) points out, an increase in social assistance has been one of the factors behind the small/fragile recovery that has taken place in the region since 2007. It should be noted that there are also a number of differences in some important socio-economic indicators in West Bank and Gaza. For example, households in Gaza are consistently larger than those in West Bank (6.4 household members compared to 5.8). Additionally, labor market indicators suggest that over the past ten

1 World Bank (2011). Coping with Conflict? Poverty and Inclusion in the West Bank and Gaza. Report No. 61293-GZ. 2 It is about 20 percent lower than the poverty line.

8

years conditions in Gaza have been consistently worse than in the West Bank.

4. The Palestinian Flagship program operated by MoSA is the Cash Transfer Program (CTP) which was launched in mid-2010. The CTP is a result of the merger of the two main cash assistance programs in place at that time: the Special Hardship Case program (SHC) financed by the European Union (EU) and the Social Safety Net Reform Project (SSNRP) financed by the World Bank. As of today, the CTP reaches almost 95,000 beneficiaries families, with approximately half of these living in West Bank and half in Gaza, and represents approximately 1% of national GDP. CTP implementation started with the update of the database of applicants and expansion of coverage in the West Bank. Last fall Gaza accepted the updated formula, joined the online database and recertified all households based on the new formula, and is currently merging the two former program beneficiaries into the CTP caseload, as well as adding new applicants to the database.

5. With the launch of the CTP, coordination in the area of social assistance has been strengthened among relief and development partners, i.e., WFP, UNRWA and the United Nations Children’s Fund (UNICEF). They, along with others, are relying on the CTP unified database to identify beneficiaries for their programs and to improve their targeting approaches. Therefore, given the expansion of the CTP, both Government and stakeholders have expressed an interest in evaluating the efficiency of the CTP’s current targeting mechanism. As a result, the CTP targeting system is becoming the core of a broad Social Safety Net (SSN) strategy where the CTP database of applicants can work as the central registry for an SSN system that can deliver a bundle of services to the population, in a more structured manner, which can include in addition to cash transfers, job opportunities through public works, energy allowances, health insurance, etc. Further, this research is important in ensuring that approaches and policies related to the CTP, and the social protection sector, more generally are well-informed. The need for research to inform policies and projects was underscored in the Social Protection Sector Strategy (2010). 6. This report focuses on the quality of the targeting method used to identify the extremely (deep) poor and the poor population in WBG. Currently, the CTP uses administrative targeting where relevant MoSA staff determine who is eligible for the CTP based on a proxy means test formula (PMTF) which is based on a set of household characteristics and ownership of assets. Each household obtains a score to be compared to a defined cut-off point where the household would be eligible to participate in the CTP if its score is less than or equal to this cut-off point. This study used a representative sample of households - extracted from the pool of applicants of the CTP, which includes households classified as extremely poor (CTP beneficiaries), as well as poor and non-poor households according to the PMTF – to explore whether the actual consumption/expenditure levels were correlated with the scores derived from the PMTF. In order to measure household consumption/expenditure, the survey was conducted using a short version of the PCBS consumption/income module designed by PCBS and WFP for the Socio-Economic and Food Security Assessment (SEFsec) (2011) survey, which includes also a special module for estimating food consumption scores. The survey data was collected by two independent firms – Engineering & Management Consulting Center (EMCC) in Gaza and Al Markaz for Development and Marketing Consultancies in the West Bank - between March and April 2012. Among other things, the firms confirmed that the CTP functionality and administrative processes are effective and that payments are received in a timely manner. Further, the reports indicated that the large majority of respondents used the cash transfer primarily for food purchases and did not, for the most part, save or invest the transfer. 7. Analysis of study data indicates that the PMTF looks reliable as a common targeting mechanism to identify deep (extremely) poor and poor households in WBG; targeting errors look small but larger errors are found in the

9

West Bank given that it has lower poverty rates than Gaza; compared to international literature estimated coverage and beneficiary incidence are large meaning that CTP targeting performance is quite high; and that small changes in current MoSA CTP rules/administration would lead to better results. Some such modifications might be: the provision of better training to social workers at the muderia level; imposition of a cut-off point even for the cases selected to join the vulnerable program; and improvement of the appeal/grievance mechanism by deploying a team to deal with the local social workers demands/cases. 8. The paper is organized as follows: in the next section the CTP is described followed by a description of the study data. Section 4 presents the theoretical framework for the targeting assessment and Section 5 describes study results. The conclusion to the report can be found in Section 6.

II. The Cash Transfer Program (CTP) 9. Recently, the Palestinian Authority (PA) designed a National Cash Transfer Strategy: the Palestinian National Program for Social Protection "Cash Transfer Strategy", defining the Government’s short, medium and long term strategy to fight poverty. It was agreed that Government had to improve the quality of life for poor people and marginalized groups through the establishment of an efficient, effective and comprehensive social protection program aimed at reducing poverty. MoSA became an important actor in this strategy because it was managing a number of social protection programs, including:

• Combating Poverty Programs: SSNRP, SHC Program and the Emergency Assistance Program; • Economic Empowerment Programs; • Rehabilitation for the Disabled Program; • Social Care and Defense Program and Family and Child Care and Protection Program; and • Enhancement of partnership with Charitable Organizations Program

10. In addition, the Strategy highlighted the need for institutional and program development, transparency, governance and accountability and coordination and integration to reduce fragmentation and ineffectiveness. As a result, in the short term MoSA worked to develop a strategy to unify the cash assistance programs into one unified program. As a consequence, the Palestinian National CTP emerged as a result of a merger of the two largest cash transfer programs: the SHC Program and the SSNRP. 11. The SHC Program was the largest cash transfer program in WBG in terms of coverage and budget allocation. It provided cash/in-kind food assistance to 50,000 families in WBG who were selected through a social worker’s assessment, desk and field study/verifications and a categorical classification – where children, at-risk women, as well as the elderly and vulnerable groups (e.g., persons with disabilities) were given priority consideration. To conclude the process, the applicants needed to receive the approval of the authorized Regional Director, which allowed for subjective measurements of poverty to hold significant weight. The Program provided regular cash assistance to families and in-kind food transfers were also made. 12. The SSNRP was initially established to reform the SHC Program and to enhance its efficiency and effectiveness. The SSNRP was also designed to provide regular cash assistance to approximately 5,000 families in WBG. As a targeting mechanism, the SSNRP used a PMTF to estimate the expected household consumption of applicants and those below the poverty line were classified as poor. The main rationale behind the choice of

10

targeting method was related to the fact that the PMTF was seen as a more transparent method than that used by the SHC Program which relied on a categorical approach and subjective input. 13. Based on assessments of both programs, MoSA requested support from the main donors, the EU and the World Bank who provided support through the SHC Program and the SSNRP, respectively. This was requested in line with MoSA’s Strategy which indicated that “As such, the SHCP maintained its status with poor targeting, (a) non transparent working mechanism, inequality and unjust eligibility and benefit. Furthermore, it did not contribute to reducing poverty or to enabling the poor people and the vulnerable groups to develop adaptation strategies that might reduce the impact of economic, seasonal and social risks and shocks due to its low coverage and low financing.” In this way, the unification strategy of MoSA built on the strengths of the SHC Program and the SSNRP, mainly using the SSNRP’s targeting and eligibility mechanisms. 14. The CTP rapidly became the flagship program in the fight against poverty – a pillar of the Social Protection Sector Strategy. However, MoSA faced some challenges in improving targeting, coverage, and management of resources and overall financing, and moreover, in enhancing coordination, internally and externally with the donor community. 15. The CTP objective is to mitigate poverty by providing regular cash transfers to families with a PMTF score below the extreme poverty line. Given that the CTP is a result of a merger, MoSA had to migrate former beneficiaries of the SSNRP and SHC Program beneficiaries into the new program and those households who met the new conditions were transferred into the CTP; while households that did not meet the new criteria were scheduled to either graduate from their respective programs or to be transferred to a temporary program. 16. As of today, CTP has merged the two databases into one single registry (database of applicants) with Gaza applicants joining the database last fall. The CTP has around 113,000 applicants (Table 1) with approximately 61,000 from the West Bank and 52,000 from Gaza. All applicants are subjected to the PMTF that determines their eligibility and some households not classified as extremely poor are still beneficiaries of the CTP under the vulnerability umbrella. In mid-2011, MoSA added the classification or category of ‘vulnerable’ to the CTP. This ‘vulnerable’ classification includes: (1) households falling between the deep (or extreme) poverty line and the national poverty line; (2) households who were previously receiving benefits under the Transitional Assistance Program (or TAP); and (3) households characterized as 'vulnerable' where the head of household is female, elderly, unemployed, disabled and/or chronically ill. 17. The CTP budget is supported by the EU, the World Bank and the PA. The overall budget represents approximately 1% of GDP and its allocation across contributors is: 6.6% from the World Bank, 73.1% from the EU and 20.3% from the PA.

18. While the CTP is the largest program in terms of assistance provided and number of individuals assisted there are a number of other social protection programs. Some of those programs which are operating in order to ensure protection and assistance for poor households include: WFP Program under MoSA which provides support to approximately 45,000 households to improve nutritional conditions and reduce the effects of malnutrition; the Orphans Program which provides cash benefits to almost 13,000 orphans under the age of 18; the Emergency Assistance Program which provides support to help poor families cope with their current situation; the Disabled Rehabilitation Fund which aims to promote the economic empowerment of persons with disabilities; and the

11

Economic Empowerment Program that aims to empower disadvantaged Palestinian families to earn an income and to become self-sufficient.

Table 1: MoSA Caseload as of January 2012

Total % WB % Gaza %

Applicants 113,184 100 61,116 100 52,068 100 CTP Extreme Poor beneficiaries 74,579 65.9 31,320 51.2 43,259 83.1

Vulnerable beneficiaries 19,200 17.0 16,536 27.1 2,664 5.1

Poor 10,154 9.0 8,304 13.6 1,850 3.6

Non Poor 9,046 8.0 8,232 13.5 814 1.6

Total beneficiaries 93,779 82.9 47,856 78.3 45,923 88.2

Not beneficiaries 19,487 17.2 13,342 21.8 6,145 11.8

Poor 6,850 6.1 3,528 5.8 3,322 6.4

Non Poor 12,637 11.2 9,814 16.1 2,823 5.4

III. Data 19. In order to assess the targeting efficiency of the PMTF, the study identified a random sample from the pool of CTP applicants: households that were extremely poor, poor and non-poor. The sample was selected using a two-stage cluster sample approach to assure 80% power and to produce a 95% confidence interval. In the first stage, a fixed number of districts (105 out 268 in West Bank, and 20 out of 23 in Gaza) was randomly selected, while in the second stage a fixed number of households (20 in West Bank and 90 in Gaza) was randomly selected from each district selected in the first stage. In addition, the consulting firms identified 158 households in the West Bank and 60 in Gaza that, according to their perception, were poor but who were not included in the MoSA database in order to evaluate the effectiveness of MoSA’s outreach in identifying poor households.

12

20. The total sample size for the study was around 3,400 households, basically one-half of these from Gaza (1,693) and the other half from West Bank (1,671). The sample expansion factors are defined as follows:

• The First Stage (district) Weight o The first stage weight represented the inverse of the selection probability assigned to a sampled

district, that is, is the inverse of the probability selection. The selection of probability is simply represented by:

Where N is the total number of districts, and n is the number of districts selected.

• The Second Stage (household) Weight o Since the subsampling of households occurred within sampled districts, then the basic second stage

weight is defined as:

Where ki is the number of households in the selected district and si is the number of sampled households within district i.

• The Final household Weight o Therefore the final household weight is defined as:

· ··

21. As a welfare variable, the adult equivalent (AE) expenditure used was measured by the total expenditure of the household divided by the household specific adult equivalence scale. The total household expenditure is defined as the sum of cash expenditures on: (a) food and the estimated value of in-kind receipts or own production consumption in the past 7 days; (b) fuel for cars/motorcycles and on public transportation in the past 7 days; (c) prepared meals and drinks outside the home, housing, tobacco, utility bills, medicine, health insurance, clothing, shoes and hygiene products in the past month; (d) large medical expenses (e.g., hospitals, etc.), car repairs, and recreational/cultural activities in the past six months; and cash expenses on (e) education and school fees, licensing, property tax and other general expenses in the past year. For the adult equivalence scale the PCBS formula was used:

# 0.46 · # . where #adult is the total number of adults age 18+ living in the household; and #children is the total number of children age 0-17 years old living in the household.

22. The interaction between the last payment and data collection was explored to evaluate whether a pre-transfer household expenditure was more appropriate to be used in the targeting assessment. Almost all households declared receiving their last payment from the CTP in December 2011, and a large majority immediately spent the transfer on food. Therefore, it was decided to use the current household consumption/expenditure without adjustment, that is, the total household expenditure was used without subtracting the transfer amount received by beneficiaries, given that the survey data were collected in March-April 2012 (almost three months after the last payment).

13

23. Classification of a household as extremely poor or poor was done in two ways. First, each household has its household expenditure compared to the extreme poverty line (NIS 1,898.72 a month for a family with 2 adults and 4 children) and to the poverty line (NIS 2,278.47 a month for the same type of family). These two values of the poverty lines are expressed in 2009 prices. Therefore, since the data were collected in March-April 2012, the PCBS consumption price index (CPI) was applied to have the lines presented in 2012 values. Households with total expenditure below the extreme poverty line were classified as extremely poor, households with total expenditure in between the extreme poverty line and the poverty line were classified as poor; and households with total expenditure above the poverty line are classified as non-poor. Second, each household was previously classified by MoSA according to their PMTF score. A PMTF score below 6.39 classifies a household as extremely poor; a PMTF score in between 6.39 and 6.57 classifies a household as poor; and PMTF score above 6.57 classifies a household as non-poor. 24. The next sections will focus on the PMTF MoSA classification of extremely poor households and compare it with the consumption poverty levels.

IV. Assessment tools 25. Before beginning the assessment of the formula, it is useful to review targeting. Targeting is a tool to channel resources into the hands of those in need to achieve greatest impact for a given budget, since it helps reduce a waste of funds. Targeting helps to concentrate programs on the desired population and even in budget-constrained settings it allows the Government to use the available, but limited, resources most efficiently. 26. But targeting incurs costs. There are, for example: administrative costs to manage the program, gather information of potential beneficiaries, monitor distribution of resources, etc.; private costs as some applicants may have financial, time or mobility constraints in going to a place and applying for the program; incentive costs since some applicants may change their behavior in an attempt to become beneficiaries; and social costs because beneficiaries can be stigmatized or politicians can decide to use the program in their own interest affecting the program administration. 27. Targeting also incurs errors. A limited budget affects the ability of the Government to ensure outreach to all individuals who constitute the desired population leaving a significant share of this population without access to the program. Gathering applicants is costly and time-consuming and is vulnerable to information errors if enumerators or social workers are not well-trained. Regardless of the choice of targeting method, two types of errors will always occur:

• Error of Inclusion: Not-targeted population selected to participate in the program; and • Error of Exclusion: Targeted population not selected to participate in the program.

14

28. In summary, targeting is an important tool that needs to be evaluated since it has significant cost and errors. Usually more attention is paid to errors forgetting that the errors can come from different sources and not only from the selection method. Main source of errors can be seen in Figure 2 below.

Figure 2: Main source of targeting errors

• Budget • Outreach • Method • Private costs

29. To address the quality of the targeting different indicators will be looked at such as:

• Beneficiary incidence o Percentage of program beneficiaries relative to the total number of beneficiaries by

decile/quintile of the post-transfer welfare distribution • Benefit incidence

o Percentage of program benefits relative to the total budget allocated by decile/quintile of the post-transfer welfare distribution

• Cost-benefit ratio o A NIS 1 reduction in the poverty gap for each NIS 1 spent in transfers.

30. Additionally, a sensitivity-specific analysis adapted from Hoddinott et al. (2002) will be performed, which systematically examines how a model works with reference to the benchmark variable based on a 2 by 2 cross-table computed as follows:

Cross-tabulation for sensitivity-specificity analysis

Current Household Consumption Extreme Poor (Y ≤ Extreme poverty line) Non-Extreme Poor (Y > Extreme poverty line)

PMTF groups

Selected PMTF ≤ 6.39

True positives False positivesa

Not selected PMTF > 6.39

False negativesb True negatives

31. From this table, the following indicators are computed: • Sensitivity (Coverage) – the proportion of Extremely Poor households selected by the PMTF, that is the

number of true positives divided by the number of Extremely Poor households.

15

• Specificity – the proportion of non-extremely poor households also selected by the PMTF, that is the number of true negatives divided by the number of all non-poor.

• Exclusion (Undercoverage)- the proportion of Extremely Poor households not selected by the PMTF, that is the number of false negatives divided by the number of Extremely Poor households (100% - Coverage)

• Inclusion - the proportion of non-extremely poor households not selected by the PMTF, that is the number of false positives divided by the number of non-extremely poor households selected by the proxy.

• Positive predictive value – the proportion of households selected by the PMTF that are Extremely Poor, that is the number of true positives divided by the number of all those classified as such by the PMTF.

• Misclassification – the proportion of cases that do not match, that is, is the sum of false positives and false negatives divided by the entire number of households.

32. Based on these indicators a good proxy indicator strongly associated with welfare levels will have: high sensitivity (coverage) that implies low errors of exclusion; high specificity that implies that non-poor households are not classified as poor which implies low errors of inclusion meaning that few selected cases are non poor, high positive predictive value, and low misclassification. There is gold no standard by which to compare the achievements of this targeting mechanism but this set of indicators will highlight good features of the model.

V. Findings 33. Based on the sample and using expansion factors (weighted estimates to represent the full set of 110,000 applicants) extreme poverty levels is estimated at 68.7%, meaning that a large majority of MoSA applicants are extremely poor. Around 10.3% has consumption levels in between the extreme poverty and poverty lines and only 21% of applicants had an adult equivalent household expenditure above the poverty line. Breaking down these estimates by region (West Bank and Gaza) the study found that 83% of Gaza applicants are extremely poor compared to 56% in West Bank, reinforcing the message that the prevalence of poverty is higher in Gaza and that it is more difficult to identify an extremely poor household in the West Bank. In other words, larger errors are expected in the West Bank.

V.a. Impact on poverty 34. The overall budget needed to eradicate extreme poverty of applicants is estimated at NIS 87.7 million a month, while NIS 61.5 million of this budget should be allocated to Gaza given that it has a larger poverty gap, 42.7% compared to 17.78% in the West Bank (the term larger poverty gap implies that people are, on average, farther away from the poverty line). However, MoSA had transferred, NIS 29.9 million a month (one-third of the budget needed) in the region, from which NIS 16.6 million of this amount went to Gaza, for the CTP and Vulnerable population program. Therefore, MoSA allocations are far from optimum for two reasons: first, the program transferred one-third of the budget needed to eradicate extreme poverty in Gaza and the West Bank; second, despite budget allocation in the region, targeting errors were expected in both regions. 35. Then, in order to measure the CTP’s efficiency in reducing extreme poverty, the poverty reduction and the cost-benefit ratio were estimated. For the pool of applicants, transfers can immediately reduce the extreme poverty headcount and extreme poverty gap ratio. In Gaza, extreme poverty would be reduced by 9% and the poverty gap by 21%, compared to a 20% reduction of extreme poverty in the West Bank and a 33% reduction in the poverty gap.

16

Comparing the CTP’s impact on poverty with international experience, it can be concluded on the basis of these data that the CTP is performing quite well. 36. As an international benchmark, the ASPIRE3 data can be used, which compiles information for more than 60 countries. ASPIRE shows that poverty reduction for the poorest 20% population in flagship programs such as the Brazilian Bolsa Família is 6% with a decline in the poverty gap of 14%. In terms of the impact among beneficiaries only, Skoufias and Di Maro (2010) showed that the Mexican PROGRESA (former Oportunidades program) reduced poverty among beneficiaries by 16.5% and the poverty gap by 24.3%. 37. These reductions in poverty and the poverty gap are translated into a cost-benefit ratio where the current impact of the program can be measured in terms of the poverty gap reduction for each NIS 1 spent in transfers by MoSA. For the CTP, this analysis found that NIS 0.66 out of each NIS 1 spent in benefits goes to extreme poverty gap reduction. Breaking it down by region (West Bank and Gaza), in Gaza the cost-benefit ratio is NIS 0.77 while in the West Bank it is NIS 0.52. The fact that the impact on poverty is higher in the West Bank but the cost-benefit is lower indicates that selected beneficiaries in the West Bank are closer to the extreme poverty line than those in Gaza. 38. Comparing these numbers with the ASPIRE data, which estimate the cost-benefit ratio of cash transfer programs (both conditional and unconditional) for the poorest 20% population, it is evident that the program indicator is among the best practices (see Figure 4).

Source: ASPIRE 2011, author’s calculation

3 ASPIRE: The Atlas of Social Protection – Indicators of Resilience and Equity. Downloadable at www.worldbank.org/spatlas

17

V.b. Sensitivity-specific analysis 39. The findings on poverty are a good indication that the overall targeting approach of MoSA that combines careful selection and revision of cases by social workers and a PMTF to rank applicants by a predicted welfare levels is quite efficient. To better understand how this process works, it is important to spend some time on the analysis of the PMTF model alone using the sensitive specific analysis to discuss the magnitude of errors and predictive quality of the current PMTF model4. 40. In Table 2 the analysis for the full set of applicants is presented. It is a good matching (true positives (TP) and true negatives (TN)) model –identifying correctly almost 70% of the cases (sum of main diagonal divided by total number of cases or 100% minus misclassification). As a result of such a high level of correct matching, the current PMTF model has significant sensitivity, 76%, and 53% of specificity that indicates that about one-half of the non-extremely poor are correctly classified as non-extremely poor. In addition, exclusion and inclusion errors are around 20% and positive predictive value is estimated at 78%. All of these indicators imply that the PMTF model is fairly predicting the extremely poor population and that is strongly associated with actual consumption.

Table 2: Sensitivity analysis

Source: Authors’ calculation

41. Breaking down the indicators by region and selecting only refugees, the main findings hold (see Table 3). However, it is important to note the low sensitivity in Gaza and larger inclusion and exclusion errors in the West Bank. 42. Low specificity means that a large majority of the non-poor households in Gaza are classified as poor. However, since the number of non-poor households in Gaza is small, the low specificity does not translate into large inclusion errors.

4 Current PMTF model was designed on the basis of the PECS 2007 data and it has a unique set of weights for both the West Bank and Gaza.

18

43. Larger inclusion and exclusion errors in the West Bank were expected. In the West Bank, the prevalence of extremely poor is estimated at 8% according to the World Bank Poverty Assessment (2011), meaning that it is hard to identify the extremely poor among the population. However, the errors, 39% and 31%, are just a reflection of the difficulty in finding the extremely poor population. Notice on the other hand that sensitivity is estimated at 61% and specificity at 65%, meaning that the extremely poor (sensitivity) are likely to be classified as extremely poor and that the non-extremely poor (specificity) are likely to be classified as non-extremely poor. In other words, the model’s ability to correctly classify is quite significant. 44. As a strategy to reduce errors, MoSA has a verification process which explores PMTF findings and provides an opportunity to bring some households previously excluded by the PMTF into the CTP case load. Some cases are classified as “vulnerable” and for those a “concept” of poor, not extremely poor, is considered. By adding this verification element and using the poverty line instead, Table 4 shows that sensitivity (coverage) increases while inclusion and exclusion errors decrease significantly, mainly in the West Bank.

Table 3: Summary of sensitivity analysis

Source: Authors’ calculation

Table 4: Summary of sensitivity effect MoSA verification

Source: Authors’ calculation

45. In summary, the main findings of this sensitivity analysis are very encouraging because international benchmarks of key and flagship programs around the world have sensitivity (coverage) levels of around 50% for cash transfer (conditional or unconditional) programs and even family allowances (see Table 5). Only in Chile which also provides in-kind transfers does the coverage for the poorest 20% reach about 90%. In the next section other findings are presented that corroborate the main study results presented here.

19

Table 5: Sensitivity (coverage) of flagship programs for the poorest 20% population

Source: ASPIRE 2011

V.c. Beneficiary (Benefit) Incidence Analysis 46. Another question that is often asked about a program like the CTP is: who received the benefits from this program or where are the beneficiaries of this program? This question was somehow answered previously when findings on the Program’s impact on poverty were presented. But in this section the overall welfare distribution is explored to measure the seriousness of exclusion and inclusion errors in targeting. That is to say, does a 31% inclusion error in the West Bank imply that these households are wealthy or are they around the extreme poverty line? In other words, while this study is interested in determining whether the extremely poor or non-extremely poor households are covered by a program, it is necessary to look at the benefit incidence of the program across the welfare distribution, as a finer analysis. By calculating the share of benefits going to deciles of the welfare distribution, which gives a finer grid than the poverty analysis, a benefit (beneficiary) incidence analysis is provided. 47. It should be noted in Figure 4 that among the applicants 70% of those classified as extremely poor by the PMTF are among the poorest 10%, and 84% among the poorest 20% of the population. Looking separately at Gaza and West Bank, there is a better benefit incidence in Gaza as already mentioned but even in the West Bank around 60% of applicants classified as extremely poor are indeed among the poorest 10% and 78% are among the poorest 20% of the population. Notice that beyond the poorest 20% the percentages are quite small and basically the wealthier deciles are not eligible. In Figure 5, the benefit incidence analysis is presented to explore the amount of benefit received in each group of the welfare distribution. For the West Bank, more than 50% of resources allocated to those classified as extremely poor by the PMTF are going to the poorest 10% and 71% to the poorest 20%. These incidence indicators are quite large because the international figures for the poorest 20% population incidence analysis summarized in Figures 6 and 7 are on average between 30-40%. Therefore, in terms of benefit incidence, the CTP is among the world’s best targeted programs.

20

Source: Author’s calculation source: Author’s calculation

Source: ASPIRE 2011; Author’s calculation Source: ASPIRE 2011; Author’s calculation VI. Conclusion 48. This assessment evaluated the CTP which is the flagship program operated by MoSA to mitigate poverty in WBG. The CTP was launched in mid-2010 as a result of the merger of the two main cash assistance programs in place at that time: the SHC Program financed by the EU and the SSNRP financed by the World Bank. The CTP selects as beneficiaries extremely poor households according to a PMTF model that predicts the welfare of each applicant household. Based on a sample of applicants for the CTP model, the actual consumption level of applicants was compared with the PMTF score to determine the effectiveness and accuracy of the PMTF model. 49. The main finding of the assessment is that the large majority of applicants classified as extremely poor by the PMTF are indeed extremely poor. Results differ across the two regions, Gaza and West Bank, but in general the

21

model is generating indicators that are among the best in the world. The CTP targeting approach reflects its objectives of reducing the poverty gap with its emphasis on assisting extremely poor households. 50. Both the poverty rates and poverty gap can be reduced significantly if transfers are made regularly to families. Poverty among current applicants would drop by 10-20% and the poverty gap by 20-30%. In addition, the efficiency of the program measured by the cost-benefit ratio shows that the CTP is quite effective because for each NIS 1 spent in transfers by MoSA, NIS 0.66 goes to reduce the extreme poverty gap (NIS 0.77 in Gaza and NIS 0.52 in the West Bank). 51. In terms of model accuracy, the PMTF correctly identified almost 70% of the cases. As a result of such a high level of correct matches, the current PMTF model has significant sensitivity (coverage) (76%) for the extremely poor. Exclusion and inclusion errors are around 20% which are below those errors found in analyses of important programs that are widely considered successful such as Mexico’s Oportunidades and Brazil’s Bolsa Familia. Given the lower levels of extreme poverty in West Bank, errors are larger but they do not affect the CTP’s efficacy. The current MoSA strategy of verification that brings some households that are not classified as extremely poor by the PMTF into the Program under the ‘vulnerable’ classification helps MoSA to reduce errors. 52. Errors are indeed minor and do not affect the Program’s effectiveness. The beneficiary (benefit) incidence analysis findings shows that more than 70% of those classified as extreme poor (67% of benefits of current beneficiaries) goes to the poorest 10% of the population, which is quite a high level when compared to international experience. 53. Therefore, the current PMTF is efficiently identifying the extremely poor households but as with any model there is room for improvement. Despite good overall findings, the current model can be improved to reduce errors but more importantly than redefining the current model, a more in-depth analysis of this current dataset can provide more information to guide such revisions. MoSA’s verification strategy is efficient and it could be that improving this process would lead to even better results.

54. Given that the CTP targeting mechanism is largely effective and has targeting accuracy similar to the best programs in the world, extending coverage to poor households would likely yield large efficiency gains. First, by extending outreach to ensure inclusion of extremely poor in the poverty-targeting database, information on such households can be shared with other relief and development organizations to further coordinate efforts avoiding duplication and providing support to the poorest populations. Second, a larger portion of total social assistance can be directed to the poorest of the poor. For example, prior to the National CTP it was found that 40% of the poor were not receiving any formal assistance (Palestinian Household Expenditure and Consumption Survey (PECS), 2007). Further, analysis of leakages to the non-poor in the distribution of amounts of benefits shows that these amounts were not insignificant. Half of the monetary value of all social assistance accrued to the non-poor, and almost four in every ten dollars were captured by the richest 40% of the population. For MoSA, at the time this figure was about 35 percent, providing some indication that there is potential for efficiency gains in extending CTP coverage as for every two dollars distributed through MoSA’s assistance only one dollar went to the poor (World Bank Poverty Assessment, 2010). 55. These study data also indicated that the CTP processes are effective, though there is some room for improvement in terms of communication to ensure information about the Program reaches the poorest of the poor

22

and to ensure that adequate and accurate information is provided to CTP applicants and beneficiaries (e.g., on targeting, procedures, complaint mechanisms, etc.). Further, while the study found that timely payments are received in a majority of cases, some delay was experienced (for example, 15% of respondents in the West Bank reported delays) in receiving the cash transfer from the CTP. It was emphasized, moreover, that the time lag between applying and receiving assistance is significant and that this might be an area where the process might be modified to reduce this time lag. Finally, in terms of priority spending, it was noted that the majority of households identified food as the top priority expenditure for the cash transfer they received and did not save or invest the cash transfer.

VII. Recommendations 56. The following policy recommendations are made on the basis of the study findings:

• Given that this targeting assessment demonstrated the effectiveness and efficiency of the PMTF, no modifications should be made to the PMTF until further analysis is carried out in the follow on technical assistance (TA) which will look at issues related to the energy affordability, food consumption, etc. It is still necessary to:

o Explore further the target population (extremely poor households) using information obtained from EMCC and Al Markaz for Development and Marketing Consultancies through this survey.

o Verify whether such characteristics are presented in the PMTF and whether the current model identifies this population.

o Explore current data to understand the cost of having a person with disability in the household (requires use of PECS data). Data from Gaza indicates that approximately 30% of the households sampled for this study had one household member who was disabled; and

o Explore characteristics of households that were erroneously excluded or included. • Explore special price differences between the West Bank and Gaza to evaluate whether cut-off points

should be different. • Explore the role of the PMTF in targeting for other social protection programs. For example: Do those

households identified as extremely poor by the PMTF also need energy subsidies? Are these households also food insecure?

• Explore further the use of the Food Consumption Score indicator to complement the PMTF given the high correlation coefficient between PMTF and Food Consumption Score in comparison to the coefficients between PMTF and other indicators.

• Design training modules for MoSA muderia staff on the CTP targeting approach. • Improve the appeal/grievance mechanisms to deal with mismatch cases. • Use, always, a cut-off point for assessing eligibility, even for the cases classified as vulnerable. • Explore the predictive power of any changes to the PMTF model on the targeting database. • Explore the possibility of applying different models for the West Bank and Gaza in determining the PMTF

for CTP eligibility • Address and explore in further detail other key issues relevant to modifying the CTP approach and

appropriately targeting and effectively assisting the poor, specifically looking at: o The impact of the energy tariff on poor households from the data collected through this survey to

understand characteristics of these households, financial burden and consider how this might be addressed to support poor households.

23

• Address the time lag between applying and joining the CTP and improve outreach and communication related to the CTP.

57. In light of study findings the Bank will continue to provide technical support to identify potential approaches to further enhance/improve the CTP's targeting mechanism that should only be geared towards the chronic poor while looking into effective ways of diverting the vulnerable and less poor to non-cash assistance programs. Further, in order to promote the development of a comprehensive and collaborative approach to social assistance in WBG, the Bank will continue to foster dialogue and collaboration among and between interested partners in technical discussions related to formula revisions and by identifying actions that can be taken to ensure that all partners can benefit from this important research.