Lesotho: Targeting and accountability mechanisms in a cash ... · Targeting process • Identification and ranking of households according to poverty criteria applying Proxy Means

Lesotho: Targeting and accountability

mechanisms in a cash transfer system

Ms Selloane M. Qhobela

Principal Secretary - Social Welfare Lesotho Child Grant Programme

Luca Pellerano (Oxford Policy Management)

Food Security Governance Workshop

5-6 December 2011 Rome, Italy

Lesotho Child Grants Programme (CGP)

•  Unconditional cash transfer •  Target Group

–  Poor households looking after orphans and vulnerable children

–  Child headed households •  Goal

–  Improve nutritional and health status and school enrolment among OVC

•  Context –  Half the population under poverty line –  Third highest HIV prevalence in the world –  One of highest Gini’s in the world

CGP—the Pilot

•  Objective: To develop and test systems for –  targeting –  enrolment –  payment to beneficiaries –  public information and education –  M&E

•  Implemented by: Department of Social Welfare, government of Lesotho

•  Financial support: EU €12m •  Technical assistance: UNICEF •  Period: 2007-2011

Benefit level and coverage

•  Maloti 360 per family per quarter –  (1US$ = 7 Maloti) –  The montly equivalent of 13% of the average

household consumption expenditure

•  9,915 Beneficiary Households –  Caring for over 27,000 vulnerable children –  Chosen from selected Community Councils of 5

districts

Targeting process

•  Identification and ranking of households according to poverty criteria applying Proxy Means Test (PMT) •  Census data collected for all households prior to

program roll-out •  Households are assigned to one of 5 poverty levels

(NISSA 1 to 5) •  Validation of eligible households by Village

Assistance Committees (VAC) •  Only households NISSA 1or2 AND validated

by the VAC AND have at least one child 0-17 are enrolled in CGP

Governance Structure

•  Village Assistance Committees –  Community Councilor, Village Chief and two

community representatives –  Facilitation, Validation, Support at Village Level

•  Case Management System –  Community Councilors, Village representative

and activists at Community Council level –  Appeal and Grievance Redressal mechanism at

Community Council level

•  District Child Protection Teams

Village Assistance Committees

•  Main role of the VAC: –  Support data collection; teams prepare maps of

corresponding villages –  Assist enumerators in community visits –  Assist community mobilisation by raising

awareness and making information available to communities

–  Conduct validation process for both eligible household and enrolment list

–  Support enrolment event and payment processes, and

–  Support community with filing updates, appeals and complaints

Evaluation

•  Rapid Assessment of Phase 1 of Pilot (early 2011)

•  Rigorous impact and targeting evaluation of Phase 2 of Pilot

–  NISSA census in Apr-May 2011 –  Baseline survey in Jul-Aug 2011 –  Random allocation of the program across

Electoral Divisions –  Enrolment and first payment in September

2011 –  Follow-up survey in 12/24 months

Baseline Evaluation Sample

District Community Council

Called to Enrolment

Non-Called to Enrolment

Total

Group A/B Group C/D Maseru Quiloane 157 (99.4%) 158 (97.5%) 315 (98.4%) Rapoleboea 130 (98.5%) 131 (99.2%) 261 (98.9%) Leribe Malaoaneng 128 (96.2%) 139 (99.3%) 267 (97.8%) Litjotjela 200 (94.8%) 226 (102.3%) 426 (98.6%) Berea Tebe-Tebe 200 (100.5%) 194 (96.5%) 394 (98.5%) Kanana 211 (97.2%) 225 (100.9%) 436 (99.1%) Mafeteng Metsi-Maholo 211 (98.1%) 221 (99.1%) 432 (98.6%) Malakeng 172 (97.2%) 177 (98.3%) 349 (97.8%) Qacha’s Neck White-Hill 35 (92.1%) 70 (97.2%) 105 (95.5%) Mosenekeng 40 (100.0%) 28 (100.0%) 68 (100.0%) Total 1484 (97.6%) 1569 (99.2%) 3053 (98.4%)

Context •  Roughly 70% of households did not have enough food to

meet their needs at least for 1 month in the last 12 months

Indicator Selected for the CGP

Non-selected for the CGP

Proportion of households in w hich any child (0-17) household member had to eat a smaller meal than felt needed during the last 3 months because there w as not enough food

69.4*** 43.7 50.3

Proportion of households in w hich any child (0-17) household member had to eat a few er meals than felt needed during the last 3 months because there w as not enough food

68.0*** 40.8 47.8

Proportion of households in w hich any child (0-17) household member w ent to sleep hungry during the last 3 months because there w as not enough food

33.9*** 16.3 20.9

By beneficiary status Overall

Targeting Assessment

•  Focus on two complementary dimensions: –  Targeting effectiveness (inclusion and exclusion

errors) – mainly quantitative –  Quality of the targeting process – mainly

qualitative

Main Targeting Outcomes   Percent   Frequency  NISSA  1  and  2  Validated   22.1   4,553  NISSA  1  and  2  Non  Validated   27.13   5,590  NISSA  3,  4  and  5   27.15   5,595  No  children   22.8   4,697  Missing  Information   0.83   170  Total   100   20,605  

Targeting Effectiveness •  Are selected households poorer than non

selected households?

    Selected   Non-­‐selected   Overall  Mean  monthly  consumption  expenditure  per  adult  equivalent  (Maloti)   205***   338   308  Proportion  of  households  below  poverty  line  (%)   74***   43   50  

Targeting Effectiveness

Target population Programme beneficiaries

E1 = eligible beneficiaries E2 = eligible non-beneficiaries N1 = non-eligible beneficiaries Inclusion error (leakage) is defined as the proportion of beneficiary households that are not in the target population • Inclusion error = N1/(E1+N1) Exclusion error (under-coverage) is defined as the proportion of the target population that are not benefiting from the programme • Exclusion error = E2/(E1+E2)

N1 Inclusion

error E1

Correctly targeted

households E2

Exclusion error

Inclusion  error                                            (%  of  selected  households  that  are  not  poor)   26  

Exclusion  error                                          (%  of  poor  households  that  are  not  selected)   67  

Drivers of Targeting Effectiveness

•  How different components of the combined targeting design are contributing to targeting effectiveness?

•  Poor households are 48% more likely to have been selected for the programme under CGP targeting than they would have been under random or universal targeting.

    Proportion  of  households  

Mean  monthly  consumption  expenditure  

(p.a.e.)  

Poverty  rate   CGH  index  

Marginal  contribution  to  targeting  efficiency  

 All  households      

 100  

 308  

 50  

 1.00  

Households  with  children    

78   292   53   1.06   0.06  

a)  Households  with  children  that  pass  NISSA  test  (ONLY  PMT)    

49   239   65   1.30   0.24  

b)  Households  with  children  that  pass  Comm.  Validation  (ONLY  VALIDATION)    

35   229   69   1.38   0.32  

c)  Selected  households    (children,  pass  NISSA  test,  validated)  

22   205   74   1.48   0.42  

International Benchmark

•  Coady et al. (2004) review targeting efficiency and outcomes for 122 anti-poverty interventions in 48 countries from various parts of the world. The study showed that the median targeting programme had an CGH index of 1.25

Targeting Process

•  Six key steps: 1.  Awareness raising, including the Public

Information Campaign and the Community Mobilisation

2.  Setting up Village Assistance Committees 3.  Community census, including problems of

coverage 4.  Community validation 5.  Household selection and enrolment, and 6.  Case management

Awareness Raising

•  For both the NISSA census and CGP –  Mostly via public gatherings

•  Main bottlenecks: –  Pitso ‘fatigue’ –  Relationship between local and traditional structures –

Villagers less likely to attend public gatherings organised by councillors 

–  Limited use of other information tools (no radio announcements, brochures and handouts)

–  Rural/urban – In urban areas Pitsos not taken as seriously –  Mobilisation team’s incomplete understanding of selection

process and partial information given to households at public gatherings

Proportion of households that: Selected Non-selected

Overall

Were aw are of targeting in the community 78.4*** 72.2 73.6

Declare that the programme's objectives w ere explained to them

52.3 50.5 50.9

Declare to know how programme beneficiaries are chosen

8.9 10 9.7

Village Assistance Committees

•  Selection Process –  Individual households attending the community mobilisation

gathering were asked to nominate two members from within their community

–  Hesitation from those present to volunteer for being nominated for selection. Reluctance stemmed from future political and relational ramifications of being part of process. Members worried that they would potentially be accused of biased selection if they participated.

–  The chiefs on the other hand were reportedly not very happy with the establishment of VAC and its members.

–  People were in general happy about those who were selected as part of the VAC. The process was considered to be fair even by the individuals who did not attend the Pitso

•  All focus group respondents (both recipients and non-recipients) were not aware of the VAC playing any role in the selection process.

Community Census

•  Coverage of the NISSA census has important ramifications on targeting effectiveness and more specifically exclusion errors

•  Overall 76% of households recorded in the national census in targeted community councils in 2006 were reported to have been registered in the NISSA in 2011

•  Two potential sources of exclusion –  Entire villages excluded within selected community councils

(inconsistency of information between lists taken from BoS) –  Pockets of households being excluded from the villages that were

visited (limited cases—key factors: previous announcements, logistics with second visits)

Community Validation

•  Key factors affecting type and level of inclusion and exclusion errors at validation

–  nature and composition of the VAC during validation process (chiefs were present in roughly half of the validation cases)

–  understanding of criteria for Validation –  adherence to criteria for Validation –  level of supervision provided by mobilisation facilitators

•  Role of VACs in validating household list was kept confidential. Villagers generally thought that targeting was undertaken at the central level with aid of computers

–  Controversial

Community Validation Criteria •  A household IS among the poorest if their house needs serious repairs or has not been

renovated for a long time (i.e. poor conditions of roof/walls/floor, lots of damage). •  A household is NOT among the poorest if the house has recently been renovated, is new, or

is in good condition (i.e. has recently been painted, has strong windows/doors/floors, sturdy roof).

•  A household IS among the poorest if it does not have a regular source of income or support. •  A household is NOT among the poorest if it is receiving a constant and significant amount of

money from relatives, pensions, or other organisations. •  A household IS among the poorest if it cannot afford to buy decorations, tools or appliances

which are in good condition. •  A household is NOT among the poorest if it has tools and appliances which are in new or

good condition (i.e. TV, DVD player, microwave). •  A household IS among the poorest if it is among the poorest now and has been for a while. •  A household is NOT among the poorest if it has experienced a recent change in situation (i.e.

recent unemployment, death of a family member, etc.) causing it to be in financial difficulty as of late.

•  A household IS among the poorest if it has no assets or livestock to sell. •  A household is NOT among the poorest if it could sell assets (e.g. refrigerator) or livestock to

afford necessities. •  A household IS among the poorest if it normally does not have enough to eat. •  A household is NOT among the poorest if it only sometimes does not have enough to eat. Source: Community Validation Guidelines – The Lesotho Child Grant Programme.

Enrolment

•  Technical error resulted in incorrect call for enrolment and certificates for enrolment lists being generated for 3 community councils

•  Certificates for enrolment were given to many ineligible households, with instructions to attend the enrolment event

•  This error had significant negative impact on community perception of programme legitimacy and has caused anger and resentment

•  Moreover this error has further diluted and confused households’ understanding of selection process

Case Management

•  At the time of qualitative fieldwork there was no formal complaints mechanism in place.

•  Although in the process of development, households were reportedly told that if they have any grievances or issues with selection that they would be able to voice these from August/September onwards through established complaints mechanisms.

Proportion of households that: Selected Non-selected

Overall

Know w ho to complain to for any problems w ith the w ay in w hich households w ere selected

11.1** 7.1 8

Conclusions

•  Overall good targeting effectiveness… –  Combine methods to minimize inclusion errors –  The design of the PMT can be largely improved

•  … but process is still crucial, particularly at community level –  Need to increase transparency and accountability –  The devil is in the operational details (at grassroots level)! –  Main issue so far: weak household understanding of selection

process –  Low attendance at public gathering during the initial community mobilisation –  Lack of full understanding of selection process by community mobilisation

officers –  Community validation process not transparent –  Errors during the enrolment process

–  Some immediate outcomes despite non-optimal governance framework, but more transparency and accountability is likely to have substantial consequence in terms of impact (community level) and sustainability (?)