1 Assessing the Impact of Microfinance: A Methodological Study Using Evidence from India Maren Duvendack Procedural Paper Presentation 23 May 2008 Supervisors:

1

Assessing the Impact of Microfinance:

A Methodological Study Using Evidence from India

Maren DuvendackProcedural Paper Presentation

23 May 2008

Supervisors: Arjan Verschoor & Nitya Rao

2

Introduction to Microfinance

What is microfinance? Provision of financial and non-financial services to low-income

households

Microfinance important strategy in the fight against poverty

Importance of microfinance recognised by United Nations and Nobel Prize Committee

No clear empirical evidence yet that microfinance has positive impacts

Impact assessments crucial for donors and microfinance institutions

3

Introduction of Research Project Challenge of every impact assessment:

Measurement of counterfactual Elimination of biases (i.e. selection & attrition bias)

Limited number of rigorous impact studies exist

Study intends to focus on methodological challenges of microfinance impact assessment studies

Suggest solutions to bias problem

4

Research Questions

What is the impact of microfinance on the households’ economic and social well-being?

How are microfinance assessment studies measuring the impact of microfinance?

What are the methodological challenges of microfinance impact assessments?

How can a rigorous treatment of biases, in particular drop-outs, improve the accuracy of impact assessment studies?

5

Research Context

Financial exclusion of India’s poor recurring problem for almost 100 years

Access to finance poverty reduction, thus Indian government launched various policy initiatives aimed at financial inclusion

BUT: Most government-run subsidised credit programmes had negative effects

Emergence of microfinance in India mainly due to lack of effective government policies

6

Research Context

Emergence of microfinance in India in the 1990s

Tremendous growth of Indian microfinance in terms of outreach and loan disbursements

BUT: Only 8 impact assessment studies conducted in India

Studies vary significantly in terms of scope and approach

They investigate one or more of the following impacts: Poverty reduction Financial services Women’s empowerment

Studies provide conflicting results, impact of microfinance unclear

Thus, more systematic approach to impact assessments needed

7

Conceptual Approaches Core elements of conceptual frameworks in impact assessments:

The impact chain model The units of assessment The impact type

Agent

Agent

Behaviours and practices over a period of time

Modified behaviours and practices over a period of time

Outcomes for the agent and/or other

agents

Program Intervention

Modified outcomes for the agent and/or

other agents

The difference between outcomes is the impact

Mediating Processes

Mediating Processes

Impact

Source: Hulme, 2000.

The impact chain model:

8

Conceptual Approaches

Units of assessment: Individual, enterprise, household, community and

institutional level Majority of studies examine impact at multiple levels

Identification of impact type: Economic, social or socio-political impacts Early impact studies mainly investigated economic impacts,

using indicators such as income, assets and expenditure In the 1980s, focus on social impacts, using indicators such

as education, health, housing and sanitation More recently, shift towards socio-political indicators such

as women’s empowerment

9

Paradigms of Impact Assessments Attribution additional challenge of impact

assessments

Two main paradigms can be extracted commonly used to demonstrate attribution:

Scientific Method Humanities Tradition

10

Scientific Method Typically attempts to attribute effects of an intervention to its

causes by utilising either…

Design Pros Cons…an experimental design

Free from biases Delivers robust results

Extensive cooperation from MFI needed Time & cost intensive Raises ethical questions

…a quasi-experimental design

Attempts to mimic experimental design Most popular design among MF impact assessors

Identification of identical control group difficult without introducing biases

…a non-experimental design

Less time & cost intensive than other two designs

Not particularly practitioner-friendly due to application of econometrical techniques

11

Humanities Tradition

Humanities tradition seeks to explain & interpret the underlying processes of an intervention

Dual function:

Triangulation to crosscheck quantitative data

Provides understanding of changes in social relationships

Difficulties in demonstrating attribution due to lack of control group approach

Causality inferred by collecting data on causal chain by interviewing programme participants, then comparison to data from areas which did not have access to programme

12

Methodological Challenges: Biases Biases common occurrence in impact evaluations

adversely effect impact results, thus solution crucial Typically the following biases occur in the context of

microfinance: Selection bias: self-selection & non-random programme

placement Attrition bias

Only handful of rigorous impact studies exist that control for biases: Hulme and Mosley (1996) Coleman (1999) Pitt and Khandker (1998) Alexander and Karlan (2007)

13

Selection Bias – Hulme & Mosley, ColemanHulme and Mosely (1996) study of microfinance programmes in seven

different countries Controlled for self-selection bias but not non-random programme

placement bias Novelty: sampling of prospective clients as a control group Mixed results, depending on programme design and country context

Coleman (1999) study on Thailand, uses village-level fixed-effects to control for non-random programme placement bias

Also, he uses Hulme & Mosley’s (1996) approach of sampling prospective clients as a control group

Difference-in-difference approach employed Little impact found, more importantly microfinance led to vicious

circle of bad debts

14

Selection Bias – Pitt & Khandker Until today, most rigorous attempt at controlling for selection bias

Quasi-experiment & eligibility requirements used to measure programme impact Primary eligibility criterion: landownership

“Treatment” Village “Control” Village

Overall findings: microcredit has positive impacts BUT: accuracy of results disputed due to lax enforcement of eligibility criteria Econometric debate between Pitt & Khandker and Morduch, not resolved until today

Eligible but do not participate

Participants

Not eligible

Would be eligible

Would not be eligible

Source: Armendáriz de Aghion and Morduch, 2005.

15

Selection Bias – Solution? Propensity score matching (PSM) popular method used to

eliminate selection bias Works by matching participants to non-participants based

on predicted probability of programme participation or the “propensity score”

Basis for matching: observable characteristics drawback Underlying assumption: no selection bias due to

unobservables Combine PSM with difference-in-difference, picks up on

unobservables but baseline data set required PSM results good approximation to those obtained under

experimental approach

16

Attrition Bias

Drop-out rates estimated to be between 3.5% to 60% in microfinance programmes worldwide

Two different types of clients exiting: Graduates

Drop-outs

Attrition bias neglected by majority of studies, Alexander and Karlan (2007) one of the few recognising its importance

Solution to attrition bias: Better sampling

Systematic interviews with drop-outs

17

Methodology – Research Design

Mainly a quantitative study with selected qualitative elements Questionnaire survey of 500 households Semi-structured interviews with selected key borrowers, in

particular drop-outs

Study proposes to employ propensity score matching (PSM) as a means to control for selection bias as well as attrition bias novelty in the context of microfinance Requirement: sampling of participants and non-

participants as well as drop-outs

18

Methodology – Overview (1)

Research Questions Steps Methods Challenge

1) What is the impact of microfinance on the households’ economic and social well-being?

Household survey, n=500, data collection on economic & social indicators, e.g. income, assets, education, health, etc.

Administration of questionnaires, max. 1.5 hours and as pre-coded as possible

Collection of appropriate income data

2) How are microfinance assessment studies measuring the impact of microfinance?

Control group needed to identify counterfactual – what would have happened had the programme not existed - requires sampling of “treated” (i.e. part.) and “non-treated” (i.e. non-part.)

PSM helps to create control group which is very similar to treatment group, only difference: control group did not participate

Identification of counterfactual

19

Methodology – Overview (2)

Research Questions Steps Methods Challenge

3) What are the methodological challenges of microfinance impact assessments?

Simple comparison of impact indicators between part. and non-part. leads to distorted results due to differences in observable & unobservable characteristics

PSM, eliminates selection bias due to observables Possibly conduct semi-structured interviews with selected part. and non-part. to understand role of unobservable characteristics

Correction for selection bias

4) How can a rigorous treatment of biases, in particular drop-outs, improve the accuracy of impact assessment studies?

Sampling of drop-outs in addition to part. and non-part.

PSM Semi-structured interviews with selected key drop-outs to understand reasons for attrition

Tracing drop-outs

20

Methodology – Sampling Procedure

Study proposes to employ multistage cluster sampling, as illustrated by figure

Sampling in stages: first, identify large areas, then narrow them down by selecting smaller areas within those larger ones

Research location: Andhra Pradesh

Sample selection criteria:

Mature microfinance programmes preferred, at least 5 years of operation

Participants: 4-5 loan cycles required

Region:Telangana

District:Khammam

Mandal:tbd

Village:tbd

State:Andhra Pradesh

Household:tbd

21

Methodology – Ethics

Oral and/or written consent of research participants shall be obtained before embarking on data collection

Data collected shall be kept confidential and will be anonymised

Reliance on research assistant and translators is expected, they shall be treated with the utmost respect and their expenses shall be covered by the researcher

22

Timeline

Preparation Fieldwork

Preparation Procedural Paper

Procedural Paper Presentation

Fieldwork Recce

23 May

Fieldwork in India

Data Analysis

Writing-Up

2008 2009 2010Jan Mar May Jul Sep Nov Jan Mar May Jul Sep Nov Jan Mar May Jul Sep Nov

23

Q & A Session