The Evaluation of Programs Combating Female Genital Cutting: Data Sources and Options February 6, 2007 P. Stanley Yoder Macro International.

The Evaluation of Programs Combating Female Genital Cutting:

Data Sources and Options

February 6, 2007

P. Stanley YoderMacro

International

Overview of presentation

• Assessing long-term impact

• DHS data

• Single survey data

• Challenges in designing evaluations

Current situation: Anti-FGC programs

• Great variety of program types

- Public awareness media campaigns

- Social mobilization in specific populations

- Lobbying for legal action

- Enlisting help from traditional and political leaders

• Very little empirical evidence of impact

(Diop & Askew, book chapter in R.M. Abusharaf, ed. 2006)

Need for data on FGC program impact

• Statistical evidence for changes in FGC prevalence

• Possible use of DHS data?

• Surveys of sample populations

- Baseline and follow-up surveys

- Comparison of intervention and

control regions

Challenges in collection of relevant data

• Conducting a survey in program areas only

• Need to have data on events 10 years ago

• Temptation to rely on program personnel rather than outside observers to evaluate

• DHS provides data on women 15-49 years old and not on young girls

• Need to describe situation taking account of secular trend

Use of DHS data for FGC evaluations

• Data on prevalence by age group of women

• Data on daughters (most recently cut)

- Age at FGC

- Identity of practitioner

- type of cutting

• Does not provide data on all daughters

DHS data on prevalence by age group

Senegal Kenya Nigeria Tanzania

15-19 24.8% 20.3% 12.9% 9.1%

20-24 28.0% 24.8% 17.0% 13.7%

25-29 28.4% 33.0% 20.8% 15.2%

30-34 30.1% 38.1% 19.4% 16.0%

35-39 30.5% 39.7% 22.2% 16.0%

40-44 30.3% 47.5% 22.2% 18.8%

45-49 30.6% 47.7% 28.4% 22.9%

Total 28.2% 32.2% 19.0% 14.6%

DHS data on distribution of FGC

• Prevalence by region

• Prevalence by ethnicity

• Prevalence by rural/urban residence

• Prevalence by religion (sometimes)

Example of DHS data for Senegal: Ethnicity

FGC prevalence

Percent of total pop.

Wolof 1,6% 40%

Serer 1,8% 16%

Diola 60% 5%

Fulani 62% 25%

Mandinka 74% 5%

Soninke 78% 3%

Limitations in use of DHS data for evaluations

• No complete data on FGC among young girls

• Sample size limits aggregation by small units such as those covered by an FGC program

• Provides data on FGC events 10-15 years ago and more only (women 15-24 years old)

Assessing long-term impact: Stand alone surveys

• Baseline and follow-up surveys

- relatively expensive to design and implement

- results after more than five years

• One survey with intervention & control areas

- much cheaper

- difficult to identify intervention & control areas, for they don’t follow administrative units

- very hard to match intervention & control areas

Example: Tostan evaluation for UNICEF New York

• Commissioned and funded by Child Protection Division of UNICEF New York

• Implemented through contract between UNICEF and Macro International

• Additional funding by USAID

• Two components: Quantitative and Qualitative

- Quantitative: Centre de Recherche pour le Développement Humain (CRDH)

- Qualitative: Population Council, Dakar

Quantitative component: intervention and control areas

• Designed and implemented by the CRDH

• Questionnaire for households and for women 15-49 years old

• Questionnaire similar to DHS

• Questions on knowing about Tostan, participation in Tostan, knowing about a Public Declaration, participation in a PD

Basic questions to answer

• How has Tostan affected:

- average age of marriage?

- cutting of young girls?

• How has participation in a Public Declaration affected:

- average age of marriage?

- cutting of young girls?

Three types of villages where FGC is practiced

Basic comparison:

Data on age of first marriage and rates of FGC in

three types of villages:

• Type A: Tostan operated and population

participated in a Public Declaration (PD)

• Type B: Population participated in PD without Tostan

• Type C: Control villages: No Tostan, no PD

Assessment of program impact

• What are the starting points for rates of FGC and for age of marriage in these three types of villages?

• What is the secular trend in these areas?

• How do the intervention and the control villages differ?

• To what extent can differences be attributed to Tostan activities?

Village type by prevalence of FGC for women

• Type A villages – FGC prevalence 64%

• Type B villages – FGC prevalence 81%

• Type C villages – FGC prevalence 87%

• Implication: The point of departure is different for the assessment of any change that may have occurred

Major challenges for village comparisons

• Matching control and intervention villages by:

- ethnic composition

- practice of FGC

- literacy

- development of social services

• Locating villages near Tostan villages that had no Tostan influence

Reporting on an evaluation: Being clear

• The questions to be answered

• The indicators to be followed

• How and why they are important

• The expectations of the program personnel

• The ways evaluators would assess the program

• The ways evaluators would measure success

Setting up a new evaluation

• Questionnaire that matches program activities

• Sample population that covers program areas plus control areas

• Choosing indicators that show process as well as impact

• Identifying a neutral agency to conduct survey

• Articulating how assessments will be made

• Plan final report that provides all this information