AADAPT Workshop South Asia Goa, December 17-21, 2009 Survey work Maria Isabel Beltran 1
Mar 22, 2016
1
AADAPT Workshop South AsiaGoa, December 17-21, 2009
Survey work Maria Isabel Beltran
2
Type of data
For evaluation purposes: Administrative data Surveys our focus, we can complement
with other sources of information▪ Household▪ Plot▪ Associations▪ Community
Census and other country surveys
3
Data collection: Who does it? Who collects the data? 2 main cases:
The ministry▪ Hiring of enumerators? Who are they going to be?▪ People inside the project have incentives to
present a better or worse picture for their areas▪ A lot of effort to follow the process
An agency (statistical office or private firm)▪ OK, this is the type of work they do, but STILL A
LOT OF EFFORT is needed to ensure quality (TORs, sample, questionnaire, training, supervision)
4
Data collection
Questionnaire design Training Pilot test (and re-training) Field work Supervision Data entry & data cleaning
5
Data collection: Questionnaire
Who defines it? YOU (the IE team, not the firm) Purpose of survey? Define: respondents, indicators,
level, modules. Time & quantity trade off Internal consistency Omission of key issues & skip patterns Clear and explicit questions for all circumstances Avoid open questions (pre-code) / recall period Respondent burden, sensitive issues last
Data collection: Training & Pilot Test
Often underestimated part of the process. Training reduce variability in data collection Pilot ensures the questionnaire is collecting all
information needed to answer questions, all correct information, flows and logic of the questionnaire.
Test the instruments cover all conceivable situations Involve the enumerators in the project the
importance of the data collected.
Training…
8
Data collection: Field Work Almost always, it is better if organized in
groups of enumerators (2-3) Time Vs. quality Have a clear field work plan and division of
responsibilities among the group Daily targets Gambia: Enumerator 1 Enumerator 2 Enumerator 3
Talks to head teacher Children, math test
Classroom observation
Head teacher question.
Children, reading test Teacher tests
Oral tests Oral tests Oral tests
9
Data collection: Supervision
Supervision protocol, 1 per 2 teams? Have a supervision strategy: 10% of
the sample, 100% ? Only non valid responses?
Use an independent firm or team; that has received the training
Supervise the supervisors
10
Data collection: Data Entry and Clean-up No need to wait for data collection to finish to
start data entry. Make corrections while the data is still being collected. (Missing values, inaccuracies)
Integrated concurrent data entry Vs. Concurrent Centralized data entry Vs. Computer assisted interviews
Data entry: ONE TIME NOT ENOUGH double entry at the same time, one after the other, one with supervision, … etc
If not planned… data cleaning = long & frustrating
Data is lost, quality decreases (decisions not documented)
11
Data collection: example from India Integrate the data collection and data
entry. Timely data Feedback on field work on real time Early detection of errors (like lack of uniform
criteria)
The Medical Advice, Quality and Absenteeism in Rural India project of the Center for Policy Research, New Delhi
The Medical Advice, Quality and Absenteeism in Rural India
3 separate firms: data collection, supervision, data entry
Define all possible error per questions and program them
type0: No
errortype1:
ID errortype2:
Formatting error
type3: entered
skip code but not skipped
type4: skipped but no
skip code
type5: cross check error
type6: header ID does
not match page 1
type7: blank
instead of -99
type8: one digit
instead of two
Total
s1q1 855 0 0 0 0 0 0 0 0 0s1q10 838 0 17 0 0 0 0 0 0 17s1q11 831 0 24 0 0 0 0 0 0 24s1q12 828 0 27 0 0 0 0 0 0 27s1q13 855 0 0 0 0 0 0 0 0 0s1q14 852 3 0 0 0 0 0 0 0 3s1q15 854 0 1 0 0 0 0 0 0 1
The Medical Advice, Quality and Absenteeism in Rural India
14
Useful Data
Relevant data Reliable data Data that is ready when needed… ON TIME, to
answer operational and policy questions.
Need to have staff dedicated to the project in all phases (design, preparation, implementation, dataset documentation & validation) field coordinator.
15
Thank You