MICS4 Survey Design Workshop Multiple Indicator Cluster Surveys Survey Design Workshop Survey Quality Control
Mar 27, 2015
MICS4 Survey Design Workshop
Multiple Indicator Cluster SurveysSurvey Design Workshop
Survey Quality Control
MICS4 Survey Design Workshop
Objectives
• Identify factors affecting the accuracy and reliability of survey data
• How to prevent and correct errors
• The essential role of supervision in the field and providing feedback to the team
MICS4 Survey Design Workshop
Overview of presentation
• Preparation for fieldwork• Data collection• Field supervision• Data processing
MICS4 Survey Design Workshop
Overview of presentation
• Preparation for fieldwork• Data collection• Field supervision• Data processing
MICS4 Survey Design Workshop
Preparation for fieldwork• Questionnaire
– Adaptation (relevant, tailored to needs)– Translation (for uniformity)– Pretest (good flow, capture relevant information)
• Sample design and size– For reliability, accuracy, feasibility
• Interviewers– Selection (systematic process, have surplus)– Training (trainers, length, practice)
MICS4 Survey Design Workshop
Overview of presentation
• Preparation for fieldwork• Data collection• Field supervision• Data processing
MICS4 Survey Design Workshop
Data Collection
• Logistics & discipline• Implementation of sample• Monitoring of fieldwork
MICS4 Survey Design Workshop
Data Collection:
• Material & equipment (questionnaires/forms)• Transport• Accommodation• Payments to field staff• Discipline & reasonable expectations• Keep the morale of the “troops”
Logistics & discipline
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
• Non-response: failure to obtain information for selected households, eligible women, or children
• A potentially serious bias that can be minimized
• Interviewers will need to make return visits to households (call-backs)
MICS4 Survey Design Workshop
1. Interviewer is unable to do the selected household
2. Interviewer unable to meet eligible respondents
3. Respondent refuses to be interviewed
Types of non-response:
Data Collection: Implement. of sample
MICS4 Survey Design Workshop
1. Interviewer is unable to do the selected household. Why?
Types of non-response:
– Structure not found– Occupied structure inaccessible– Structure non-residential, vacant, or demolished
Data Collection: Implement. of sample
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
2. Interviewer unable to meet eligible respondents. Why?
Types of non-response:
– No one at home at time of call– Respondent temporarily absent Will need call-backs
MICS4 Survey Design Workshop
Call-backs
• Interview that is not completed requires a “call-back” or follow-up visit
• Three call-backs required (different times and days)
• Supervisors and interviewers keep track using control sheets
• Requires good tracking of work to ensure that all planned interviews are completed before leaving cluster
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
3. Respondent refuses to be interviewed. What to do?
Types of non-response:
– Approach respondent from her point of view– Postponed interview to another day– Have field editor carry out the interviewThe number of refusals should be closely
monitored. Need to find out reasons for frequent refusals.
MICS4 Survey Design Workshop
Data Collection: Monitoring field work
• Training is a continuous process
• Observation and supervision throughout the fieldwork are a part of the training
• Team supervisors and field editors play very important roles in continuing this training and in ensuring the quality of MICS4 data
MICS4 Survey Design Workshop
Data Collection: Monitoring field work
• Fieldwork control sheets• Direct observation of interviews• Review of completed questionnaires (editing)• Spot-checking households
MICS4 Survey Design Workshop
Fieldwork Control Sheets
• Interviewers’ work is monitored and evaluated by keeping accurate record of assignments
• Both supervisors and interviewers have control forms to maintain
• These forms should be returned to the director of field operations along with the completed questionnaires
• Interviewer is responsible for ensuring that control sheet is up-to-date
MICS4 Survey Design Workshop
Observing Interviews, part 1
• To evaluate and improve interviewer performance
• To look for errors and misconceptions that cannot be detected through editing
• Why? Precise but inaccurate answers • Who? The supervisors or field editors (females
only for women’s questionnaire)
MICS4 Survey Design Workshop
Observing Interviews, part 2
• How often? 5-6 interviews per week, more at start of fieldwork
• How? Just take notes without disrupting the interview
• What to do after? Supervisors speaks to interviewers
MICS4 Survey Design Workshop
Editing Questionnaires in the Field
• Fundamental to survey quality
• Need to ensure:
– Accurate and complete information in each questionnaire
– Correct count of questionnaires
MICS4 Survey Design Workshop
Editing Questionnaires in the Field
• Done daily by field editor; supervisor can assist
• Finish all questionnaires before leaving cluster
• Provides an opportunity to identify and correct mistakes before leaving cluster
• Results and errors are discussed with interviewers
MICS4 Survey Design Workshop
Systematic Spot Checking
• How to spot-check household composition?– Supervisors should complete columns 2-8 in the
HH questionnaire and compare with that of the interviewer
– Check about 5% of households (5-6 per week)– All team members must be spot-checked;
provide feedback if necessary
MICS4 Survey Design Workshop
Evaluating Interviewers Performance
• Daily discuss the quality of interviewers work • Point out mistakes discovered during observation of
interviews or noticed during editing• Discuss examples of actual mistakes, but be careful not to
embarrass individual interviewers. • Re-read relevant sections from the Interviewer's Manual
with the team to resolve problems. • Encourage the interviewers to talk about any situations
they encountered in the field • Discuss whether situations are handled properly, and how
to do it in the future
MICS4 Survey Design Workshop
Overview of presentation
• Preparation for fieldwork• Data collection• Field supervision• Data processing
MICS4 Survey Design Workshop
Field supervision• Staff from survey headquarters
– Senior staff
– Quality control team
• Purpose
– Visit teams to observe interviews, review work
– Re-visit selected clusters; spot-check households
• Very important during first few weeks of fieldwork! (Possible to start teams in one location?)
MICS4 Survey Design Workshop
Overview of presentation
• Preparation for fieldwork• Data collection• Field supervision• Data processing
MICS4 Survey Design Workshop
Data Processing
• Integral aspect of survey quality (DP programmes, double-entry, secondary editing)
• Problems detected in DP should be communicated back to teams (phone calls and visits)
• Field-check tables
• Observations from reviewed questionnaires
MICS4 Survey Design Workshop
GENERAL INSTRUCTION TO EDIT QUESTIONNAIRES
According to MICS Manual
Under no circumstances should we make up an answer