SURVEY QUALITY CONTROL. Objectives § Identify factors affecting the accuracy, validity and reliability of survey data § How to prevent and correct errors,

SURVEY QUALITY CONTROL

Objectives

Identify factors affecting the accuracy, validity and reliability of survey data

How to prevent and correct errors, How to avoid common causes of mis-reporting The essential role of supervision and team

interaction in the field

Main Components of Survey Quality!!

Supervision of interviewersNon-responsePending interviewsFieldwork control sheetsMonitoring interviewer performance

Spot checking Observing interviews

Editing of questionnaireEvaluating interviewersReturning questionnaires to central office

Supervision of Interviewers

Training is a continuous process. Observation and supervision throughout the

fieldwork are a part of the training Team supervisors and field editors play

very important roles in continuing this training and in ensuring the quality of MICS3 data.

NON RESPONSE

Non Response

One of the most serious problems for MICS No response is failure to obtain information for selected

households, eligible women, or children. A serious bias to be minimized by supervisors and editors Interviewers will need to make return visits to households

in the evening or on the weekends

Types of Non Response

• No response may be classified into three basic types:

– Type 1: the interviewer is unable to locate the selected household;

– Type 2: the interviewer is unable to locate the eligible woman or the caretaker of the children under five for the individual interview;

– Type 3: the respondent refuses to be interviewed.

TYPE 1

The interviewer is unable to locate the household

– Occupied structure inaccessible– Structure not found.

– Structure non-residential, vacant, or demolished.

TYPE 2

The interviewer is unable to locate the respondent for an individual interview

• No one home at time of call

• Respondent temporarily absent

TYPE 3

The respondent refuses to be interviewed Number of refusals should be closely monitored Reasons for frequent refusals

What to do? • Approach respondent from her point of view

• Postpone interview to another day.

• Have field editor carry out the interview

PENDING INTERVIEWS

Handling Pending Interviews

Pending interview-interview that was not completed and requires a call back

Who keeps track of pending interviews and how? Supervisors and interviewers via control sheets

Planning and completing call backs for pending interviews before leaving the cluster

Requires a good division of work to ensure that all planned interviews are completed

Maintaining Fieldwork Control Sheets

The work of interviewers is monitored and evaluated by keeping accurate record of assignments

Both supervisors and interviewers have control forms to maintain.

These forms should be returned to the director of field operations along with the completed questionnaires

Interviewer is responsible for ensuring that control sheet is up-to-date y giving it back to the supervisor

Systematic Spot Checking of Household Composition

Spot checking is used to improve data quality by identify common problems and poor skills

How to spot-check household composition? Supervisors should complete again columns 2-8 in the

HH questionnaire and compare with that of the interviewer

Conduct about 5% of households (5-6 per week) All team members must be spot-checked. If problems exist, the supervisor should review the

household schedule with interviewer, review the MICS3 manual with the team

Take more serious actions if problem persists with individual interviewer

Monitoring Interviewer Performance

How? By observing interviews and carrying out field editing

When to observe interviews? Throughout field work but more frequently at the beginning of the survey and again toward the end

Observing Interviews

To evaluate and improve interviewer performance To look for errors and misconceptions that cannot be

detected through editing Why? Precise but inaccurate answers Who? The supervisors but only by female supervisors for

the women’s questionnaire How often? 5-6 interviews per week When? At any time during the day How to observe and interview? without disrupting the

interview, taking notes about the observed issues What to do after? Supervisors speaks to interviewers to

provided the needed feedback

Household Visits and Individual Interviews

At the end of each day field editor should: First, review the Household and Individual Questionnaires Second, using the questionnaires, verify the information

about the results of the interview Check that all the households or dwellings are listed on the

Interviewer's Assignment Sheet and agrees with those selected from household listing form or map for cluster

There can never be fewer household questionnaires than selected households or dwellings, but there can be more households questionnaire than selected ones

Evaluating Interviewers Performance Daily discuss the quality of interviewers work Point out mistakes discovered during observation of

interviews or noticed during editing Discuss examples of actual mistakes, but be careful not to

embarrass individual interviewers. Re-read relevant sections from the Interviewer's Manual

with the team to resolve problems. Encourage the interviewers to talk about any situations

they encountered in the field Discuss whether situations are handled properly, and how

to do it in the future

EDITING QUESTIONNAIRES

Editing Questionnaires in the Field To ensure that women and children are not omitted from

the Household Questionnaire To ensure that woman’s and children’s dates of birth are

properly recorded As explained before this is the work of the field editor Where and When? Throughout fieldwork but specially at

the beginning and end of fieldwork Who can help? The supervisor can assist Results and errors are discussed with interviewers When necessary, interviewers may need to contact

respondents to correct errors or inconsistencies Disciplinary actions should be considered and used

Editing Questionnaires in the Office

A computer program checks each questionnaire and print out a listing of all errors found

If the errors are major ones, an entire questionnaire may be omitted from the analysis.

Errors and inconsistencies observed in the office can not be corrected only imputed (!)

Feedback to the field teams can be provided on errors found during office editing

GENERAL INSTRUCTION TO EDIT QUESTIONNAIRES

According to MICS3 Manual

Under no circumstances should we make up an answer

Returning Questionnaires to the Central Office Checking the Household Structure

Include all individual questionnaires into the household Questionnaires.

Verify if more than one individual questionnaire (women or children under five) pertain to the household?

Check household questionnaires for the number of eligible women and children, and their corresponding line numbers

Verify number of Individual Questionnaires, and that identification lines are correct and consistent between questionnaires

Follow procedures of communication with central office For incomplete interviews, include blank questionnaires

with only the identification information and the result code

Forwarding Questionnaires to the Central Office

Instructions from the director of field operations Packaging and protection of questionnaires Confidentiality of data in questionnaires Proper instruction developed and implemented for

transportation of questionnaire to avoid their loss or information