Transcript
Questionnaires and Structured Interviews
Chapter 9
Introduction
Questionnaires and structured interviews are the two most widely used methods of data collection in social science research.
Introduction
Questionnaire A data collection instrument with questions
and statements that are designed to solicit information from respondents
Introduction
Structured interview A data collection method in which an
interviewer reads a standardized list of questions to the respondent and records the respondent’s answers
Introduction
Survey A study in which the same data are collected from
all members of the sample Usually in the form of answers to questions Often use large probability samples and cross-
sectional study design
Introduction
Respondents The participant in a survey who completes a
questionnaire or interview
Introduction
Self-report method Another name for questionnaires and interviews
because respondents are most often asked to report their own characteristics, behaviors, and attitudes.
The Uses of Questionnaires and Interviews Questions can be asked about a variety of topics It is common to use the self-report method to gather
information on attitudes, beliefs, values, goals, and expectations
Questions can also be used to gather a person’s level of knowledge on something or behavior
Introduction
Stop and Think The surveys by Public Agenda and Pew
focused on attitudes towards national energy policies and health care, respectively.
Suggest a list of other topics that you think you could ask questions about using a questionnaire.
Would you be concerned about the accuracy of the answers on any of these topics?
Focal Research
Environmentalism Among College Students: A Sociological Investigation by Brandon Lang and Christopher W. Podeschi Asked college students questions about
environmental issues Interested in whether the students’ attitudes
were connected to some background characteristics and to behavior patterns
Descriptive and explanatory purposes
Focal Research
Ethics The researchers submitted a proposal to their
university’s Institutional Review Board (IRB) where it was approved.
The respondents were informed about the study’s sponsor, were assured that the surveys were anonymous and that they could skip any of the questions.
The Uses of Questionnaires and Interviews
The Uses of Questionnaires and Interviews
Although surveys are widely used there are important concerns about their validity
When using self-report methods it is based on the implicit assumption that people have the information being asked and they will answer based on their core beliefs and values
The Uses of Questionnaires and Interviews
Answers to questions may be compromised if the participants Do not understand Do not take the time to adequately answer the
question Influenced by current mood, events, etc.
For example, research has shown that people who are in more positive moods answer more favorably compared to those who are feeling more negative (Martin, Abend, Sedikes, & Green, 1997)
The Uses of Questionnaires and Interviews Measurement error
Error that occurs when the measurement we obtain is not an accurate portrayal of what we tried to measure
The giving of inaccurate answers to the questions
The Uses of Questionnaires and Interviews
Measurement error may also be affected by question wording respondent’s level of knowledge on the topic whether or not the respondent has an opinion about
the topic respondent having an accurate memory respondent wanting to present a good image
Participant Involvement
Response rate The percentage of the sample contacted that
actually participates in a study
Participant Involvement
Response rate is affected by The number of people who cannot be
reached for the study (noncontacts) The number who choose not to participate
(refusals) The number who are incapable of performing
the tasks required of them For example, due to illness or language barriers
Participant Involvement
Nonresponse error Results from differences between nonresponders and
responders to a survey
Participant Involvement
When participants and nonparticipants differ in social characteristics, opinions, attitudes, values, or behavior – then generalization to a larger population is much more difficult
Some researchers have questioned whether higher rates of refusal automatically means more bias
Participant Involvement
Stop and Think Have you ever been asked to complete a
questionnaire or an interview? What factors did you consider when deciding
whether or not to participate? Think about what you would do if you were on
the other side. What would you do to try and convince someone to participate in a survey?
Participant Involvement
Participation in studies can best be understood within a social exchange context. Social exchange – means a cost/benefit
approach Once potential sample members are
contacted, they must decide about cooperation after thinking about the costs and benefits.
Participant Involvement
Respondents are asked to give up their time, engage in interactions controlled by the interviewer, think about issues or topics that might cause discomfort, and take the risk of being asked to reveal embarrassing information.
Potential participants might worry about privacy and lack of control over the information
Participant Involvement
Conversely, participants might want to participate because of an interest in a topic, a desire to share their views, or knowledge that their information will be useful to science and society
Participant involvement
Possible ways to increase participant rates Advance mailings Incentives
Giving small gifts Token cash payments
Focusing on the interesting aspects of participation Re-contacting participants to encourage participation Minimize the costs of participation
For example, time and possible embarrassment
Self-Administered Questionnaires
Self-administered questionnaires A questionnaire that the respondent completes by him or
herself Can be written or web-based survey
Self-Administered Questionnaires
Interview A data collection method in which respondents answer
questions asked by an interviewer Can be done by phone or in person
Self-Administered Questionnaires
Group-administered questionnaires Questions administered to respondents in a group setting
Self-Administered Questionnaires
Group-administered questionnaires Advantages
Allows the researcher to explain the instructions and answer questions.
Provides the researcher some control over the setting in which the questionnaire is completed
Allows the respondents to participate anonymously Helps in getting honest answers to sensitive questions
Usually results in a good response rate Inexpensive
Self-Administered Questionnaires
Group-administered questionnaires Disadvantages
There might be no group setting for the population the researcher wants to study
A group setting might raise concerns over the violation of voluntary participation
Groups typically have limits for the amount of time they spend on a survey
Self-Administered Questionnaires
Mailed questionnaires Questionnaires mailed to the respondent’s residence or
workplace
Self-Administered Questionnaires
Mailed questionnaires Advantages
Inexpensive Reasonably effective No time requirement, allows the participant to not
feel rushed Answering in private, creates fewer social pressures
and expectations
Self-Administered Questionnaires
A questionnaire can be individually administered or administered as an Internet questionnaire
An individually administered questionnaire is hand delivered to the respondent and picked up after completion Similar to using mailed questions, but the expense is
higher and has a better response rate
Self-Administered Questionnaires
An internet questionnaire is sent by e-mail or posted on a website
Self-Administered Questionnaires
Coverage error A sampling error that arises when the
sampling frame is different from the intended population
Self-Administered Questionnaires
Cover letter The letter accompanying a questionnaire that
explains the research and invites participation
Interviews
Structured interview Interviewer reads a standardized set of questions
and the response option for closed-ended questions.
The interview has some similarities to a conversation, except that the interviewer controls the topic, asks the questions, and does not share experiences or opinions.
Interviews
The set of instructions to the interviewer, the list of questions, and the answer categories make up the interview schedule
The use of structured interviews allows for some flexibility in administration, clarification of questions, and the use of follow-up questions.
Interviews
Interviews typically have good response rates Interviews are more expensive than questionnaires,
because interviewers have to be hired and trained. Using an interviewer adds the potential of
interviewer effect, or the changes in respondents’ behaviors or answers that result from some aspect of the interview situation
Interviews
In-person interviews The dominant mode of data collection from 1940 to 1970. An in-person interview is conducted face to face. In-person interviews are a good choice for questions
involving complex reports of behavior, for groups difficult to reach by phone, or for respondents who need to see material or to consult records as part of the data collection, or when the interview is long, and high response rates are essential.
Interviews
In-person interviews Rapport
A sense of interpersonal harmony, connection, or compatibility between an interviewer and a respondent
Interviews
Phone interviews The dominant mode of survey data collection since the
1970’s Preferred because it can yield close to the same results as
an in-person interview at half the cost Good for people who feel too busy for face-to-face
interviews New technology allows makes it cost effective to do
computer-assisted telephone interviews in which data are collected, stored, and transmitted during the interview
Constructing Questions
Researchers make a series of decisions based on assumptions about how respondents read or hear the questions that are asked.
Constructing Questions
Types of questions Open-ended questions
Questions that allows respondents to answer in their own words
Constructing Questions
Types of questions Closed-ended questions
Questions that include a list of predetermined answers
Answer categories must be exhaustive and mutually exclusive
Constructing Questions
Types of Questions Deciding whether to use open- or closed-
ended questions involves several issues. Answer choices can provide a context for the
question, they can make the completion and coding of questionnaires and interviews easier.
Respondent might not find the response that best fits what they want to say, and answer categories can be interpreted differently by different respondents.
Constructing Questions
Types of questions Open-ended questions
All open-ended responses must be categorized before the researcher does statistical analysis
A limited number of answer categories must first be created for each question, so the data may be coded
Constructing Questions
Types of questions Coding
The process of assigning data to categories Coding is a time consuming and expensive task that
can result in the loss of data richness.
Constructing Questions
Types of questions Screening questions
Question that asks for information before asking the question of interest
Example Have you applied to a graduate or professional program?
Yes or No If yes, please list the program(s)
Example It is important to ask someone if they voted in November,
before asking whom they voted for.
Constructing Questions
Types of questions A screening question is often followed with a
contingency question A contingency question
A question that depends on the answer to the previous question.
Example If yes, go to question #___
Example Do you work for pay outside the home? If yes, how
many hours per week are you employed?
Constructing Questions
Types of questions Occasionally researchers use indirect questions, where
the link between the information the information desired and the question is not obvious
This is done by creating scenarios or vignettes about people or situations that the researcher creates to use as part of the data collection method.
Constructing Questions
How to ask questions Keep the questionnaire or interview as short as
possible Only the necessary questions planned for data
analysis should be asked A preliminary draft or pilot test should be used
with a small sample A pilot test is a preliminary draft of a set of
questions that is tested before the actual data collection.
Constructing Questions
How to ask questions Avoid loaded words – words that trigger an
emotional response or strong association by their use
Avoid ambiguous words – words that can be interpreted in more than one way
Don’t use double negative questions – questions that can ask people to disagree with a negative question
Constructing Questions
How to ask questions Don’t use leading questions – questions that
encourage the respondent to answer in a certain way, typically by indicating which is the “right” or “correct” answer
Avoid threatening questions, or questions that make respondents afraid or embarrassed to give an honest answer
Constructing Questions
How to ask questions Don’t use double-barreled or compound questions –
questions that ask two or more questions in one Ask question in the language of your respondents, using
the idioms and language appropriate to the sample’s level of education, vocabulary of the region, etc.
Review Box 9.9 Examples of poorly worded questions and the suggested
revisions
Constructing Questions
How to ask questions Question order & context
Responses to questions can be affected by the question order as earlier questions provide a context for later ones and people may try to be consistent in their answers to questions on the same topic.
Constructing Questions
How to ask questions Question order & context
1. Consider a logical order that makes participation easy
2. To encourage participation start with interesting, no threatening questions, and save questions about sensitive topics for the middle or end
3. Conclude with “cool down” to minimize discomfort of participant
Summary
Considerations when choosing a data collection method
Quiz – Question 1
Open-ended questions allow for respondents to answer
a. in their own words.
b. according to pre-determined categories.
c. as often as they wish.
d. skip contingency questions
e. none of the above
Quiz – Question 2
If close-ended questions are used, it is important that the responses are
a. mutually exclusive.
b. exhaustive.
c. created by respondents.
d. a and b
e. All of the above
Quiz – Question 3
Which type of survey tends to have the highest response rate?
a. Mailed questionnaire
b. Phone interview
c. In-person interview
d. All three tend to yield high response rates
e. None of these yield high response rates
top related