Adler clark 4e ppt 09

Questionnaires and Structured Interviews

Chapter 9

Introduction

Questionnaires and structured interviews are the two most widely used methods of data collection in social science research.

Introduction

Questionnaire A data collection instrument with questions

and statements that are designed to solicit information from respondents

Introduction

Structured interview A data collection method in which an

interviewer reads a standardized list of questions to the respondent and records the respondent’s answers

Introduction

Survey A study in which the same data are collected from

all members of the sample Usually in the form of answers to questions Often use large probability samples and cross-

sectional study design

Introduction

Respondents The participant in a survey who completes a

questionnaire or interview

Introduction

Self-report method Another name for questionnaires and interviews

because respondents are most often asked to report their own characteristics, behaviors, and attitudes.

The Uses of Questionnaires and Interviews Questions can be asked about a variety of topics It is common to use the self-report method to gather

information on attitudes, beliefs, values, goals, and expectations

Questions can also be used to gather a person’s level of knowledge on something or behavior

Introduction

Stop and Think The surveys by Public Agenda and Pew

focused on attitudes towards national energy policies and health care, respectively.

Suggest a list of other topics that you think you could ask questions about using a questionnaire.

Would you be concerned about the accuracy of the answers on any of these topics?

Focal Research

Environmentalism Among College Students: A Sociological Investigation by Brandon Lang and Christopher W. Podeschi Asked college students questions about

environmental issues Interested in whether the students’ attitudes

were connected to some background characteristics and to behavior patterns

Descriptive and explanatory purposes

Focal Research

Ethics The researchers submitted a proposal to their

university’s Institutional Review Board (IRB) where it was approved.

The respondents were informed about the study’s sponsor, were assured that the surveys were anonymous and that they could skip any of the questions.

The Uses of Questionnaires and Interviews


Although surveys are widely used there are important concerns about their validity

When using self-report methods it is based on the implicit assumption that people have the information being asked and they will answer based on their core beliefs and values


Answers to questions may be compromised if the participants Do not understand Do not take the time to adequately answer the

question Influenced by current mood, events, etc.

For example, research has shown that people who are in more positive moods answer more favorably compared to those who are feeling more negative (Martin, Abend, Sedikes, & Green, 1997)

The Uses of Questionnaires and Interviews Measurement error

Error that occurs when the measurement we obtain is not an accurate portrayal of what we tried to measure

The giving of inaccurate answers to the questions


Measurement error may also be affected by question wording respondent’s level of knowledge on the topic whether or not the respondent has an opinion about

the topic respondent having an accurate memory respondent wanting to present a good image

Participant Involvement

Response rate The percentage of the sample contacted that

actually participates in a study


Response rate is affected by The number of people who cannot be

reached for the study (noncontacts) The number who choose not to participate

(refusals) The number who are incapable of performing

the tasks required of them For example, due to illness or language barriers


Nonresponse error Results from differences between nonresponders and

responders to a survey


When participants and nonparticipants differ in social characteristics, opinions, attitudes, values, or behavior – then generalization to a larger population is much more difficult

Some researchers have questioned whether higher rates of refusal automatically means more bias


Stop and Think Have you ever been asked to complete a

questionnaire or an interview? What factors did you consider when deciding

whether or not to participate? Think about what you would do if you were on

the other side. What would you do to try and convince someone to participate in a survey?


Participation in studies can best be understood within a social exchange context. Social exchange – means a cost/benefit

approach Once potential sample members are

contacted, they must decide about cooperation after thinking about the costs and benefits.


Respondents are asked to give up their time, engage in interactions controlled by the interviewer, think about issues or topics that might cause discomfort, and take the risk of being asked to reveal embarrassing information.

Potential participants might worry about privacy and lack of control over the information


Conversely, participants might want to participate because of an interest in a topic, a desire to share their views, or knowledge that their information will be useful to science and society

Participant involvement

Possible ways to increase participant rates Advance mailings Incentives

Giving small gifts Token cash payments

Focusing on the interesting aspects of participation Re-contacting participants to encourage participation Minimize the costs of participation

For example, time and possible embarrassment

Self-Administered Questionnaires

Self-administered questionnaires A questionnaire that the respondent completes by him or

herself Can be written or web-based survey


Interview A data collection method in which respondents answer

questions asked by an interviewer Can be done by phone or in person


Group-administered questionnaires Questions administered to respondents in a group setting


Group-administered questionnaires Advantages

Allows the researcher to explain the instructions and answer questions.

Provides the researcher some control over the setting in which the questionnaire is completed

Allows the respondents to participate anonymously Helps in getting honest answers to sensitive questions

Usually results in a good response rate Inexpensive


Group-administered questionnaires Disadvantages

There might be no group setting for the population the researcher wants to study

A group setting might raise concerns over the violation of voluntary participation

Groups typically have limits for the amount of time they spend on a survey


Mailed questionnaires Questionnaires mailed to the respondent’s residence or

workplace


Mailed questionnaires Advantages

Inexpensive Reasonably effective No time requirement, allows the participant to not

feel rushed Answering in private, creates fewer social pressures

and expectations


A questionnaire can be individually administered or administered as an Internet questionnaire

An individually administered questionnaire is hand delivered to the respondent and picked up after completion Similar to using mailed questions, but the expense is

higher and has a better response rate


An internet questionnaire is sent by e-mail or posted on a website


Coverage error A sampling error that arises when the

sampling frame is different from the intended population


Cover letter The letter accompanying a questionnaire that

explains the research and invites participation

Interviews

Structured interview Interviewer reads a standardized set of questions

and the response option for closed-ended questions.

The interview has some similarities to a conversation, except that the interviewer controls the topic, asks the questions, and does not share experiences or opinions.

Interviews

The set of instructions to the interviewer, the list of questions, and the answer categories make up the interview schedule

The use of structured interviews allows for some flexibility in administration, clarification of questions, and the use of follow-up questions.

Interviews

Interviews typically have good response rates Interviews are more expensive than questionnaires,

because interviewers have to be hired and trained. Using an interviewer adds the potential of

interviewer effect, or the changes in respondents’ behaviors or answers that result from some aspect of the interview situation

Interviews

In-person interviews The dominant mode of data collection from 1940 to 1970. An in-person interview is conducted face to face. In-person interviews are a good choice for questions

involving complex reports of behavior, for groups difficult to reach by phone, or for respondents who need to see material or to consult records as part of the data collection, or when the interview is long, and high response rates are essential.

Interviews

In-person interviews Rapport

A sense of interpersonal harmony, connection, or compatibility between an interviewer and a respondent

Interviews

Phone interviews The dominant mode of survey data collection since the

1970’s Preferred because it can yield close to the same results as

an in-person interview at half the cost Good for people who feel too busy for face-to-face

interviews New technology allows makes it cost effective to do

computer-assisted telephone interviews in which data are collected, stored, and transmitted during the interview

Constructing Questions

Researchers make a series of decisions based on assumptions about how respondents read or hear the questions that are asked.


Types of questions Open-ended questions

Questions that allows respondents to answer in their own words


Types of questions Closed-ended questions

Questions that include a list of predetermined answers

Answer categories must be exhaustive and mutually exclusive


Types of Questions Deciding whether to use open- or closed-

ended questions involves several issues. Answer choices can provide a context for the

question, they can make the completion and coding of questionnaires and interviews easier.

Respondent might not find the response that best fits what they want to say, and answer categories can be interpreted differently by different respondents.


Types of questions Open-ended questions

All open-ended responses must be categorized before the researcher does statistical analysis

A limited number of answer categories must first be created for each question, so the data may be coded


Types of questions Coding

The process of assigning data to categories Coding is a time consuming and expensive task that

can result in the loss of data richness.


Types of questions Screening questions

Question that asks for information before asking the question of interest

Example Have you applied to a graduate or professional program?

Yes or No If yes, please list the program(s)

Example It is important to ask someone if they voted in November,

before asking whom they voted for.


Types of questions A screening question is often followed with a

contingency question A contingency question

A question that depends on the answer to the previous question.

Example If yes, go to question #___

Example Do you work for pay outside the home? If yes, how

many hours per week are you employed?


Types of questions Occasionally researchers use indirect questions, where

the link between the information the information desired and the question is not obvious

This is done by creating scenarios or vignettes about people or situations that the researcher creates to use as part of the data collection method.


How to ask questions Keep the questionnaire or interview as short as

possible Only the necessary questions planned for data

analysis should be asked A preliminary draft or pilot test should be used

with a small sample A pilot test is a preliminary draft of a set of

questions that is tested before the actual data collection.


How to ask questions Avoid loaded words – words that trigger an

emotional response or strong association by their use

Avoid ambiguous words – words that can be interpreted in more than one way

Don’t use double negative questions – questions that can ask people to disagree with a negative question


How to ask questions Don’t use leading questions – questions that

encourage the respondent to answer in a certain way, typically by indicating which is the “right” or “correct” answer

Avoid threatening questions, or questions that make respondents afraid or embarrassed to give an honest answer


How to ask questions Don’t use double-barreled or compound questions –

questions that ask two or more questions in one Ask question in the language of your respondents, using

the idioms and language appropriate to the sample’s level of education, vocabulary of the region, etc.

Review Box 9.9 Examples of poorly worded questions and the suggested

revisions


How to ask questions Question order & context

Responses to questions can be affected by the question order as earlier questions provide a context for later ones and people may try to be consistent in their answers to questions on the same topic.


How to ask questions Question order & context

1. Consider a logical order that makes participation easy

2. To encourage participation start with interesting, no threatening questions, and save questions about sensitive topics for the middle or end

3. Conclude with “cool down” to minimize discomfort of participant

Summary

Considerations when choosing a data collection method

Quiz – Question 1

Open-ended questions allow for respondents to answer

a. in their own words.

b. according to pre-determined categories.

c. as often as they wish.

d. skip contingency questions

e. none of the above

Quiz – Question 2

If close-ended questions are used, it is important that the responses are

a. mutually exclusive.

b. exhaustive.

c. created by respondents.

d. a and b

e. All of the above

Quiz – Question 3

Which type of survey tends to have the highest response rate?

a. Mailed questionnaire

b. Phone interview

c. In-person interview

d. All three tend to yield high response rates

e. None of these yield high response rates

Adler clark 4e ppt 09

Documents