a.k.a. “paper-pencil” measures or “self- report” measures represents the dominant paradigm for social science research in the last 30 years advantages:

a.k.a. “paper-pencil” measures or “self-report” measures

represents the dominant paradigm for social science research in the last 30 years

advantages: expedience in gathering data yields interval level data

Disadvantages People may not know their own minds People may misreport information

problems: problem of

“mindfulness,” verbal reports of mental processes are suspect

problem of “non-attitudes,” people tend to venture an opinion whether they have one or not

unreliability of self-reports data: even factual information can be misreported

intentional misreporting of information.

social desirability biasProviding socially correct

responses

reasons for non-disclosure embarrassment fear of negative sanctions shame, public humiliation privacy

sometimes helpful to include a “lie scale”

phrasing of questions is critical

avoid vague, nebulous questions Bad example: “What do

you think about abortion?” Bad example: “How many

children do you have?” questions must be clear,

unambiguous Good example: “Which of

the following statements best represents your attitude toward elective abortions in the first trimester?”

Conduct a “readability analysis”

avoid lengthy questions; keep questions short, succinct Bad example: If she runs

for president in 2016 and wins Hillary Clinton would become the first female President of the United States. Do you think her gender would be a problem in negotiating with the Arab world, which is a male-dominated culture?

Good example: Would Hillary Clinton make a good Supreme Court justice?

avoid double-barreled questions (compound questions) Bad example: “Do you

favor stricter hand gun controls and mandatory minimum sentences for carjackers?”

avoid loaded language (push polling) Bad example: “Don’t you

think that...?” “Isn’t it true that...?”

Bad example: Emotionally charged words: “ gang member,” “welfare mother,” “extremist groups,” “spin doctor,” etc.

avoid slang, jargon, abbreviations and acronyms Bad example: “Should

states regulate PETA and the ALF?”

Bad example: “Do you think hip hop is whack?”

avoid or minimize negative wording Bad example: Do you

disagree with Romney’s decision not to disclose which tax loopholes he would close?

Good example: Should Romney disclose the specific tax loopholes he would close?

Schuman, Ludwig, & Krosnick (1986): 60% of respondents selected one of four options in a close-ended format, but only 2.4% mentioned any of the same four responses in an open-ended format. open-ended: allows subjects more

leeway, flexibility

“What is your primary ethnic/cultural background?”

requires a content analysis of responses

close-ended or “forced-choice” ties respondents’ hands somewhat

easy to code the data always include an “other_____________”

category

even slight variations in wording can alter respondents’ answers. “occupied territory” versus “contested

territory” “assisted suicide” versus “mercy killing” “fetus” versus “unborn child”

make questions concrete, come down the “ladder of abstraction”

Use negatively worded questions or statements sparingly sometimes necessary to include reverse-

valenced items to identify a “response set”

Avoid evaluative language Phrasing of questions should not imply

approval, disapproval Follow-ups should not suggest surprise,

liking, disliking, etc. Bad example: Do you think the House

of Representatives, which is dominated by the Tea Party, should lift the harsh restrictions on stem cell research?

Bad example: Despite its poor track record in crisis intervention, do you think the United Nations should intervene in Sudan?

Respondents must be capable and willing to answer the questions May be unaware of topic/issue May be unwilling to disclose May lie, exaggerate, under-report or over-report example: If you ask people what nonverbal cues

they leak when they lie, people may not know. example: if you ask people what they would do

in a hostage crisis, how would they know?

order effects: sequencing of questions can be a factor “halo effect”: carry-over from one item to

the next use multiple versions of questionnaire “response set”: checking the same response

all the way down a questionnaire use “reverse scaling” to detect this

gender of researcher/respondent can influence responses

ethnicity of researcher/respondent can influence responses

medium can influence responses face-to-face interview, telephone

survey, mail survey, online survey anonymous versus non-anonymous

questionnaire fatigue: minimize length of surveys

Type of survey

Obtaining sample

Cooperation rate

Cost per respondent

advantages

disadvantages

in person difficult medium high interviewer rapport, nonverbal cues

respondent apprehension, expensive, time consuming

telephone easy high medium fast, random sample (random dialers)

call screening, limited to oral questions

mall intercept

easy medium medium to high

fast, in-person

can be costly, non-random sample

mail easy low medium respondents aren’t rushed, random sample

costly, low return rate

Online or computer-assisted

easy high low automatic data entry

requires computer literacy, non-random sample