Class # 6 Survey Research & Questionnaire Design.

Class # 6

Survey Research

&

Questionnaire Design

Types of Surveys- Audience Surveys & Consumer Surveys

- DESCRIPTIVE

to document conditions ... identify variables

- ANALYTICAL

to describe and analyze WHY

SurveyAdvantages

Advantages

1) can be done in realistic settings

2) reasonable costs ... get lots of info

3) large amount of data collected

4) data helpful for surveys already exists

Survey Disadvantages ...

Disadvantage

1) cannot manipulate the independent variable

2) inappropriate wording/placement of questions

3) who did we talk with anyway

4) those who misuse surveys hurt us all

So HOW do we obtain data through SURVEY research ?

- MAIL SurveysAdvantages --

Disadvantages --

- PHONE SurveysAdvantages --

Disadvantages --

- In-PERSON Surveys / InterviewsAdvantages --

Disadvantages --

Types of QUESTIONS on Surveys ...

Open Ended --

Where is the best place to obtain a Journalism degree?

Forced Choice --

Which of the following is the primary reason you selected ___.

Types of QUESTIONS on Surveys ...

Ranking --

Please rank the following from 1-5 as reasons why you chose ASU?

Creating the Questionnaire

Surveys...Scale development--- DeVellis

Scale items

Generate an item pool—

Every possible question you can think to write out.

Consider the latent variable—concepts and constructs

Consider the manifest variable- the obvious variables

Item Pool

Devillis (1991):

Choose items that reflect the scale’s purpose (p.54).

Each item is a test of the strength of the latent variable (p.54).

Consider readability of scale.

In word 2007, go to review..spelling

Question Wording ...

1 - Make Sure Question Wording is Clear ...

2 - Keep Questions Short ...

3 - Remember Purpose of Questionnaire/Survey ...

4 - Avoid DOUBLE-BARRELING ...

Question Wording ... I. Ambiguous Reference: When a reader does not know which of two or three earlier nouns a pronoun stands for

Examples

Ethel told Lucy that her pie was wonderful.

The files arranged by the temporary workers were out of order, so we sent them back to the main office.

Question Wording ... Misplaced modifiers A modifier is a word or a phrase that describes something else.

The young girl was walking the dog in a short skirt.

The dog was chasing the boy with the spiked collar.

The correct versions are:

The young girl in a short skirt was walking the dog.

The dog with the spiked collar was chasing the boy.

Question Wording ...

5 - Avoid BIASED Wording ...

6 - Avoid LOADED Questions ...

7 - Don’t ASSUME they know ...

8 - Put OPERATIVE clauses at the end ...

9 - Place Similar Type Questions/Responses together

Question Wording ...

Pre-testingChecking for variation...meaning..task difficulty..respondents interest and attention....

Writing the questions or statements

• Scale formats.• Thurstone: measuring parent ideas about

children.. Difficult to make happen. – Achieving success is the only way for my child to

repay my efforts as a parent. Agree or Disagree– Going to college and getting a job are important

but not essential to my child’s happiness. Agree or disagree.

Writing the questions or statements

• Scale formats.

• Guttman scale:• I would permit a child of mine to marry an immigrant.

• I believe that this country should allow more immigrants in.

• I would be comfortable if a new immigrant moved next door to me.

• I would be comfortable with new immigrants moving into my community.

• It would be fine with me if new immigrants moved onto my block. • I would be comfortable if my child dated a new immigrant.

– http://www.socialresearchmethods.net/kb/scalgutt.php

Seldom used

Writing the questions or statements

• Likert --• Please indicate your agreement with the

following statements.

Writing the questions or statements

• Likert statements: Use even number scale.– Responses to likerts– Strongly agree Moderately agree mildly agree– Mildly disagree moderately disagree strongly

disagree

Item Pool-Likert

Technical aspects--- Likert

I like weather forecasts that put the clouds into motion.

I like weather that use national radar to show where rain or snow is occuring in the United States.

When I can see where its raining in the United States I have a better understanding of when my home town will receive rain.

Item Pool-Likert

Informational Likert

 TV weather gives me more information about the weather than any other source.

 When the weather is bad the weather segment should be longer.

 Extended forecast should be used during a weather segment.

Today's forecast is important to me.

Tomorrow's forecast is important to me.

Item Pool-Likert

Personality Likert

I like a weather person who uses humor to present the weather.

I would rather see my weather person giving the weather in someone's back yard than in the TV studio.

Men are better weather announcers than women.

Item Pool-Demographics

Demographic---- Closed ended

I get most of my weather information from:

a. radio b. TV c. newspaper d. weather phone e. cable services

When are you most likely to be interested in the weather forecast?

a. morning b. afternoon c. evening 

Which media outlet is the most reliable source of weather information? 

Where should the weather forecast appear in the newscast?

Demographic Item Pool

Do you work? yes or no Can be a filter question.

What applies to you?

construction ___ professional ___

farming ___ retired ___

house husband ___ sales ___

housewife ___ small business ___

managerial ___ student ___

marketing ___ Other_________ ___

not employed ___

Semantic differential

• Semantic Differential -- EDUCATION

• DIFFICULT ___:___:___:___:___ EASY• IMPORTANT ___:___:___:___:___ UNIMPORTANT

• Used for attitude research.– Uses adjectives

• Opposites ends of a continuum

• Points along a continuum.

• Good_ _ _ _ _ _ Bad Bipolar

• Quiet_ _ _ _ _ _Noisy Bipolar

• Honest_ _ _ _ _ _Dishonest Unipolar

•

Writing the questions or statements

Visual Analog

• Similar to semantic differential.

• Best example– Level of pain

Writing the questions or statements

• Binary options...two response scales.

...Two answer... Variability limited

- Instructions ... what are you asking them to do ... tell how to do it

- Layout ... keep it clean, readable, professional ...

- Question Ordering ... put most difficult last ...

- Length ... how much is enough ? (info on each page)

- Pre-test Questionnaire ... Always TEST to make sure it works ...

Organizing the Survey Instrument/Coding Form

- Pre-test Questionnaire ... Always TEST to make sure it works ...

Get a expert to look at scales

Organizing the Survey Instrument/Coding Form

- Insure ‘FORM’ fits ‘CIRCUMSTANCES’ ... mail survey ... phone survey ... who’s completing?

- Remember ‘FILTER” Questions ... must respond certain way to continue ...

- Questions Should LINK to Res Questions ... wording must be parallel if it is going to work ...

- COVER LETTER ... don’t tell respondent too much ... stated deadlines & resultant problems ... promise of anonymity ... what to do if you have questions ....

Organizing the Survey Instrument/Coding Form

- Insure ‘FORM’ fits ‘CIRCUMSTANCES’ ... mail survey ... phone survey ... who’s completing?

- Remember ‘FILTER” Questions ... must respond certain way to continue ...

- Questions Should LINK to Res Questions ... wording must be parallel if it is going to work ...

- COVER LETTER ... don’t tell respondent too much ... stated deadlines & resultant problems ... promise of anonymity ... what to do if you have questions ....

Organizing the Survey Instrument/Coding Form

. . . Study other questionnaires to learnsequencing ... response options ...and statement flow ...