Constructing Questionnaires Interviews

7/28/2019 Constructing Questionnaires Interviews

1/29


2/29

What do questionnaire/interviews measure?

What someone knows?

What someone likes?

What someone thinks?

KNOWLEDGE &INFORMATION

VALUES &

PREFERENCES

ATTITUDES &

BELIEFS


3/29

How to assess questionnaire/interviews validity?

Influence respondents to show themselves in

a good light?

Influence respondents to attempt to anticipate

what researchers what to hear or learn?

Ask for information about respondents that

they may not know about themselves?

TO WHAT EXTENT

MIGHT A QUESTION:


4/29

How to ask the questions?

DIRECTQUESTIONS

INDIRECT

QUESTIONS

Ask someone whether or nothe likes his job

Ask what he thinks of his job or

selected aspects of it

LIES HOW OBVIOUSLY THE QUESTIONS SOLICIT

SPECIFIC INFORMATION

Frank and open responses more Q


5/29


SPECIFICQUESTIONS

NON-SPECIFIC

QUESTIONS

Focus on object, idea as inputto attitude belief, or concept

Questions that probe for more

general areas

SPECIFIC QUESTIONS MAY CAUSE RESPONDENTS TO

BECOME CAUTIOUS TO GIVE LESS-HONEST ANSWERS

How he feels with a particular class

taught by the teacher

How he likes a particular teacher


6/29


PROVIDEFACTS

REQUEST

OPINIONS

Do not always elicit factualanswers

Do not necessarily elicit honest

answers

DUE TO FAULTY MEMORY OR A CONSCIOUS DESIRE TO

CREATE A PARTICULAR IMPRESSION, FACTUAL OR

OPINION QUESTIONS DO NO ELICIT HONEST ANSWERS

SO MINIMIZE SOURCES OF BIAS

Preference for Toyota or Honda models

What type of car he owns


7/29


QUESTIONFORM

STATEMENT

FORM

Respondent provides anappropriate answer

Respondent indicates whether

statement is true or false

The OSEC should be shortened.

____ Agree ____ Disagree

Do you think that the OSEC should be

lengthened? ___ Yes ___ No

FORMATS ARE INDISTINGUISHABLE IN THEIR

POTENTIAL FOR ELICITING HONEST RESPONSES

SELECT FORMAT BASED ON RESPONSE MODE


8/29


PREDETERMINEDQUESTIONS

RESPONSE-KEYED

QUESTIONS

Require respondents tocomplete list of questions

Subsequent questions may or

may not call for answers

Are you a college graduate?

____ Yes ____ No (skip next Q)

Age _____: Gender _____:

Educational Attainment _____:

POSSIBLE QUESTIONS KEYED TO THE RESPONSE

WHAT? WHY? HOW?


9/29

How to answer the questions?

UNSTRUCTUREDRESPONSES

What led you to go to OSEC?

________________________________________________________

Describe your feelings as you think of your partner.

________________________________________________________

Allows respondent to give a responsein whatever form he chooses

EXERT LITTLE CONTROL OTHER THAN BY ASKING

QUESTIONS AND LIMITING THE AMOUNT OF SPACE

PROVIDED FOR THE ANSWER

RESPONSE MODE


10/29


STRUCTUREDRESPONSES

Place a check opposite each item in the following list to indicate your

answer.

What are your chances of reaching your goal?

____ Excellent ____ Good _____ Fair _____ Poor _____ Very Poor

Offer answers and ask respondent tochoose one

CENTERS ON THE TYPE OF RESPONSE THAT THE

RESPONDENT IS ALLOWED TO MAKE

RESPONSE MODE


11/29


FILL-IN RESPONSE

What is your occupation? ___________________________________

In what school did you do your undergraduate work? ______________

Looking at the above picture, what word best describe the way it

makes you feel? __________________________________________

Transitional mode betweenstructured & unstructured questions

ALTHOUGH REQUIRING RESPONDENT TO GENERATE

RATHER THAN CHOOSE A RESPONSE, IT TYPICALLY

LIMITS RANGE OF POSSIBLE RESPONSES BY LIMITING

THE ANSWER TO A SINGLE WORD OR PHRASE

RESPONSE MODE


12/29


TABULARRESPONSE

Suppose you were offered an opportunity to make a substantial

advance in a job. Indicate how important it would be in stopping you in

making that advance.

Would Might stop me from Wouldnt matter

stop me making change at all

Work harder

Leave family

Requires numbers, words, phrasesor degree of agreement along a scale

RESPONSE THAT INCLUDE A VARIETY OF INFORMATION

AND ORGANIZES EITHER FILL-IN OR SCALED

RESPONSES

RESPONSE MODE


13/29


SCALEDRESPONSE

Degree of likelihood:

________________________________________________

Excellent Good Fair Poor Very Poor

Degree of frequency:

Always Often Seldom Never

Respondent express endorsement orrejection of an attitude statement

SCALED RESPONSES CAN YIELD INTERVAL DATA

RESPONSE MODE


14/29


FREQUENCY

QUESTIONNAIRE

Instructions: All of the questions below are about what you actually do.

If you Always do what the statement says, circle the 1 for A. If you

Often do what the statement says, circle the 2 for O. If you Seldom

do what the statement says, circle the 3 for S. If you Never do whatthe statement says, circle the 4 for N.

There are no right or wrong answers for these questions. We are

interested only in what you actually do.

a. I think about what I will do when I finish OSEC.1. A 2. O 3. S 4. N

b. I talk to officers to learn about their jobs.

1. A 2. O 3. S 4. N

RESPONSE MODE


15/29


RANKINGRESPONSE

Rank the following activities in terms of their usefulness to you as you

write your research objectives. (Assign the numbers 1 through 5, with

5 indicating the most useful activity and 1 indicating the least useful. Ifany activity gave you no help at all, indicate this by 0).

___ Initial discussion of research lecturer

___ Initial small-group activity

___ Weekly student sessions___ Individual session with adviser

___ Examples of research objectives

Respondent to rank order series ofstatements based on a criterion

RANKING FORCES RESPONDENT TO GIVE CRITICAL

ESTIMATES OF THE VALUES OF ALTERNATIVES

RESPONSE MODE


16/29


CHECKLISTRESPONSE

I get most of my professional and intellectual stimulation from:

Check one of the following blanks:

___ A. Lecturers

___ B. Academic Dean

___ C. Director

Respondent select one of thepossible choices in a checklist item

FORM OF RESPONSE DOES NOT REPRESENT A SCALE,

RATHER THEY ARE NOMINAL CATEGORIES

RESPONSE MODE


17/29


CATEGORICALRESPONSE

Are you a veteran?

____ Yes ____ No

Research counseling does not begin early enough.

____ True ____ False

Offers respondent only twopossibilities for each item

YES-NO DICHOTOMY IS OFTEN USED IN THE

CATEGORICAL RESPONSE MODE.

ATTITUDE-RELATED ITEMS MAY GIVE TRUE-FALSE

ALTERNATIVES.

RESPONSE MODE


18/29

How to construct questionnaire/interviews?

WHAT QUESTIONS SHOULDYOU ASK?

WHAT FORMATS SHOULD

YOU USE?

WHAT RESPONSE MODES

SHOULD YOU EMPLOY?


19/29

SPECIFYING VARIABLES TO MEASURE

STUDY RELATING SOURCE OF OCCUPATIONAL

TRAINING TO WORK PERFORMANCE:

---------------------------------------------------------------------

Occupational Training PNPA, PMA, University

Work Performance Level High, Medium, Low

QUESTIONS REFLECT INFORMATION RELATED TO HYPOTHESES OR

RESEARCH QUESTIONS

SPECIFY VARIABLES BY NAME


20/29

CHOOSING THE QUESTION FORMAT DECISION IS WHETHER TO PRESENT ITEMS IN A WRITTEN

QUESTIONNAIRE OR AN ORAL INTERVIEW

CONSIDERATION INTERVIEW QUESTIONNAIRE

Personnel to collect data

Major expense categories

Opportunities for response keying

Opportunities for askingOpportunities for probing

Magnitude of data reduction

Number of respondents reached

Rate of return

Sources of error

Overall reliability

Emphasis on writing skill

Interviewers

Payments to interviewers

Extensive

Extensive Possible

Great

Limited

Good

Interviewer, instrument,coding, sample

Quite limited

Limited

Enumerators

Postage/printing

Limited

Limited Difficult

Limited to roster

Extensive

Poor

Instrument,sample

Fair

Extensive


21/29

CHOOSING THE RESPONSE MODE KIND OF INFORMATION DETERMINE THE MOST SUITABLE MODE

CHOICE OF RESPONSE MODE SHOULD BE BASED ON

THE MANNER IN WHICH THE DATA WILL BE TREATED

RESPONSEMODE TYPE OFDATA ADVANTAGES DISADVANTAGES

Fill-in Nominal Limited bias

Expanded flexibility

Difficult to score

Scaled Interval Easy to score Time-consuming

Potential for bias

Ranking Ordinal Easy to score

Forces discrimination

Difficult to complete

Checklist or

Categorical

Nominal* Easy to score

Easy to respond

Limited data and option

* May be interval when totaled


22/29

CHOOSING THE RESPONSE MODE

FILL-IN

RULE OF THUMB USE THE SCALED RESPONSE MODE

SINCE THE INTERVAL DATA CAN ALWAYS BE

TRANSFORMED INTO ORDINAL OR NOMINAL SCALE

SCALED

RANKING

Group into ranges to provide nominal data

for chi-square analysis

Applicable to parametric statistical

analysis since considered as interval data

May provide less information since they

generate ordinal data


23/29


24/29

PREPARING QUESTIONNAIRE ITEMS

1. Check one of the following to show how much of the time

you feel satisfied with your job.

A ___ most of the time B ___ about half of the time

2. Describe your health since entering the police force.CIRCLE ONE.

Excellent Good Average Poor

3. Did you receive training for the job which you are now

employed. (CHECK ONE BELOW)___ high school ___ college ___ armed forces

___ others (please specify) _______________________

IS THIS WHAT I WANT TO MEASURE?


25/29


Instructions: All questions are statements to which we seek your

agreement. If you Strongly Agree with any statement, circle the 1. If

you Agree but not strongly with any statement, circle the 2. If you

Disagree but not strongly, circle the 3. If you Strongly Disagree with

any statement, circle the 4.

There are no right or wrong answers for these questions. We areinterested only in how you feel about the statements.

I believe it is important for me to participate in OSEC.

1. SA 2. S 3. D 4. SD

2. I believe OSEC can offer many achievements.

1. SA 2. S 3. D 4. SD

3. OSEC schoolwork bores me.

1. SA 2. S 3. D 4. SD



26/29


Name ________________________ Course Name ________________

Lecturer ___________________

SATIFACTION SCALE

Do you feel like skipping the class?

____1_________2___________3___________4___________5_____

Never Rarely Sometimes Often Always

2. Do you like this class?

____1_________2___________3___________4___________5_____

Very much Quite a bit Its all right Not so much Hate it

3. How much do you feel you have learned in this class?

____1_________2___________3___________4___________5_____

A great deal Quite a bit A fair amount Not much Nothing



27/29

INSTRUMENT DEVELOPMENT PROCESS

MANAGEMENT

QUESTION

QUESTION HIERARCHY

RESEARCH

QUESTION

INVESTIGATIVE

QUESTIONS

MEASUREMENT

QUESTIONS

Problem which management must answer

Answer to contribute solution to management

problem

Specific questions to answer research question

Detailed questions which respondents must

answer


28/29


CONTENT

QUESTION CONSTRUCTION

SHOULD THE QUESTION BE ASKED?

(significance to research topic)

IS THE QUESTION OF PROPER SCOPE?

(coverage)

CAN THE RESPONDENT ANSWER ADEQUATELY?

(subject of question)

WILL RESPONDENT ANSWER ADEQUATELY?

(question sensitivity)


29/29


WORDING

QUESTION CONSTRUCTION

Is the question stated in terms of shared vocabulary?

(common knowledge)

Is the question clear?

(understandable)

Is there biased wording?

(choice of word)

Is there the right degree of personalization?

(first or third person)

Constructing Questionnaires Interviews

Documents