Design of questionnaire Holger Diessel University of Jena [email protected] http://www.holger-diessel.de/
Dec 28, 2015
Interview
Questionnaire
Experiment
Steps
1. Establish the goals of the project - What you want to find out
2. Determine your sample - Whom you will ask
3. Choose the medium – How will you ask
4. Create your questionnaire - What you will ask
5. Pre-test the questionnaire - Test the questions
6. Run the study and enter data - Ask questions and categorize
answer.
7. Analyze the data - Produce the report
The goal of your study
State a precise research question (or hypothesis)
The sample
Who do you want to ask?
How many subjects do you want to ask?
How do you select your subjects?
Convenient sample Simple random sample Stratified random sample Cluster sample
Sampling
• Age• Gender• Educational background• Profession• Regional background• Native language• Special interests
Features to stratify
• Reading questions aloud• Written questionnaire in face-to-face situation• Written questionnaire by mail or email• Internet survey
The medium
• Keep your questionnaire short• Keep it simple
Form
• Welcome message• Brief explanation as to why you conduct the study• Warming-up
Introduction
Question types
(1) What do you
like?
(2) Do you like fish?
Open-ended vs. close-ended
(1) What kind of music do you like?
(2) What kind of music do you like? (a) classical, (b) pop, (c) rock, (d) country
Rating scales
How would you rate this product?
1. Excellent2. Good3. Fair4. Poor
Rating scales
On a scale where 10 means you have a great amount of interest in a subject and 1 means you have none at all, how would you rate your interest in each of the following topics?1. Domestic politics2. Foreign affairs3. Science and health4. Business
Rating scales
Strongly agree
Agree Disagree Strongly disagree
The teacher provides constructive criticism
The teacher follows the course schedule
I would prefer to study more at home
How much do you agree with each of the following statements?
Rating scales
Jena is a beautiful city.
Strongly agree
Strongly disagree
Rating scales
How often do you speak dialect?
Always Frequently Sometimes Rarely Never
Rating scales
Compared to other students, how would you rate this student’s performance?
Lower 50% Upper 50% Upper 25% Upper 10% Upper 5%
Rating scales
Rating scales are also often used to elicit frequency information. How often do you call your Mom?
__ Less than once a month__ About once a month__ About twice a month__ Every week__ Twice a week__ Every other day
Rating scales
Always present <agree> before <disagree> and >positive> before <negative>.
Rating scales
How many scores should your rating scale include?
Rating scales
Jena is a beautiful city.
Strongly agree
Strongly disagree
Strongly agree
1 2 3 4 5 Strongly disagree
Rating scales
What do you think about product X?
It's the best on the market It's about averageIt's the worst on the market
Direct vs. indirect questions
Direct: ‘What is the best instance of X?’ ‘What is the second best instance of X?’
Indirect: ‘What is more similar to X?’
Don’t know / not applicable
For some questions it may be appropriate to allow for a “Don't Know” or “Not Applicable” response.
Ordering of questions
• Motivation• Influence
Ordering of questions: Motivation
Ideally, the early questions in a questionnaire should be easy and pleasant to answer.
Ordering of questions: Influence
• Priming• Habituation
Wording of questions
Make your questionnaire simple.
Wording of questions
Avoid ‘double-barreled’ questions’.
The subject in German is the first element of the clause and sometimes appears in dative case.
Wording of questions
Avoid emotionally charged words or leading questions that point towards a certain answer.
1. What do you think of the XYZ proposal? 2. What do you think of the Republican XYZ proposal?
Wording of questions
Make sure your questions accept all the possible answers.
Is the NP the subject or object?
Coding
If you ask open questions you have to develop a coding scheme .
Layout
Always consider the layout of your questionnaire. You want to make it attractive, easy to understand and easy to complete. Pictures and answering cells can serve this purpose.
Pre-test
Run a small pre-test before you conduct the study.