Questionnaire Construction

Questionnaire Construction

Assessment Team

November 6, 2007

PowerPoint by JD White and Joel H. Scott

When using a Questionnaire…

Think about the purpose of your assessment.

Then, think about the data you want to collect.

Next, ask yourself if a questionnaire is the right tool for the job.

Remember, good questionnaires take time to develop. They are not just questions on a page.

Steps for Developing Questionnaires

Develop the Purpose

Create the Conceptual Framework

Write Questions

Design the Questionnaire

Developing the Purpose

This step likely started with your assessment plan.

Think about what, specifically, you want to know, why a questionnaire is appropriate, and what you want to do with the data.

Your purpose will assist in keeping you focused while developing your questionnaire.

Creating the Conceptual FrameworkThis is a multi-step process and will include a

great deal of brain storming.

Step 1: List all the questions/areas you might want to knowStep 1: List all the questions/areas you might want to know

Step 2: Organize these questions into categories and Step 2: Organize these questions into categories and subcategories subcategories

Step 3: Define these categoriesStep 3: Define these categories

Step 4: Make sure that you are asking items that will provide Step 4: Make sure that you are asking items that will provide enough data to fully answer your research assessment enough data to fully answer your research assessment questions.questions.

Can you name the Seven Wonders of the World?Did you start to list the Seven Wonders of the World or did you just say Did you start to list the Seven Wonders of the World or did you just say

“yes” or “no”? “yes” or “no”?

The most appropriate answer would be either yes or no. Question wording The most appropriate answer would be either yes or no. Question wording matters because it will affect the data you will collect.matters because it will affect the data you will collect.

Drafting Questions

This step can be challenging because it asks us to rethink how we ask questions.

Tip: Think about why you are asking a question and Tip: Think about why you are asking a question and if you really need the information.if you really need the information.

Questions have two distinct parts.Questions have two distinct parts.

1. Question Wording1. Question Wording

2. Response Format2. Response Format

Question Wording

Be as short and simple as possible.

Ask one question at a time.Ask one question at a time.

Don’t use jargon and the negative Don’t use jargon and the negative wording of a question.wording of a question.

Define/clarify items as needed. Define/clarify items as needed.

Question Wording cont.

Respondents don’t have the best memories, Respondents don’t have the best memories, so be specific and try not to ask about so be specific and try not to ask about generalities.generalities.

Be careful when asking threatening questions.Be careful when asking threatening questions.

Response Formats

Open response formats don’t limit/influence responses, but they can create data management issues.

Use open-ended responses strategically.Use open-ended responses strategically.

-Before asking specific questionsBefore asking specific questions- As summary for missed informationAs summary for missed information

Provide labels where possibleProvide labels where possible

Ex. How many days…? Ex. How many days…? ___days___days

Closed response formats can limit/influence responses, but they provide some control.

Response Formats

Categories: Categories: Use to measure factual information and Use to measure factual information and behavior. Make sure options are exhaustive and mutually behavior. Make sure options are exhaustive and mutually exclusive.exclusive.

Ordinal and Interval: Ordinal and Interval: Use the items to measure the intensity of Use the items to measure the intensity of attitudes or opinions and think about what scaling you should use attitudes or opinions and think about what scaling you should use (e.g., Likert, Uni-Dimensional, Semantic Differential).(e.g., Likert, Uni-Dimensional, Semantic Differential).

Issues to Consider

To Neutral or Not to Neutral.

Think about how a respondent enters a response scale and how many decisions the person must make.

Strongly Disagree

Disagree Neither Agree nor Disagree

Agree Strongly Agree

1 2 3 4 5

Issues to Consider cont.

Pre-coding can save time with data entry.

Has this presentation increased your understanding of questionnaire development?

1 Yes 2 No

There is more than one way to ask a question. Use a variety of question formats.

Pre-code

Types of Questions

Yes/No: Are you currently a member of the A-Team (Assessment Team)?

o Yes o No

Uni-dimensional:

How interested are you in being a member of the A-Team (Assessment Team) beginning fall 2007?

Not At All Moderately Interested

Very Interested

1 2 3 4 5Semantic Differential:

This presentation is:

Boring 1 2 3 Interesting

Designing the Questionnaire

Begin with “easy” questions first and more to Begin with “easy” questions first and more to more “challenging” questions.more “challenging” questions.

Think about placing demographic questions at the Think about placing demographic questions at the end if possible.end if possible.

Create a conversational tone in the opening text Create a conversational tone in the opening text and throughout the questionnaire.and throughout the questionnaire.

Limit demographic questions to only those that Limit demographic questions to only those that you you needneed..

Designing Questionnaires

Create sections to break up longer questionnaires.

Ask about behaviors before attitudes or opinions.

Use page space effectively.

Design can make a difference.

Formatting the Questionnaire Margin size, layout, and font size can make Margin size, layout, and font size can make a big difference.a big difference.

Tables are a good tool for helping to format Tables are a good tool for helping to format your questionnaire.your questionnaire.

Bullets and symbols can provide response Bullets and symbols can provide response circles, boxes, or other options.circles, boxes, or other options.

Shading can assist the respondent in Shading can assist the respondent in completing the questionnaire.completing the questionnaire.

What do you think?

Were you satisfied with the quality of our services and programs?

A. Yes

B. No

Double Barrel: Ask only one question at a time

What do you think?

What brand of television do you own?

A. Samsung

B. Panasonic

Too exclusive: What about other TVs?

What about the use of “Other”?

What do you think?

In the past two years, approximately how many times have you been to the gym?

A. 1-10 times

B. 11-20 times

C. 21-30 times

D. More than 30 times

Can you recall things from 2 years ago? What about the past two weeks?

What do you think?

Where did you grow up?

A. country

B. farm

C. city

Confusing Response Options: What if I live on a farm?

What do you think?

Are you in favor of Proposition 13?

____ Yes

____ No

____ Undecided

Assumptive Question: Does everyone know what Prop 13 is? Also, what if you “don’t know”?

What do you think?

What percent of your annual budget do you spend on advertising for student programs?

______%

Too difficult: You don’t want your respondents to pull their hair out.

What do you think?

Don’t you think this administration is spending too much money in the Iraq war?

A. Yes

B. No

Leading Question: Be careful of sharing your agenda.

What do you think?

What income tax bracket do you report under to the IRS?

A. under 30,000

B. between 30,000-99,000

C. 100,000 to 500,000

Threatening question/Be careful of using abbreviated terms-spell them out!

What do you think?

1. Do you currently own a vehicle? If no, go to question 3.

A. Yes

B. No

2. How many miles do you drive on a daily basis? ______

Be careful of BRANCHING Q’s

Why not this…

How many miles do you drive on a daily basis?

A. none, I do not own a vehicle

B. 1-10 miles

C. 11-20 miles

D. 21 or more miles

E. Other________________miles

What do you think?

What do you think about this report?

A. It’s the worst report I’ve read

B. It’s somewhere between the worst and best

C. It’s the best report I’ve read

Little variability in responses…How are you going to measure this?

Why not this…

Please respond to the degree you agree or disagree with the following statements.

I think this report is well written.

I think this report should be shared with everyone in the company.

I think this report has the potential to improve working conditions.

Questions?