Chapter10

Chapter Ten

Questionnaire Design

Copyright © Houghton Mifflin Company. All rights reserved. !2

Chapter Objectives

• Discuss questionnaire design as a series of interrelated steps

• List and describe the different question forms and discuss when and how they can be used most effectively

• Recognize and reword problem questions. • State the suggested guidelines for proper question

sequencing • Discuss the importance of questionnaire appearance

and layout


Chapter Objectives (Cont’d)

• Explain why pretesting the questionnaire can be critical

• Describe the design benefits inherent in computerized and online questionnaires

• List the features of a good cover letter • Create appropriate openers for personal and

telephone interviews • Discuss the questionnaire guidelines that would be

relevant in the design of an observation form


Circuit City – Bizrate Customer Satisfaction Study

• Shopzilla.com is a division of Biz-Rate.com that collects data from previous shoppers at a particular online merchant

• Shopzilla reports satisfaction ratings for a variety of online retailers so that shoppers can compare merchants across 15 service dimensions


Circuit City – Bizrate Customer Satisfaction Study (Cont’d)

• Circuit City has extended this survey to its in-store customers through the use of checkout receipts that invite shoppers to the Biz-Rate survey site

• The shoppers are them asked to rate Circuit City on ten shopping dimensions

• This data can be used by both shoppers and Circuit City to improve customer satisfaction


Shopzilla.com

Courtesy of Shopzilla


Questionnaire

• A set of questions designed to generate the data necessary for accomplishing a research project's objectives


Complexity of Questionnaire Design

• Consider the following question that appeared in a mail questionnaire from an presumably experienced researcher

• Do you consider the many marketing research texts adequate for most of your business majors at the undergraduate level ? _____ Yes ____ No If no; briefly,why not?

• What do you think? Easy or difficult to answer?

• What exactly do the words many, adequate, and most refer to in this question?


Questionnaire's Impact on Data Accuracy

• It must communicate to the respondent what the researcher is asking

• It must communicate to the researcher what the respondent has said


Sources of Distortion

• A sloppy questionnaire can lead to a great deal of distortion

• The entry of an intermediary, or the interviewer, into the communication channel can also distort the questionnaire


Exhibit 10.1 Questionnaire: Link Between Researcher and Respondents


Questionnaire Design Elements

• Does each question have the most appropriate form (e.g., structured versus nonstructured)?

• Is each question relevant and properly worded to obtain meaningful, valid responses?

• Is the sequencing of the questions likely to introduce any bias?

• Are the layout and appearance of the questionnaire conducive to accurate and easy data collection?


Exhibit 10.2 The Questionnaire Design Process


Question Form

• Nonstructured questions – Open-ended

• Structured questions – Fixed-response


Nonstructured Questions

• Consider the following illustrative questions How old are you?___________________ What do you like most about owning your own

home? _____________ Will you please describe your thoughts about a

person who shoplifts items from a grocery store to keep from going hungry? ______________________________________________________________________________________


Structured Questions

• In which of the following categories does your age fall? _______ Less than 18 _______ 18 to 30 _______ 31 to 45 _______ 46 to 60 _______ Over 60


Structured Questions: Dichotomous Questions

• Do you smoke cigarettes? ___Yes ___No

• Have you ever watched CNN News? ___Yes ___No


Multiple-Category Questions

• In your opinion, which product category is the most suited for making purchases on the Internet? ____Automobiles ____Books ____Electronic Items (TVs, video cameras, VCR and DVD players, etc) ____Videos, CDs ____Computer-related items ____Travel (airlines, car rental, hotel reservations, etc.)


Multiple-Category Questions (Cont’d)

• What do you like about Bank of America’s online banking service?

____24 hour service ____Transaction privacy ____No need to travel – do it from home ____Paying bills online– no need to mail bills ____Other ____(please specify)


Response Category Sequence

• Approximately how many long-distance telephone calls do you make per week? ______ 0 to 1 call ______ 2 to 3 calls ______ 4 to 5 calls ______ 6 to 7 calls ______ More than 7 calls

• The response categories in this question follow a natural sequence


Response Category Content

• Response choices should be – Collectively exhaustive

• taken together – Mutually exclusive

• not overlap


Response Category

• The response choices are collectively exhaustive but not mutually exclusive

• On the average, how many cans of cola do you drink per week ____0 to 3 cans ____3 to 6 cans ____More than 6 cans

• Changing the 3 in the second category to a 4 will make the response choices mutually exclusive


Response Category (Cont’d)

• What do you like about Bank of America’s online banking service?

____24 hour service ____Transaction privacy ____No need to travel – do it from home ____Paying bills online – no need to mail bills ____Other ____(please specify)

• What is wrong with this question ?


Response Category (Cont’d)

• Modify the question to read, "Which of the following do you like most about Bank of America’s Online Banking Service?"

• With this revision each respondent would, or should, check only one response category

• What do you like about Bank of America’s Online Banking Service? (Please check as many categories as apply.)


Number of Response Categories

• Multiple-category questions fall between dichotomous questions & open-ended questions


Question Relevance and Wording

Can the research objective(s) be

fulfilled without asking this question?

The question may still be a candidate for revision or

deletion and must be further examined

question should be deleted

Yes No


Can the Respondent Answer the Question?

• In your opinion, how many students in your marketing courses will become potentially successful marketing managers? Ten percent ______ Twenty percent ______ Thirty percent ______ ______percent ______


Can the Respondent Answer the Question? (Cont’d)

• Consider the following question taken from a questionnaire used by a marketing research firm in conducting a face-to-face survey of female household heads – How much has your family spent in the last 12

months on cookware? $ _______


Writing a Questionnaire for International Markets

• Caution – Use of the same word can have different

meanings in different cultures • Word Equivalence

– In Japan, China and non-English speaking European countries

• “Very Satisfied” means “Somewhat Satisfied” • “Very Satisfied” should be replaced by “Totally

Satisfied”


Will the Respondent Answer the Question?

• Questions about personal financial matters or sexual behavior are examples of sensitive questions

• Words used in the questions should not be ambiguous to the respondents


Double-Barreled Questions

• Suppose the following question is to be used in conducting a survey of the general public – Do you feel firms today are concerned about their employees

and customers? _____ Yes _____ No _____ No opinion

• A “no” response can be interpreted three different ways – The respondent feels firms are concerned about neither

employees nor customers – The respondent feels firms are concerned about employees

but not customers – The respondent feels firms are not concerned about

employees although they are concerned about customers


Avoiding Double-Barreled Questions

• This revision is usually accomplished by breaking one question into several questions

• Thus our illustrative question can be reworded as two separate questions – Do you feel firms today are concerned about their

employees? ___ Yes ___ No ___ No opinion – Do you feel firms today are concerned about their

customers? ___ Yes ___ No ___ No opinion


Leading Questions

• Don't you think offshore drilling for oil is environmentally unsound? ___Yes ___ No ___ No opinion

• Do you think the quality of products on the market today is as high as it used to be ten years ago? ___ Yes ___ No ___ No opinion


Avoiding Leading Questions

• The two leading questions we saw earlier can be modified for neutrality as follows – What is your feeling about the environmental

impact of offshore drilling for oil? ___ Offshore drilling is environmentally sound ___ Offshore drilling is environmentally unsound ___ No opinion


Avoiding Leading Questions (Cont’d)

• How do you feel about the quality of products on the market today compared with the quality of products on the market ten years ago? ___Product quality is better now than ten years ago. ___Product quality is worse now than ten years ago. ___Product quality is the same now as ten years ago. ___ No opinion.


One-Sided Questions

• Do you feel firms today are concerned about their employees? ___Yes ___ No ___ No opinion

• Would you agree or disagree that the deregulation of the airline industry has benefited customers? ___Yes ___ No ___ No opinion


Avoiding One-sided Questions

• Do you feel firms today are concerned about their employees? ___Concerned ___ Unconcerned ___ No opinion

• Do you feel the deregulation of the airline industry has benefited consumers, has had no impact on consumers, or has hurt consumers? ___Has benefited customers ___Has had no impact on customers ___Has hurt customers ___No opinion


Unbalanced Questions

• How important is price to you in buying a new car? ___More important than any other factor ___Extremely important ___Important ___Somewhat important ___Unimportant


Balanced Questions

• How important is price to you in buying a new car? ___Very important ___Relatively important ___Neither important nor unimportant ___Relatively unimportant ___Very unimportant


Avoiding Questions with Implicit Assumptions

• Are you favorable, indifferent, or unfavorable toward a 10% increase in city taxes?

___Favorable ___Indifferent ___ Unfavorable • The above question can be improved as

– Are you favorable, indifferent, or unfavorable toward a 10% increase in city taxes to repair potholes in the city's streets?

___Favorable ___Indifferent ___Unfavorable


Avoiding Questions with Implicit Assumptions (Cont’d)

• Do you drink coffee while watching football, baseball, and so on? ____ Yes ____ No

• The above question can be reworded as – During which of the following activities do you drink

coffee?(Check as many as apply.) ___ Attending ballgames ___Attending sporting events other than ballgames ___Watching ballgames on TV ___Watching sporting events other than ballgames on TV


Guarding Against Errors

• How often do you eat eggs for breakfast? _____ Frequently _____ Occasionally _____ Rarely _____ Never


Guarding Against Errors (Cont’d)

• To guard errors due to misinterpretations, the previous question can be revised as – On average, how many days per week do you

eat eggs for breakfast? ___ Every day ___ 5 or 6 days ___ 3 or 4 days ___ 1or 2 days ___ Less than 1 day per week ___ Never eat eggs for breakfast


Complex Questions

• In which of the following do you typically invest your liquid assets? ___Insured accounts ___Stock market ___Insured accounts and stock market ___Other accounts

• Of the total number of miles you drove during the past month, approximately what percentage was traveled in driving to and from work? ___Percent


How Does USA Today Avoid Biases in Polling?

• Question posed to Jim Norman in Hackensack, New Jersey

• “With all of the little idiosyncrasies available to pollsters to potentially skew the answers of the respondents, e.g., voice tone, making statements vs. asking questions (a.k.a. push polling), or the historical success of polling where the last choice offered automatically becomes the preferred answer,with all of these potential manipulations, how does a polling unit like yours avoid ANY appearance of partisanship or impropriety?”


How Does USA Today Avoid Biases in Polling? (Cont’d)

• Jim Norman Responds – “Voice Tone: There can be a wide range in the

professionalism of the interviewers from polling outfit to polling outfit, but the best outfits use people who are TRUE professionals. With them voice tone is not an issue.”

– “Making statements vs. asking questions: If you sat through just one of our often excruciating conference calls to determine question wording, you would know how much the people involved are concerned that we wring out any bias in our questions. It still can happen,but I think it’s rare.”


How Does USA Today Avoid Biases in Polling? (Cont’d)

• Jim Norman Responds (Cont’d) – “Last choice offered automatically becomes the

preferred answer: we rotate the responses, so that each possible response is the last one offered to an equal number of respondents.”


Sequencing of Questions

• Position of Demographic and Sensitive Questions

• Arrangement of Related Questions • Funnel and Inverted-Funnel Sequences • Skip Patterns


Position of Demographic and Sensitive Questions

• Respondents' personal or demographic characteristics, i.e., age, education level, or income are placed at the end of the questionnaire


Arrangement of Related Questions

• Cluster questions that focus on the same topic – In a survey of business executives

• Views on the economy • Competition • Employee turnover

• Grouping questions into meaningful clusters can increase respondents' ease in answering the questions and reduce the chance of response errors


Funnel Sequences

• Which of the following types of TV shows do you watch? (Check all categories that apply.)

____ News shows ____ Quiz shows____ Game shows

• Which of the following types of TV shows do you like the most? (Check as many categories as apply.)

____ News shows ____ Quiz shows____ Game shows


Funnel Sequences (Cont’d)

• Which of the following specific shows did you watch during the past seven days? (Check "yes" or "no" for each show listed.) Deal or No Deal ___ Yes ___ No Wheel of Fortune ___ Yes ___ No CBS Evening News ___ Yes ___ No


Inverted-Funnel Sequences

• A survey of consumers to measure the image they have of Netgrocer.com

• What do you think of the specific attributes of Netgrocer.com? ___ Website ___ Products ___ Customer Service


Skip Patterns

• Proper sequencing of questionnaire items is essential to avoid complicated skip patterns that may confuse interviewers and respondents


Question Sequence Needing Improvement

• 11. Do you own or rent your current place of residence? Own Rent

• 12. How long have you lived in this state? Less than one year One year to less than 5 years 5 years or more

• 13. If you have lived outside this state, in which state did you live immediately before moving here?


Question Sequence Needing Improvement (Cont’d)

• 14. If you have lived in this state 5 years or more and if you currently rent your residence, do you intend to buy a home within the next two years? Yes No Don’t know

• 15. How long have you lived in your current place of residence? Less than one year One year to less than 5 years 5 years or more


Improved Question Sequence

• 11. Have you always lived in this state? Yes (Go to Question 12) No (Go to Question 11a)

• 11a. In which state did you live immediately before moving into this state?

• 12. How long have you lived in this state? Less than one year One year to less than 5 years 5 years or more


Improved Question Sequence (Cont’d)

• 13. Do you own or rent the place where you live? Own (Go to Question 15) Rent (Go to Question 14)

• 14. Do you intend to buy a home within the next two years? Yes No Don’t know

• 15. How long have you lived in your current place of residence? Less than one year One year to less than 5 years 5 years or more


Questionnaire Appearance And Layout

• Version 1 – How old are you?

___Less than 18 ___18 to 25 ___26 to 40 ___Over 40

• Version 2

– How old are you? ___Less than 18 ___18 to 25 ___26 to 40 ___Over 40


International Marketing Research

• First Step – A professional who is fluent in both the languages

should translate the original questionnaire • Second Step

– A different translator must translate the local language questionnaire back to the original version

• Third Step – Researchers should compare the original version and

the translated original version to ensure that they are identical in meaning


International Marketing Research (Cont’d)

• Fourth Step – The local language version should also be

checked by the marketing research professionals in the country of interest

• Fifth Step – The local language version should be

pretested to make sure that the questionnaire serves the intended purpose


Pretesting

• Pretesting is administering a questionnaire to a limited number of potential respondents and other individuals capable of pointing out design flaws


Questionnaires for Computerized and Online Interviewing

– Randomizing response choices – Checking for response consistency – Incorporating complex skip patterns – Personalization – Ability to draw questions from computer

libraries – Adding “new” response categories


Randomizing Response Choices

• The computer can be programmed to randomize the order of presentation of the categories separately for each respondent for multiple-category questions in which category sequence may influence response choice


Checking for Response Category

• The computer can be programmed to check for consistency between the response given to a question at hand and responses given to certain key questions asked earlier


Incorporating Complex Skip Patterns

• Computerized online interviewing can handle very complex skip patterns

• The computer can check as many “if then” statements and previous responses as necessary,decide which question should be asked next and pose that question on the computer screen instantaneously


Personalization

• When a respondent’s name is entered into the computer keyboard at the start of an interview, that name can be automatically inserted into key questions and instructions throughout the questionnaire to provide a degree of personalization that would be very time consuming to achieve in non-computerized interviewing


Ability to Draw Questions from Computer Libraries

• A questionnaire writer simply clicks and drags a standard questionnaire from an online library and modifies it as necessary to create a desired questionnaire


Adding New Response Categories

• In a multiple-category question with an “other” category, when a prespecified number of respondents provide the same open-ended response to the “other” category, that response can be automatically converted into an explicit check off category by adding it to the set of prespecified categories


Designing Cover Letters For Mail Questionnaires

• Primary purpose – To win the cooperation of respondents

• A good cover letter should be – Concise – Objective


Exhibit 10.5 Illustrative Cover Letter and Key Features


Openers for Personal and Telephone Interviews

• Features – An appropriate salutation, such as "Good

morning! I am Ralph Johnston with Marketing Research Associates Company.”

– A brief statement about the project and its purpose, such as "We are conducting a survey of heads of household regarding their feelings about modern home appliances.”


Openers for Personal and Telephone Interviews (Cont’d)

• Features (Cont’d) – An indication of how long the interview might

last, such as "This survey should take no more than ten minutes.”

– A polite request for permission to conduct the interview, such as "May I please talk to the male or female head of your household for a few minutes?"


Designing Observation Forms

• Guidelines – Wording – Sequencing – Layout


Exhibit 10.6 Observation Form for Recording Characteristics and Behavior of Customers Stopping at a Special Display


Creating Online Questionnaires

• Creating an online questionnaire is simplified by a growing number of websites that allow researchers to design and launch their survey from one site

• SurveyZ.com is an example of one such site


SurveyZ

• SurveyZ allows users to register for a free trial and sample the functionality of the website

• There are also a number of sample surveys posted for users to view

• The site is a powerful tool that allows anyone to create a professional looking surveys very quickly


Using SurveyZ

• Surveyz! The Ultimate Survey Software Solution - Online Survey Software - Customer Satisfaction Survey - Employee Survey - Web Survey Software

• Follow the screen shots in your text to explore the website