Questionnaire Construction

Questionnaire design, to be effective, should be done with the respondent in mind.

Language

The first and foremost question we have to ask ourselves as a researcher is –

What language is the respondent going to understand and respond in?

The questionnaire must be designed such that it can be used in the language concerned. This does not necessarily mean it has to be printed in each language in which it has to be administered.

For instance, a questionnaire printed in English could be administered to the respondent in the local language he speaks, by a trained interviewer who could translate each question on-line. The answers can be recorded in the given English language form if the interviewer is fluent in both languages. This makes it easier to tabulate.

Alternatively, the numerical codes for the answers can be in usual numbers, and the questionnaire could be translated into any language required for the respondent to understand. But the translation must be as consistent as possible with the original.

Difficulty Level

Avoid marketing jargon or difficult words unless the respondent is a postgraduate or an experienced executive. In other words, keep the language as simple and straightforward as possible.

Fatigue

Avoid unnecessary questions. The golden rule is to keep the questionnaire as short as possible, and the ideal maximum interview time is probably about 20 minutes per interview.

Cooperation with Researcher

Encourage the respondent to respond.

In personal interviews, introduce the subject of the research and the agency represented, before starting the interview.

In questionnaires which are filled by respondents themselves, there must be a two-three line introduction and request for respondent’s cooperation at the top of the questionnaire.

In mailed questionnaires, a covering letter detailing the purpose of the study and explaining what use its results will be put to, along with a return pre-paid/stamped envelope, is likely to increase manifold the response rate.

Slide 2

Social Desirability Bias

There is a tendency on the part of respondents to give wrong, but “socially acceptable” answers to even the most ordinary, innocuous questions. For example, the socially desirable answer to the question “Do you read the daily newspaper?” is “yes”. It is as likely to be wrong as right.

There are many ways to verify the accuracy of responses and to deal with them. Some of the techniques are –

•.Repeating the same or similar question in the questionnaire at different places.•.Asking indirect questions•.Asking follow up questions to probe if the respondent is really truthful.

For example, we could ask the respondent to state one important headline, or describe one important story he remembers, if he states that he reads the daily newspaper. This could be from the same day’s or previous day’s, newspaper..Deliberately introducing non-existent periodicals, or advertisements, and asking the respondent if he/she has seen them.

Slide 3

Ease of Recording

A questionnaire, that it has to be carried on the field, and data may be recorded on it while standing in awkward postures. The questionnaire design should ensure it is easy to carry, visible in different kinds of light, and the distance between different answer categories should be sufficient so that there is no confusion or mistake while placing a tick over the actual response for a given question.

Coding

If the questionnaire is coded before doing the field work (as most questionnaires are these days), it must be ensured that the field staff knows where to mark the answers – on the code or on the actual answer choice. This should be done during the briefing and mock interview.

Instructions for Navigation

Frequently, a questionnaire contains printed instructions for the interviewer. This includes ‘Go To’ statements, such as “If respondent is a non-user of Brand X”, then Go To Q.5. If not, Go To Q.9.

Slide 4

Sequencing of Questions

Questions in a questionnaire should appear in a sequence starting from non-threatening or ice-breaking or introductory questions, and then proceed to the main body of questions.

Generally, the age, income, occupation, education and similar demographic questions should appear at the end of a questionnaire, after an interviewer has established a rapport or familiarity with the respondent. If these are asked in the beginning, there is a high likelihood of suspicion and non-cooperation resulting in a wasted effort in many cases.

As far as possible, questions should follow a logical sequence, and must be phrased appropriately.

Biased and Leading Questions

The questions should be carefully worded to avoid bias. It is not a good practice to ask questions such as “Don’t you think liberalisation is a good idea?” You could be better off getting an unbiased reply asking a question like “Some people think liberalisation is a good thing, and some think it is bad. What do you think?”

Slide 5

Monotony

One indicator that a questionnaire is monotonous for the respondent is if he answers “Agree” to every question or “Disagree” to every question, for four to five questions in a row.

If this happens, the researcher must find a way to overcome the potential problem, by re-sequencing the questions which force the respondent to think before he answers, or by changing the scale, or by some other method.

Analysis Required

A questionnaire design is dependent on the analysis required from it. But the most important effect of the analysis required is in the scale of measurement that must be used. So we will deal with this topic – the scale of measurement – next.

Slide 6

Scales of Measurement Used in Marketing Research

Marketing research uses the following four major types of scales –Nominal, Ordinal, Interval and Ratio.

Nominal Scale

A nominal scale uses numbers as labels, with no numerical sanctity. For example, if we want to categorise male and female respondents, we could use a nominal scale of 1 for male and 2 for female.

But 1 and 2 in this case do not represent any order or distance. They are simply used as labels. For instance, we could easily label females as ‘1’ and males as ‘2’, and it could still be a valid nominal scale.

We can use the nominal scale to indicate categories of any variable which is not to be given a numerical significance. For example, demographic variables such as religion, education level, languages spoken, and other variables like magazines read, T.V. shows watched, user or non-user of a brand, brands bought, etc. can be nominally scaled.

Slide 7

Nominally scaled variables cannot be used to perform many of the statistical computations such as mean, standard deviation etc., because such statistics do not have any meaning when used with nominal scale variables.

However, counting of number of responses in each category and computation of percentages after division by the sample size is allowed. Also, nominal scale variables can be used to do cross tabulations, one of the most popular methods of routine analysis. The chi-squared test can be performed on a cross tabulation of nominal scale data.

To repeat, simple tabulations (also called frequency tables) and cross tabulations can be done with nominal scale variables.

Slide 7 contd...

Structured and Unstructured Questionnaires

Structured questionnaires are those where the questions to be asked are standardised, and no variation is permitted in terms of the wording of the questions between different interviewers. Standardisation in a structured questionnaire usually extends to the answers also. In effect, then, we can standardise either (1) questions only, or (2) both questions and answers.

Structured Questions Structured questions improve the reliability of the study, by ensuring that every respondent is asked the same question, word for word.

For example, the question " Do you live in Delhi?" may be construed differently from the question " Are you a resident of Delhi?" by some respondents, even though it appears that both questions are asking for the same information.

A person who is normally not resident in Delhi but is living there at present on a short visit may answer "yes" to the first question but "no" to the second one. It is best to keep the question exactly the same (either version 1 or version 2), when asked by different interviewers.

Slide 11

Structured Answers

Structuring or standardising answers which a respondent can choose from in a questionnaire also achieves consistency of form. Additionally, it makes the interpretation of answers, analysis and tabulation, easier than in the case of unstructured answers.

Unstructured answers become difficult to categorise after the study, and different analysts may interpret them differently - so they may lend themselves to subjective interpretations.

Subjectivity by itself is not bad, but it becomes difficult to defend it if the sponsors(clients) of the study are quantitatively oriented. Most large scale studies in marketing research therefore, choose the less risky, and easier to manage, structured-answer approach.

Slide 12

Open ended and Closed ended Questions

Questions which permit any answer from the respondent in his own words are called open-ended questions. Questions which structure the possible answers beforehand are known as closed-ended questions.

An example of an open-ended question is " What do you like about Surf detergent?"____________________________________

The respondent can say whatever he wants to, in response to this question.

On the other hand, a closed-ended question which elicits similar information could be "What do you like about Surf detergent?"

•.Its cleaning power•.Its Price•.Its fragrance•.That it dissolves easily•.Its stain-removing ability•.Any other, (please specify)____________________________________

Here, options "a" to "e" are pre-determined, but "f" provides for anything else the respondent wants to add.

Slide 13

Disguised Versus Undisguised Questions

Sometimes questions that are disguised (rather than direct) can elicit more accurate replies. For example, we may ask a person if he/she is a good parent. This is a direct question.

Or, we may ask for the respondent's opinion on the deficiencies they have observed in how others bring up their children- say, their neighbours, relatives or friends. This is an indirect question, and a qualified analyst can interpret the answers to gauge how good a parent the respondent might be, from the responses given.

The problem with the direct question in this case is that most people will not admit to being a bad parent. But they may come out freely with other people's deficiencies, some of which could reflect their own shortcomings.

There are other reasons why disguised questions are sometimes needed. It is often found that respondents are biased when they know who is the sponsor of the study. To get true, unbiased opinions regarding attitudes towards brands, researchers sometimes do not let on the name of the sponsor.

For example, a well known multinational company making electrical switches for industrial application once did an anonymous survey in Mumbai among its customers (a study done by the author) and found many deficiencies in its products and service which they otherwise may not have found out. If it results in more accurate data without doing any harm to the respondent, it may be a legitimate way to do the study.

Slide 14

Completely disguised or indirect questions probing into the psyche of a person are usually used for qualitative research, as part of projective techniques, etc.

To summarise, market researchers usually ask structured, undisguised questions in a typical study done on a large sample. Most studies also tend to be of the "quantitative" type, where numbers (frequencies), percentages, averages or similar summary statistics are computed. These types of analyses are easier to do with structured formats for answers.

Even if a study is primarily based on structured responses, a couple of open-ended questions may still be included in it if they are the best suited for the task on hand. One such category of questions is called "Probing" questions in marketing research terminology. These are used as a follow up after a structured response question. An example of this use of open-ended question following a structured question is

.Which brand of mosquito mats do you use?•.Good Knight•.Mortein•.Jet

.Why do you use this particular brand?_________________________

In this question, the second part is open-ended, while the first part is closed-ended.

Slide 15

Types of Questions

The six major types of questions that most questionnaires would generally use are-

•.Open-ended•.Dichotomous (2 choices)•.Multiple Choice•.Ratings or Rankings•.Paired Comparisons•.Semantic Differential, or other special types of scales.

An open-ended question is one which leaves it to the respondent to answer it as he chooses. An example is “What do you think of the taste of Brand X of Cola?” No alternatives are suggested. The answer can be in the respondent’s own words.

Slide 16

Dichotomous questions

These are those which ask the respondent to choose between two given alternatives.

The most common example of this is the yes or no type of questions “Are you a user of Brand X toilet soap?” Yes or No are the alternatives given.

A third choice is sometimes added to dichotomous questions such as “Do you like Brand X of potato chips?” The choices given are “Yes”, “no”, and “neither like nor dislike”.

Sometimes, any other, please specify “______” is used instead of neither like nor dislike.

Slide 17

Multiple choice questions

These are extensions of dichotomous questions, except that the alternatives listed number more than two. A common example is as follows –

Please tick against the factors which made you buy this brand of car :

.Reasonable Price

.Great Looks (Appearance)

.Fuel Economy

.Easy Availability of Service

.Any Other, please specify.

In the above question, more than one category can be chosen. In some multiple choice questions, only one category is to be chosen. For example, look at the question below-

Please specify your age group-.Below 15.16-25.26-40.Above 40

Only one of the above is to be chosen. It must be clear to the respondent and the interviewer whether only one choice is allowed, or more than one are allowed for a multiple choice question.

Slide 18

Ratings or Rankings : This is a question of the type, ‘Please rate the following detergent brands on a scale of 1 to 7 in their ability to clean clothes’.

Brand A 1 2 3 4 5 6 7Brand B 1 2 3 4 5 6 7Brand X 1 2 3 4 5 6 7

This is an example of rating. Ranking would have looked as follows :

Please rank (1=Best, 2=next best, etc.) the following detergent brands on their ability to clean clothes.

Brand A -----Brand B -----Brand X -----

Slide 19

Paired Comparisons

A special type of question is the paired comparison.

This requires the respondent to choose between pairs of choices at a time. For example, there could be six brands of colour TVs, Brands A, B, C, D, E, F. A respondent may be asked to do a paired comparison to say which Brand is better, but for only two Brands at a time.

He is given a table or a card with two brands written on it, and has to choose the better brand, each time. This process has to repeat for as many pairs as exist in the given set of objects or brands.

Some special techniques such as Multidimensional Scaling need data from paired comparisons. This technique is explained later in Part II of this book.

Slide 20

Semantic Differential

Another scale commonly used by marketing researchers is called the semantic differential. This type of question is similar to the rating scale. The only additional feature is that a set of two adjectives forms the two extreme points of the scale. For example, a product is

Easy to Use |----------------------| Difficult to UseExpensive |----------------------| InexpensiveEasily Available |----------------------| Not Easily AvailableConvenient |-----|-----|-----|-----| Inconvenient

There may be several intermediate points between the two extreme values of the scale. These could be coded 1 to 5 or 1 to 7 or whatever the number of points is. A commonly used 5 point scale is from Completely Agree to Completely Disagree.

There may be questions based on other scales which are standard or specially constructed. Some scales like the Likert Scale or Thurston Scale are named after people who invented them.

Slide 21

How to Choose a Scale and Question Type

The researcher must decide on the scale and type of question based on the following factors.

•.Information Need•.Output format desired•.Ease of tabulation•.Ease of interpretation•.Ease of statistical analysis•.Reduction of various errors in understanding or use by respondents and field workers

Slide 22

Transforming Information Needs Into A QuestionnaireWe will now illustrate by developing a complete questionnaire for a given set of information needs.

Example of Information Needs : A soft drink concentrate manufacturer (such as Rasna’s manufacturer, for example) wants to know the following :

.Demographic profile of users versus non-users of soft drink concentrates.

Among users•the preference for liquid concentrate versus powder.•preference for powder with sugar added, versus powder with no added sugar.•occasions of use by self•whether served to guests•rating on convenience, taste, price and availability•brand preferred among soft drink concentrates.

Among non-users•Reasons for not using soft drink concentrate•Substitute product usage, if any, and reasons for using or consuming them

Let us attempt to develop a questionnaire for the above information needs. A possible questionnaire is shown in the next slide –

Slide 23

Double-Barrelled Questions

Inexperienced questionnaire designers have a tendency to combine two questions into a single question, such as –

Are you happy with the price and quality of Brand Y ?

Yes No

This is not a good question to ask, because the answer will be ambiguous, whether it is yes or no. It would not be clear whether the respondent has said yes for price alone, quality alone, or for both. The same problem exists for a ‘no’ answer.

It is better to rephrase the question and provide for different answer categories for each attribute, or ask two separate questions, one for price and one about quality.

Slide 26

Good Questionnaires and Bad Questionnaires

In general, a questionnaire is good if it measures what it set out to measure (ie., it is VALID) and does it in an efficient manner.

Usually, a questionnaire goes through various stages before it is used in the field. Listing of information needsConversion into questions with suitable scales of measurementSequencing of questions into a logical orderTrying it out in a pre-test on a handful of respondents in a convenience sample or a field sampleModifications in the wording, scale or sequence as a result of the pre-test, and then Preparation of the final draft for the actual study

are the usual steps involved. Most faults in a questionnaire would be ironed out in this process if followed meticulously.

Blank Output Formats/Tables

Problems in a typical study stem from a lack of sufficient thought given to the analysis required in advance.

The solution for this is to prepare blank output formats for each question on the questionnaire, before doing the field work.

In many cases, the value of the research increases manifold by slightly modifying the scale or wording of the questions asked. Remember, it is cheaper to modify the questionnaire in advance than think about what could have been done after the study is over.

Slide 28

Summary

Questionnaire design is an art, but there are certain common sense rules that can help, as we have discussed throughout this chapter.

Scales to be used should be decided on by the researcher in consultation with the study sponsor, keeping in mind the kind of output formats or tables required for decision-making.

Validity and reliability issues are of particular importance if the subject of the study is new or the researcher is inexperienced.

Practice with designing questionnaires is the best way to perfect the art.

Please do test the questionnaire on a small sample, and modify it if necessary, before going full steam ahead.

Slide 30