Questionnaires. Quantitative and Qualitative Research Questionnaires (quantitative) – do the Carbon Footprint questionnaire Creating questionnaires Tutorial.

Questionnaires

Questionnaires• Quantitative and Qualitative Research• Questionnaires (quantitative) – do the

Carbon Footprint questionnaire• Creating questionnaires• Tutorial on lit review assignment

Next week• Creating a sample• List potential participants

Five assumptions

1. The Ontological Assumption

Reality is the hardware of a computer programme

What is real and knowable depends on

the software

What is the nature of reality? What is real and knowable?

Assumption Question

Ontological What is the nature Assumption of reality?

Quantitative Qualitative

Reality is objective Reality is subjective and singular, and multiple,

apart from the as seen by participants researcher in a study

Qualitative vs Quantitative

2. Epistemological assumption

The researcher is all powerful

In order to know I must win “I want to be a Millionnaire”

What is the relationship of the researcher to that researched?

What we must do in order to know?

Assumption Question

Epistemological What is the relationship of assumption the researcher to that

researched?

Quantitative Qualitative

Researcher is Researcher interactsindependent from that with that being being researched researched

3. Axiological Assumption

Survival values – nature red in tooth and clawSocial Values – fitting in with societyTranscendent values – beyond self, for the greater good

Our values determine what we want to research

What is the role of values?

What is worth knowing and why?

Assumption Question

Axiological Assumption What is the role of values?

Quantitative Qualitative

Value-free and Value-laden and unbiased biased

4. Rhetorical Assumption

Dramatic Appealing to the personalStretching the truthLarge group of believers present

What is the language of research? How best to share our research with others.

Assumption QuestionRhetorical What is the language assumption of research?

Quantitative Qualitative

Formal InformalBased on set definitions Evolving decisionsImpersonal voice Personal voiceUse of accepted Use of accepted quantitative words qualitative words

5. Method-ological

assumptionResearch is complexThere is no single approachBefore you use any particular method you have to understand the assumptions behind itMethodology deals with the assumptions behind specific methods of research

What is the process of research?

What techniques will lead to

fruitful inquiries?

Assumption Question Methodological What is the process of assumption research?

Quantitative Qualitative Deductive process Inductive process Cause and effect Mutual simultaneous

shaping of factors Static design, categories Emerging design, categories identified before study isolated during research Context-free Context-bound Generalisations leading to Patterns, theories developed prediction and explanation for understanding Accurate and reliable Accurate and reliable through validity and through verification reliability

Qualitative Methods descriptive, theory building, meaning making

• Focus Groups• Interviews• Case Studies• Ethnographic Inquiry• Collaborative Inquiry• Grounded Theory• Qualitative analysis of documents, video,

observation, etcWhat are they?

Quantitative Methods measuring, theory confirmation, defining

• Experiments• Questionnaires and surveys• Psychometric Scaling and Tests• Quantitative analysis of

documents, video, observation, etc• Audits

What are they?

Carbon footprint questionnaire

• Do the questionnaire• Tick the answers closest to

your understanding of the way you live

• Add up your score by adding up the numbers in each box you ticked

• Discuss the outcome• Critique the questionnaire

Scores Earths

required

32-62 1.2-1.6

62-93 1.6-2.3

93-124 2.3-5.8

124-155 5.8-15.3

The importance of enhancing whānaungatanga (the feeling of close connection as between kin)

between students at Unitec Waitakere

Question Issues• What types of

questions can be asked and how?

• How complex will the questions be?

• How long will the questionnaire/ survey be?

Developing your questions - 1 Managing bias. • The researchers – how easy is it to disprove

your hypothesis, theory, bias?• The participants – will the subject and the

questions put off the participant?Opening questions.• Simple • Key open ended questions to get

uninfluenced “off the top of my head” answers.

Developing your questions - 2

7. Would you like to help with any of the following activities? yes no

being involved in support group for your community setting up a support group for your community being a driver on field trips

helping with education activities (e.g.. cooking, sewing, computer skills) helping Shakti with policy development

being involved in activities aimed at changing government policy

Response option lists. (forced choice)

Developing your questions - 3Rating questions (likert scales)Did your mental health change because of the

help you got from the hospital or the service(s)? 1=much better; 2=better; 3=didn’t change/not

sure; 4=worse; 5=much worse• What to do about don’t know/not sure/did not

answer responses• Rating option lists – putting in an “other”

option space• Question Matrices

Developing your questions - 3The “other” option

3 Community health 3c. Where did the mental 3e. Did your mental health professionals/services health consultation(s) mainly improve because of the (other than overnight take place? help you got?services). place =

clinicsurgery,office, orrooms

1 GP or Doctor If>0 If>02 Visiting nurse (Community, District, Plunket) If>0 If>03 Community mental health service If>04 Community drug and alcohol (D&A) service If>05 Psychiatrist* If>06 Psychologist* If>07 Counsellor (include psychotherapists)* If>08 Social worker If>0 If>09 Deaf Association Service Coordinator If>0 If>010 Employment or occupational service (eg. If>0 If>0

work experience, Workbridge, NZES)11 Accommodation or housing service (eg. If>0 If>0

Housing New Zealand, rest homes) 12 Maori health service (eg. marae health If>0 If>0

service, Tohunga, spiritual healer)13 Alternative health service (eg. use of herbal If>0 If>0

and natural medicines, acupuncturist, chiropractor, spiritual healer)

14 Interpreter or communicator on their own for If>0 If>0

advice or discussion of problems (do not include social or friendship meetings)

15 Other health professional (eg. audiologist If>0 If>0

physiotherapist, optician, obstetrician,ear, nose and throat specialist)

(please circle which professionals you have been to or write their professions down if they are not in the list____________________________________________)

*Differences between a psychiatrist, a psychologist and a counsellor are 1) psychiatrists prescribe medication for mental health problems, the others do not; 2) Sessions with psychologists and counsellors are

usually longer than with psychiatrists (one hour compared with less than half an hour). 3) Psychologists use more cognitive and behavioural techniques to help you control your thoughts and behaviours

occasio

nally

Ho

spita

l or A

& E

In y

ou

r h

om

e

half t

he t

ime

alw

ays

mostly

did

no

t ch

an

ge

wo

rse

mu

ch w

ors

e

mu

ch b

ett

er

be

tte

r

If you have seen a mental health or D&A professional while using a community mental health or D&A service, do not count them again under their specific profession (eg nurse, social worker, psychiatrist).

never

3d. Was there an interpreter or communicator present?

3b. When you saw a health professional, were there times when you were feeling these things? (Show illness list) How many times?

3a. Over the past 12 months how many times have you consulted with the following people or services about your health?

At

the

ir p

lace

At

an

oth

er's

pe

rso

n's

pla

ce

Developing your questions - 4• “Why” questions sparingly used after key

options questions • Mopping up questions. Solutions, answers,

ideas, “anything else you want to say”• Reliability checks.

• Check response patterns for rigidity• Vary direction of rating responses• Set up correlating questions• Set up some very unlikely options

Demographic questionsThe big three:

age, gender, ethnicity/culture.

Socio-economic status:• Income (tax brackets/income support levels), • Occupation

1=factory, unskilled;2=clerical, fishing, farming, trades; 3=administrator, professional, technician

Demographic questions - 2

Socio-economic status (continued):• Employment (full-time, part-time, student,

not working, on the benefit),• Education (SC, UE/bursary, tertiary cert,

diploma, degree)• Family: partners, dependants,

responsibilities• Health: disabling conditionsBeliefs: religion, political affiliation, sexual

orientation

Strengths of Questionnaires• Good for measuring attitudes and eliciting other content

from research participants • Inexpensive (especially mail questionnaires and group-

administered questionnaires) • Can be administered to large probability samples • Quick turnaround from data collection to report• Can be administered to groups • Perceived anonymity by respondents is high • Moderately high measurement validity for well-

constructed and well-tested questionnaires • Low dross rate for closed-ended questionnaires • Ease of data analysis for closed-ended items

• May need validation and may have poor reliability • Must be kept short • Often has missing data, particularly to open ended

questions • Possible reactive effects (e.g. response sets, social

desirability, dislike of questionnaires) • Failure to reach certain groups – low SES, low literacy• Response rates generally low for mail questionnaires • Open-ended answers are vague or reflect differences in

verbal ability, obscuring the issues of interest • Data analysis sometimes time-consuming for open-

ended items

Weaknesses of Questionnaires

Sampling

Population issues• Can the population be counted? What data is

available• Are response rates likely to be a problem?

– Is the population literate? – Are there language issues? – Will the population cooperate? – What are the geographical restrictions?

• Generalisation.– How small is the effect size we are seeking?– What sub-groups do we want to compare?– How do we represent the people who won’t

answer?

Probability sampling• Based on statistics which can describe the

similarity of a sample to the population that it is supposed represent.

• The simplest form of random sampling is called simple random sampling. In this we select participants from a given population such that each person in the population has an equal chance of being selected.

Stratified Random Sampling Involves dividing a population into homogeneous

subgroups based on one factor and then taking a simple random sample in each subgroup (e.g. culture). Participants are randomly selected within each group.

Subgroups might be based several factors (e.g. culture (two), gender and age (two groups)) which would give 8 (2x2x2) equal subgroups within which participants are randomly selected.

Quota Sample

The entire population is first divided into homogeneous sub-groups with respect to the given characteristic such as culture. A specified number of people are recruited from each strata as you come across them rather than selecting them through random procedure. The resulting samples are called quota samples.

Purposive sampling

• Purposive sampling starts with a purpose in mind and the sample is thus selected to include people of interest and exclude those who do not suit the purpose. Subjects are selected on the basis of some characteristic/s. Purposive sampling is popular in qualitative research.

Examples of Purposive Sampling• Extreme or Deviant Case - Learning from highly

unusual manifestations of the area of interest• Intensity - Information-rich cases that demonstrate

the area of interest very clearly. Critical incident case, Politically Important Case

• Maximum Variation - Purposefully picking a wide range of variation within a group

• Homogeneous – Reducing variation to get a clear group or sub-group perspective

• Typical Case - Illustrates or highlights what is typical, normal or average. (e.g. the average student). Criterion based or Theory based selection

Examples of Purposive Sampling• Random Purposeful – random selection form a pool

of participants.• Purposive Quota Sampling Interviewers are given

quota of particular groups of people to interview and the quota are organised so that final sample is representative of population. If we want our sample to represent the age of our population and 20% are between 20 and 30, and sample is to be 20 then 4 of the sample (20%) must between 20 and 30. Complex quotas can be developed so that several characteristics (e.g. age, sex, marital status) are used simultaneously.

Waitakere City 2006 Census Data (People 16 years and over)Occupation

N% of pop-

ulationparticipants

in each category

rounded participants

Managers 12,645 9% 2.84 3Professionals 16,167 11% 3.64 4Technicians and Trades Workers 13,035 9% 2.93 3Community and Personal Service Workers

6,798 5% 1.53 2

Clerical and Administrative Workers 12,570 9% 2.83 3Sales Workers 9,258 7% 2.08 2Machinery Operators and Drivers 5,622 4% 1.26 1Labourers 7,362 5% 1.66 2Emloyed Not Elsewhere Included 4,689 3% 1.05 1Unemployed 5,349 4% 1.20 1Full-time and part-time Tertiary Students 13987 10% 3.15 3Not in the Labour Force (house persons, retired, disability)

27494 19% 6.18 6

Status Unidentifiable 7,311 5% 1.64 2Total 142,287 100% 32 33

Purposive Quota sample

Examples of Purposive Sampling• Snowball or Chain - Initially contact a few

potential respondents and then ask them whether they know of anybody with the same characteristics that you are looking for in your research.

• Opportunistic –Following new leads during fieldwork, taking advantage of the unexpected.

• Convenience - The sample comprises subjects who are available in a convenient way to the researcher.