Business 205
WHAT?!
Reliability and Validity
Since this survey is a training tool, it has not been formally checked for reliability or validity. However, since I have had a lot of feedback from various training classes, other trainers, and various sources, I feel that it is fairly accurate. If I did a formal reliability and validity analysis, I would probably have to charge for it. Please feel free to send me your feedback ([email protected]).
Purpose of a Survey
To be able to place a number on a construct
This enables a researcher to engage in statistical analysis
Constructs (DVs) are composed of MANY questions whose scores are averaged togetherThe average (mean) result is the number
statistics are performed on
Survey Ethics
Nazi Medical War CrimesTuskegee Syphilis StudyJewish Chronic Disease Hospital Study
Articles
Congress
National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research (1974)Belmont ReportCommon Rule
Belmont
Three ethical principles are outlined in the Report:Respect for Persons
informed consentBeneficence
Do no harm, and maximize benefitsJustice
Equitable distributions of the burdens and benefits of research.
Common Rule
Requirements for assuring compliance by research institutions
Requirements for researchers’ obtaining and documenting informed consent
Requirements for Institutional Review Board (IRB) membership, function, operations, review of research, and record keeping.
Human Subjects Review Board (HRB) = IRB
Serves to ensure studies follow ethical guidelines
Controls research aspectsAllows those projects that do no harmMay stop a project at any given time
Administering a Survey
Pre-Survey Introduction
Name, general background of what your are studying, side-effects, VOLUNTARY, participation reward, contact information
Post-SurveyConcluding Remarks
Debriefing about your study, thank-you, give participation reward
Open-Ended Questions
Respondents asked to provide their own answers to the question
Example:
What is your favorite restaurant to eat at?
Closed-Ended Questions
Respondents are asked to select their answer from among a list provided
Example:
Please circle your age:
18 19 20 21 22 23+
Likert Scale
Strongly Agree/Strongly Disagree5 or 7 point scale (5 is more common)
1.) People who drink heavily at parties are popular.
SA A N D SD
2.) I can excuse someone more readily for obnoxious behavior because they were drunk.
SA A N D SD
* 7 pt: Strongly Agree, Somewhat Agree, Agree, Neutral, Disagree, Somewhat Disagree, Strongly Disagree
Semantic-Differential Scales
Polar oppositesNormally either a 5 or 7 point scale
Place an "X" or a check mark along the continuum between each pair of words. Placing the check mark closer to one of the words indicates that you feel that the word closely describes the language used in the message.
The language of the message is:
Strong Weak
Emotional Unemotional
*other pairs: Good/Bad; Wise/Foolish; Intelligent/Unintelligent; Honest/Dishonest; Reliable/Unreliable
Think about “splitting”
Must have question that will divide groups by IV You wanted to know about gender differences
Demographic question: I’m a Male Female
You want to know about Smokers and Non-Smokers
Demographic question: I smoke. Yes No
You want to know about people who drink 1, 2 or 3 drinks and hour
How many drinks an hour do you drink? 1 2 3
Watch for Erroneous Questions and Biases!
Double barreled questionsSlanted questionsEmotionally charged questions
Researcher biasAnswering bias
Writing Survey Questions
Writing questions that make sense to ask of the DVAsk questions that your group thinks may
be important to finding out information about the DV
Make sure that the questions are relevant
Rule of thumb: More than 3 questions per DV
Survey Reliability
Questions need to “hang together”
Reliability is from alpha = .70 or higher
To achieve this, “like” questions are added together to get the Mean for the DV
Example
You want to know if gender affects how happy people are. You write a survey based on finding out how happy people are and divide the groups by gender.
Understanding Questionnaires
Person 1
I am a F
I am a very happy person.
5
I believe myself to be very blessed.
4
I am a very optimistic person
4
Person 2
I am a Female M
I am a very happy person.
3
I believe myself to be very blessed.
5
I am a very optimistic person
2
Milestone 4
Compose a surveyHave several demographic questions
Write at least 15 questions per DVThese questions can be a mix of all types of
survey questionsMake sure you have at least 10 that are of
Likert-type or Semantic-Differential