Copyright © Allyn & Bacon 2008 CHAPTER NINE Methods and Tools of Research This multimedia product and its contents are protected under copyright law. The.

Copyright © Allyn & Bacon 2008

CHAPTER NINECHAPTER NINE

Methods and Tools of ResearchMethods and Tools of Research

This multimedia product and its contents are protected under copyright law. The following are prohibited by law:• Any public performance or display, including transmission of any image over a network;• Preparation of any derivative work, including the extraction, in whole or in part, of any images;• Any rental, lease, or lending of the program.


ScalesScales

►Nominal (gender, race)Nominal (gender, race)►Ordinal (rank)Ordinal (rank)► Interval (Exams, scores, Likert type)Interval (Exams, scores, Likert type)►Ratio (Age, height, number of)Ratio (Age, height, number of)


Important ElementsImportant Elements

ValidityValidity ReliabilityReliability EconomyEconomy Interest (motivating, interesting)Interest (motivating, interesting)

ReliabilityReliability

►Refers to the extent to which Refers to the extent to which assessments are consistent. assessments are consistent.

► If you weigh five pounds of potatoes in If you weigh five pounds of potatoes in the morning, and the scale is reliable, the morning, and the scale is reliable, the same scale should register five the same scale should register five pounds for the potatoes an hour laterpounds for the potatoes an hour later

Types of reliabilityTypes of reliability

►Test-retestTest-retest►Parallel or equivalent formsParallel or equivalent forms► Internal consistencyInternal consistency

Split halvesSplit halves Kuder Richardson (Kuder Richardson (is a measure of internal consistency reliability is a measure of internal consistency reliability

for measures with dichotomous choices)for measures with dichotomous choices)

Cronbach’s alpha (Cronbach’s alpha (Cronbach's alpha is used for non-dichotomous Cronbach's alpha is used for non-dichotomous or continuous measures)or continuous measures)

► Inter-scorer or inter-rater reliabilityInter-scorer or inter-rater reliability►Standard error of measurementStandard error of measurement

Difference between Difference between Validity & ReliabilityValidity & Reliability

► If a test is reliable it is accurate but we If a test is reliable it is accurate but we don’t know it measures what we want don’t know it measures what we want to measure.to measure.

► If a test is valid we know it measures If a test is valid we know it measures what we want it to measure.what we want it to measure.


SoSo

►Even if a test is reliable, it may not be Even if a test is reliable, it may not be validvalid

► If a test is not reliable it cannot be If a test is not reliable it cannot be validvalid

► If a test is valid, it is almost always If a test is valid, it is almost always reliable. reliable.

Types of ValidityTypes of Validity

Type of Validity

Definition Example/Non-Example

Content The extent to which the content of the test matches the instructional objectives.

A semester or quarter exam that only includes content covered during the last six weeks is not a valid measure of the course's overall objectives -- it has very low content validity.

Predictive The extent the test can predict an external criterion

High school GPA and success at college

Concurrent The extent to which scores on the test are in agreement with an external criterion.

If the end-of-year math tests in 4th grade correlate highly with the statewide math tests, they would have high concurrent validity.

Construct

The extent to which an assessment corresponds to other variables, as predicted by some rationale or theory.

If a test is supposed to evaluate mathematical ability then those who took the math course should score higher than those who didn’t

Types of tests and inventoriesTypes of tests and inventories

► Achievement testsAchievement tests► Aptitude testsAptitude tests► Interest inventoriesInterest inventories► Personality inventoriesPersonality inventories► Projectile DevicesProjectile Devices

AssociationAssociation CompletionCompletion Role-playingRole-playing Creative or constructiveCreative or constructive


Achievement testsAchievement tests

►Most tests used in schoolsMost tests used in schools►Used in placing, advancing, and Used in placing, advancing, and

retaining studentsretaining students►DiagnosticsDiagnostics►Standardized tests (compare schools & Standardized tests (compare schools &

districts) IGAP, CATdistricts) IGAP, CAT


Aptitude testsAptitude tests

►To predict future achievement in a To predict future achievement in a particular fieldparticular field

►We cannot measure capacity or aptitude We cannot measure capacity or aptitude but we can measure performance in but we can measure performance in unplanned learningunplanned learning

►Example, music aptitude test, reading Example, music aptitude test, reading readiness, readiness,


IQ testsIQ tests

►Aptitude tests are limited to a Aptitude tests are limited to a particular field like music, math, ...etcparticular field like music, math, ...etc

► IQ is general ability to learn fast and to IQ is general ability to learn fast and to learn mentally difficult taskslearn mentally difficult tasks

►Stanford-Binet, WAIS, WPPSI, WISC_RStanford-Binet, WAIS, WPPSI, WISC_R


Interest inventoriesInterest inventories

►Kuder Preference RecordKuder Preference Record►Strong-Campbell Interest InventoryStrong-Campbell Interest Inventory

(type of jobs interested)(type of jobs interested)


Personality inventoriesPersonality inventories

►Self-reportSelf-report

Problem ?Problem ?

Individuals’ inability or unwillingness to Individuals’ inability or unwillingness to report their own reactions or their own report their own reactions or their own attitudes = Low validity attitudes = Low validity

Implicit MeasuresImplicit Measures


Projective DevicesProjective Devices

► Association TestsAssociation TestsThe respondent is asked to indicate what he/she thinks when The respondent is asked to indicate what he/she thinks when

presented with a picture , cartoon, ink blot, word, or phrase.presented with a picture , cartoon, ink blot, word, or phrase.


Thematic Apperception TestThematic Apperception Test


Rorschach Ink Bolt TestRorschach Ink Bolt Test


Projective DevicesProjective Devices

► Association TestsAssociation TestsThe respondent is asked to indicate what he/she thinks when The respondent is asked to indicate what he/she thinks when

presented with a picture , cartoon, ink blot, word, or phrase.presented with a picture , cartoon, ink blot, word, or phrase.

Thematic Apperception TestThematic Apperception Test

Rorschach Ink Bolt TestRorschach Ink Bolt Test

► Completion Completion (I get very angry when….)(I get very angry when….)

► Role-Playing Role-Playing (act like your mom)(act like your mom)

► CreativeCreative or Constructive or Constructive (draw or paint, play with (draw or paint, play with toys, write something)toys, write something)


ObservationObservation

►Checklist Checklist (the presence or absence of (the presence or absence of a behavior)a behavior)

►Rating Scale Rating Scale (5-7 categories Excellent, (5-7 categories Excellent, good, average)good, average)

►ScorecardScorecard (overall score for a school, a (overall score for a school, a textbook, a department)textbook, a department)

►Scaled specimen Scaled specimen (handwriting scale, (handwriting scale, Goodenough-Harris drawing test)Goodenough-Harris drawing test)


Attitude assessment (opinionnaire)Attitude assessment (opinionnaire)


Attitude assessment (opinionnaire)Attitude assessment (opinionnaire)

►Thurstone technique of scaled values Thurstone technique of scaled values (judges rate items in their strength of (judges rate items in their strength of opinion)opinion)

►Likert MethodLikert Method►Osgood semantic differential (two Osgood semantic differential (two

adjectives)adjectives)►Guttman Scaling


Thurstone techniqueThurstone technique

► people get AIDS by engaging in immoral behavior people get AIDS by engaging in immoral behavior ► you can get AIDS from toilet seats you can get AIDS from toilet seats ► AIDS is the wrath of God AIDS is the wrath of God ► anybody with AIDS is either gay or a junkie anybody with AIDS is either gay or a junkie ► AIDS is an epidemic that affects us all AIDS is an epidemic that affects us all ► people with AIDS are bad people with AIDS are bad ► people with AIDS are real people people with AIDS are real people ► AIDS is a cure, not a disease AIDS is a cure, not a disease ► you can get AIDS from heterosexual sex you can get AIDS from heterosexual sex ► people with AIDS are like my parents people with AIDS are like my parents ► you can get AIDS from public toilets you can get AIDS from public toilets ► women don’t get AIDS women don’t get AIDS ► I treat everyone the same, regardless of whether or not they have AIDS I treat everyone the same, regardless of whether or not they have AIDS ► AIDS costs the public too much AIDS costs the public too much ► AIDS is something the other guy gets AIDS is something the other guy gets ► living with AIDS is impossible living with AIDS is impossible


Thurstone techniqueThurstone technique


You should select statements that are at equal intervals across the range of medians.

attitudes).

Likert MethodLikert Method


Osgood semantic differentialOsgood semantic differential

Guttman Scaling

Cumulative scalingCumulative scaling or or scalogram analysisscalogram analysis

Gives us a set of items or statements so that a respondent who Gives us a set of items or statements so that a respondent who agrees with any specific question in the list will also agree agrees with any specific question in the list will also agree with all previous questionswith all previous questions

► (1) "I am willing to be near ice cream"; (2) "I am willing to (1) "I am willing to be near ice cream"; (2) "I am willing to smell ice cream"; (3) "I am willing to eat ice cream"; and (4) "I smell ice cream"; (3) "I am willing to eat ice cream"; and (4) "I love to eat ice cream". love to eat ice cream".


Guttman Scaling


Q methodologyQ methodology

►Q- sortQ- sort

►A large number of items (opinions) are A large number of items (opinions) are sorted into different (9-11) categories sorted into different (9-11) categories (piles)(piles)

Most important Most important least importantleast important

Most approveMost approve least approveleast approve

Most liberalMost liberal least liberalleast liberal

Most favorableMost favorable least favorableleast favorable


Social Scaling (stereotypes)Social Scaling (stereotypes)

►SociometrySociometry►SociogramSociogram►Guess-Who techniqueGuess-Who technique►Social distance scaleSocial distance scale


SociometrySociometry

►subjects are asked to name in order of subjects are asked to name in order of preference the persons that they preference the persons that they would invite towould invite to

A partyA party

Eat lunch withEat lunch with

Sit next toSit next to

Work on a class projectWork on a class project

Have as BFF Have as BFF



SociogramSociogram


Guess-Who techniqueGuess-Who technique

►Hartshorne & MayHartshorne & May►Descriptions of various roles played by Descriptions of various roles played by

children in a groupchildren in a group This one is always happyThis one is always happy This one is always picking on othersThis one is always picking on others This one is always worriedThis one is always worried This one never likes to do anythingThis one never likes to do anything This one will always help youThis one will always help you

BogardusBogardusSocial distance scaleSocial distance scale

► The scale asks people the extent to which they would The scale asks people the extent to which they would be accepting of each group (a score of be accepting of each group (a score of 1.001.00 for a for a group is taken to indicate group is taken to indicate no social distanceno social distance):):

► As close relatives by marriage (score 1.00)As close relatives by marriage (score 1.00)► As my close personal friends (2.00)As my close personal friends (2.00)► As neighbors on the same street (3.00)As neighbors on the same street (3.00)► As co-workers in the same occupation (4.00)As co-workers in the same occupation (4.00)► As citizens in my country (5.00)As citizens in my country (5.00)► As only visitors in my country (6.00)As only visitors in my country (6.00)► Would exclude from my country (7.00)Would exclude from my country (7.00)


Similar to Guttman’s scaleSimilar to Guttman’s scale

►Measure to what degree an individual Measure to what degree an individual or group of individuals is accepted or or group of individuals is accepted or rejected by another individual or rejected by another individual or group.group.

► Complete acceptanceComplete acceptance I’d like to have this I’d like to have this student as my BFstudent as my BF

► Partial acceptancePartial acceptance I wouldn’t mind sitting near this I wouldn’t mind sitting near this studentstudent

► Rejection Rejection I wish this student weren’t in our I wish this student weren’t in our classclass



The QuestionnaireThe Questionnaire► Inquiry Forms:Inquiry Forms:

Open FormOpen Form Closed FormClosed Form

Open formOpen form

►Calls for free response in the person’s Calls for free response in the person’s own wordsown words

► In-depth but return rate is lowIn-depth but return rate is low►Mostly used in qualitative researchMostly used in qualitative research

Why did you choose to do your graduate work at this Why did you choose to do your graduate work at this university?university?



Closed FormClosed FormTeacher Morale Questionnaire


Closed FormClosed Form

Improving Questionnaire ItemsImproving Questionnaire Items

►Clarify your items (Clarify your items (no raising handsno raising hands))►The same words mean different things The same words mean different things

to different people.to different people.► WereWere you there last night? you there last night?► Were Were youyou there last night? there last night?► Were you Were you therethere last night? last night?► Were you there Were you there lastlast night? night?► Were you there last Were you there last nightnight??


Improving Questionnaire ItemsImproving Questionnaire Items

►Define or qualify terms Define or qualify terms ► (what is the value of your house?)(what is the value of your house?)► Age (age at the last birthday, or date of birth)Age (age at the last birthday, or date of birth)► What work are you doing now? (filling out your foolish What work are you doing now? (filling out your foolish

questionnaire)questionnaire)► One respondent’s occasionally maybe another’s rarelyOne respondent’s occasionally maybe another’s rarely► Double negative- Are you opposed to not requiring students to Double negative- Are you opposed to not requiring students to

take showers after gym class? = Are you in favor of requiring take showers after gym class? = Are you in favor of requiring students to take showers after gym class? students to take showers after gym class?

► Inadequate alternatives (Married? Yes No)Inadequate alternatives (Married? Yes No)


What is wrong with this question?What is wrong with this question?

►How late at night do you permit your How late at night do you permit your children to watch television?children to watch television?



►"do you think that students should "do you think that students should have more classes about history and have more classes about history and culture?"culture?"


Underline KeywordUnderline Keyword

►Should all schools offer a Should all schools offer a modernmodern foreign language?foreign language?



►How would you rate this student How would you rate this student teacher’s classroom demeanor?teacher’s classroom demeanor?

Superior Average Below Superior Average Below average average



►Are you satisfied with the salary raise Are you satisfied with the salary raise that you received last year?that you received last year?


►Design questions that will give a Design questions that will give a complete response.complete response.

Do you read the Indianapolis Star?Do you read the Indianapolis Star?



►What are your favorite television programs?What are your favorite television programs?


Academic senate1st choice 5 points2nd choice 4 points

►Classify the responses yourself rather Classify the responses yourself rather than having the respondent creating than having the respondent creating their own categories.their own categories.

► Unskilled laborUnskilled labor► Skilled laborSkilled labor► Clerical workClerical work► ProfessionalProfessional


Characteristics of a good Characteristics of a good QuestionnaireQuestionnaire

► Significant topic to warrant respondingSignificant topic to warrant responding► Seeks info not available elsewhereSeeks info not available elsewhere► As short as possibleAs short as possible► Attractive in appearanceAttractive in appearance► Directions clearDirections clear► Objective- No leading suggestionsObjective- No leading suggestions► Psychological order More general to specifics and personalPsychological order More general to specifics and personal► Tabulate an interpret possible answers with code sheetTabulate an interpret possible answers with code sheet


Copyright © Allyn & Bacon 2008 CHAPTER NINE Methods and Tools of Research This multimedia product and its contents are protected under copyright law. The.

Documents

cat copyright allyn

r copyright allyn bacon

number of slide

interesting slide

types of validity slide

copyright law

iq tests aptitude tests

types of tests