tau equivalent.pdf

7/25/2019 tau equivalent.pdf

1/65

ComparingClassicalTestTheorywithCFA

and

HowToUseTestScoresin

SecondaryAnalyses

LatentTraitMeasurementand

StructuralEquationModels

Lecture#8

March6,2013

PSYC948:Lecture#8


2/65

TodaysClass

ComparingclassicaltesttheorytoCFA Theuseandmisuseofsumscores

ReliabilityforsumscoresunderCFA

HowtouseCFAtotestassumptionsinCTT

WhattodowhenSEMisntanoption Secondaryanalyses

PSYC948:Lecture#8 2


3/65

DataforTodaysClass

Datawerecollectedfromtwosources: 144experiencedgamblers

Manyfromanactualcasino

1192collegestudentsfromarectangularmidwestern state

Manynever

gambled

before

Today,wewillcombinebothsamplesandtreatthemas

homogenous onesampleof1346subjects

Laterwewilltestthisassumption measurementinvariance(calleddifferentialitemfunctioninginitemresponsetheoryliterature)

Wewillbuildascaleofgamblingtendenciesusingthefirst24items

oftheGRI Focusedonlongtermgamblingtendencies

PSYC948:Lecture#8 3


4/65

PathologicalGambling:DSMDefinition

Tobediagnosedasapathologicalgambler,anindividualmustmeet5of10definedcriteria:

PSYC948:Lecture#8

1. Ispreoccupiedwithgambling

2. Needstogamblewithincreasingamountsofmoneyinorderto

achievethe

desired

excitement

3. Hasrepeatedunsuccessfuleffortstocontrol,cutback,orstopgambling

4. Isrestlessorirritablewhenattemptingtocutdownorstopgambling

5. Gamblesasawayofescapingfromproblemsorrelievingadysphoricmood

6. Afterlosingmoneygambling,oftenreturnsanotherdaytogeteven

7. Liestofamilymembers,therapist,orotherstoconcealtheextentofinvolvementwithgambling

8.

Hascommitted

illegal

acts

such

as

forgery,fraud,theft,orembezzlementtofinancegambling

9. Hasjeopardizedorlostasignificantrelationship,job,educational,orcareeropportunitybecauseof

gambling10. Reliesonotherstoprovidemoney

torelieveadesperatefinancialsituationcausedbygambling

4


5/65

Final12ItemsontheScale

Item Criterion Question

GRI1 3 Iwouldliketocutbackonmygambling.

GRI3 6

IfIlostalotofmoneygamblingoneday,Iwouldbemorelikelytowanttoplay

againthefollowingday.

GRI5 2

Ifinditnecessarytogamblewithlargeramountsofmoney(thanwhenIfirst

gambled)forgamblingtobeexciting.

GRI6 8 Ihave gonetogreatlengthstoobtainmoneyforgambling.

GRI9 4 IfeelrestlesswhenItrytocutdownorstopgambling.

GRI10 1 ItbothersmewhenIhavenomoneytogamble.

GRI11 5 Igambletotakemymindoffmyworries.

GRI13 3 Ifinditdifficulttostopgambling.

GRI14 2 IamdrawnmorebythethrillofgamblingthanbythemoneyIcouldwin.

GRI15 7 Iamprivateaboutmygamblingexperiences.

GRI21 1 Itishardtogetmymindoffgambling.

GRI23 5 Igambletoimprovemymood.

PSYC948:Lecture#7 5


6/65

The12itemanalysisgavethismodelfitinformation:

GRI12ItemAnalysis

PSYC948:Lecture#7 6

Themodelindicatedthemodeldidnotfitbetterthanthesaturatedmodel butthis

statisticcanbeoverlysensitive

ThemodelRMSEAindicatedgoodmodelfit

(wantthistobe.95)

TheSRMRindicatedthefitwell(wantthisto

be


7/65

CLASSICALTESTTHEORY

PSYC948:Lecture#8 7


8/65

ClassicalTestTheory(CTT)

WhatyouhavelearnedaboutmeasurementsofarlikelyfallsunderthecategoryofCTT:

Writingitemsandbuildingscales

Itemanalysis

Scoreinterpretation

Evaluatingreliabilityandconstructvalidity

Bigpicture:WewillviewCTTasmodelwitharestrictivesetof

assumptionswithinamoregeneralfamilyoflatenttrait

measurementmodels

ConfirmatoryFactorAnalysisisameasurementmodel

PSYC948:Lecture#8 8


9/65

DifferencesAmongMeasurementModels

Whatisthenameof

the

latent

traitmeasuredbyatest?

ClassicalTestTheory(CTT) = TrueScore(T)

ConfirmatoryFactorAnalysis(CFA) = FactorScore(F)

ItemResponseTheory(IRT) = Theta()

Fundamentaldifferenceinapproach:

CTT unitofanalysisistheWHOLETEST(itemsumormean)

Sum=latenttrait,andthesumdoesntcarehowitwascreated

Onlyusingthesumrequiresrestrictiveassumptionsabouttheitems CFA,IRT,andbeyond unitofanalysisistheITEM

Modelofhowitemresponserelatestoanestimatedlatenttrait

Differentmodelsfordifferingitemresponseformats

Providesaframeworkfortestingadequacyofmeasurementmodels

PSYC948:Lecture#8 9


10/65

ClassicalTestTheory(CTT)

InCTT,theTESTistheunitofanalysis: TruescoreT:

Bestestimateoflatenttrait:Meanoverinfinitereplications

Errore:

Expectedvalue(mean)of0,expectedtobeuncorrelatedwithT

esaresupposedtowashoutoverrepeatedobservations

SotheexpectedvalueofTisYtotal Intermsofobservedvarianceofthetestscores:

Observedvariance=truevariance+errorvariance

GoalistoquantifyreliabilityReliability=truevariance/(truevariance+errorvariance)

Because

the

CTT

model

does

not

include

individual

items,

itemsmustbeassumedexchangeable(andmoreitemsisbetter)

PSYC948:Lecture#8

Ytotal

TrueScore

error

?

?

10


11/65

ClassicalTestTheory,continued

CTTunitofanalysisistheWHOLETEST(sumofitems) Wanttoascertainhowmuchofobservedtestscorevariance

isduetotruescorevarianceversuserrorvariance

Quantifyerrorvarianceinvariousways

ErrorisaunitaryconstructinCTT(anderrorisbad) Goalisthentoreduceerrorvarianceasmuchaspossible

Standardizationoftestingconditions(makeconfoundsconstants)

Aggregation=moreitemsarebetter(errorsshouldcancelout)

Itemsareexchangeable;propertiesarenottakenintoaccount

Followedbygeneralizabilitytheorytodecomposeerror

e.g.,ratervariance,personvariance,timevariance

PSYC948:Lecture#8 11


12/65

AdvantagesofCFAoverCTT

Morereasonableassumptionsaboutitems CTTassumestauequivalentitems

Tau equivalentitems:equalfactorloadings

CFAallowsatestofwhethereachitemrelatestothefactor,aswellaswhether

differentfactorloadingsacrossitemsareneeded

Wouldindicatesomeitemsarebetterthanothers

Comparabilityacrosssamples,groups,andtime CTT:Noseparationofobserveditemresponsesfromtruescore

Sumacrossitems=truescore;itempropertiesarefor thatsampleonly

CFA:Latenttraitisestimatedseparatelyfromitemresponses

Separatespersontraitsfromspecificitemsgiven

Separatesitempropertiesfromspecificpersonsinsample

Advantagesapplytoanylatenttraitmodel



13/65

ReliabilityMeasuredbyAlpha

Forquantitativeitems(itemswithascale althoughusedoncategoricalitems),thisisCronbachs Alpha

OrGuttmanCronbach alpha(Guttman 1945>Cronbach 1951)

Anotherreducedformofalphaforbinaryitems:KR20

Alphaisdescribedinmultipleways:

Isthemeanofallpossiblesplithalfcorrelations

Isexpectedcorrelationwithhypotheticalalternativeformofthe

samelength

Islowerboundestimateofreliabilityunderassumptionthatallitemsaretau

equivalent(moreaboutthatlater)

Asanindexofinternalconsistency

Althoughnothingabouttheindexindicatesconsistency!



14/65

WhereAlphaComesFrom

Thesumof

the

item

variancesisgivenby:

Var(I1)+Var(I2)+Var(I3).+Var(Ik)(justtheitemvariances)

ThevarianceofthesumoftheitemsisgivenbythesumofALLthe

itemvariancesandcovariances:

Var(I1+I2+I3)=Var(I1)+Var(I2)+Var(I3)

+2Cov(I1,I2)+2Cov(I1,I3)+2Cov(I2,I3)

Wheredoesthe2comefrom?

Covariancematrixissymmetric

Sumthewholethingtogettothe

varianceofthesumoftheitems

PSYC948:Lecture#8

I1 I2 I3

I1

12

12

13

I2

21

22

23

I3 31 32 3

2

14


15/65

GuttmanCronbach AlphaforReliability

Numeratorreducestojustthecovarianceamongitems

Sum

of

the

item

variances

Var(X)+Var(Y)=Var(X)+Var(Y)justtheitemvariances

Variance

of

total

Y

(the

sum

of

the

items)

Var(X+Y)=Var(X)+Var(Y)+2Cov(X,Y) PLUScovariances

So,iftheitemsarerelatedtoeachother,thevarianceofthetotalYitemsum

shouldbebiggerthanthesumoftheitemvariances

Howmuchbiggerdependsonhowmuchcovarianceamongtheitems theprimaryindexofrelationship

PSYC948:Lecture#8

Covariance

Version:

k=#items

15


16/65

AssessingReliabilityinOur12ItemGamblingScale

TogettheGuttmanChronbach Alphaofour12itemscale,weneedthecovariancematrix

ThiscanbefoundbytheSAMPSTAToptionunderthe

OUTPUTstatement

Sumofitemvariances=11.834

Sumofitemcovariances=21.575

VarianceofTotalY=11.834+2*21.575=54.984

Alphareliability: ..

. .852



17/65

Reliabilityforwhat?

Thealphareliabilityisthereliabilityfor: Thetotaltestscore

Undertheassumptionthattheitemsaretauequivalent

Tauequivalentmeanseachitemcontributesequally

Inafewslides,wewillseehowthistranslatestoCFA

Whatalphaisnot: Anindexofmodelfit(unidimensionality)



18/65

MeasurementLanguage:DontSayThese

Often,peoplerefertoitemsastappingsomelatenttrait Ithinkthismakestheprocesslesstransparent itemsmeasurethetrait

Whenalphaisused,youcansometimeshearpeoplesaysomething

abouthow

well

the

items

hang

together

Thisiscertainlynottrue



19/65

HowtoGetAlphaUP



20/65

AlphaasReliabilityWhatcouldgowrong?

Alphadoesnotindexdimensionality itdoesnotindextheextenttowhichitemsmeasurethesameconstruct

Thevariabilityacrosstheinteritemcorrelationsmatters,too!

Weuseitembasedmodels(CFA)toexaminedimensionality



21/65

CaseInPoint:All24Items

Lastclassweshowedthatthe24itemsoftheGRIdidnotfitaonefactormodel whatwouldhappenifweneglectedtocheckmodel

fitandusedthetotalscoreasourestimateofgamblingtendency?

Thereliabilityestimate fromthecovariancematrixofalltheitems

(thesaturatedmodelH1)was .861 Wewouldhaveconcludedwehadagoodscaleforgambling

But,fromCFAlastweek,wefoundthatonefactordidntdescribeall

theitems Anysubsequentanalysiswillhavethemisfitbiastheresults



22/65

TestingCTTAssumptionsinCFA

Alphaisreliabilityassumingtwothings: Allfactorloadings(discriminations)areequal,orthattheitemsare

truescoreequivalentortauequivalent

Localindependence(dimensionalitynowtestedwithinfactormodels)

Wecantesttheassumptionoftauequivalencetoovianestedmodel

comparisonsinwhichtheloadingsareconstrainedtobeequal

doesmodelfitgetworse?

Ifso,dontusealpha usemodelbasedreliability(omega)instead.Omegaassumesunidimensionality,butnottauequivalence

Researchhasshownalphacanbeanoverestimateoranunderestimatedependingon

particulardatacharacteristics

TheassumptionofParallelitemsisthentestablebyconstraining

itemerrorvariancestobeequal,too doesmodelfitgetworse? Parallelitemswillhardlyeverholdinrealdata

Notethatiftauequivalencedoesnthold,thenneitherdoesparallel



23/65

AnotherBlastfromthePast:ParallelItems

AnotherCTTmodelthatexiststhatofparallelitems Allitemshavethesamecovarianceandvariance

Goesonestepfurtherthantauequivalence(equalcovariancesbutunequalvariances)

Undertheparallelitemsmodel,thealphareliabilityforthetotaltestscoreiscalledthe

SpearmanBrownreliability Usedtoprophesythenumberofitemsneededtoincreasereliabilitytoadesiredlevel

SpearmanBrownProphesyFormula

ReliabilityNEW=ratio*relold/[(ratio1)*relold+1]Ratio=ratioofnew#itemstoold#items

Forexample:

Oldreliability=.40Ratio=5timesasmanyitems(had10,whatifwehad50)

Newreliability=.77



24/65

Reliabilityvs.ValidityParadox

GiventheassumptionsofCTT,itcanbeshownthatthecorrelationbetweenatestandanoutsidecriterioncannotexceedthereliabilityofthetest(seeLord&

Novick1968)

Reliabilityof.81?Noobservedcorrelationspossible>.9,

becausethatsallthetruevariancetheretoberelatable!

Inpractice,thismaybefalsebecauseitassumesthattheerrorsareuncorrelatedwith

thecriterion(andtheycouldbe)

Selectingitemswiththestrongestdiscriminations(orthestrongestinter

correlations)canhelptopurifyorhomogenizeatest,butpotentiallyattheexpenseofconstructvalidity

Canendupwithabloatedspecific

Itemsthatareleastinterrelatedmaybemostusefulinkeepingtheconstructwell

definedandthusrelatabletootherthings



25/65

UsingCTTReliabilityCoefficients:BacktotheScoreEstimates

ReliabilitycoefficientsareusefulfordescribingthebehaviorofthetestintheoverallsampleVar(Y)=Var(T)+Var(e)

Butreliabilityisameanstoanendininterpretingascoreforagivenindividual

weuseittogettheerrorvariance

Var(T)=Var(Y)*reliability;soVar(e)=Var(Y) Var(T)

95%CIforindividualscore=Y1.96*SD(e)

Givesanindicationofhowprecisethetruescoreestimateisonthemetricofthe

originalvariable

Example:

Y

=

100,

Var(e)

=

9

95%

CI

94

to

106Y=100,Var(e)=25 95%CI90to110

Notethisassumesasymmetricdistribution,andthuswillgooutofboundsofthescale

forextremescores

NotethisassumestheSD(e)ortheSEforeachpersonisthesame

Cuemind

blowing

GRE

example



26/65

95%ConfidenceIntervals:Quantitative

SEMrangesfrom9to55



27/65

REVISITINGCTTFROMA

CFAPERSPECTIVE



28/65

ClassicalTestTheoryfromaCFAPerspective

InCTTtheunitofanalysisisthetestscore:,

InCFAtheunitofanalysisistheitem:

TomapCFAontoCTT,wemustputthesetogether:

,



29/65

FurtherUnpackingoftheTotalScoreForumla

BecauseCFAisanitembasedmodel,wecanthensubstituteeachitemsmodelintothesum:

,

MappingthisontotruescoreanderrorfromCTT:

and



30/65

FamiliarTerms

Thetauequivalentmodelassumes: Allitemsmeasurethefactorthesame: Eachitemhasitsownuniquevariance: 0,

Theparallelitemsmodelsassumes: Allitemsmeasurethefactorthesame: Allitemshavethesameuniquevariance: 0,

Assuch,eachofthesemodelscanbetestedbyusingtheCFAapproach eacharenestedwithinthefullCFAmodel



31/65

TauEquivalence:ModelImpliedCovarianceMatrix

TheCFAmodelimpliesaveryspecificformforthecovariancematrixoftheobserveditems:

Thevarianceofanitem

was:

Thecovarianceofapairofitemsandwas:

UnderTauEquivalence,allloadingsarethesame,meaning: Theitemvariancescanbedifferent(becauseof) Allitemcovariancesarethesame()

Thisiscalledthecompoundsymmetryheterogeneousmodel Wecanactuallyachievethesamemodelwithoutthefactor



32/65

TauEquivalenceModelfor12ItemGamblingScale

ThefollowingtwopiecesofMplussyntaxresultinthesameequivalentmodel: TauEquivalenceasaFactorModel:

TauEquivalenceasaCompundSymmetryHeterogeneousVariancesModel:



33/65

ModelImpliedCovarianceMatrix

Allcovariancesequal/allvariancesdifferent



34/65

TestingforTauEquivalence

TheTauEquivalencemodel(assumedwhenyousumitems)canbetestedagainstthefullCFAmodel

Themodelsarenested,sowecanusealikelihoodratiotest

LoglikelihoodfromCFAmodel: 18,988.425;SCF=2.4309 36parameters(12itemintercepts,11factorloadings,1factorvariance,12uniquevariances)

LoglikelihoodfromTEmodel: 19,051.350;SCF=2.5172 25parameters(12itemintercepts,1factorloading,12uniquevariances)

MLRLikelihoodratiotest: 56.315, .001

Therefore,werejectthetauequivalentmodelinfavoroftheCFAmodel this

meansthe

simple

sum

of

the

items

isnot

sufficient

WeshouldusetheCFAmodelfactorscoreinsteadofasumscore



35/65

ParallelItems:ModelImpliedCovarianceMatrix

TheCFAmodelimpliesaveryspecificformforthecovariancematrixoftheobserveditems:

Thevarianceofanitemwas: Thecovarianceofapairofitemsandwas: UnderParallelItems,allloadingsanduniquevariancesarethesame:

Allitemvariancesarethesame( ) Allitemcovariancesarethesame()

Thisiscalledthecompoundsymmetrymodel Wecanactuallyachievethesamemodelwithoutthefactor

Becauseparallelitemsarenestedwithintauequivalentitems,wedonothaveto

testthismodelasweknowitwillnotfitwhencomparedtotheCFAmodel



36/65

TestScoreReliabilityUndertheCFAModel

CoefficientalphagavereliabilityforthetotaltestscoreundertheTauEquivalentItemsModel

WerejectedthatmodelinfavoroftheCFAmodel

Therefore,coefficientalphawillnotbecorrectforourtotaltestscore(ifwewereto

stillsum

up

the

items)

ThenotionsoftestscorereliabilityundertheCFAmodelnowinvolve

thefactorloadings Butstillcomebacktoclassicalnotionofreliabilitybeingtheproportionofvariancedue

totruescore:



37/65

DerivingReliabilityForSumScores UndertheCFAModel

ToshowwheretotalscorereliabilityundertheCFAmodelcomesfrom,recallourCFAmodelforthetotalscore:

,

MappingthisontotruescoreanderrorfromCTT:

and

WenowmustderivethevarianceforTandE


S V i U d h C A d l


38/65

TrueScoreVarianceUndertheCFAModel

Thevarianceforthetruescore:


E V i U d th CFA M d l


39/65

ErrorVarianceUndertheCFAModel

BecausetheCFAmodelallowsfortheestimationoferrorcovariances(althoughyoushouldntdothat),theerror

varianceundertheCFAmodelbecomes:

Whenerrorcovariancesarenotestimated,thelasttermis

zero,leaving PSYC948:Lecture#8 39

R li bilit f T t l S U d CFA


40/65

ReliabilityforTotalScoreUnderCFA

ThereliabilityofthetotalscorefromCFA,isthen:

2, ThisreliabilitycoefficientiscalledcoefficientOmega() Ifthetauequivalentmodeldoesnotholdisthereliabilityofa

totaltestscore(sumscore) TypicallyishigherthanAlpha

Ifunidimensionalmodelholds,coefficientswillbeclose


Calculating Omega for Our Test


41/65

CalculatingOmegaforOurTest

Wecan

use

Mplus

to

calculate

Omega

for

our

test:

PSYC948:Lecture#8

Here,Omegais.855

41

Omega Under Tau Equivalent Items


42/65

OmegaUnderTauEquivalentItems

Omegaequal

to

Alpha

when

you

use

the

tau

equivalent

items

model

OmegaistheSpearmanBrownreliabilityunderparallelitems

PSYC948:Lecture#8

Here,Omegais.852

whichisequaltothe

Alphawecalculatedusing

thecovariancematrix

42

Recapping: CTT using CFA


43/65

Recapping:CTTusingCFA

Classicaltesttheory andmorespecifically,totaltestscores,isthedominantwaytoassesssubjects

ThisistrueevenunderCFA

Thekeyistobesuretocheckifaonefactormodelfitsthedatabeforeusinganytypeofreliabilitycoefficient

Ifnot,donotuseatestscore

Iftheonefactormodelfits thenasinglescorecanrepresentthetest

Thenextworryisaboutrepresentingtheerrorinthetestscore

(relatedtoreliability) Ifreliabilityishigh(?Howhigh,standardof.8),thenusingthetestscoreina

subsequentanalysisisacceptedpractice


Secondary Analyses with Factor Scores


44/65

SecondaryAnalyseswithFactorScores

Ifyouwanttouseresultsfromasurveyinanewanalysis Best: UseSEM errorinfactorscoresisalreadypartitionedvariance

Similarlygood: Useplausiblevalues(repeateddrawsfromposterior

distributionofeachpersonsfactorscore) essentiallywhatSEMdoes butwithfactorscoresthatvarywithinaperson

CanbedoneinMplus notdescribed

SlightlyLessGood: UseSEMwithsingleindicatorfactorsusingsumscores

Thefocusofthenextsection

Makeerrorvariance=(1reliability)*Variance(Sumscore);factorloading=1

Okay(butwidespread):forscalesthatareunidimensional(andverifiedin

CFA),usesumscoresAssumesunidimensionalityandhighreliability

NotCool: Usefactorscoresonly


What about Using Factor Scores?


45/65

WhataboutUsingFactorScores?

AlthoughCFAfactorscoreshavefewerproblemsthanEFAfactorscores(becausethereisnorotationinCFA),theystillhaveissues:

Theywillbeshrunken(i.e.,pushedtowardsthemean,suchthattheobservedvariance

ofthefactorscoreswillbelessthantheoriginalfactorvariance)

Cangetestimatesoffactordeterminacy howcorrelatedestimatedfactorscores

arewithtruefactorscores(basicallyhowmucherrorisintroducedbyestimatingthe

factorscoresasobservedvariables)

Theyarejustestimatesof

central

tendencyfromadistributionforeachperson,not

knownvalues andusingestimatesasknownvaluesinanothermodelmakesthe

relationshipswithinthatmodellookmoreprecisethantheyare(likeSE=0)

YouCANNOTcreatefactorscoresbyusingtheloadingsassuch:

F=11y1+21y2+21y3 ThisisaCOMPONENTmodel,notaFACTORmodel.



46/65

SINGLEINDICATORMODELS


Single Indicator Models


47/65

SingleIndicatorModels

SingleindicatormodelsareCFAlikemodelswhereafactorismeasuredbya

singleindicator:

Shownhereforthegamblingfactor

PSYC948:Lecture#8

Gambling

Tendencies

Sumof10

GRIItems

,

1

1

47

IdentificationinSingleIndicatorModels


48/65

g

Howisthis

possible?

Isnt

asingle

indicator

factor

model

unidentified?

Wefixthefactorvariance,factorloading,anduniquevariance

Factorvariancerepresentsreliableportion

Singleindicatormodelparameters:

factorvariance; factorloading;itemuniquevariance(assumefactormeanfixedtozeroanditeminterceptissettoitmean)

Ourconstraintsare:

1 (theportionofYthatisreliable) 1 (theportionofYthatisleftover)


AssumptionsinaSingleIndicatorAnalysis


49/65

p g y

Touseasingleindicatoryoumustassume: Theindicatorisunidimensional(onlyonefactor)

ThisistestableinCFA(butifyouhaveasmallsampleishardtodo)

Ifnotpossibletotest,youmustassumeyouhaveonefactor Thisisanassumptionthatthetestis*as*dimensionalinyoursample/population

Thereliabilityoftheindicatorisknown

AlsoobtainablefromCFA

Ifnotpossibletoobtain,thenyoumustuseapreviouslyreportedreliability

coefficient Thisisanassumptionthatthetestis*as*reliableinyoursample/population


SingleIndicatorExampleAnalysis


50/65

g p y

Todemonstrateasingleindicatorexampleanalysis,wewillusethe12itemGRItopredicttheSOGSscore

SOGS=SouthOakGamblingScreen(wecollectedthis)

Note:weassumethishasreliabilityof1.0

The12itemGRIisthesingleindicatorofthegamblingfactor

Step#1:determinethatthesingleindicatorisunidimensional The1factorCFAmodelfitthe12itemGRI

Step#2:getthesingleindicatorreliability FromtheCFAanalysiswefoundthatthereliabilityofthe12itemGRIwas.855

Step#3:estimatethevarianceofthe12itemGRItotalscore WecandothisinMplus foundthevariancetobe55.050


SingleIndicatorAnalysis


51/65

NowthatwehaveourreliabilityoftheGRIandthevarianceoftheGRI,wecanputthese

intothesingleindicatormodel:


SingleIndicatorModelResults:


52/65

Note:47.067+7.982=55.05(thevarianceoftheGRI)

PSYC948:Lecture#8

Gambling

Tendencies

Sumof10

GRIItems

,

1

47.067

(3.459)

7.982

SOGSScore

.167(.011)

2.440(.213)

52

SingleIndicatorModelInterpretation


53/65

Thestandardizedregressionslopeforthegamblingfactor,predictingtheSOGSwas.591

asgamblingwentup1SD,theSOGSscorewentup.591SD

Correlationbetween

gamblingandSOGS

Thegambling

factor

accountedfor35.0%ofthe

varianceintheSOGSscore


ComparisonwithNonSingleIndicator


54/65

Withoutusingthesingleindicator:



55/65

ITEMPARCELING


ItemParceling


56/65

Frequently,sumscoresareusedinSEMunderadifferentlabel:asitemparcels

Evidentlyparcelsoundsmorepolitestuffthatdidntfit

ItemparcelsaresumsofsetsofitemsthatareinsertedintoaSEMwithoutanyfurtherinspection

Frequently,itemparcelswillhidebadfitofmodel Blindparceling=cheating

Asparcelsaresums andtodaysclassisaboutusingsums,wecan

nowdiscuss

parcels

under

CTT

with

CFA


ApplyingOurUnderstandingofTotal/SumScorestoParcels


57/65

Aswe

have

seen

today,

atotal

score

isastatistical

model

Tauequivalentitems

Aswithanystatisticalmodel,ifthemodeldoesnotfit(adequately

representthedata),misleadingresultsoccur

Parcelingitemsmakesanimplicitassumptionabouttheirstructure

thattheytooaretauequivalent Ifthatassumptionisnotvalid,resultscannotbebelieved

Mostusesofparcelingmakenoattempttodetermineifthetau

equivalenceassumption

iscorrect


Revisitingour24ItemGRI


58/65

Asyou

will

recall,

our

24

item

GRI

did

not

fit

the

one

factor

model

Todemonstrateparceling,wewilltakethe12misfittingitemsand

createaparcel(sumtheirscorestogether)

WewillthenaddtheparceltoaCFAmodelwiththeother12items


ThemodelRMSEAindicatedthemodeldidnotfitwell(wantthistobe.95)

TheSRMRindicatedthemodeldidnotfitwell

(wantthistobe


59/65

Thesyntax:

Themodelfitstatistics(adequatemodelfit):

Note:12itemGRIhadRMSEAof.045


HowParcelingHidesPoorModelFit


60/65

Theitem

parcel

hides

poor

model

fit

by

using

anumbers

game

toitsadvantage

Modelwith24itemshad300elementsinsaturatedcovariance

matrix(but48parametersforthatmatrix)

Modelwith12itemsplusparcel(12items)had91elementsin

saturatedcovariance

matrix

(and

26

parameters)

Therelativeratioofparameterstosaturatedcovariancesmakesthe

parcelhidethefitissues Especiallywhentheremainderoftheitemsfitwellalready


ParcelingDoneRight


61/65

Toaddaparcelyoumustfirstexaminethefitofaonefactormodeltotheitems

oftheparcel:

Themodelfitsuggestsaonefactormodeldoesntfit


WhyParcelingisCheatingandWhyYouShouldntDoIt


62/65

Ifyou

didnt

check

the

parcel

before

adding

itto

the

12

item

GRI

modelyouwouldconclude1factormodelfitthedatawell Ifaonefactormodelfits,thenwhatcomesnextistypicallytheuseofitssumscore

Thesumscorefromthe12good+1bad(parcel)modelisjustthesumscorefromthe24itemGRI whichdidntfitaonefactormodel

Thecaveat:the12+1modelsumscorehadanomegareliabilityof.516!

Mostofthelackofreliabilitycomesfromtheestimateduniquevarianceoftheparcel

Youcannotmakeagoodfactorbycheatingwithaparcel!



63/65

CONCLUDINGREMARKS


WrappingUp


64/65

Todaywas

spent

on

comparing

classical

test

theory

(synonymous

withsumscores)toCFA

UnderstandinghowCTTandCFAarerelatedisimportant ManypeoplebelievethatsumscoresareAOK

Theyonlyareiftheyfita1factormodelandhaveahighreliability

Manypeopledontthinkparcelinginvolvessumscores

Thelabelmustbetheproblem

Singleindicatormodelscanbeagoodwaytousesumscoresif: The1factormodelfits

Thereisahighdegreeofreliability

WewillreturntothisoncewediscussSEMmorethoroughly


ComingUp


65/65

Nextweekslecture:multidimensionalCFAmodels Morethanonefactor

Reliabilityforatotaltestscorenolongerapplies(eachfactoris

wherereliabilityisimportant)

Timepermitting:anintroductiontoExploratoryFactorAnalysis

FollowedbyacomparisonofCFAandEFA

AndwhyyoualsoshouldntbedoingEFA ButcouldexplorethedatabetterusingCFAtechniques


tau equivalent.pdf

Documents