Principal Components Analysis; Exploratory Factor Analysis ...€¦ · The Logic of Exploratory Analyses • Exploratory analyses attempt to discover hidden structure in data with

PrincipalComponentsAnalysis;ExploratoryFactorAnalysis;

ConductingExploratoryAnalyseswithCFA

(or…WhyIHateEFA)

EPSY905:MultivariateAnalysisSpring2016

Lecture#13– April27,2016

EPSY905:PCA,EFA,andCFA

Today’sClass

• Methodsforexploratoryfactoranalysis(EFA)Ø PrincipalComponents-based (TERRIBLE)Ø MaximumLikelihood-based Exploratory FactorAnalysis(BAD)Ø ExploratoryStructuralEquationModeling(ALSOBAD)

• ComparisonsofCFAandEFA

• HowtodoexploratoryanalyseswithCFAØ StructureofnoitemsknownØ Structureofsomeitemsknown

EPSY905:PCA,EFA,andCFA 2

TheLogicofExploratoryAnalyses• Exploratoryanalysesattempt todiscover hiddenstructure indatawithlittletono

user inputØ Asidefromtheselectionofanalysisandestimation

• Theresults fromexploratoryanalysescanbemisleadingØ IfdatadonotmeetassumptionsofmodelormethodselectedØ IfdatahavequirksthatareidiosyncratictothesampleselectedØ IfsomecasesareextremerelativetoothersØ Ifconstraintsmadebyanalysisareimplausible

• Sometimes,exploratory analysesareneededØ MustconstructananalysisthatcapitalizesontheknownfeaturesofdataØ Therearebetterwaystoconductsuchanalyses

• Often,exploratoryanalysesarenotneededØ Butareconductedanyway– seealotofreportsofscaledevelopmentthatstartwiththeideathat

aconstructhasacertainnumberofdimensions


ADVANCEDMATRIXOPERATIONS


AGuidingExample

• Todemonstratesomeadvancedmatrixalgebra,wewillmakeuseofdata

• IcollecteddataSATtestscoresforboththeMath(SATM)andVerbal(SATV)sectionsof1,000students

• Thedescriptivestatisticsofthisdatasetaregivenbelow:


MatrixTrace

• Forasquarematrix𝚺 withp rows/columns,thetraceisthesumofthediagonalelements:

𝑡𝑟𝚺 =%𝑎''

(

')*• Forourdata,thetraceofthecorrelationmatrixis2

Ø Forallcorrelationmatrices, thetrace isequaltothenumberofvariablesbecausealldiagonalelements are1

• Thetracewillbeconsideredthetotalvarianceinprincipalcomponentsanalysis

Ø Usedasatargettorecoverwhenapplyingstatisticalmodels


MatrixDeterminants• Asquarematrixcanbecharacterizedbyascalarvaluecalledadeterminant:

det𝚺 = 𝚺

• CalculationofthedeterminantbyhandistediousØ Ourdeterminant was0.3916Ø Computers canhavedifficulties withthiscalculation(unstable incases)

• Thedeterminantisusefulinstatistics:Ø ShowsupinmultivariatestatisticaldistributionsØ Isameasureof“generalized”varianceofmultiplevariables

• Ifthedeterminantispositive,thematrixiscalledpositivedefiniteØ Isinvertable

• Ifthedeterminantisnotpositive,thematrixiscallednon-positivedefinite

Ø NotinvertableEPSY905:PCA,EFA,andCFA 7

MatrixOrthogonality

• Asquarematrix𝚲 issaidtobeorthogonalif:𝚲𝚲/ = 𝚲/𝚲 = 𝐈

• Orthogonalmatricesarecharacterizedbytwoproperties:1. Thedotproductofallrowvectormultiplesisthezerovector

w Meaning vectorsareorthogonal (oruncorrelated)2. Foreach rowvector, thesumofallelements isone

w Meaning vectorsare“normalized”

• ThematrixaboveisalsocalledorthonormalØ Thediagonal isequal to1(eachvectorhasaunit length)

• Orthonormalmatricesareusedinprincipalcomponentsandexploratoryfactoranalysis


EigenvaluesandEigenvectors

• Asquarematrix𝚺 canbedecomposedintoasetofeigenvalues𝛌 andasetofeigenvectors𝐞

𝚺𝐞 = λ𝐞

• EacheigenvaluehasacorrespondingeigenvectorØ Thenumberequaltothenumberofrows/columnsof𝚺Ø Theeigenvectors areallorthogonal

• Principalcomponentsanalysisuseseigenvaluesandeigenvectorstoreconfiguredata


EigenvaluesandEigenvectorsExample

• InourSATexample,thetwoeigenvaluesobtainedwere:𝜆* = 1.78𝜆9 = 0.22

• Thetwoeigenvectorsobtainedwere:

𝐞* =0.710.71 ; 𝐞9 =

0.71−0.71

• Thesetermswillhavemuchgreatermeaningprincipalcomponentsanalysis


SpectralDecomposition

• Usingtheeigenvaluesandeigenvectors,wecanreconstructtheoriginalmatrixusingaspectraldecomposition:

𝚺 =%𝜆'𝐞'𝐞'/(

')*• Forourexample,wecangetbacktoouroriginalmatrix:

𝐑* = 𝜆*𝐞*𝐞*/ = 1.78 .71.71 .71 .71 = .89 .89

.89 .89

𝐑9 = 𝐑* + 𝜆9𝐞9𝐞9/

= .89 .89.89 .89 + 0.22 .71

−.71 .71 −.71 = 1.00 0.780.78 1.00


SpectralDecomposition inR


AdditionalEigenvalueProperties

• Thematrixtraceisthesumoftheeigenvalues:

𝑡𝑟𝚺 =%𝜆'

(

')*Ø Inourexample, the𝑡𝑟𝐑 = 1.78 + .22 = 2

• Thematrixdeterminantcanbefoundbytheproductoftheeigenvalues

𝚺 =B𝜆'

(

')*Ø Inourexample 𝐑 = 1.78 ∗ .22 = .3916


ANINTRODUCTIONTOPRINCIPALCOMPONENTSANALYSIS


PCAOverview

• PrincipalComponentsAnalysis(PCA)isamethodforre-expressingthecovariance(oroftencorrelation)betweenasetofvariables

Ø There-expression comesfromcreatingasetofnewvariables(linearcombinations) oftheoriginalvariables

• PCAhastwoobjectives:1. Datareduction

w Movingfrommanyoriginalvariablesdowntoafew“components”

2. Interpretationw Determiningwhichoriginalvariablescontributemosttothenew“components”


GoalsofPCA

• ThegoalofPCAistofindasetofkprincipalcomponents(compositevariables)that:

Ø Ismuchsmaller innumberthantheoriginalsetofV variablesØ Accountsfornearlyallofthetotalvariance

w Totalvariance=traceofcovariance/correlationmatrix

• Ifthesetwogoalscanbeaccomplished,thenthesetofk principalcomponentscontainsalmostasmuchinformationastheoriginalV variables

Ø Meaning – thecomponents cannowreplacetheoriginalvariables inanysubsequentanalyses


QuestionswhenusingPCA

• PCAanalysesproceedbyseekingtheanswerstotwoquestions:

1. Howmanycomponents(newvariables)areneededto“adequately”representtheoriginaldata?Ø Thetermadequately isfuzzy(andwillbeintheanalysis)

2. (once#1hasbeenanswered):Whatdoeseachcomponentrepresent?Ø Theterm“represent” isalsofuzzy


PCAFeatures

• PCAoftenrevealsrelationshipsbetweenvariablesthatwerenotpreviouslysuspected

Ø Newinterpretations ofdataandvariables oftenstemfromPCA

• PCAusuallyservesasmoreofameanstoanendratherthananendititself

Ø Components (thenewvariables) areoftenusedinotherstatistical techniques

w Multiple regression/ANOVAw Clusteranalysis

• Unfortunately,PCAisoftenintermixedwithExploratoryFactorAnalysis

Ø Don’t.Pleasedon’t.Pleasemakeitstop.


PCADetails

• Notation:𝑍 areournewcomponentsand𝐘 isouroriginaldatamatrix(withN observationsandV variables)

Ø Wewillletp beourindexforasubject

• Thenewcomponentsarelinearcombinations:𝑍(* = 𝐞*/𝐘 = 𝑒**𝑌(* + 𝑒9*𝑌(9 + ⋯+ 𝑒K*𝑌(K𝑍(9 = 𝐞9/𝐘 = 𝑒*9𝑌(* + 𝑒99𝑌(9 + ⋯+ 𝑒K9𝑌(K

⋮𝑍(K = 𝐞K/𝐘 = 𝑒*K𝑌(* + 𝑒9K𝑌(9 + ⋯+ 𝑒KK𝑌(K

• Theweightsofthecomponents(𝑒NO) comefromtheeigenvectorsofthecovarianceorcorrelationmatrixforcomponent𝑘 andvariable𝑗


DetailsAbouttheComponents

• Thecomponents(𝑍) areformedbytheweightsoftheeigenvectorsofthecovarianceorcorrelationmatrixoftheoriginaldata

Ø Thevariance ofacomponent isgivenbytheeigenvalue associatedwiththeeigenvector forthecomponent

• Usingtheeigenvalueandeigenvectorsmeans:Ø Eachsuccessive componenthaslowervariance

w Var(Z1)>Var(Z2)>…>Var(Zv)Ø Allcomponents areuncorrelatedØ Thesumofthevariances oftheprincipalcomponents isequaltothetotalvariance:

%𝑉𝑎𝑟 𝑍T = 𝑡𝑟𝚺 = % 𝜆T

K

T)*

K

T)*


PCAonourExample

• WewillnowconductaPCAonthecorrelationmatrixofoursampledata

Ø Thisexample isgivenfordemonstration purposes – typicallywewillnotdoPCAonsmallnumbersofvariables


PCAinR

• TheRfunctionthatdoesprincipalcomponentsiscalledprcomp()


GraphicalRepresentation

• PlottingthecomponentsandtheoriginaldatasidebysiderevealsthenatureofPCA:

Ø ShownfromPCAofcovariancematrix


TheGrowthofGamblingAccess

• Inpast25years:Ø Anexponential increase intheaccessibilityofgambling

Ø Anincreased rateofwithproblemorpathologicalgambling(Volberg,2002,Welte etal.,2009)

• Hence,thereisaneedtobetter:Ø Understand theunderlyingcausesofthedisorderØ ReliablyidentifypotentialpathologicalgamblersØ Provideeffective treatment interventions


PathologicalGambling:DSMDefinition

• Tobediagnosedasapathologicalgambler,anindividualmustmeet5of10definedcriteria:

1. Ispreoccupiedwithgambling2. Needs togamblewithincreasing

amountsofmoneyinordertoachieve thedesiredexcitement

3. Hasrepeated unsuccessful efforts tocontrol,cutback,orstopgambling

4. Isrestlessorirritablewhenattemptingtocutdownorstopgambling

5. Gamblesasawayofescapingfromproblemsorrelievingadysphoricmood

6. After losingmoneygambling,oftenreturnsanotherdaytogeteven

7. Liestofamilymembers, therapist, orothers toconcealtheextentofinvolvementwithgambling

8. Hascommittedillegalactssuchasforgery, fraud,theft,orembezzlement tofinancegambling

9. Hasjeopardizedorlostasignificantrelationship, job,educational,orcareeropportunitybecauseofgambling

10. Reliesonothers toprovidemoneytorelieveadesperate financialsituationcausedbygambling


ResearchonPathologicalGambling

• Inordertostudytheetiologyofpathologicalgambling,morevariabilityinresponseswasneeded

• TheGamblingResearchInstrument(Feasel,Henson,&Jones,2002)wascreatedwith41Likert-typeitems

Ø Itemsweredevelopedtomeasureeachcriterion

• Exampleitems(ratings:StronglyDisagreetoStronglyAgree):Ø IworrythatIamspendingtoomuchmoneyongambling (C3)Ø TherearefewthingsIwouldratherdothangamble (C1)

• TheinstrumentwasusedonasampleofexperiencedgamblersfromariverboatcasinoinaFlatMidwesternState

Ø Casinopatronsweresolicitedafterplayingroulette


TheGRIItems

• TheGRIuseda6-pointLikert scaleØ 1:StronglyDisagreeØ 2:DisagreeØ 3:SlightlyDisagreeØ 4:SlightlyAgreeØ 5:AgreeØ 6:StronglyAgree

• Tomeettheassumptionsoffactoranalysis,wewilltreattheseresponsesasbeingcontinuous

Ø Thisistenuousatbest,butoftenisthecase infactoranalysisØ Categorical itemswouldbebetter….but you’dneedanothercourse forhowtodothat

w Hint: ItemResponse Models


TheSample

• Datawerecollectedfromtwosources:Ø 112“experienced” gamblers

w ManyfromanactualcasinoØ 1192college students froma“rectangular”midwestern state

w Manynevergambledbefore

• Today,wewillcombinebothsamplesandtreatthemashomogenous– onesampleof1304subjects


Final10ItemsontheScale

Item Criterion Question

GRI1 3 Iwouldliketocutbackonmygambling.

GRI3 6IfIlostalotofmoneygamblingoneday,Iwouldbemorelikelytowanttoplayagainthefollowingday.

GRI5 2Ifinditnecessarytogamblewithlargeramountsofmoney(thanwhenIfirstgambled)forgamblingtobeexciting.

GRI9 4 IfeelrestlesswhenItrytocutdownorstopgambling.

GRI10 1 ItbothersmewhenIhavenomoneytogamble.GRI13 3 Ifinditdifficulttostopgambling.

GRI14 2 IamdrawnmorebythethrillofgamblingthanbythemoneyIcouldwin.

GRI18 9 Myfamily,coworkers,orotherswhoareclosetomedisapproveofmygambling.GRI21 1 Itishardtogetmymindoffgambling.

GRI23 5 Igambletoimprovemymood.


PCAwithGamblingItems

• ToshowhowPCAworkswithalargersetofitems,wewillexaminethe10GRIitems(theonesthatfitaone-factorCFAmodel)

• TODOTHISYOUMUSTIMAGINE:Ø THESEWERETHEONLY10ITEMSYOUHADØ YOUWANTEDTOREDUCETHE10ITEMSINTO1OR2COMPONENTVARIABLES

• CAPITALLETTERSAREUSEDASYOUSHOULDNEVERDOAPCAAFTERRUNNINGACFA– THEYAREFORDIFFERENTPURPOSES!


Question#1:HowManyComponents?

• Toanswerthequestionofhowmanycomponents,twomethodsareused:

Ø Screeplotofeigenvalues (lookingforthe“elbow”)Ø Varianceaccounted for(shouldbe>70%)

• Wewillgowith4components:(varianceaccountedforVAC=75%)

• Varianceaccountedforisforthetotalsamplevariance


PlotstoAnswerHowManyComponents


Question#2:WhatDoesEachComponentRepresent?

• Toanswerquestion#2– welookattheweightsoftheeigenvectors(hereistheunrotated solution)


FinalResult:FourPrincipalComponents

• Usingtheweightsoftheeigenvectors,wecancreatefournewvariables– thefourprincipalcomponents

• EachoftheseisuncorrelatedwitheachotherØ Thevariance ofeachisequaltothecorresponding eigenvalue

• WewouldthenusetheseinsubsequentanalysesEPSY905:PCA,EFA,andCFA 34

PCASummary

• PCAisadatareductiontechniquethatreliesonthemathematicalpropertiesofeigenvaluesandeigenvectors

Ø Usedtocreatenewvariables (smallnumber)outoftheolddata (lotsofvariables)

Ø Thenewvariables areprincipalcomponents (theyarenotfactorscores)

• PCAappearedfirstinthepsychometricliteratureØ Many“factoranalysis”methodsusedvariants ofPCAbefore likelihood-basedstatisticswereavailable

• Currently,PCA(orvariants)methodsarethedefaultoptioninSPSSandSAS(PROCFACTOR)


PotentiallySolvableStatisticalIssuesinPCA• ThetypicalPCAanalysisalsohasafewstatisticalconcerns

Ø SomeofthesecanbesolvedifyouknowwhatyouaredoingØ Thetypicalanalysis(usingprogramdefaults)doesnotsolvethese

• Missingdataisomittedusinglistwise deletion– biasespossibleØ CoulduseMLtoestimatecovariancematrix,butthenwouldhavetoassumemultivariatenormality

Ø CoulduseMItoimputedata

• Thedistributionsofvariablescanbeanything…butvariableswithmuchlargervarianceswilllookliketheycontributemoretoeachcomponent

Ø Couldstandardizevariables– butsomecan’tbestandardizedeasily(thinkgender)

• Thelackofstandarderrorsmakesthecomponentweights(eigenvectorelements)hardtointerpret

Ø Canusearesampling/bootstrapanalysistogetSEs(butnoteasytodo)


My(Unsolvable)IssueswithPCA• MyissueswithPCAinvolvethetwoquestionsinneedofanswersforanyuseofPCA:

1. ThenumberofcomponentsneededisnotbasedonastatisticalhypothesistestandhenceissubjectiveØ VarianceaccountedforisadescriptivemeasureØ Nostatisticaltestforwhetheranadditionalcomponentsignificantlyaccounts

formorevariance

2. TherelativemeaningofeachcomponentisquestionableatbestandhenceissubjectiveØ Typicalpackagesprovidenostandarderrorsforeacheigenvectorweight(can

beobtainedinbootstrapanalyses)Ø Nodefinitiveanswerforcomponentcomposition

• Insum,Ifeelitisveryeasytobemisled(orpurposefullymislead)withPCA


EXPLORATORYFACTORANALYSIS


PrimaryPurposeofEFA

• EFA:“Determinenatureandnumberoflatentvariablesthataccountforobservedvariationandcovariationamongsetofobservedindicators(≈itemsorvariables)”

Ø Inotherwords,whatcauses theseobserved responses?Ø Summarizepatterns ofcorrelation amongindicatorsØ Solutionisanend(i.e., isofinterest) inandofitself

• ComparedwithPCA: “Reducemultipleobservedvariablesintofewercomponentsthatsummarizetheirvariance”

Ø Inotherwords,howcanIabbreviate thissetofvariables?Ø Solutionisusuallyameanstoanend


MethodsforEFA

• Youwillseemanydifferenttypesofmethodsfor“extraction”offactorsinEFA

Ø ManyarePCA-basedØ Mostweredeveloped beforecomputersbecame relevantorlikelihood theorywasdeveloped

• Youcanignoreallofthemandfocusonone:

OnlyUseMaximumLikelihoodforEFA

• ThemaximumlikelihoodmethodofEFAextraction:Ø Usesthesamelog-likelihood asconfirmatoryfactoranalyses/SEM

w Defaultassumption:multivariatenormaldistributionofdataØ Providesconsistent estimates withgoodstatistical properties(assuming youhavealargeenoughsample)

Ø Missing datausingall thedatathatwasobserved(MAR)Ø Isconsistent withmodernstatistical practices


QuestionswhenusingEFA

• EFAsproceedbyseekingtheanswerstotwoquestions:(thesamequestionsposedinPCA;butwithdifferentterms)

1. Howmanylatentfactorsareneededto“adequately”representtheoriginaldata?Ø “Adequately”=doesagivenEFAmodelfitwell?

2. (once#1hasbeenanswered):Whatdoeseachfactorrepresent?Ø Theterm“represent” isfuzzy


TheSyntaxofFactorAnalysis• Factoranalysisworksbyhypothesizingthatasetoflatentfactorshelpstodetermineaperson’sresponsetoasetofvariables

Ø Thiscanbeexplainedbyasystemof simultaneous linearmodelsØ HereY=observed data,p=person, v=variable,F=factorscore(Qfactors)

𝑌(* = 𝜇VW + 𝜆**𝐹(* + 𝜆*9𝐹(9 +⋯+ 𝜆*Y𝐹(Y + 𝑒(*𝑌(9 = 𝜇VZ + 𝜆9*𝐹(* + 𝜆99𝐹(9 +⋯+ 𝜆9Y𝐹(Y + 𝑒(9

⋮𝑌(K = 𝜇V[ + 𝜆K*𝐹(* + 𝜆K9𝐹(9 +⋯+ 𝜆KY𝐹(Y + 𝑒(K

• 𝜇V\ =meanforvariable𝑣• 𝜆T^ =factorloadingforvariablevontofactorf(regressionslope)

Ø FactorsareassumeddistributedMVNwithzeromeanand(forEFA)identitycovariancematrix(uncorrelated factors– tostart)

• 𝑒(T =residualforpersonpandvariablevØ ResidualsareassumeddistributedMVN(acrossitems)withazeromeanandadiagonalcovariancematrix𝚿 containing theunique variances

• Often,thisgetsshortenedintomatrixform:𝐘( = 𝝁a + 𝚲𝐅(/ + 𝐞𝐩


HowMaximumLikelihoodEFAWorks

• MaximumlikelihoodEFAassumesthedatafollowamultivariatenormaldistribution

Ø Thebasisforthelog-likelihood function (samelog-likelihoodwehaveusedineveryanalysistothispoint)

• Thelog-likelihoodfunctiondependsontwosetsofparameters:themeanvectorandthecovariancematrix

Ø Meanvector issaturated (justusestheitemmeans foritemintercepts) – soitisoftennotthoughtofinanalysis

w Denotedas𝝁a = 𝝁d

Ø Covariancematrixiswhatgives“factorstructure”w EFAmodelsprovideastructureforthecovariancematrix


TheEFAModelfortheCovarianceMatrix

• Thecovariancematrixismodeledbasedonhowitwouldlookifasetofhypothetical(latent)factorshadcausedthedata

• Forananalysismeasuring𝐹 factors,eachitemintheEFA:Ø Has1uniquevarianceparameterØ Has𝐹 factorloadings

• Theinitialestimationoffactorloadingsisconductedbasedontheassumptionofuncorrelatedfactors

Ø Assumption isdubiousatbest– yetisthecornerstone oftheanalysis


ModelImpliedCovarianceMatrix

• Thefactormodelimpliedcovariancematrixis𝚺a =𝚲𝚽𝚲/ +𝚿

Ø Where:w 𝚺a =model implied covariancematrixoftheobserveddata(size𝐼 x𝐼)w 𝚲 =matrixoffactorloadings(size𝐼 x𝐹)

– InEFA:all termsin𝚲 areestimatedw 𝚽 =factorcovariancematrix(size𝐹 x𝐹)

– InEFA:𝚽 = 𝐈 (allfactorshavevariancesof1andcovariancesof0)– InCFA:thisisestimated

w 𝚿 =matrixofunique(residual)variances(size𝐼 x𝐼)– InEFA:𝚿 isdiagonalbydefault(noresidual covariances)

• Therefore,theEFAmodel-impliedcovariancematrixis:𝚺a = 𝚲𝚲/ +𝚿


EFAModelIdentifiability

• UndertheMLmethodforEFA,thesamerulesofidentificationapplytoEFAastoPathAnalysis

Ø T-rule:TotalnumberofEFAmodelparameters mustnotexceeduniqueelements insaturated covariancematrixofdata

w Forananalysiswithanumberoffactors𝐹 andasetnumberofitems𝐼 thereare𝐹∗𝐼 + 𝐼 = 𝐼 𝐹 + 1 EFAmodelparameters

w Aswewillsee,theremustbeg gh*9

constraints forthemodel towork

w Therefore,𝐼 𝐹 + 1 − g gh*9

< d dj*9

Ø Local-identification: eachportionofthemodelmustbelocallyidentifiedw Withallfactorloadingsestimated localidentification fails

– Nowayofdifferentiating factorswithoutconstraints


ConstraintstoMakeEFAinMLIdentified

• TheEFAmodelimposesthefollowingconstraint:𝚲/𝚿𝚲 = 𝚫

suchthat𝚫 isadiagonalmatrix

• Thisputsg gh*9

constraintsonthemodel(thatmanyfewerparameterstoestimate)

• Thisconstraintisnotwellknown– andhowitfunctionsishardtodescribe

Ø Fora1-factormodel,theresultsofEFAandCFAwillmatch

• Note:theothermethodsofEFA“extraction”avoidthisconstraintbynotbeingstatisticalmodelsinthefirstplace

Ø PCA-basedroutinesrelyonmatrixpropertiestoresolveidentificationEPSY905:PCA,EFA,andCFA 47

TheNatureoftheConstraintsinEFA

• TheEFAconstraintsprovidesomedetailedassumptionsaboutthenatureofthefactormodelandhowitpertainstothedata

• Forexample,takea2-factormodel(oneconstraint):

%𝜓T9B𝜆T^

Y)9

^)*

K

T)*

= 0

• Inshort,somecombinationsoffactorloadingsanduniquevariances(acrossandwithinitems)cannothappen

Ø Thisgoesagainstmostofourstatistical constraints – whichmustbejustifiableandunderstandable (therefore testable)

Ø Thisconstraint isnottestable inCFAEPSY905:PCA,EFA,andCFA 48

TheLog-LikelihoodFunction

• Giventhemodelparameters,theEFAmodelisestimatedbymaximizingthemultivariatenormallog-likelihood

Ø Forthedata

log 𝐿 = log = 2𝜋 hrK9 𝚺 hr9 exp %−𝒀( − 𝝁V

/𝚺h* 𝒀( − 𝝁V2

r

()*

=

−𝑁𝑉2 log 2𝜋 −

𝑁2 log 𝚺 − %

𝒀( − 𝝁V/𝚺h* 𝒀( − 𝝁V2

r

()*

• UnderEFA,thisbecomes:log 𝐿

= −𝑁𝑉2 log 2𝜋 −

𝑁2 log 𝚲𝚲/ +𝚿

−%𝒀( − 𝝁𝑰

/𝚲𝚲/ +𝚿 h* 𝒀𝒑 − 𝝁𝑰

2

r

()*EPSY905:PCA,EFA,andCFA 49

BenefitsandConsequencesofEFAwithML

• TheparametersoftheEFAmodelunderMLretainthesamebenefitsandconsequencesofanymodel(i.e.,CFA)

Ø Asymptotically(largeN)theyareconsistent, normal,andefficientØ Missingdataare“skipped”inthelikelihood,allowingforincompleteobservations tocontribute (assumedMAR)

• Furthermore,thesametypesofmodelfitindicesareavailableinEFAasareinCFA

• AswithCFA,though,anEFAmodelmustbeacloseapproximationtothesaturatedmodelcovariancematrixiftheparametersaretobebelieved

Ø Thisisamarkeddifference between EFAinMLandEFAwithothermethods–qualityoffitisstatisticallyrigorous


MLEFAWITHBASERFUNCTIONFACTANAL(THEBADWAY)


MLEFAUsingthefactanal()Function• ThebaseRprogramhasthefactanal()functionthatconductsML-basedEFA

Ø Butitisverylimited

• AlthoughthefunctionuseML,youstillcannothavemissingdataintheanalysis

Ø BADR!

• Wewillremovecaseswithanymissingdata(listwisedeletion)andproceed

• WewillalsonotusearotationmethodatfirstastoshowhowdefaultconstraintsinEFAwithMLareridiculous


Step#1:DetermineNumberofFactors• TheEFAfactanal()functionprovidesarudimentarytestformodelfit

• Rememberthesaturatedmodelfrompathanalysis?

Ø Allcovariances estimated

• ThemodelfitteststhesolutionfromEFAvsthesaturatedmodel

Ø EFA1-factormodelshown

• Thegoalistofindamodelthatfitswell


Step#1inR:ModelFitTests

• Onefactor:

• Twofactors:

• Threefactors:

• Fourfactors:


SideNote:DefaultConstraintsinMLEFA

• TheEFAmodelimposesthefollowingconstraint:𝚲/𝚿𝚲 = 𝚫

suchthat𝚫 isadiagonalmatrix• Herearethe𝚫matricesfromeachanalysis:


Step#2:InterpretingtheBestModel

• Asthefour-factorsolutionfitbest,wewillinterpretit• Unrotated solutionoffactorloadings:

What???


FACTORLOADINGROTATIONSINEFA


RotationsofFactorLoadingsinEFA

• Transformationsofthefactorloadingsarepossibleasthematrixoffactorloadingsisonlyuniqueuptoanorthogonaltransformation

Ø Don’tlikethesolution?Rotate!

• Historically,rotationsusethepropertiesofmatrixalgebratoadjustthefactorloadingstomoreinterpretablenumbers

• Modernversionsofrotations/transformationsrelyon“targetfunctions”thatspecifywhata“good”solutionshouldlooklike

Ø Thedetailsofthemodernapproacharelackinginmosttexts


TypesofClassicalRotatedSolutions

• Multipletypesofrotationsexistbuttwobroadcategoriesseemtodominatehowtheyarediscussed:

• Orthogonalrotations:rotationsthatforcethefactorcorrelationtozero(orthogonalfactors).Thenameorthogonalrelatestotheanglebetweenaxesoffactorsolutionsbeing90degrees.Themostprevalentisthevarimax rotation.

• Obliquerotations:rotationsthatallowfornon-zerofactorcorrelations.Thenameorthogonalrelatestotheanglebetweenaxesoffactorsolutionsnotbeing90degrees.Themostprevalentisthepromax rotation.

Ø These rotationsprovide anestimateof“factorcorrelation”EPSY905:PCA,EFA,andCFA 59

HowClassicalOrthogonalRotationWorks

• Classicalorthogonalrotationalgorithmsworkbydefininganewrotatedsetoffactorloadings𝚲∗ asafunctionoftheoriginal(non-rotated)loadings𝚲 andanorthogonalrotationmatrix𝐓

𝚲∗ = 𝚲𝐓where: 𝐓𝐓/ = 𝐓/𝐓 = 𝐈

• Theserotationsdonotalterthefitofthemodelas𝚺a = 𝚲∗𝚲∗/ + 𝚿 = 𝚲𝐓 𝚲𝐓 / + 𝚿 = 𝚲𝐓𝐓/𝚲/ +𝚿= 𝚲𝚲/ +𝚿


ModernVersionsofRotation

• MoststudiesusingEFAusetheclassicalrotationmechanisms,likelyduetoinsufficienttraining

• Modernmethodsforrotationsrelyontheuseofatargetfunctionforhowanoptimalloadingsolutionshouldlook

FromBrowne(2001)


RotationAlgorithms

• Givenatargetfunction,rotationalgorithmsseektofindarotatedsolutionthatsimultaneously:

1. Minimizes thedistancebetween therotatedsolutionandtheoriginalfactor loadings

2. Fitsbest tothetarget function

• Rotationalgorithmsaretypicallyiterative– meaningtheycanfailtoconverge

• RotationsearchestypicallyhavemultipleoptimalvaluesØ Needmanyrestarts


RotatedFactorLoadings:OrthogonalRotationviaVarimax

• TheVarimax rotationbroughtaboutthefollowingloadings

• Arethesebetterforinterpretation?

• Alsonote:nofactorcorrelation


RotatedFactorLoadings:ObliqueRotationviaPromax

• ThePromax rotationbroughtaboutthefollowingloadings:

• Italsobroughtaboutthefollowingfactorcorrelations:


EFAVIACFA


CFAApproachestoEFA

• WecanconductexploratoryanalysisusingaCFAmodelØ Needtoset therightnumberofconstraints foridentificationØ Weset thevalueoffactorloadings forafewitemsonafewofthefactors

w Typicallytozero(myusualthought)w Sometimes toone(Brown,2002)

Ø Wekeep thefactorcovariancematrixasanidentityw Uncorrelated factors(asinEFA)withvariancesofone

• BenefitsofusingCFAforexploratoryanalyses:Ø CFAconstraints remove rotational indeterminacyoffactor loadings– norotatingisneeded (orpossible)

Ø Defines factors withpotentially lessambiguityw Constraints areeasytosee

Ø Forsomesoftware (SASandSPSS),wegetmuchmoremodelfitinformation


EFAwithCFAConstraints

• TodoEFAwithCFA,youmust:Ø Fixfactor loadings(settoeither zeroorone)

w Use“rowechelon” form:w Oneitemhasonlyonefactorloadingestimatedw Oneitemhasonlytwofactorloadingsestimatedw Oneitemhasonlythreefactorloadings estimated

Ø Fixfactorcovariancesw Setall to0

Ø Fixfactorvariancesw Setall to1


EFAViaCFAExample

• Wecanuselavaan todoCFA…hereisthesyntaxfortheonefactormodel

Ø The~=isthekeyà Factornametotheleft,itemsmeasuring ittotheright


One-FactorResultsfromlavaan


LookFamiliar?TheyareIdentical totheOne-Factor EFAfromfactanal()


CFALogic…Applied toEFA

• Becauseourone-factormodelfitwell,wecanstop!

• CFAhasmoreindicesofmodelfit– whichcanmakefindinganappropriatesolutioneasier

• CFAalsogivesyouthestandarderrorsforeachfactorloading,leadingtoaWaldtesttoseeifitisnon-zero

Ø Noneedtousearbitrary .3cutoffØ Smallnote:AlthoughmostEFAroutines (likefactanal)don’tgiveSEstheyarecertainlyattainable underMLtheory

• Althoughweshouldstophere…We’llcontinuewiththetwo- andthree-factorversionstocomparewithEFA


Two-FactorSyntax


CFATwo-FactorResults


CFAThree-FactorSyntax


CONCLUDINGREMARKS


WrappingUp

• Todaywediscussedtheworldofexploratoryfactoranalysisandfoundthefollowing:

Ø PCAiswhatpeople typicallyrunwhentheyareafterEFA

Ø MLEFAisabetteroptiontopick(likelihoodbased)w Constraints employed arehidden!w Rotations canbreakwithoutyourealizingtheydo

Ø MLEFAcanbeshowntobeequaltoCFAforcertainmodels

Ø Overall,CFAisstillyourbestbet


Principal Components Analysis; Exploratory Factor Analysis ...€¦ · The Logic of Exploratory Analyses • Exploratory analyses attempt to discover hidden structure in data with

Documents