Factor Analysis and Structural equation modelling Herman Adèr Previously: Department Clinical Epidemiology and Biostatistics, VU University medical center, Amsterdam Stavanger July 4–13, 2006 Herman Adèr 1
Factor Analysis and Structural equation
modelling
Herman Adèr
Previously:Department Clinical Epidemiology and Biostatistics,
VU University medical center, Amsterdam
Stavanger July 4–13, 2006
Herman Adèr 1
Overview
1 E ←→ C
2 EFA
3 CFA
4 SEM
Exploratory ←→ Confirmatory methods
Exploratory Factor Analysis
Confirmatory Factor Analysis
Structural Equation Modelling
Herman Adèr 2
Overview
1 E ←→ C
2 EFA
3 CFA
4 SEM
Exploratory ←→ Confirmatory methods
Exploratory Factor Analysis
Confirmatory Factor Analysis
Structural Equation Modelling
Herman Adèr 2
Overview
1 E ←→ C
2 EFA
3 CFA
4 SEM
Exploratory ←→ Confirmatory methods
Exploratory Factor Analysis
Confirmatory Factor Analysis
Structural Equation Modelling
Herman Adèr 2
Overview
1 E ←→ C
2 EFA
3 CFA
4 SEM
Exploratory ←→ Confirmatory methods
Exploratory Factor Analysis
Confirmatory Factor Analysis
Structural Equation Modelling
Herman Adèr 2
Overview
1 E ←→ C
2 EFA
3 CFA
4 SEM
Exploratory ←→ Confirmatory methods
Exploratory Factor Analysis
Confirmatory Factor Analysis
Structural Equation Modelling
Herman Adèr 2
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Part VI
Factor analysis and Structural equation modelling
Herman Adèr Factor Analysis and SEM 3
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Detective ←→ Judge
Herman Adèr Factor Analysis and SEM 5
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Characterization of Tukey (1977)
Exploratory data analysis is detective in character.
Confirmatory data analysis is judicial or quasi-judicial in
character. . . Unless the detective finds the clues, judge or jury
has nothing to consider. Unless exploratory data analysis
uncovers indications, usually quantitative ones, there is likely to
be nothing for confirmatory data analysis to consider.
On the other hand:
Exploratory data analysis can never be the whole story, but
nothing else can serve as the foundation stone – the first step.
Herman Adèr Factor Analysis and SEM 7
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Characterization of Tukey (1977)
Exploratory data analysis is detective in character.
Confirmatory data analysis is judicial or quasi-judicial in
character. . . Unless the detective finds the clues, judge or jury
has nothing to consider. Unless exploratory data analysis
uncovers indications, usually quantitative ones, there is likely to
be nothing for confirmatory data analysis to consider.
On the other hand:
Exploratory data analysis can never be the whole story, but
nothing else can serve as the foundation stone – the first step.
Herman Adèr Factor Analysis and SEM 7
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
F = M∪U ∪ E ,
of which M indicate the meaningful factors, U so-called
‘unique’ factors (factors on which only one item loads) and E
error factors.
Questions we try to settle using EFA
1 How many meaningful dimensions are present?
2 What is the structure of those dimensions?
Herman Adèr Factor Analysis and SEM 8
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
F = M∪U ∪ E ,
of which M indicate the meaningful factors, U so-called
‘unique’ factors (factors on which only one item loads) and E
error factors.
Questions we try to settle using EFA
1 How many meaningful dimensions are present?
2 What is the structure of those dimensions?
Herman Adèr Factor Analysis and SEM 8
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
�
�
determinemeaningful factors
�� ��rotate
��
��determine factor meaning
�� ��unrotated
��
��Exploratory Factor Analysis
Items
k meaningful factors
final factor solution
factor structure
factor structure
Subject
Data set:
D =
{ }
?
?
?
Three steps
1 Determine the meaningful factors
2 Rotate
3 Interpret the factor structure
Herman Adèr Factor Analysis and SEM 9
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
determine meaningful factors:��
��scree plot
��
��total variance explained more than 50%�
�
�
�
select factors withintelligible
loading patternsand name them�
���communalities ≈ reliabilities
��
��eigenvalues larger than 1
�� ��doubtful�
�
�
GoF changesignificant
between solutions
Herman Adèr Factor Analysis and SEM 10
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
CES-D: A multidimensional scale to assess Depression
FACTOR
/VARIABLES cesd1 cesd2 cesd3 cesd4 cesd5
cesd6 cesd7 cesd8 cesd9 cesd10
cesd11 cesd12 cesd13 cesd14 cesd15
cesd16 cesd17 cesd18 cesd19 cesd20
/MISSING LISTWISE
/ANALYSIS cesd1 cesd2 cesd3 cesd4 cesd5
cesd6 cesd7 cesd8 cesd9 cesd10
cesd11 cesd12 cesd13 cesd14 cesd15
cesd16 cesd17 cesd18 cesd19 cesd20
/PRINT all
/FORMAT BLANK(.30)
/PLOT EIGEN
/CRITERIA MINEIGEN(0) ITERATE(100) factors(3)
/EXTRACTION pc
/ROTATION norotate.
Herman Adèr Factor Analysis and SEM 12
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
CES-D: A multidimensional scale to assess Depression
FACTOR
/VARIABLES cesd1 cesd2 cesd3 cesd4 cesd5
cesd6 cesd7 cesd8 cesd9 cesd10
cesd11 cesd12 cesd13 cesd14 cesd15
cesd16 cesd17 cesd18 cesd19 cesd20
/MISSING LISTWISE
/ANALYSIS cesd1 cesd2 cesd3 cesd4 cesd5
cesd6 cesd7 cesd8 cesd9 cesd10
cesd11 cesd12 cesd13 cesd14 cesd15
cesd16 cesd17 cesd18 cesd19 cesd20
/PRINT all
/FORMAT BLANK(.30)
/PLOT EIGEN
/CRITERIA MINEIGEN(0) ITERATE(100) factors(3)
/EXTRACTION pc
/ROTATION norotate.
Herman Adèr Factor Analysis and SEM 12
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Scree Plot
Component Number
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
Eig
envalu
e
10
8
6
4
2
0
Herman Adèr Factor Analysis and SEM 13
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
8.516 42.578 42.578
1.775 8.876 51.454
1.229 6.147 57.600
1.040 5.201 62.802
.944 4.719 67.520
.767 3.836 71.356
.738 3.689 75.045
.684 3.419 78.465
.606 3.030 81.495
.577 2.886 84.381
.508 2.542 86.923
.442 2.211 89.134
.426 2.129 91.263
.339 1.697 92.960
.318 1.592 94.552
.286 1.429 95.981
.274 1.371 97.352
.202 1.010 98.362
.178 .891 99.253
.149 .747 100.000
Component1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Total % of Variance Cumulative %
Initial Eigenvalues
Extraction Method: Principal Component Analysis.
Herman Adèr Factor Analysis and SEM 14
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Interpretation of the unrotated factor structure
.525 .372 -.373
.757 -.318
.802 .389
-.573 .538
.687 .336
.689
.726 .349
-.635 .531
.721 -.353
.672 .398
.623 .415
-.690 .489
.599 -.513
.710
.432 .490 -.627
-.758 .383
.657 .500
.763 .349
.482 -.447 -.376
.784 .359
CESD1
CESD2
CESD3
CESD4
CESD5
CESD6
CESD7
CESD8
CESD9
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
1 2 3 4 5 6
Component
Extraction Method: Principal Component Analysis.
Herman Adèr Factor Analysis and SEM 15
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Communalities: four factors and all factors
Communalities
1.000 .418
1.000 .751
1.000 .798
1.000 .683
1.000 .492
1.000 .526
1.000 .533
1.000 .763
1.000 .695
1.000 .543
1.000 .597
1.000 .734
1.000 .655
1.000 .546
1.000 .442
1.000 .721
1.000 .699
1.000 .709
1.000 .631
1.000 .624
CESD1
CESD2
CESD3
CESD4
CESD5
CESD6
CESD7
CESD8
CESD9
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Initial Extraction
Extraction Method: Principal Component Analysis.
Communalities
1.000 .966
1.000 .970
1.000 .868
1.000 .817
1.000 .919
1.000 .912
1.000 .830
1.000 .834
1.000 .812
1.000 .883
1.000 .896
1.000 .841
1.000 .861
1.000 .829
1.000 .961
1.000 .829
1.000 .828
1.000 .793
1.000 .963
1.000 .773
CESD1
CESD2
CESD3
CESD4
CESD5
CESD6
CESD7
CESD8
CESD9
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Initial Extraction
Extraction Method: Principal Component Analysis.
Herman Adèr Factor Analysis and SEM 16
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Communalities: two and three factors
Communalities
1.000 .414
1.000 7.631E-02
1.000 .795
1.000 .617
1.000 .481
1.000 .489
1.000 .531
1.000 .686
1.000 .544
1.000 .539
1.000 .399
1.000 .716
1.000 .389
1.000 .540
1.000 .427
1.000 .721
1.000 .434
1.000 .587
1.000 .291
1.000 .615
CESD1
CESD2
CESD3
CESD4
CESD5
CESD6
CESD7
CESD8
CESD9
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Initial Extraction
Extraction Method: Principal Component Analysis.
Communalities
1.000 .418
1.000 .650
1.000 .796
1.000 .639
1.000 .491
1.000 .517
1.000 .532
1.000 .720
1.000 .669
1.000 .541
1.000 .571
1.000 .733
1.000 .392
1.000 .546
1.000 .437
1.000 .721
1.000 .449
1.000 .587
1.000 .490
1.000 .621
CESD1
CESD2
CESD3
CESD4
CESD5
CESD6
CESD7
CESD8
CESD9
CESD10
CESD11
CESD12
CESD13
CESD14
CESD15
CESD16
CESD17
CESD18
CESD19
CESD20
Initial Extraction
Extraction Method: Principal Component Analysis.
Herman Adèr Factor Analysis and SEM 17
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Goodness-of-fit criterium
# Facts GoF χ2 Df χ
2 change Df χ2
α=.05
1 444.399 1702 278.177 151 166.222 19 30.1443 218.422 133 59.755 18 28.8694 170.845 116 47.577 17 27.5875 126.361 100 44.484 16 26.296
As one can see by comparing the 0.05 threshold χ2 in the last column
Herman Adèr Factor Analysis and SEM 18
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Unrotated Rotated
Factor Factor
Item I II III I II III
CESD01 + + +
CESD02 + +
CESD03 + + +
CESD04 − + −
CESD05 + + +
CESD06 + + +
CESD07 + + +
CESD08 − + −
CESD09 + − + +
CESD10 + +
CESD11 + + + +
CESD12 − + −
CESD13 + +
CESD14 + +
CESD15 + + +
CESD16 − + −
CESD17 + + +
CESD18 + + +
CESD19 + +
CESD20 + + +
Herman Adèr Factor Analysis and SEM 19
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Interpretation of table I
Unrotated Rotated
Factor Factor
Item I II III I II III
CESD01 + + +
CESD02 + +
CESD03 + + +
CESD04 − + −
CESD05 + + +
CESD06 + + +
CESD07 + + +
CESD08 − + −
CESD09 + − + +
CESD10 + +
CESD11 + + + +
CESD12 − + −
CESD13 + +
CESD14 + +
CESD15 + + +
CESD16 − + −
CESD17 + + +
CESD18 + + +
CESD19 + +
CESD20 + + +
All items with high positive loadings on
the unrotated Factor I still load on this
factor after rotation, but some of them
(5, 6, 7, 9, 17, 18 and 20) also load on
factor II. This makes that Factor I is
now separated into two sublists, of
which the above could be called the
pure depression scale while (1, 3, 10,
11, 13, 14, 15, 19) includes a sublist
indicating less severe symptoms of
depression.
Herman Adèr Factor Analysis and SEM 20
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Interpretation of table II
Unrotated Rotated
Factor Factor
Item I II III I II III
CESD01 + + +
CESD02 + +
CESD03 + + +
CESD04 − + −
CESD05 + + +
CESD06 + + +
CESD07 + + +
CESD08 − + −
CESD09 + − + +
CESD10 + +
CESD11 + + + +
CESD12 − + −
CESD13 + +
CESD14 + +
CESD15 + + +
CESD16 − + −
CESD17 + + +
CESD18 + + +
CESD19 + +
CESD20 + + +
Factor II now also contains all the
items that loaded negatively on Factor
I before (4, 8, 12 and 16). We called
this a factor that measures:
general attitude towards life.
Herman Adèr Factor Analysis and SEM 21
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Questions and answers
Three steps
Determining the meaningful factors
Rotation
Interpretation of the factor structure
Interpretation of table III
Unrotated Rotated
Factor Factor
Item I II III I II III
CESD01 + + +
CESD02 + +
CESD03 + + +
CESD04 − + −
CESD05 + + +
CESD06 + + +
CESD07 + + +
CESD08 − + −
CESD09 + − + +
CESD10 + +
CESD11 + + + +
CESD12 − + −
CESD13 + +
CESD14 + +
CESD15 + + +
CESD16 − + −
CESD17 + + +
CESD18 + + +
CESD19 + +
CESD20 + + +
After rotation, Factor III now contains
only item 2 (‘I did not feel like eating:
my appetite was poor’) and item 11
(‘My sleep was restless’), suggesting
that it is a factor that has to do with
physical aspects of depression.
Herman Adèr Factor Analysis and SEM 22
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example of EQS input
Example of EQS input
Conclusions of EFA and CFA combined
Example of EQS input
/TITLE Cesd;
/SPECIFICATIONS
VARIABLES=6; CASES= 200;
DATAFILE=’cesd.ess’; MATRIX=RAW;
ME = ML;
/EQUATIONS
V1 = 2 * F1 + E1;
V2 = 2 * F1 + E2;
V3 = 2 * F2 + E3;
V4 = 2 * F2 + E4;
V5 = 2 * F3 + E5;
V6 = 2 * F3 + E6;
F1 = 2 * F2 + 2*F3+ D1;
/VARIANCES
E1 to E10 = 0.2*;
D1, D2, D3 = 0.2*;
/COVARIANCES
F2,F3 = .5*;
/END
Herman Adèr Factor Analysis and SEM 23
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example of EQS input
Example of EQS input
Conclusions of EFA and CFA combined
CES-D: Confirmatory factor analysis
Conclusions of the CFA
The varimax solution is not fully confirmed: item 5, 6, 7, 18
and 20 have negligible coefficients on the second factor.
The assumption of a orthogonal factor structure is
unfounded: The correlations between the factors are
(F1, F2) = 0.241; (F1, F3) = 0.382 and (F2, F3) = −0.025.
Herman Adèr Factor Analysis and SEM 24
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example of EQS input
Example of EQS input
Conclusions of EFA and CFA combined
CES-D: Confirmatory factor analysis
Conclusions of the CFA
The varimax solution is not fully confirmed: item 5, 6, 7, 18
and 20 have negligible coefficients on the second factor.
The assumption of a orthogonal factor structure is
unfounded: The correlations between the factors are
(F1, F2) = 0.241; (F1, F3) = 0.382 and (F2, F3) = −0.025.
Herman Adèr Factor Analysis and SEM 24
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example of EQS input
Example of EQS input
Conclusions of EFA and CFA combined
Conclusions of EFA and CFA combined
1 The first factor is a general depression factor (item 1, 3, 5,
6, 7, 10, 13, 14, 15, 18, 19 and 20)
2 The second factor contains the positively formulated items
(4, 8, 9, 12, 16, 17). It represents a general attitude
towards life.
3 The third factor contains items that have to do with
physical aspects of depression.
Herman Adèr Factor Analysis and SEM 25
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example: Post-traumatic stress disorder
Example of a postulated structure
Draw pictures!
Herman Adèr Factor Analysis and SEM 26
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example: Post-traumatic stress disorder
Example of a postulated structure
Draw pictures!
Herman Adèr Factor Analysis and SEM 26
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Example: Post-traumatic stress disorder
Example of a postulated structure
Draw pictures!
Herman Adèr Factor Analysis and SEM 26
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Postulated structure
Structure found
Herman Adèr Factor Analysis and SEM 27
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Recommended literature
Factor analysis: Friendly (1995)
(http://www.psych.yorku.ca/lab/psy6140/fa/factorbi.htm)
Principal component analysis: Jackson (1991)
Structural equation modelling: Bollen (1989), Kaplan (2000)
Herman Adèr Factor Analysis and SEM 28
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Summary
Herman Adèr Factor Analysis and SEM 29
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Summary
In factor analysis the contrast between exploratory and
confirmatory approaches is quite clear: the available
techniques are different although related.
Exploratory factor analysis can well be performed with
principal component analysis, possibly combined with
Maximum likelihood factor analysis. PCA is first used to
determine the meaningful factors, MLFA for the final factor
structure.
Structural Equation Modelling makes it possible to analyze
a research problem that has been represented as a
diagram.
Herman Adèr Factor Analysis and SEM 30
Exploratory and confirmatory methods
Exploratory factor analysis
Confirmatory factor analysis
Structural equation modelling
References
Bollen, K. A. (1989). Structural equations with latent variables. New
York: John Wiley and Sons.
Friendly, M. (1995). Annotated Factor Analysis Bibliography.
Jackson, J. E. (1991). A user’s guide to principal components. New
York: Wiley.
Kaplan, D. (2000). Structural Equation Modeling. Foundations and
Extensions. Thousand Oaks London New Delhi: Sage
Publications.
Tukey, J. W. (1977). Exploratory data analysis. Reading, MA:
Addison Wesley.
Herman Adèr Factor Analysis and SEM 31