Kai Kristensen, IMPS´03 PLS structural Equation Modeling for Customer Satisfaction -Methodological and Application Issues- Kai Kristensen, J. Eskildsen, H.J. Juhl, P. Østergaard Centre for Corporate Performance The Aarhus School of Business, Denmark
Kai Kristensen, IMPS´03
PLS structural Equation Modeling for Customer Satisfaction
-Methodological and Application Issues-
Kai Kristensen, J. Eskildsen, H.J. Juhl, P. ØstergaardCentre for Corporate PerformanceThe Aarhus School of Business, Denmark
2
Kai Kristensen, IMPS´03
Agenda
• The EPSI Rating Model• Latent Structure• Manifests
• A few recent results• The Danish car market• External validity
• Practical problems and observations• The choice of scale• Reliability:The choice of manifests• Explanatory power• Missing values• Multicollinearity
• Some results from a simulation study
3
Kai Kristensen, IMPS´03
The EPSI Rating Model: Latent structure
Perceived Quality“Software”
Perceived Quality“Hardware”
Expectations
Image
LoyaltyPerceived Value Customer Satisfaction
4
Kai Kristensen, IMPS´03
EPSI Rating model
• Generic model with 7 latent constructs· 4 latent exogenous constructs (Image,
expectation, quality of ”hardware” and ”software)
· 3 endogenous constructs (perception of value, satisfaction and loyalty)
• Each construct is determined by 3-6 manifest measurements.
• The model is estimated by use of PLS (Partial Least Squares estimation techniques.
5
Kai Kristensen, IMPS´03
Examples of manifest measurements
• Image: General perception of company image with regard to:
• Reliability• Being customer
focussed• Giving value for money• Innovation in products
and services• Overall image
• Satisfaction:• Overall satisfaction• Comparison to ideal• Disconfirmation
• Loyalty:• The customer's intention to
repurchase, • Intention of cross buying
(buy another product from the same company),
• Intention to recommend the brand/company to other consumers.
6
Kai Kristensen, IMPS´03
Example from the Danish car industry
74
88
77 77
7275 7474
84
77 77
7274 73
50
55
60
65
70
75
80
85
90IM
AG
E
EX
PEC
TA
TIO
NS
QU
AL
ITY
OF
"HA
RD
WA
RE
"
QU
AL
ITY
OF
"SO
FTW
AR
E"
VA
LU
E
SAT
ISFA
CT
ION
LO
YA
LT
Y
Inde
x
2001 2002
7
Kai Kristensen, IMPS´03
Individual brands
7473
75
81
74
70
75
68
76
67
70
73
79
72
68
77
71
76
50
55
60
65
70
75
80
85
Peugeot VW Ford Toyota Opel Citroen Mazda Fiat Other
Inde
x
SATISFACTION 2001 SATISFACTION 2002
8
Kai Kristensen, IMPS´03
Inner coefficients for the 2002 model
UNSTANDARDISED INNER COEFFICIENTS
IMAGEEXPECTA-
TIONSQUALITY OF
"HARDWARE"
QUALITY OF "HUMAN WARE" VALUE
SATISFAC-TION LOYALTY
IMAGEEXPECTATIONS
QUALITY OF "HARDWARE"QUALITY OF "HUMAN WARE"
VALUE 0,36 -0,06 0,35 0,32 SATISFACTION 0,44 -0,03 0,23LOYALTY 0,29 -0,11 0,24 0,27 0,53
T-VALUES FOR INNER COEFFICIENTS
IMAGEEXPECTA-
TIONSQUALITY OF
"HARDWARE"
QUALITY OF "HUMAN WARE" VALUE
SATISFAC-TION LOYALTY
IMAGEEXPECTATIONS
QUALITY OF "HARDWARE"QUALITY OF "HUMAN WARE"
VALUE 14,72 -3,38 11,30 11,21 SATISFACTION 20,93 -1,80 9,06 10,82 5,54 LOYALTY 7,10 -4,08 5,22 6,10 13,84
9
Kai Kristensen, IMPS´03
The impact of drivers on satisfaction and loyalty 2001 & 2002
0,53
0,01
0,31
0,22
0,47
-0,03
0,26
0,29
0,57
0,01
0,33
0,33
0,54
-0,13
0,38
0,42
-0,2 -0,1 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7
Image
Expectations
Quality of "hardware"
Quality of "software"
Dri
vers
Impact
Satisfaction 2001 Satisfaction 2002 Loyalty 2001 Loyalty 2002
10
Kai Kristensen, IMPS´03
External validity: Relation to actual service performance
Other
VW
Toyota
Peugeot
Opel
Mazda
FordFiat
Citroen
y = -36,036x + 103,99R2 = 0,4554
70
72
74
76
78
80
82
84
60,0% 65,0% 70,0% 75,0% 80,0% 85,0% 90,0%
Percentage cars with defects
Inde
x on
"H
uman
war
e" in
200
2
11
Kai Kristensen, IMPS´03
External validity: Relationship between satisfaction and complaints
OtherFiat
Mazda
Citroen
Opel
Toyota
Ford
VW
Peugeot
y = 6E+17x-11,008
R2 = 0,643
0
0,0005
0,001
0,0015
0,002
0,0025
0,003
0,0035
0,004
65 67 69 71 73 75 77 79 81 83 85
Average satisfaction 2001 & 2002
Com
plai
nts:
Pro
port
ion
of p
opul
atio
n m
entio
ned
on se
lfrep
ortin
g ho
mep
age
Kai Kristensen, IMPS´03
Practical problems and observations
The Choice of Scale
13
Kai Kristensen, IMPS´03
The experiment
• In order to test the effect of scale choice on the results of customer satisfaction studies a controlled experiment was set up.
• Under totally identical conditions two samples were drawn from the population. The only difference between the samples was that in the first sample a 5-point scale was used and in the second a 10-point scale was used.
• The questionnaires were the standard customer satisfaction questionnaires used for a given company.
• The size of the samples was 545 for the 10-point scale and 563 for the 5-point scale.
14
Kai Kristensen, IMPS´03
Mean value of latent variables
Ten points Five pointsMean Mean
Expectations 73,3 75,1 0,13Products 64,2 64,3 0,88Service 66,9 66,4 0,70Value 54,4 54,4 0,96Satisfaction 65,2 65,2 0,97Loyalty 57,5 58,7 0,36Image 63,6 64,0 0,74
Data source
Significance, two sidedVariable
15
Kai Kristensen, IMPS´03
Conclusion: Mean values
•There is no significant difference between the mean values of the aggregate variables.
•This means that the choice of scale has no influence on the level of the customer satisfaction index or the loyalty index.
16
Kai Kristensen, IMPS´03
Standard deviation of aggregate variables
Ten points Five pointsStd Deviation Std Deviation
Expectations 19,2 20,1 0,476Products 19,1 20,5 0,274Service 21,2 23,4 0,014Value 19,7 22,4 0,005Satisfaction 19,3 21,5 0,013Loyalty 21,7 23,6 0,054Image 18,1 19,5 0,069
Data source
Variable Significance
17
Kai Kristensen, IMPS´03
Conclusion: Latent variable standard deviations
•As expected the standard deviation of the 10-point scale is smaller than the standard deviation of the 5-point scale with Image, Expectations and Products as possible exceptions.
•The difference is on the average app. 10%.•The reason for this difference is, that the underlying distributions are discrete.
18
Kai Kristensen, IMPS´03
5- and 10-point scales
Shape of the distribution
19
Kai Kristensen, IMPS´03
Satisfaction: Distribution 10 point scale
0 20 40 60 80 100Satisfaction
Mean: 65.2Std. dev.: 19.2
0.0280.072
0.250
0.439
0.213
20
Kai Kristensen, IMPS´03
Comparison of observed and theoretical distributions. (10-point scale)
0
0,1
0,2
0,3
0,4
0,5
-20 20-40 40-60 60-80 80-100
ObservedNormalBeta 10
21
Kai Kristensen, IMPS´03
Satisfaction: Distribution 5 point scale
0 20 40 60 80 100Satisfaction
0.0360.075
0.281
0.369
0.240Mean: 65.2Std. dev.: 21.4
22
Kai Kristensen, IMPS´03
Comparison of observed and theoretical distributions. (5-point scale)
0
0,1
0,2
0,3
0,4
-20 20-40 40-60 60-80 80-100
ObservedNormalBeta 5
23
Kai Kristensen, IMPS´03
A comparison of satisfaction distributions
05
1015202530354045
%
0-20 20-40 40-60 60-80 80-100
10-point5-point
24
Kai Kristensen, IMPS´03
Satisfaction: A general comparison of the distribution of 5- and 10-point scales
Kolmogorov-Smirnov Z Asymp. Sig. (2-tailed)
Image 1,08 0,19Expectations 2,53 0,00Products 1,41 0,04Service 1,63 0,01Value 1,95 0,00Satisfaction 1,77 0,00Loyalty 1,53 0,02
25
Kai Kristensen, IMPS´03
Conclusion
•In general the standardized distributions are not identical with Image as a possible exception. This is to be expected due to the discrete underlying distributions.
•The beta distribution or the doubly truncated normal distribution seem to give the closest approximation to the distribution but even here we have a significant difference in both cases.
26
Kai Kristensen, IMPS´03
5- and 10- point scales
Are demographics and scale interacting?
27
Kai Kristensen, IMPS´03
Variables and factors for the analysis of variance
•Dependent variables:•All aggregate variables
•Explanatory variables•Data Source (5-point, 10-point)•Age (-25, 26-35, 36-45, 46-55, 56-65. 66-)•Education ( 8 groups from high school to university)
•Gender (Male, Female)•Location (Copenhagen, Sealand, Funen, Jutland)
28
Kai Kristensen, IMPS´03
Analysis of variance (10% significance)
Image Expectation Product Service Value Satisfaction Loyalty
Main AgeLoca-tionEduca-tionGender
Location Location AgeLoca-tionEduca-tionGender
NONE AgeLocationEducation
Loca-tionEduca-tion
Two-wayinter-action
NONE Age x source
NONE NONE Age x source
NONE NONE
29
Kai Kristensen, IMPS´03
Conclusion
•In the case of Image, Product, Service, Satisfaction and Loyalty there is no effect from the data source.
•Only in the case of Expectation and Value we can trace an effect. In these cases there is a tendency that the age groups are using the scales differently.
•Based on this our general conclusion is, that the demographic interpretation of customer satisfaction studies will not be seriously affected by the choice of scale.
•When it comes to satisfaction there seems to be a universal main effect of Age, Location and Education.• Satisfaction is increasing with age and decreasing with
education.• Satisfaction is decreasing with the degree of urbanization.
30
Kai Kristensen, IMPS´03
Conclusions concerning scales
•In general terms a 10-point scale is preferable to a five point scale:•Smaller variance.•Closer approximation to a continuous variable.•10-point scales are used by all the major national customer satisfaction studies.
•In general it is possible to compare studies using 5 and 10-point scales since the mean values (on a 100-point scale) are not affected.
•Demographics have a small but not very important effect on the results from the scales.
Kai Kristensen, IMPS´03
Practical problems and observations
Reliability and prediction
32
Kai Kristensen, IMPS´03
Internal reliability and validity of the results: The car example
IMAGEEXPECTA-
TIONS
QUALITY OF
"HARDWARE"
QUALITY OF
"HUMAN WARE" VALUE
SATISFAC-TION LOYALTY
R-SQUARE FOR LATENT VARIABLES 0,56 0,76 0,55COMPOSITE RELIABILITY 0,92 0,95 0,89 0,86 0,97 0,90 0,91AVERAGE VARIANCE EXPLAINED BY LATENT VARIABLES 0,69 0,86 0,73 0,68 0,91 0,75 0,84
R-SQUARE FOR LATENT VARIABLES 0,62 0,72 0,56COMPOSITE RELIABILITY 0,92 0,94 0,91 0,90 0,97 0,90 0,91AVERAGE VARIANCE EXPLAINED BY LATENT VARIABLES 0,71 0,84 0,77 0,75 0,91 0,74 0,84
2001
2002
( )=
=
λ=
λ + Θ
∑
∑
p2i
i 1p
2i i
i 1
AVE =
= =
⎛ ⎞λ⎜ ⎟
⎝ ⎠ρ =⎛ ⎞
λ + Θ⎜ ⎟⎝ ⎠
∑
∑ ∑
2p
ii 1
c 2p p
i ii 1 i 1
33
Kai Kristensen, IMPS´03
Reliability and choice of manifests
• Automobiles• Reasonable reliability: No reason for changes.
• Petrol stations• High reliability: No reason for changes.
• Banks:• In the satisfaction construct the “comparison to ideal” may cause a
problem. Much lower level than the two other questions.• Supermarkets
• In the satisfaction construct the “comparison to ideal” may cause a problem. Much lower level than the two other questions.
• The value for money indicator and the assortment indicator may cause a problem since they reflect the type of supermarket.
• The question about opening hours which is classified as belonging to the service block should possibly be re-classified
34
Kai Kristensen, IMPS´03
Reliability and choice of manifests: Conclusions
•For most of the areas covered by the Danish Customer Satisfaction Index the manifest questions are working well.
•The only area where we have observed a necessity for changes is Supermarkets.
•Other conclusions may apply when we have discussed the problem of missing values.
35
Kai Kristensen, IMPS´03
Explanatory power
•The general observation is, that the explanatory power of the model is rather good.
•There is no problem in obtaining an R2 beyond .65 for the satisfaction construct as required by the ECSI Technical Committee. In general R2 is somewhere between .70 and .80.
•The degree of explanation for value and loyalty is usually a little lower.
Kai Kristensen, IMPS´03
Practical problems and observations
Missing values and multicollinearity
37
Kai Kristensen, IMPS´03
Missing values
Supermarkets Banks Automobiles Petrol Stations
•Below 10% for all items
•Relative comparisons are problematic. 40-50% missing values
•19 out of 22 items have missing values below 5%
•13 out of 22 have missing values below 10%.•8 have missing values between 10% and 20%
38
Kai Kristensen, IMPS´03
Multicollinearity (Latent variables)
•In general the degree of multicollinearity is rather high.
•Banks: Correlations between .54 (expectation and service) and .82 (product and service).
•Petrol stations: Correlations between .42 (expectations and service) and .69 (product and service).
•Automobiles: Correlations between .48 (expectations and service) and .85 (product and service).
•Mobile telephones: Correlations between .44 (expectations and service) and .76 (product and service).
•Supermarkets: Correlations between .52 (expectations and image) and .71 (image and product).
Kai Kristensen, IMPS´03
Simulation study
A study of some of the implications of the empirical findings
40
Kai Kristensen, IMPS´03
Background
• To get insight into the consequences of some of the empirical problems based on a true model which is very close to the actualmodels observed. Our model is reflective for all latent variables.
• To formulate some simple rules of thumb.• To supplement and verify the simulation study conducted by
Cassel, Hackl and Westlund (1999, 2000). These authors investigated the effect of the following factors on the estimation of an EPSI like model with formative exogenous and reflective endogenous latent variables:
• Skewness of manifest variables• Multicollinearity between latent variables• Misspecification (omission of relevant regressors or regressands, or
manifests within a measurement model)• Sample size• Size of the path coefficients
41
Kai Kristensen, IMPS´03
Simulation setup
• STAGE 1(Screening): Orthogonal main effect plan with 7 factors in 27 runs with 25 replications for each run. Each replication has a number of observations varying between 50 and 1000.
• Exogenous distribution (Beta vs. Normal)• Multicollinearity between latent exogenous variables• Indicator validity (bias)• Indicator reliability (standard deviation within a block)• Structural model specification error• Sample size• Number of indicators in each block
• STAGE 2: Full factorial design with 4 factors in 54 runs with 25replications for each run
• Multicollinearity, reliability, sample size and number of indicators
42
Kai Kristensen, IMPS´03
Stage 2 factor levels and response variables
•FACTOR LEVELS:•Multicollinearity: ρ={0.2;0.8}.•Reliability: σ={1; 10; 20}.•Sample size: n={50; 250; 1000}.•Number of indicators: p={2; 4; 6}.
•RESPONSE VARIABLES:•Absolute bias for indices•Standard deviation for indices•Bias for path coefficients•R2, AVE and RMSE.
43
Kai Kristensen, IMPS´03
The simulation model (A simplified customer satisfaction model)
4G3G
1G
2G
β =12 .50
γ =11 .50
γ =21 .25
γ =22 .75
1y
2yy21λ
y11λ
x
2x
3x
4x
x11λ
x21λ
x42λ
3y 4y
y32λ y
42λx32λ
1ζ
2ζ
y1ε
y2ε
y3ε
y4ε
x1ε
x2ε
x3ε
x4ε
12φ
122.5*Beta(4,3)
44
Kai Kristensen, IMPS´03
Simulation results
F1 F2 F3 F4Multicollinearity Indicator reliability Sample size # indicators F1*F2 F1*F3 F1*F4 F2*F3 F2*F4 F3*F4
g1 ** ** ** ** ** **g2 ** ** ** ** ** **g3 ** ** ** ** ** **g4 ** ** ** ** ** **stdg1 ** ** ** ** **stdg2 ** ** ** ** **stdg3 ** ** ** ** ** **stdg4 ** ** * ** ** **gamma21 ** ** ** ** * ** **gamma22 ** ** ** ** **gamma11 ** ** ** ** ** **beta12 ** ** ** ** ** **stdgam21 ** ** ** ** ** ** ** ** **stdgam22 ** ** ** ** ** ** ** ** **stdgam11 ** ** ** ** ** ** ** ** **stdbet12 ** ** ** ** ** ** ** ** **rsq1 ** ** ** **rsq2 ** ** ** ** **ave1 ** ** **ave2 ** ** *ave3 ** ** * *ave4 ** ** ** *avetot ** ** *rmse ** ** ** ** ** ** ** **
45
Kai Kristensen, IMPS´03
Bias of indices: Multicollinearity and indicator reliability
Multicollinearity
phi=0.8phi=0.2
Mea
n ab
solu
te b
ias
,80
,60
,40
,20
0,00
G1
G2
G3
G4
Indicator reliability
sigma=20sigma=10sigma=1
Mea
n ab
solu
te b
ias
,8
,6
,4
,2
0,0
G1
G2
G3
G4
46
Kai Kristensen, IMPS´03
Bias of indices: Sample size and number of indicators
Sample size
n=1000n=250n=50
Mea
n ab
solu
te b
ias
,8
,6
,4
,2
0,0
G1
G2
G3
G4
Number of indicators
642
Mea
n ab
solu
te b
ias
,8
,6
,4
,2
0,0
G1
G2
G3
G4
47
Kai Kristensen, IMPS´03
Example of mean relative bias for Gamma 21
Sample size
n=1000n=250n=50
Mea
n R
elat
ive
bias
Gam
ma2
1 (%
)
8,0
7,5
7,0
6,5
6,0
5,5
5,0
4,5
4,0
Number of indicators
642
Mea
n R
elat
ive
bias
Gam
ma2
1 (%
)
8,0
7,5
7,0
6,5
6,0
5,5
5,0
4,5
4,0
48
Kai Kristensen, IMPS´03
Example of mean relative bias for Beta 12
Sample size
n=1000n=250n=50
Mea
n R
elat
ive
Bias
Bet
a12
(%)
-4,0
-5,0
-6,0
-7,0
-8,0
-9,0
-10,0
-11,0
Number of indicators
642
Mea
n R
elat
ive
Bias
Bet
a12
(%)
-4
-5
-6
-7
-8
-9
-10
-11
49
Kai Kristensen, IMPS´03
Standard deviation of Gamma 21 as a function of multicollinearity and indicator reliability
Multicollinearity
phi=0.8phi=0.2
Mea
n St
d. d
ev. G
amm
a 21
,060
,040
,020
0,000
Indicator reliability
sigma=20sigma=10sigma=1
Mea
n St
d. d
ev. G
amm
a 21
,06
,04
,02
0,00
50
Kai Kristensen, IMPS´03
Standard deviation of Gamma 21 as a function of sample size and number of indicators
Sample size
n=1000n=250n=50
Mea
n St
d. d
ev. G
amm
a 21
,06
,04
,02
0,00
Number of indicators
642
Mea
n St
d. d
ev. G
amm
a 21
,060
,040
,020
0,000
51
Kai Kristensen, IMPS´03
Degree of explanation.
Indicator reliability
sigma=20sigma=10sigma=1
Mea
n R
SQ
1,0
,9
,8
,7
,6
RSQ1
RSQ2
Number of indicators
642
Mea
n R
SQ
1,00
,90
,80
,70
,60
RSQ1
RSQ2
52
Kai Kristensen, IMPS´03
Average variance extracted.
Number of indicators
642
Mea
n AV
ETO
T
1,00
,90
,80
,70
,60
Indicator reliability
sigma=20sigma=10sigma=1
Mea
n AV
ETO
T
1,0
,9
,8
,7
,6
53
Kai Kristensen, IMPS´03
A couple of rough rules of thumb concerning the absolute bias of the indices
•Let σ be the standard deviation of the manifest variables, n the sample size, and p the number of indicators, then:
· BIAS(kσ,n,p) = k BIAS(σ,n,p).· BIAS(σ,kn,p) = (1/√k) BIAS(σ,n,p).· BIAS(σ,n,kp) = (1/k) BIAS(σ,n,p).
54
Kai Kristensen, IMPS´03
Conclusion to the simulation study
• Basically our results support The Cassel, Hackl, Westlund results where comparable:
• Misspecification is in general a serious problem with severe parameter bias.• Skewness of distribution is of minor importance to the PLS estimates.• Multicollinearity between the latent variables is without importance for the
estimated indices. It has a significant but small impact on the bias of the path coefficients. It has a significant effect on all standard deviations.
• Size of the sample has no influence on the bias of the path coefficients. It has a large effect on all standard deviations.
• In addition:• Indicator reliability has an enormous influence on all measured responses,
i.e. bias, standard deviation and fit measures. Furthermore several cases of two-factor interaction with both multicollinearity, sample size, and thenumber of indicators were found.
• Likewise the number of indicators has a strong impact on all responses, and also a strong two-factor interaction with sample size and reliability.
55
Kai Kristensen, IMPS´03
General conclusion
• PLS provides reasonably robust estimates of a customer satisfaction index in a usual practical setting where the sample size is n=250, the standard deviation around σ=20, and the average multicollinearity around ρ=.60.
• In a usual practical setting the bias of the indices is low and usually not larger than .50 (on a 100 point scale).
• The parameter estimates are in general biased. The bias can be both positive and negative depending on the model structure. Therelative bias will in a usual practical setting be in the area of 10-20%.