-
Jiang et al./Measuring IS Service Quality
The application of the measure to the IS field hasgamered a
great deal of recent debate (Kettingerand Lee 1997; Pitt et al.
1997; Van Dyke et al.1997). There is a psychometric concern
ofoperationalizing a single concept as the differenceof two
separate elicitations and also empiricalambiguity of the construct
structure. The use ofthe difference scores presents a number
ofpotential flaws, including reduced reliabiiity, poorconvergent
validity, and unstable dimensionality(Van Dyke et ai. 1997).
SERVQUAL as adoptedfor information systems has been inconsistent
inlerms of dimensional structure, reliability, andvalidity (Cronin
and Taylor 1992; Kettinger andLee 1997; Kettinger et al. 1995;
Parasuraman etai. 1994). The question is whether the effects
ofthese issues are serious enough to exclude theuse of SERVOUAL in
the IS setting.
Using an IS professional sample populationmatched to a sample of
IS users, we re-examineSERVOUAL issues from the IS professional
side:(1) the dimensionality of the instrument, (2) theconvergent
validity, and (3) the reliability mea-sures of the difference
scores. We then examineIhe expectation gap between the IS user and
ISprofessional according to the same criteria. Sinceexpectation
gaps are expected to impact per-ceptions (Ginzberg 1981 ), we
compare the resultsof the expectation gap to the dimensions of
themore common user satisfaction scale (Baroudiand Orlikowski
1988).
Ci Empirical Support I
. To addressthe difference score concerns involved-.: in
SERVQUAL, empirical analysis is necessary.{::: Pitt et al. (1997),
based upon user samples,
calculated the reliability adjusted for differencesiH and
demonstrated no reliability problem asso-
ciated with the SERVQUAL. Kettinger and Lee(1997 addressed the
dimensionality problemusing student samples across different
campusesand found consistent dimensions existed in the IS-
adapted SERVOUAL. Qthers found a differenti; number of
dimensions depending on the popula-
tion involved (Cronin and Taylor, 1992: Kettingeret al. 1995;
Parasuraman et al. 1994; Pitt et al.1995). Further studies of user
populations areclearly needed as are studies examining
theappropriateness of using SERVQUAL from theperspective of IS
professionals to analyze gapsbetween providers and customers.
Sampie
To obtain a sample of IS professionals andmatched IS users, the
SERVOUAL and the usersatisfaction (UIS) questionnaires were mailed
to200 managers in different organizations in theU.S. The 200
managers selected were those whoagreed to participate from 612
contacts made withdifferent organizations. The list of
organizationsand managers for contact was extracted from amore
comprehensive listing of organizations main-tained by an economic
development center at aMidwestern university.
These managers were first contacted directly bythe authors or
graduate assistants. Each mana-ger was asked to secure a response
from an ISprofessional for the SERVQUAL instrument(Appendix A). The
manager was also asked tosecure a response from an IS user for
theSERVOUAL instrument (Appendix 8) and for theUIS instrument
(Appendix 0). Managers whoreturned both the IS professional version
and theuser versions were considered to have returnedmatched sets.
All of the respondents wereassured that their responses would be
keptconfidential. A total of 186 questionnaires werereturned, which
included 168 matched sets. Thedemographic information of these
respondents isshown in Table 1.
Before any analysis was conducted on thedimensionality or
scales, the data was examinedfor potential biases. An ANOVA was
conductedby using service quality (as the dependentvariable)
against each demographic categoryshown in Table 1 (independent
variable). Resultsdid not indicate any significant relationships.
Non-response bias was examined by comparing our
MIS Quarterly Vot. 26 No. 2/June 2002
-
Jiang et al /Measuring IS Service Quality
Table 1. Demographics
1. GenderMaleFemaleNo Response
2. AgeUnder 2525 to 3435 to 4445 and overNo response
3. Work ExperienceUnder 5 years5 to 9 years10 to 14 years15 to
19 years20 years or more
4. Experience in Different Applications1 to 3 areas4 to 6
areasMore than 6 areasNo response
5. Total Number of Employees in OrganizationLess than 50
people50 to 99 people100 to 249 people250 to 499 people500 to 999
people1,000 to 2,499 people2,500 people or moreNo response
IS Professionals
111662
226346363
39464114272
5158627
4332262210121310
IS Users
661011
437134191
66442317272
expectation measures on the SERVQUAL scalesand the UIS scales to
previous studies (BaroudiandOriikowski 1988, Pitt et al. 1998).
Chi-squaretests found no difference between the means ofour sample
to those in the other studies oncenormalized tc a five-point scale.
Additionally, thesample was split into early and late
respondentsand t-tests found no difference in the means ofany
SERVQUAL dimension. Non-response biasdid not arise as an issue
based on these tests.
Dimensionality of SERVQUALfrom the Other Side
If the measurement model provides a reasonablygood approximation
to reality, confirmatory factoranalysis (CFA) accounts for observed
relation-ships in a data set. The chi-square test providesa
statistical test of the null hypothesis that themodel fits the
data. In addition, other fit indicesare typically used to identify
overall goodness of
148 M/s Quarterly Vol. 26 No, 2/June 2002
-
Jiang et al./Measuring IS Service Quality
Table 2. Confir
Fit Index
RMRChi-square
d.f.
Chi-square/d.f.
CFINNFI
GFIAGFI
matory Factor Analysis fo
Threshold(s .10)
( i 5.0)( .90)U .90)( i .90)(> .80)
Model 1.057
196.7965
3.030.880.850.840.78
r SERVQUAL ModeModel 2
.048106.6
641.670.950.940.910.87
S
Model 3.041
125.1684
1.490.950.940.910.87
Mod9l 4.041
86.8459
1.470.960,940.920.88
Notes:(1) RMR = Root Mean Square Residual(2) CFI = Comparative
Fit Index(3) NNFI ^ Bagozzi (1980) Non-normed Index(4) GFI =
Goodness of Fit Index(5) AGFI = GFI Adjusted for Degrees of
Freedom(6) Model 1 = (Responsiveness, Assurance, Empathy, and
Reliability) as one dimension(7) Model 2 = (Responsiveness,
Assurance, and Empathy) and Reliability as two dimensions(S) Model
3 = Responsiveness, Assurance, Empathy, and Reliability as four
dimensions, 16 tem
as in Parasuraman et al. (1994)(9) Model 4 = Responsiveness,
Assurance, Empathy, and Reliability as four dimensions, 13 item
as in Kettinger and Lee (1994) as shown in Figure 1
fit. Previous studies of rigor have found theSERVQUAL tangibles
dimension to be weak(Cronin and Taylor1992,1994; Kettinger and
Lee1994,1997; Parasuraman etal, 1991). We beginouranaiysis with the
four dimensional model usedin other studies of IS service quality
because of
^ the recency of the results and the IS orientation ofthe
instrument (Kettinger and Lee 1994, 1997).We test one-, two-, and
four-dimensional models
I found in other recent studies, including a newermodel proposed
by the developers of SERVQUAL(Parasuraman et al. 1994). The model
we carry
' fonward in the analysis (model 4) compares;. favorably to the
remaining models. The preferred levels of each index for the CFA
and the results of
the models are shown in Table 2, Analysis wasconducted with
LISREL 8.51 using maximum
r; likelihood estimation on the covariance matrix.
The correlations and descriptive statistics forthese dimensions
appear in Tables 3 and 4.Patterns of mean, median, skewness, and
kurtosisin Table 4 were examined according to
convention(Ghisellietal. 1981). The responses had reason-able,
skewness (less than 2), and kurtosis (lessthan 5). This indicates a
lack of bias in thesample in the measured variables. The
four-dimensional model is highlighted in Figure 1.
SERVQUAL Validity fromthe Other Side
Convergent validity and discriminant validity wereexamined.
Empirically, convergent validity can beassessed by reviewing the
t-tests for the factor
MIS Quarterly Vot. 26 No. 2/June 2002
-
Jiang et al./Measunng IS Service Ouaiity
Tabie 3. Corr
Reliability
elation of Dimensions m Model 4
Reliability
1.00
Responsiveness 0,82Std, Errort-value
AssuranceStd, Errort-vaiue
EmpathyStd, Errort-value
(0.05)17.34
0.81(0.06)14.12
0.65(0.07)9.96
Responsiveness
1.00
0.95(0.05)17.55
0.84(0.06)15.28
Assurance
1.00
0.91(0.06)15.20
Empathy
1,00
iptive Statistics of tiie 4D SERVQUAL Model (Model 4)
MeanVarianceMedianSkewnessKurtosis
Reliability
.64
.91
.33
.721.22
Responsiveness
.46
.69
.33
.69
.88
Assurance
.36
.50
.33
.39
.65
Empathy
.23
.39
.00
.911.52
loadings. If all factor loadings for the indicatorsmeasuring the
same construct are statisticallysignificant (greater than twice
their standarderror), this can be viewed as evidence supportingthe
convergent validity of those indicators(Andersen and Gerbing 1988),
All t-tests Vi^ eresignificant (Table 5) showing that ali
indicators areeffectiveiy measuring the same construct, or
highconvergent vaiidity.
Empirically, discriminant validity is achieved whenthe
correlations betv i^een any two dimensions aresignificantly
different from unity (Bagozzi andPhillips 1982), Evidence regarding
discriminantvalidity can be obtained by using the chi-square
difference test. The chi-square difference testcompares an
unconstrained model that estimatesthe correlation between a pair of
constructs and aconstrained modei which fixes the value of
theconstruct correlation to unity. The difference inchi-square
between these models is a chi-squarevariate with degrees of freedom
equal to one. Asignificant chi-square difference implies that
theunconstrained model is a better fit for the data,thereby
supporting the existence of discriminantvalidity
(BagozziandPhillips,1982), The results ofthe chi-square difference
tests generally supportthe discriminant validity of the scales:
howeverthe ASSURANCE scale exhibits some historicaldiscriminant
validity problems (see Table 6),
150 MIS Quarteriy Voi. 26 No. 2/June 2002
-
Jiang et ai./Measuring IS Service Quaiity
jrement ModelFour Factors
' SERVQUAL Reliability fromthe Other Side
Reliability refers to consistency of measurement.A construct is
reliable if, for example, it providesessentially the same set of
scores for a group ofsubjects upon repeated testing. There are
anumber of different ways that reliability can beexamined. In the
present study, the compositefeliability, variance extracted
estimates, andCronbach alpha values were examined.
Composite reliability reflects the internal con-sistency of the
indicators measuring a given factor(Fornel! and Larcker 1981). The
compositereliability can be computed by taking the square ofthe sum
of standardized factor loadings for thatfactor divided by the sum
of the error varianceassociated with the individual indicator
variablesand the square of the sum of the standardizedfactor
loadings (Forneli and Larcker 1981). Thecomposite reiiabilities for
each SERVQUALdimension are shown in Table 7. Results indicate
M/S Ouarterly Vol. 26 No. 2/June 2002 151
-
Jiang e at./Measuring IS Ser
Table 5. Convergent Validity of Model 4 |Constructs and
Indicators
ReliabilityDRELl (item 5)DREL3(item7)DREL4 (item 8)
ResponsivenessDRESP2 (item 11)DRESP3 (item 12)DRESP4 (item
13)
AssuranceDASSl (item 14)DASS3(item16)DASS4 (item 17)
EmpathyDEMP1 [item 18)DEMP3(item20)DEMP4(item21)DEMP5 (item
22)
Standardized Loadings
0.810.800.89
0.740.730.69
0.710.470.65
0.650,610,620.68
adings significant at p < .01 level.
Table 6. Discriminant Validity of Model 4
Construct PairREL-RESPREL-ASSREL-EMP
RESP-ASSRESP-EMPASS-EMP
AChi-Square22.43
14.84
77.81
0.08
10.24
2.75
A Degrees ofFreedorr
1
1
1
1
1
1
Discriminant Validity
Yes-
Yes*
Yes*
No
Yes*
No
'Indicates significant at p ^ .01 li
152 MIS Quarterly Vot. 26 No. 2/June 2002
-
Jiang e a/,/Measuring IS Service Ouatity
ConstructIndicators
Fieliability
Responsiveness
Assurance
Empathy
CoR
Service
mposJteliability
.87
.76
.64
.73
Gap of IS ProfessionalsVariance Extracted
Estimate
.70
.52
.38
.41
CronbachAlpha
.87
.76
.65
.74
Adjusted(Johns 1981)
,84
.67
.64
.67
an acceptable level of reliability, although theassurance scale
is lower than desired for empi-rical analysis (Carmines and Zeiler
1988). In
__ addition, the traditional Cronbach alpha values foreach ofthe
SERVQUAL dimension are shown forcomparison. The Johns (1981)
adjusted formulator difference score alpha value was also
applied.
Variance extracted estimates, as discussed by_ Fornel and
Larcker, assess the amount of
variance that is captured by an underlying factorin relation to
the amount of variance due tomeasurement error. Fornel and Larcker
suggest
Ilhat it is desirable a construct exhibit estimates of.50 or
larger, because estimates less than .50indicate that variance due
to measurement erroris larger than the variance captured by the
factor.
'^ However, this test is quite conservative. Very;. often,
variance extracted estimates will be below-^ .50, even when
reliabilities are acceptable. Thei^ variance extracted estimates
for each dimension,, of SERVQUAL are also shown in Table 7.
Expectation Gap and Validation
. One premise of the SERVQUAL model is that thegaps are produced
by a series of prior gapsIZeithamI et al. 1990). One of these is a
gapbehween the expectation of the user and the abilityotthe service
provider to understand their desires.Pitt et al. (1998) found the
gaps to be present andmeaningful in the interpretation of the
service gap.Ginzberg (1981) presented a similar concept
thatproposes that a gap in expectations between IS
professionals and IS users will lead to a lack ofsatisfaction on
the part of the user, a predictiveform of final perceptions. We
explore this premiseby examining the relationship between the
expec-tation gap and a common measure of user satis-faction, the
UIS (Baroudi and Qriikowski 1988).
First, the gap scores are taken for the expec-tations of the
users and the IS professionals forthe SERVQUAL instrument items. We
restrictourselves to the items in the four dimensionslocated in the
previous analysis. The results forthe expectation gap measures are
tested to thesame rigor as the service gap measures. Table 8shows
the CFA fit results to the four-dimensionalmodel. Figure 2 shows
the model as fit by theCFA, Table 9 presents the correlations of
the fourdimensions and Table 10 has the descriptivestatistics.
Table 11 has the results of the conver-gent validity tests and
Table 12 shows the resultsof the tests for discriminant validity.
Reliabilityfigures are in Table 13.
To examinetherelationshipofthe expectation gapto UIS, the UIS
instrument was first validatedaccording to the same rigor as the
SERVQUALinstrument. UIS is a more widely accepted instru-ment and
results from this data followed expecta-tions (Baroudi and
Orlikowski 1988), The struc-ture found by Baroudi and Oriikowski
held in thissample, with the three dimensions of
informationproduct, staff and services, and knowledge/involvement
present. Due to the acquisition oftheexpected structure, the CFA,
reliability, andvalidity results of the UIS are not presented
herefor the sake of brevity.
MIS Quarterly Vol 26 No. 2/June 2002
-
Jiang et al./Measuring IS Service Quality
Fit Index
RMR
Chi-square
d.f.
Chi-square/d.f.
CFI
NNFI
GFi
AGFI
Threshold
(. .10)
(: 5.0)(.. .90)
(.. .90)(.: .90)
( .80)
Modei 4
0.54
120.30
59
2.03
0.92
0.89
0.90
0.85
Figure 2. Expectation Gaps Measur
154 M/S Quarterly Voi. 26 No. 2/June 2002
-
Jiang et al./Measuring IS Sen/ice Quality
FAFF
1
i
iII
UISK
NEm
path
y
11I
ura
nce
If,
enes
s
1
Resp
abi
lityRe
l
_
1,00
Rel
iabi
litySt
d, Er
ror
t-valu
e
o
-0,
079,
42
Resp
onsiv
enes
sSt
d, Er
ror
t-valu
e
oq
001-
,00-
:0
0,90
-0.
0713
.85
Assu
ranc
eSt
d. Er
ror
t-valu
e
1,00
0,84
-0.
0613
,43
0.6'
-0-
010
,5
0,68
-0,
079,
42
Empa
thy
Std,
Erro
rt-v
alue
.00
-0.1
5-0,
09-1.
58
-0,
19-0,
10-1.
89
'--' 9 T
0,01
-0,
100,
14
UISK
NOW
Std.
Erro
r
S
' o CM
-0,
06-0.
10-0,
63
-0.
16-0,
10-1,
51
CO o -O-
d d cri
0,07
-0,
10 0,73
UISI
Pst
d. Er
ror
t-vai
ue1,
00
10,
580,
078,
55
-0.
19-0,
09-2.
04
-0.
31-0,
10-3,
27
-0,
1
-1,
[
9 9 9
UIST
AFF
Std.
Erro
rt-v
alue
M/s Ouanerly Vol. 26 No. Z-June 2002 155
-
Jiang et al./Measuring IS Ser
Fit Index
RMR
Chi-squared.f.
Chi-square/d.f.CFI
NNFI
GFI
AGFI
Threshold
(= .10)
(; 5.0)(. .90){, .90)(i .90)(> .80)
Model 4 ^
0.54 i f120.30
59
2.03
0.92
0.89
0.90
0.85
Figure 2. Expectation Gaps Measurement Model
154 MIS Quarterly Vol. 26 No 2/June 2002
-
Jiang et ai.measuring K
liabi
lity
Std.
Erro
rv
alue
00
o CTJ
sp
onsiv
enes
sSt
d. Er
ror
t-valu
e
1.00
r2 o 5o o o
ssur
ance
Std.
Erro
r