Page 1
ORGANIZATIONALRESEARCH METHODSSimsek, Veiga / ELECTRONIC SURVEYTECHNIQUE
ZEKI SIMSEKJOHN F. VEIGAUniversity of Connecticut
Even though e-mail is the most widely used computer-mediated communication
medium, its considerable potential as a survey technique has received little atten-
tion from management scholars. Using a three-dimensional framework focused
on sampling issues, nonsampling errors, and comparative performance, the
authors review and integrate previous research on the electronic survey technique
and provide an assessment of the comparative trade-offs vis-à-vis other tech-
niques. Moreover, they provide recommendations for future researchers inter-
ested in using this approach. Finally, they conclude that although this approach
poses unique challenges and drawbacks, when an unbiased sampling frame exists
or can be constructed, it allows researchers to inexpensively gather data with less
effort than other available approaches.
Surveying techniques—usually classified by the communication medium used, such
as face-to-face, telephone, mail, or electronic—rely on questioning individuals to
elicit particular information to look for patterns among facts, values, behaviors, and so
on to make generalizations about a population from which only some individuals are
surveyed. Over the years, the use of such techniques has been, by far, the most com-
mon method of data collection in several fields, and this is anticipated to remain such,
at least for the foreseeable future (Aaker, Kumar, & Day, 1995; Chadwick, Bahr, &
Albrecht, 1984; Malhotra, 1993; Synodinos & Brennan, 1988). Despite its well-
known inherent weaknesses relative to experimental methods, gathering data via sur-
veys has been more prevalent in management research arguably because of costs and
obstacles associated with carrying out experiments and because the basic locus of
many research questions has involved phenomena in the field. Simply put, if you want
to find out what managers are thinking, you need to ask them (Zikmund, 1994).
Albeit sketchy, the history of surveys and thus surveying techniques can be traced
back thousands of years (Erdos, 1983; Rossi, Wright, & Anderson, 1983). However,
until recently, mail questionnaires, field interviews, and telephone surveys were the
Authors’ Note: We wish to thank Monica Maciel Lopes and Melissa Foreman for their help during the
preparation of this manuscript. We also thank two anonymous reviewers for their suggestions.
Organizational Research Methods, Vol. 3 No. 1, January 2000 93-115© 2000 Sage Publications, Inc.
93
Page 2
only convenient techniques to collect survey information. Gates and Jarboe (1987)
argue that developments in electronic technology, computer software, and environ-
mental forces that oppose traditional data collection techniques have contributed to
the change in data collection techniques today. Of these, computer technology has
been a fundamental force behind the growth of alternative survey techniques and has
led to improved data collection (Malhotra, 1993). Indeed, the emergence of this tech-
nology has affected not only data collection techniques but also has had a dramatic
impact on almost every phase of survey research, including instrument design, sam-
pling, field monitoring, coding and data editing, data capture, data cleaning, scale
index construction, database organization, database retrieval, data analysis, and docu-
mentation (Anderson & Gansender, 1995; Karweit & Meyers, 1983; Neal, 1989;
Saltzman, 1995).
It has been aptly acknowledged that currently there are probably as many surveying
techniques as there are different forms of communication technology (Aaker et al.,
1995). For example, computer-assisted personal interviewing (Baker, 1992; Couper &
Burt, 1994), computer-assisted telephone interviewing (Havice, 1990), fully auto-
mated telephone interviewing (Dacko, 1995), computer disk by mail (Higgins, Dim-
nik, & Greenwood, 1987; Saltzman, 1995), fax surveys (Vazzana & Bachmann,
1994), online World Wide Web (WWW) surveys, and focus groups (Gaiser, 1997) are
only a few of the new techniques that have evolved. This evolution is the result of rapid
developments in computer and communication technologies. The ever-increasing
preference for computer-mediated communication, the opening of the Internet to the
public, the introduction of WWW in 1989 at the European Particle Physics Laboratory
in Europe, and the low-cost dispersion of software and hardware are fundamental
forces that have shaped the viability of the e-mail survey technique (EST).
EST holds considerable promise to obliterate the time and geographical constraints
usually associated with surveys, facilitate interaction between surveyors and respon-
dents, and reduce cost, time, and data entry errors per response (Bachmann, Elfrink, &
Vazzana, 1996; Kiesler & Sproull, 1986; Mehta & Sivadas, 1995; L. Parker, 1992).
Nonetheless, despite this potential, our review of the literature revealed that research
on EST (a) is dispersed across the literature of almost two decades and several fields;
(b) has not been systematically evaluated, let alone integrated; (c) consists mostly of
empirical studies dealing with either response rates or quality of collected data or com-
mentaries that juxtapose the pros and cons; and (d) has not embraced replications and
lacks theoretical arguments and a conceptual framework. Indeed, to date, no one has
attempted to integrate and assess both the theoretical and practical concerns (cf.
Kiesler & Sproull, 1986; Kittleson, 1995; Oppermann, 1995; Schuldt & Totten, 1994).
Thus, it seems that EST is, like the weather, something about which everybody is talk-
ing but nobody is doing much about it.
We believe that this state of research on EST is unfortunate for several important
reasons. First, literally hundreds of organizations conduct e-mail or web-based sur-
veys for private and organizational consumers who in turn base their decisions on
these data. Second, on any given day, numerous researchers conduct surveys using
some conventional techniques, some of which could be done more efficiently and
effectively using EST. Third, although e-mail is the most widely used computer-
mediated communication medium, its full utility in sample surveying is not thor-
oughly assessed, let alone realized. Indeed, it took 2,500 years until basic postal serv-
Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE 94
Page 3
ices became available to most individuals after being established by King Cyrus of the
Persian Empire in the 6th century B.C. (Kochmer & Northwest, 1993, p. 47), but it took
only 50 years until the numbers of computers rose to 1 computer per 44 persons from
no more than dozens in the world (Perl & Whitley, 1995, p. 6). According to a recent
estimate, electronic messages sent in the United States went from 776 billion to 2.6
trillion from 1994 to 1997 and are projected to reach 6.6 trillion by 2000 (Gwyne &
Dikerson, 1997). Fourth, with the exception of earlier work by Kiesler and Sproull
(1986), there is a paucity of focused discussion on EST in the management literature.
Accordingly, using a three-dimensional framework focused on sampling issues,
nonsampling errors, and comparative performance, we review and integrate previous
research on EST and provide an assessment of the comparative trade-offs vis-à-vis
other techniques. Moreover, we provide recommendations for future researchers
interested in using this approach.
Although the basic notion for an electronic mail system has been around since the
1840s, e-mail has only emerged in the past three decades as a result of the convergence
of computer and communication technologies (Helliwell, 1986; Mortensen, 1985).
The use of e-mail was first started on the ARPAnet during the 1960s. At this early
stage, the limited access and primitive nature of the systems hampered widespread
usage, and it was not until local-area networks (LANs) were developed that e-mail
acquired its popularity.
E-mail is simply a combination of software, hardware, and communication tech-
nologies that allows a user to send and receive messages or documents to and from a
user or set of users. The Electronic Mail Association (EMA), a Washington-based
trade association, defines e-mail as “the generic term for the non-interactive commu-
nication of data, images or voice messages between a sender and designated recipi-
ent(s) by systems utilizing telecommunication links.” Although this definition encom-
passes technologies such as facsimile, telex, and communicating word processors, in
this article, e-mail refers to the transmission of text message and (in some advanced
systems) graphics, video, and sound over telephone lines or wireless technology from
computer to computer.
Within this context, EST can simply be defined as a computerized self-
administered questionnaire in which the researcher sends a questionnaire and the
respondents receive, complete, and return the questionnaire through e-mail systems
that bring together capabilities of both computers and telecommunication networks
(Rice, 1990). The researcher can send a separate e-mail with the survey embedded to
each respondent or multiple respondents, or the researcher can ask each respondent
to access a web site where the survey is housed. In this latter case, on completion, the
survey is submitted either as an e-mail to the researcher, or it can be downloaded to a
data file. Anyone with a computer, modem, and telephone line can use EST if he or
she has access to an online service, a commercial carrier, a LAN, or an Internet serv-
ice provider.
EST differs from WWW surveys that depend on the transmission of a questionnaire
over the Internet to a database located at the site of the study (Subramanian, McAfee,
& Getzinger, 1997) but rely on chance that somebody might come across the question-
95 ORGANIZATIONAL RESEARCH METHODS
Page 4
naire, become interested in it, copy the document, complete it, and then return it (Swo-
boda, Muhlberger, Weitkunat, & Scheneeweib, 1997). Assuming no specialized con-
trols such as passwords are used, anyone who has access to the Internet could
potentially respond to such a survey. As such, although a WWW survey does not have
the sampling controllability as EST (Stanton, 1998), it has the potential, with protec-
tion, to provide greater anonymity. Furthermore, unlike EST, WWW surveys can
incorporate images, graphics, sound, and so on into the survey. Another advantage of a
WWW survey is that it is always present and available, whereas EST is inherently epi-
sodic (Stanton, 1998). EST, however, does not require the researcher to develop
appropriate data-screening methods and develop links that respondents should access
to participate in the survey.
EST also differs from surveys that are done through newsgroups or list servers,
although sending e-mail questionnaires through newsgroups and list servers is clearly
another form of EST. Like WWW surveys, these surveys suffer from low sampling
control as well as self-selection bias because individuals on newsgroups or list servers
are usually interested in particular issues. For example, a list server survey typically
relies on surveying individuals with a special interest in a certain topic (Stanton,
1998). The potential receivers are often unknown to the sender and are characterized
only by their interest in a particular subject (Batinic, 1997). There is virtually no con-
trol over individuals who are to complete the survey by the researcher (Swoboda et al.,
1997).
Once the target population is identified, determination of the sampling frame,
selection of a sampling procedure (probability vs. nonprobability), and computation
of the sample size are important sampling-related issues that must be addressed while
preparing for collecting data through EST. Although the latter two issues are rather
straightforward statistical topics, we will attend primarily to the availability or con-
struction of the sampling frame. The sampling frame can significantly reduce difficul-
ties involved in the sampling process because it determines the sampling control and
guides the direction of the inquiry in EST.
The sampling frame is a master listing of population members usually used to draw
a random sample from which data will be collected. Depending on the objective of the
research, a sampling frame can be a listing of all the managers who work at a company,
all the executives from Fortune 500 companies, and so forth. The quality of such a list
primarily determines sampling biases. The ideal sampling frame is one in which every
element of the population is not only represented but also only represented once.
Clearly, when one is interested in sampling an appreciable segment of the human
population, there will be problems with this ideal (Sudman, 1996; Tull & Hawkins,
1993). With respect to telephone and mail survey techniques, problems such as mail-
ing lists being out of date or incomplete and phone directories not including unlisted
numbers have been addressed. However, mostly because of lack of reliable documen-
tation on addresses and profiles of e-mail users among different segments of the soci-
ety, problems associated with sampling frames of EST have not been documented yet.
Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE 96
Page 5
To our best knowledge, there is no good frame that lists individuals or households
using or even having access to e-mail, although it has been reported that the Internet’s
Network Information Center is working to produce a master directory of all users
called InterNIC (T. Parker, 1995). We also do not know of any private firm that can
currently offer unbiased frames for many populations. Indeed, there are many list bro-
kers, list servers, and sources of opt-in lists on the Internet, the lists that are developed
from people who have agreed to receive unsolicited e-mail. Yet, it has been suggested
that commercial lists tend to be seriously flawed (Comley, 1996). Although it might be
possible to create a good EST sampling frame by using e-mail addresses from online
service providers or e-mail carriers, these companies observe privacy laws and poli-
cies, so this is not a viable option. Finally, even though there are several utilities such
as Finger, Whois, and Netfind, to collect the Internet e-mail addresses, it has been
claimed that only 1% or 2% of all the Internet users can be located with one of these
methods (T. Parker, 1995). Accordingly, obtaining or constructing an unbiased or at
least a useable sampling frame that allows probability sampling is currently the most
serious challenge that EST imposes on researchers. Indeed, whether EST can success-
fully use probability sampling for general populations has not yet been established.
On the other hand, even when available, many if not all e-mail frames should be
used cautiously because they usually lack universal coverage of the population. As a
rule, such claims as those echoed in the popular media on availability or accessibility
of e-mail should not be considered as a surrogate measure of feasibility. In making a
case, such aggregate numbers are of limited use because the availability of e-mail does
not guarantee acceptance, usage, or compatibility (Kerr & Hiltz, 1982; Komsky,
1991). Although researchers have generally been able to assume that people receive
their mail at their postal address, they cannot simply assume that all e-mail addressees
are active.
Moreover, although some of the problems in constructing e-mail frames pertain to
self-administration in general (e.g., literacy and blindness), others may only relate to
the ability to have and use a computer. The systematic exclusion associated with
e-mail frames is severe, particularly because of e-mail’s relation to purchasing power.
Research on computer usage reveals that computer users still share similar demo-
graphic characteristics of being young, well educated, and above average in income
(Oppermann, 1995). Likewise, Couper and Rowe (1996) found that less educated,
older respondents and those with less computer experience were less likely to com-
plete a self-administered component of a computer-assisted personal interview survey
on self-images, suggesting that the use of the computer may add additional constraints
on the willingness or ability of respondents to complete an e-mail questionnaire.
Clearly then, whether researchers can currently reach a representative sample
through EST primarily hinges on the population under investigation. For example, if
the investigation involves low-income households, minorities, or elderly populations,
even simple random sampling attempts will be flawed because of high noncoverage
error (Anderson & Gansender, 1995; Dillman, 1991). On the other hand, EST can
prove quite beneficial for obtaining opinions related to new software. Likewise,
because most large firms and their managerial/professional employees have access to
e-mail, sample surveys of these populations are possible. In any case, using a stratified
sampling approach rather than a random sample should lessen the degree of potential
noncoverage error (Oppermann, 1995). However, the researcher should keep in mind
97 ORGANIZATIONAL RESEARCH METHODS
Page 6
that almost all hypothesis testing and estimation procedures assume simple random
samples, not stratified samples.
Nonsampling errors, which are often viewed more important than sampling errors,
are all the other errors in a survey except those due to sampling method and sample size
(Sudman, 1996; Tull & Hawkins, 1993). More specifically, they include coverage
error—which has been discussed—nonresponse, and measurement error (Lavrakas,
1996).
Nonresponse errors. A high number of nonresponses raise the question of whether
those who responded to the survey are different from those who did not. Even in the
absence of sampling biases, if nonresponses are not randomly distributed, then the
data generated by EST will be biased because careful attempts at sample randomiza-
tion have been eroded. In turn, such biased data severely influence the validity of the
research and often result in invalid inferences (Gilbert, Longmate, & Branch, 1992;
Dillman, 1991; Dillon, Madden, & Firtle, 1987). Nonresponses, even when random,
may reduce what was an adequate sample to an inadequate one, forcing the researcher
to either survey additional respondents or to find a remedy for these missing responses
through postsurvey estimates (Hair, Anderson, Tatham, & Black, 1995). Nonetheless,
increasing the response rate—as opposed to postsurvey adjustments, such as weight-
ing cases by estimated probabilities of cooperation and known population quantities,
imputation, and selection bias models that can work under certain assumptions to a
limited extent (e.g., Hair et al., 1995; Kalton, 1983)—is clearly the safest strategy to
reduce nonsampling errors.
Overall, nonresponse errors in EST can generally be attributed to noncontacts (i.e.,
unreachables and refusals). It is promising to note that EST has been used as an effec-
tive means of gathering data in several academic and institutional settings (Anderson &
Gansender, 1995), especially when one takes into account that most responses were
attained without the use of any response inducement technique.
Researchers using EST have reported response rates ranging from 19.3% to 76%.
For example, Kiesler and Sproull (1986), on examining the response rate associated
with EST and a postal survey, found a higher response rate for the paper survey (75% vs.
67%). When comparing EST with face-to-face interviews in a Fortune 500 manufac-
turing company, Sproull (1986) found a participation rate of 73% for EST versus 87%
for interviews. In a survey of a major corporation’s overseas employees, L. Parker
(1992) reported that the response rate associated with EST (68%) was significantly
higher than when mail pouches (38%) were used. Although Schuldt and Totten (1994)
reported a response rate of 56.5% for a mailed survey and 19.3% for EST, Kittleson
(1995) obtained a response rate of 28.1% for EST and 76.5% for a postcard survey.
Anderson and Gansender (1995), employing a survey to assess how and why people
used a network system, obtained a response rate of 76% from 488 Free-Net users of a
metropolitan area. Walsh, Kiesler, Sproull, and Hesse (1992) attained a response rate
of 76% from a 93-item online survey of 300 science-net subscribers. Finally, in
another study involving business school deans and division chairpersons on the use of
total quality management, Bachmann, Elfrink, and Vazzana (1996) had a response rate
of 65.6% for the mail questionnaire and 52.5% for EST.
Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE 98
Page 7
Although further research is need on factors causing response rate differences
across studies, these findings overall indicate that EST has been used as an effective
means of gathering data in terms of response rates. They also indicate that EST tends to
have a lower response rate compared to mail surveys, thereby pointing to the need for
more research on response inducement techniques in the EST context. However, aug-
menting responses to EST through some incentives and procedures virtually has not
been explored. Even fewer studies have investigated the influence of such tactics and
incentives on response speed and response content. Although the researcher has no
control over the unreachables, refusals stemming from such factors as survey design
can be influenced as the research on mail survey suggests (Linsky, 1975; Yammarino,
Skinner, & Childers, 1991). In the Recommendations section of this article, we further
touch on this issue.
Measurement errors. Measurement error is simply the deviation between the “true”
and the observed responses (Dillman, 1991). Many types of errors stemming primarily
from the data collection method used, such as those committed during the transforma-
tion of the data, are always at work in a survey. Broadly speaking, there are three
sources of measurement error due to the survey instrument, the respondent, and/or the
data collection technique. With respect to EST, measurement errors due to the survey
instrument generally occur at the presurvey scale development stage. In this respect,
EST does not differ from other surveying techniques in that it also requires the
researcher to do everything that is needed in terms of developing a reliable and valid
scale. Given the newness of EST, we were able to find only one study that detailed how
the scale for a study using EST was developed (Clayton, Applebee, & Pascoe, 1996).
With respect to respondent-based errors, some researchers have examined
response content of computerized data collection techniques in general and EST in
particular. Tourangeau and Smith (1996) noted that computer-assisted, self-
administered surveys produced similar outcomes to the more conventional self-
administered techniques. They further claimed that computerization by itself had little
influence on the response quality and that better data quality often associated with the
computer-administered questionnaires may be the result of the self-administration.
However, in a comprehensive review on computerized data collection techniques,
Leeuw, Hox, and Snijkers (1995) contended that in general, computerized techniques
such as computer-assisted personal interviewing (CAPI) and computer-assisted self-
interviewing (CASI) have a positive effect on data quality. In the case of EST, these
findings are largely consistent with the findings reported on response quantity, but
they are inconclusive with respect to response quality because how responses are
influenced by e-mail communication itself has not been conclusively demonstrated.
Some researchers have asserted that EST generally conveys little social informa-
tion, so respondents experience less evaluation anxiety than when they respond using
other survey modalities (Kiesler & Sproull, 1986; Kiesler, Zubrow, & Moses, 1985;
Sproull, 1986). Kiesler (1989) and Sproull (1986) suggest that because e-mail tends to
reduce social concerns and constraints on individuals, EST respondents are less con-
cerned about reporting negative and socially inappropriate things about themselves.
Couper and Rowe (1996) found that those who completed a self-administered com-
puter interview reported a more positive self-image than those who had the inter-
viewer help, after controlling for respondent characteristics related to self-image.
Ayidiya and McClendon (1990) noted that EST might influence the acquiescence of
99 ORGANIZATIONAL RESEARCH METHODS
Page 8
respondents, thereby lessening some respondents’ propensity to agree with survey
statements more often than they would do with a pencil-and-paper survey.
In another study, Sproull (1986) observed that questionnaire data were slightly
more complete when using conventional mail than when e-mail was used. However,
when comparing these methods, it was found that although there were no differences
in the nature of answers provided by the participants, the e-mail survey elicited more
extreme responses. Perhaps this is because, as Kiesler (1989) has suggested, e-mail
communication loosens social concerns and constraints on people, so that they are less
concerned about saying negative things and/or revealing socially inappropriate
beliefs.
Indeed, in a meta-analysis of self-disclosure on computer forms in general, Weis-
band and Kiesler (1996) found that across 39 studies using 100 measures, computer
administration was associated with increased self-disclosure compared to face-to-face
interviews. The researchers speculated that this finding might be because working on a
computer creates a sense of privacy. Corman (1990) directly attempted to validate data
generated by a computerized survey by comparing them with postal survey data. To do
this, data from two separate groups were compared in three different ways, including
test-retest reliability, criterion validity, and internal consistency. The results indicated
that the computerized survey method produced considerably higher criterion validity
and slightly higher test-retest reliability than did the postal survey. Corman attributed
these findings to the novelty of the computerized data collection approach in that com-
puter respondents may have taken greater care in filling out the survey.
With respect to measurement errors introduced by the data collection technique
itself, several researchers have looked at how EST compares to the postal survey tech-
nique in terms of item completeness and responses to open-ended questions. Liefeld
(1988) compared the response effects of a computer-administered questionnaire to
self-completion and personal interview techniques. Liefeld found that with the excep-
tion of multiresponse/knowledge-type questions, there was little difference in
response patterns among techniques. The computer-assisted technique, however, pro-
duced higher means for most of the items. Bachmann et al. (1996) also found that there
were no significant differences in the responses and respondents’ tendency to leave an
item blank or to comment on questions between individuals receiving a mail or an
e-mail questionnaire. Yet, the e-mail respondents showed a greater willingness to
respond to open-ended questions (21.9% vs. 4.8%). Kiesler and Sproull (1986) found
similar responses between a paper and an electronic survey but also reported greater
item completeness resulting for the electronic survey. Finally, Schaefer and Dillman
(1998) found that EST generated greater item completion and lengthier responses to
open-ended questions. In turn, all this suggests that EST might be particularly useful to
conduct surveys involving open-ended questions.
On the other hand, some have expressed concern that self-administered question-
naires can suffer from sequence biasing because respondents are able to see the whole
questionnaire and may consider questions together rather than each question individu-
ally (Churchill, 1995, p. 371). Hence, responses to an earlier question can prime par-
ticular beliefs and make them more accessible, serve as a standard of comparison for
subsequent items, or be a source for consistency pressure (Lockhart & Russo, 1996).
Although sequencing bias is a concern for conventional paper surveys, e-mail ques-
tionnaires may be sent in such a way that the computer displays each question
Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE 100
Page 9
exactly as wished and does not display all the questions until previous ones have
been completed.
Table 1 displays some other specific possible sources of nonsampling errors and
some techniques for reducing or controlling them when using EST. Table 1 is based on
the extant research on self-administered questionnaires in general. Thus, most of our
suggestions are not unique to EST because they represent general guidelines for iden-
tifying or controlling some nonsampling errors.
The recent proliferation of data collection techniques has made it important that
researchers have a comparative knowledge of these techniques to make an appropriate
decision. For example, although an electronic survey may generate a lower response
rate compared to a postal survey, the choice of which one to use might also depend on
such issues as survey costs, speed, convenience, and the like. Several researchers have
indicated that the assessment of any data collection technique should take into account
its comparative performance as well (Lockhart & Russo, 1996, p. 125). Salant and
Dillman (1994) noted that “no single method can be judged superior to the others in the
abstract. Instead each should be evaluated in terms of a specific study topic and popu-
lation, as well as budget, staff, and time constraints” (p. 35). More specifically, Kiesler
and Sproull (1986) aptly suggest comparing EST with other available alternatives as a
good first step. Accordingly, Table 2 compares EST with personal interviewing, tele-
phone interviewing, and the mail questionnaire technique along several dimensions
considered to be important determinants of the choice of a data collection technique
(cf. Dillman, 1978, 1991; Dillon et al., 1987; Erdos, 1983; Markus, 1994; Watson,
1998). These determinants include sampling issues, cost/efficiency and convenience,
information richness, respondent issues, response outcomes, and future prospects in
terms of usage.
Given that we have already addressed sampling issues, we will first turn to issues of
cost, efficiency, and convenience. EST has the potential of radically changing the eco-
nomics of conducting surveys. With the file transfer capability of computers, EST
does not require usage of paper at any stage, thereby avoiding the costs associated with
the manual entry of raw data or electronic scanning. With EST, according to one esti-
mate, the marginal cost of storage, communication, and dissemination of a 30-page
document can be less than a penny (Kambil, 1995). In many cases, a hard copy of ques-
tionnaires may not be necessary, which in turn eliminates the need to print labels, type
addresses, purchase envelopes, and so on. Furthermore, although the costs of the other
techniques tend to be proportional to the size of the sample, the cost associated with
adding additional respondents in EST is practically zero. The primary costs of EST
include assembling and checking the e-mail list(s), creating or buying software and
supporting databases, and accessing e-mail. When these are available, the cost of EST
is trivial to the researcher. In any case, the marginal costs of collecting and communi-
cating data through EST are much lower than costs of interviewing, telephoning, and
sending questionnaires through postal services (Mehta & Sivadas, 1995).
With respect to speed, sending questionnaires out and receiving them via EST are
definitely very fast. An e-mail questionnaire can be sent to one thousand people as eas-
ily as to one person automatically, and all potential respondents immediately receive
101 ORGANIZATIONAL RESEARCH METHODS
(text continued on p. 104)
Page 10
Tabl
e1
Som
eN
onsa
mpl
ing
Err
ors
inE
ST
and
Tech
niqu
esfo
rH
andl
ing
The
m
Type
Def
initi
onS
ome
Pos
sibl
eTe
chni
ques
for
Han
dlin
g
Non
resp
onse
Failu
reto
obta
inin
form
atio
nfr
omso
me
elem
ents
ofth
esa
mpl
e.U
nrea
chab
les
Des
igna
ted
resp
onde
ntca
nnot
bere
ache
dvi
aE
ST.
1.C
heck
the
e-m
aila
ddre
ssfo
rac
cura
cy.
2.C
heck
for
tem
pora
rylo
cala
ndno
nloc
alsy
stem
wid
ee-
mai
lpro
blem
s.R
efus
als
Res
pond
ents
dono
tres
pond
toth
equ
estio
nnai
re.
1.U
sepr
ior
e-m
ailn
otifi
catio
ns.
2.A
ttem
ptto
conv
ince
the
resp
onde
ntof
the
valu
eof
the
rese
arch
and
his
orhe
rpa
rtic
ipat
ion.
3.E
nsur
ean
onym
ityan
dco
nfid
entia
lity.
4.In
crea
secr
edib
ility
thro
ugh
spon
sors
hip
man
ipul
atio
n.5.
Offe
rso
me
ince
ntiv
essu
chas
gifts
orm
oney
tom
otiv
ate.
6.S
hort
enth
equ
estio
nnai
rew
hen
poss
ible
.7.
Sen
din
divi
dual
e-m
ailq
uest
ionn
aire
sra
ther
than
forw
ardi
ngor
usin
gch
ain
send
ing.
8.M
ake
the
ques
tionn
aire
appe
alin
gth
roug
hvi
sual
aids
.9.
Use
ane-
mai
lfol
low
-up.
10.
Ove
rsam
plin
g.R
espo
nden
terr
ors
Alth
ough
resp
onde
nts
part
icip
ate
inth
est
udy,
thei
rre
spon
ses
are
notc
ompl
ete
orin
corr
ect.
Inte
ntio
nal
Res
pond
ents
fail
tote
llth
etr
uth
ordo
notp
rovi
de1.
Em
phas
ize
anon
ymity
.re
spon
dent
erro
rsre
spon
ses
toso
me
item
s.2.
Use
valid
atio
nch
ecks
.3.
Use
ath
ird-p
erso
nte
chni
que
whe
npo
ssib
le.
Uni
nten
tiona
lR
espo
nden
tsfa
ilto
prov
ide
thei
rre
alth
inki
ngbe
caus
e1.
Scr
een
surv
eyqu
estio
nsbe
fore
usin
g.re
spon
dent
erro
rsth
eym
isun
ders
tand
the
ques
tions
,or
they
gues
s.2.
Use
valid
ated
scal
e.3.
Pro
vide
good
ques
tionn
aire
inst
ruct
ions
and,
whe
npo
ssib
le,u
seex
ampl
es.
4.U
sere
vers
als
ofsc
ale
end
poin
tsor
ques
tion
stat
emen
ts.
5.U
sevi
sual
aids
whe
npo
ssib
le.
Exc
eptio
nsR
espo
nden
tsex
hibi
tape
rsis
tent
tend
ency
tore
spon
d1.
Rec
heck
the
ques
tions
’wor
ding
.fa
vora
bly
orun
favo
rabl
y,ha
veno
opin
ion,
orpr
ovid
e2.
Exc
lude
inco
nsis
tent
resp
onse
s.in
cons
iste
ntre
spon
ses.
3.Tr
eats
ome
resp
onde
nts
asou
tlier
s.
NO
TE
:ES
T=
e-m
ails
urve
yte
chni
que.
102
Page 11
Tabl
e2
ES
T,M
ail,
Per
sona
l,an
dTe
leph
one
Dat
aC
olle
ctio
nTe
chni
ques
Com
pare
d
ES
TM
ailQ
uest
ionn
aire
Per
sona
lInt
ervi
ewTe
leph
one
Inte
rvie
w
Sam
plin
gIs
sues
Sam
plin
gfr
ames
for
man
yS
ampl
ing
fram
efo
rm
any
Sam
plin
gfr
ames
are
usua
llyS
ampl
ing
fram
esfo
rso
me
popu
latio
nsdo
note
xist
and
popu
latio
nsex
ista
ndar
eea
sily
obta
ined
and
cons
truc
ted.
popu
latio
nsal
read
yex
ista
ndar
eus
ually
diffi
cult
toob
tain
usua
llyea
syto
obta
inan
dH
igh
pote
ntia
lfor
sam
plin
gar
eus
ually
easy
toob
tain
and
cons
truc
t.Li
mite
dpo
ssib
ility
cons
truc
t.Li
mite
dpo
ssib
ility
cont
rol.
and
cons
truc
t.H
igh
sam
plin
gof
sam
plin
gco
ntro
l.of
sam
plin
gco
ntro
l.co
ntro
l.
Cos
teffi
cien
cyLe
aste
xpen
sive
and
mos
teffi
cien
t.Lo
wco
st/m
oder
atel
yef
ficie
nt.
Hig
hco
stan
dle
aste
ffici
ent.
Hig
hest
cost
and
leas
teffi
cien
t.
Info
rmat
ion
richn
ess
Low
tran
smis
sion
ofno
nver
bal
Bas
edon
the
sam
efo
urcr
iteria
Bas
edon
the
sam
efo
urcr
iteria
Bas
edon
the
sam
efo
urcr
iteria
cues
,con
veyi
nglo
wse
nse
ofof
med
iaric
hnes
s,its
richn
ess
ofm
ediu
mric
hnes
s,it
isth
eof
med
ium
richn
ess,
itis
the
pers
onal
izat
ion,
timel
yfe
edba
ck,
isth
elo
wes
t.ric
hest
ofth
efo
urte
chni
ques
.se
cond
riche
stof
the
four
and
tran
smis
sion
ofm
ediu
mte
chni
ques
.va
ried
lang
uage
.
Res
pond
enti
ssue
sH
igh
resp
onde
ntco
nven
ienc
e—M
axim
umre
spon
dent
Low
resp
onde
ntco
nven
ienc
e—Lo
wre
spon
dent
conv
enie
nce—
time
and
disc
retio
nto
resp
ond.
conv
enie
nce—
time
and
notim
ean
ddi
scre
tion
tore
spon
d.no
time
and
disc
retio
nto
Som
ean
onym
ityis
poss
ible
.di
scre
tion
tore
spon
d.F
ull
resp
ond.
Req
uire
slit
erac
yan
dco
mpu
ter
anon
ymity
poss
ible
.Req
uire
ssk
ills;
com
patib
ility
ofe-
mai
llit
erac
y.pa
ckag
esco
uld
bea
prob
lem
.
103
Page 12
Res
pons
eou
tcom
esM
ediu
mto
high
.No
inte
rvie
wer
Med
ium
tolo
w.N
oin
terv
iew
erH
ighe
st.P
ossi
bilit
yof
inte
rvie
wer
Med
ium
tohi
gh.P
ossi
bilit
yof
dist
ortio
n,le
ssso
cial
lyde
sira
ble,
dist
ortio
n,le
ssso
cial
lydi
stor
tion
and
soci
alde
sira
bilit
y.in
terv
iew
erdi
stor
tion
and
thou
ghtfu
lres
pons
es,l
owde
sira
ble,
thou
ghtfu
lres
pons
es,
Hig
hlik
elih
ood
that
cont
amin
atio
nso
cial
desi
rabi
lity.
Hig
hlik
eli-
invo
lunt
ary
erro
r,an
dse
quen
celo
win
volu
ntar
yer
ror,
and
from
othe
rsav
oide
d.Lo
w-
hood
that
cont
amin
atio
nfr
ombi
asin
gpo
ssib
ility
.App
ropr
iate
sequ
ence
bias
ing
poss
ibili
ty.
sequ
ence
bias
ing
poss
ibili
ty.
othe
rsav
oide
d.Lo
w-
sequ
ence
for
elic
iting
sens
itive
resp
onse
s.A
ppro
pria
tefo
rel
iciti
ngC
ontr
olov
erits
spee
d.N
otbi
asin
gpo
ssib
ility
.Con
trol
over
No
cont
rolo
ver
resp
onse
spee
d.se
nsiti
vere
spon
ses.
No
cont
rol
appr
opria
tefo
rel
iciti
ngse
nsiti
veits
spee
d.Li
mite
dap
plic
a-bi
lity
Ver
yea
syto
proc
ess
and
anal
yze.
over
itssp
eed.
Eas
yto
data
.Ofte
ndi
fficu
ltto
proc
ess.
toel
icit
sens
itive
data
.Of-
ten
Sel
f-se
lect
ion
bias
poss
ibili
ty.
proc
ess
and
anal
yze.
Sel
f-Lo
wite
mno
nres
pons
e.ea
syto
proc
ess.
Low
item
Less
pron
eto
field
erro
rs.
sele
ctio
nbi
aspo
ssib
ility
.N
onre
spon
se.
Fut
ure
pros
pect
sH
igh
likel
ihoo
dof
wid
epo
pula
rity.
Dec
reas
ing
popu
larit
y,H
igh
likel
ihoo
dth
atits
usag
ew
illH
igh
likel
ihoo
dth
atits
us-
age
Priv
acy
conc
erns
may
limit
itspr
opor
tiona
teto
incr
ease
inno
tbe
influ
ence
dby
incr
easi
ngw
illre
mai
nhi
ghin
the
near
usag
e.E
ST
and
tele
phon
ein
terv
iew
ing.
use
ofe-
mai
lsur
veys
.How
ever
,fu
ture
aste
leph
one
com
-m
uni-
incr
easi
ngus
eof
com
pute
r-ca
tion
gets
chea
per.
Am
ong
assi
sted
inte
rvie
win
g,te
leph
one
high
e-m
ailu
ser
popu
la-
tions
,in
terv
iew
ing,
and
give
nvi
deo-
itsus
age
mig
htde
crea
se.
conf
eren
cing
capa
bilit
yof
com
pute
rs,i
tsus
age
amon
gaf
fluen
tpop
ulat
ions
mig
htde
crea
se.
NO
TE
:ES
T=
e-m
ails
urve
yte
chni
que.
104
Page 13
the questionnaire regardless of their location (Kiesler & Sproull, 1986). Likewise,
responses can flow back just as rapidly when respondents check their e-mail daily.
None of the existing survey techniques, including facsimiles, can provide researchers
with such speed in reaching specified individuals. Thus, EST promises to provide data
in a timely manner when a quick answer is being sought. Indeed, as Mehta and Sivadas
(1995) have suggested, EST generates fast data not only because e-mail is a fast com-
munication medium but also because individuals are likely to respond more quickly to
an e-mail, whereas in comparison, a mail questionnaire may remain on an individual’s
desk for a long time.
Likewise, EST is one of the most convenient data collection techniques. Once the
survey instrument is developed, it can be e-mailed easily. EST saves all the time that
regular postal survey requires for photocopying questionnaires, stuffing envelops, and
addressing outgoing mail. Because a copy of all outgoing e-mail can be saved in an
electronic mailbox, EST also makes repeated communications with respondents, such
as sending follow-up questionnaires, extremely easy. Given that EST allows the
researcher to know in a moment if the message has been received—and even when it is
opened—identification, elimination, and replacement of unreachable respondents are
easily accomplished. EST can also reduce many field and administration errors, such
as deciphering respondents’ handwriting and allowing researchers to encode data
without transcribing from paper.
According to information richness theory (Daft & Lengel, 1984; Daft, Lengel, &
Trevino, 1987), computer-mediated communications, such as electronic mail, are less
rich in information-carrying capacity than face-to-face communication. Within this
perspective, face-to-face interaction is seen as the richest medium, followed by tele-
phone, electronic mail, letters, and memos. In effect, e-mail offers limited interactivity
and informational cues compared to face-to-face interactions. Indeed, compared to
other surveying techniques such as personal and telephone interviewing, EST
involves low transmission of nonverbal cues, varied language, timely feedback, and
low sense of personalization.
With EST, lack of complete anonymity is also a concern. Truly anonymous
responses are not possible with EST. When a respondent returns a questionnaire using
the reply function in an e-mail package, his or her e-mail address, including the name
and affiliation, is automatically conveyed to the surveyor. This lack of anonymity
might in turn affect response rates as well as response content in EST. Moreover, if
respondents complete the survey at their place of employment, it is possible that an
electronic trail will remain and that their responses could be uncovered, which,
depending on the nature of the survey questions, could raise confidentiality issues as
well. Previous research has indicated that respondents’beliefs about anonymity affect
responses to computer-based surveys (Kantor, 1991). There is also a wide recognition
among researchers that whether anonymity is provided affects responses in mail sur-
veys (Albaum, 1987). However, if anonymity or confidentiality is a major concern, we
suggest one of two approaches: First, use a web-based survey because when respon-
dents submit their answers back to the researcher, their identifying information is not
automatically conveyed, and second, respondents could be directed to go through a
free Internet e-mail account such as “Hotmail” and, if necessary, use a fictitious name.
With respect to our second suggestion, note that it is extremely difficult, if not impossi-
105 ORGANIZATIONAL RESEARCH METHODS
Page 14
ble, for an outsider to obtain the identity of a respondent, given the privacy assurance
policies that such providers offer. Indeed, most providers indicate that the only excep-
tion to such a policy is in the case of an alleged crime. Hence, we believe that this
approach would provide a high degree of confidentiality. However, having said that,
irrespective of the actual degree of confidentiality achievable by such approaches,
EST researchers should be aware that as long as respondents perceive that complete
confidentiality is necessary and could be compromised, response rates are likely to
suffer.
Noncompatibility among e-mail packages is another important disadvantage of
EST compared to the other three techniques. E-mail is still not a standardized medium,
despite the growing demand of users for standardization. For instance, LAN e-mail
packages are different not only from one another but also from the Internet e-mail.
Although most online service providers and LAN-based e-mail systems permit trans-
fer of binary files, the Internet still uses an ASCII format. Nonstandardization among
various e-mail systems causes discrepancies between the form of questionnaire sent
and that received by respondents (Oppermann, 1995). Relatedly, researchers should
also be concerned with such issues as free hard disk space (either in their or their serv-
ice provider’s hard drive), e-mail bandwidth, and server capacity while using EST. For
instance, a large number of responses could create problems because returned e-mail
takes up a lot of space on a system. It is essential to download responses in a timely
manner while using EST for full-scale surveys.
Finally, with respect to the future prospects of EST, Bloom, Milne, and Adler
(1995) have noted that although new information technologies increase efficiency and
effectiveness of data collection, their haphazard use can lead to some legal difficulties.
For example, although legislation concerning privacy of e-mail communication is still
in its infancy, such legislation would effectively destroy the use of EST in many con-
texts. As e-mail addresses are considered to be more personal than mail addresses,
sending unsolicited e-mail questionnaires might be considered an intrusion on a per-
son’s seclusion or solitude or into his or her private affairs (Dyson, 1994). Although
such a case has yet to be made in the courts to our knowledge, the use of e-mail for
mass mailing, known as “spamming,” has been very much debated. A popular online
service was sued in three states for deceptive advertising because some of its cus-
tomers were using the system for mass mailing instead of personal messaging (Shan-
non & Rosenthal, 1993).
The e-mail overload that many individuals increasingly face is likely to be another
important disadvantage of EST (Clayton et al., 1996). Some researchers have indi-
cated that the increasing e-mail overload can cause individuals to feel overwhelmed
(Garton & Wellman, 1995). It is likely that the more individuals receive e-mail, the less
likely they are to spend time responding to EST. Although individuals do not have
many clues for judging the importance of e-mail they receive, it is very easy for them to
sort out and delete e-mail that they are not interested in. In turn, this makes it particu-
larly important that researchers successfully employ some incentives and response
inducement strategies to make respondents complete the survey.
In sum, researchers have to make a number of trade-offs when they decide to use
EST. Using EST requires harmony among a variety of resources, including human,
hardware, and software. It is thus essential that the decision to use EST takes into
account all relevant issues, including sampling issues, nonsampling errors, and com-
Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE 106
Page 15
parative performance that are discussed throughout this article. We next provide
some practical recommendations as to how EST can be used productively.
We suggest that researchers first go through relevant directories and then check the
following sources: The Usenet Addressees Database, Knowbot Information Server
(KIS), GOPHER, WAIS, The Usenet Newsgroup, and Netfind. Many of these sources
can be used to locate individuals’ e-mail addresses or names. For example, Mehta and
Sivadas (1995) wrote a program that collected e-mail addresses and signatures of the
people who posted articles on newsgroups; then, based on confirmation, they deter-
mined mailing addresses of individuals. Put simply, because some Internet users
already have organized themselves into mailing lists and discussion groups according
to their interests, it is possible to use these lists or combine several such lists to con-
struct specific sample frames. The directories seem to be especially beneficial if the
population of interest is academic staff because a growing number of institutions are
putting their staff and student directories online in publicly accessible formats and are
being incorporated into Gopher and WAIS (Kochmer & Northwest, 1993, p. 53).
It is also possible that the researcher can develop sampling frames using more con-
ventional sources such as mailing lists and phone directories and then use these
sources to determine e-mail addresses of individuals through several Internet search
engines or the aforementioned sources. In fact, some of the sources, such as Finger and
KIS, provide additional information such as telephone number, postal address, and so
on of individuals, thereby allowing the information obtained from the conventional
sources to be double-checked.
Based on our experience with e-mail surveys and the extant literature, we recom-
mend the following broad approaches. First, because many e-mail users have strong
concerns about the use of their e-mail boxes and consider them to be more private than
their mail addresses, it may prove beneficial to notify sample members about the
incoming e-mail questionnaire through an e-mail or postal prior notification (Emery,
1995, p. 344). The prior notification should not only ask for permission but also let the
respondent know the purpose of the survey, why their involvement is important, how
responses will be used, the sponsor of the survey, person(s) to contact for questions,
expected date of the survey, and a statement indicating the strict confidentiality of the
respondent’s e-mail address and response. Yu and Cooper (1983) suggest that a prior
notification should simultaneously include the following: (a) a social utility appeal
that emphasizes the worthiness of the survey, (b) an egoistic appeal that stresses the
respondent’s place and importance in completing the survey, and (c) an appeal to help
the researcher in completing an important project. Given concerns over anonymity in
107 ORGANIZATIONAL RESEARCH METHODS
Page 16
EST, it is also important that the cover letter assure respondents that their responses
will be held confidentially and mention some possible steps that will be taken toward
this goal. For example, the researcher may state that screen headers will be deleted
once the responses are received (Goree & Marszalek, 1995). The researcher could also
offer some options for responding anonymously such as placing the questionnaire on
the WWW or mentioning the possibility that the respondent could send the completed
questionnaire through regular mail.
Second, questionnaire layout and design issues should also be taken into account.
EST should be accompanied by very clear and simple instructions, such as how to
reply, that will not consume much of the respondents’ time. In particular, “extra” fea-
tures that would minimize questionnaire completion time and maximize respondent
convenience should be pursued. For example, scrolling, jump screen, quitting, no
automatic next, no keyboard responses, help screens, and a progress thermometer
indicating completed percentage of the questionnaire were incorporated and success-
fully used by Beebe, Mika, Harrison, Anderson, and Fulkerson (1997). Like many
other researchers (e.g., Johnston & Walton, 1995), we also believe that whenever pos-
sible, researchers should use simple graphics-animated questionnaires because many
people’s perceptions of computers are similar to that of TV rather than postal mail. But
we caution that such devices consume enormous amounts of memory and make open-
ing such messages time-consuming and frustrating if the individual is on an older
modem. Graphics, sounds, and special formatting of the questionnaire may not trans-
late across various e-mail software packages. To solve such problems, the researcher
could check the major e-mail systems used by respondents and make sure that the for-
matting and appearance of the questionnaire remain the same after transmission (Tse,
1998). However, unless the researcher knows the capabilities of the e-mail systems of
the people included in the survey, we suggest that it is best to keep the survey as simple
and short as possible. As an aside, it should be remembered that respondents who use
commercial online services effectively incur some cost in sending and receiving mes-
sages.
Only after incorporating such approaches should the researcher attempt to manipu-
late some incentives and factors to increase responses to EST without eschewing that
different populations may react differently to the factors, which the large body of
research on mail survey strongly points out (Childers, Pride, & Ferrell, 1980; Jobber &
Sanderson; 1985; Kaldenberg, 1994). A plethora of studies have been undertaken to
identify factors that might potentially influence responses to a mail survey, including
monetary offerings, lottery tickets, contributions to a charity, an offer of survey
results, cover letter, personalization, anonymity, topical interest, sponsorship, ques-
tionnaire design, prior notification, follow-up, humor, type of mailing, and deadline.
Excellent reviews of this body of research have been written by Church (1993); Fox,
Crask, and Kim (1988); Heberlin and Baumgartner (1978); Jobber (1986); Linsky
(1975); Veiga (1984); Yammarino et al. (1991); and Yu and Cooper (1983). Although
EST may resemble a postal survey and share some characteristics of it, not all of these
response inducement techniques are transferable to the EST context because EST has
its own unique features. For example, it is impossible to attach a monetary incentive,
such as a dollar bill, to an electronic survey or to attach a nonmonetary incentive such
as a pen.
Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE 108
Page 17
In a recent review of the literature on factors inducing responses to mail surveys,
Roth and BeVier (1998) concluded that prior notification, follow-up, monetary incen-
tives, personalization, and salience of the issues investigated have consistently been
found to positively influence response rates. Although not conclusive, a few recent
studies indicate that responses to EST can also be increased through these strategies
(e.g., Kittleson, 1997; Schaefer & Dillman, 1998). In addition to these factors, we also
expect that sponsorship might positively influence responses to EST. For example,
alumni may respond to a university or business school–sponsored surveys more read-
ily because of psychological indebtedness (Paxson, 1995). However, there is still lack
of focused research on determining how strategies such as sponsorship, the opportu-
nity to complete the survey through regular mail or the WWW, summary of the
research, donations to a charity, purchase credits, and discounts affect response rates
and response content in EST.
Although we have primarily focused on the use of EST as an alternative, independ-
ent surveying technique, EST can also be used in combination with some other tech-
niques or at different stages of a research project. In fact, it is sometimes desirable to
combine several techniques, thereby offsetting the strengths and limitations of any
single technique (Aaker et al., 1995; Lockhart & Russo, 1996). In addition, using EST
with some other techniques such as postal surveys can allow experimentation with
much more diverse populations, not only with populations having nearly universal
coverage. This strategy can reduce coverage error that is usually associated with EST
as well (Schaefer & Dillman, 1998).
EST can be used in combination with almost any data collection technique, includ-
ing telephone interviews, personal interviews, postal surveys, or the other Internet-
based surveys, as well as to send prior notifications and follow-ups. Schaefer and Dill-
man (1998) suggested that because of its cost and speed advantages, EST is ideal for a
first mode of contact in surveys, such that the researcher could begin with EST and use
progressively more expensive methods until enough responses are obtained. Or the
researcher could simply use EST among respondents having e-mail addresses and use
the postal technique to survey those without access. Likewise, one of the greatest
benefits of EST may be realized when it is used to send prior notifications and follow-
ups to increase responses to postal surveys and to EST itself.
EST can also be helpful in pretesting a survey instrument to increase the quality and
quantity of responses in a full-scale survey (Swoboda et al., 1997). The cost and speed
advantages of EST make it possible to conduct surveys aimed at establishing reliabil-
ity and validity of survey instruments. This initial process might also result in early
respondents commenting on the process of filling out the survey as well. Indeed, sev-
eral researchers have successfully used the pretest to get feedback for identifying the
optimal approach for conducting a mail survey (e.g., Hunt, Sparkman, & Wilcox,
1982).
Astudy by Clayton et al. (1996) demonstrates how EST can be used in combination
with other techniques, as well as for pilot surveying purposes. After developing the
survey instrument through two nominal group discussions, the researcher used the
EST to send the instrument for pilot survey purposes. They then sent a paper follow-up
survey to those who did not respond to the EST version. Once the survey instrument
109 ORGANIZATIONAL RESEARCH METHODS
Page 18
was fully developed, the researcher used EST and then used a final paper-based survey
among those who did not respond to the electronic version. Through this mixed usage
of EST, the researchers were able to increase reliability of the survey instrument as
well as response rates while reducing the cost of the survey.
When a representative sample can be formed, it is apparent that EST can be used in
several types of organizational studies employing a self-administered data collection
technique. It can provide a good opportunity for those researchers who have a limited
research budget or who are interested in fast data gathering. Because e-mail obliterates
time and zone constraints, surveying with e-mail can prove very beneficial when the
sample is scattered or mobile or consists of members from such populations as execu-
tives who will not participate in personal or phone interviewing but may respond to an
e-mail questionnaire at their convenience. Indeed, EST has provided researchers with
the ability to reach rare, hidden, and geographically disperse populations (O’Lear,
1996; Sell, 1997). Moreover, because e-mail addresses are personal, sending the ques-
tionnaire to the right person can be more effective via EST than a mailed questionnaire
sent to a position wherein it is not always clear who is responding or usually results in
questionnaires being thrown away before reaching the person who has the required
information. That is, an e-mail survey intended for an individual is more likely to be
read and answered by that individual (Mehta & Sivadas, 1995). Likewise, compared to
other noncomputerized surveying techniques, EST is inexpensive, fast, and less prone
to many known sources of nonsampling errors such as data collection and data proc-
essing. In addition, compared to surveys over the Internet such as newsgroup surveys,
EST is the easiest to use and has better sampling control.
On the other hand, EST’s brimming potential is at present inhibited by its lack of
universal coverage, biased sampling frames, incompatibility of current e-mail sys-
tems, restricted binary file transfer, and technicalities involved in sending and receiv-
ing questionnaires. Of them, noncoverage error arguably presents the most significant
impediment to the increased use of EST. It makes EST unsuitable for conducting sur-
veys of many populations. There are no e-mail lists for most populations that can serve
as sampling frames, and constructing them can be very difficult, costly, and time-
consuming. Even when they exist, such frames are usually biased, primarily because
e-mail users by gender, age, race, income, education, and other major demographic
characteristics are very different from their populations, in a sense creating a unique
population. In particular, when the research project involves sample surveying of het-
erogeneous populations such as households, the researcher should be extremely cau-
tious in the decision to employ EST in isolation. We believe that until local sites on the
Internet develop and maintain local e-mail lists of general populations, e-mail lists will
usually suffer from being incomplete and outdated, much like traditional mailing lists
and telephone directories.
With respect to future research, we feel that the best research will likely come from
taking an interdisciplinary focus because EST is a multifaceted phenomenon. Within
this context, we feel that each component of the assessment framework should and
could be scrutinized for development of theoretical arguments. Meanwhile, empirical
Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE 110
Page 19
research should investigate the comparative performance of EST, particularly vis-à-
vis postal surveying. So far, EST has mostly been used within organizations and edu-
cational institutions, so it will be fruitful and interesting to see findings from studies
conducted in different organizational and institutional settings. Some research must be
undertaken to explore different incentives to augment EST response rates because
with time, as the novelty of e-mail fades, reactions of computer users toward unsolic-
ited e-mail messages may become more negative. Because there is no way of knowing
whether approaches used to increase response rates are only initial stimulators, the
influences of these approaches on the quality of the data and the randomness of the
sample must be simultaneously investigated. Furthermore, we believe such issues
should be investigated through some experimentally designed studies. We urge
researchers to use a factorial experimental design to provide greater precision for esti-
mating overall variable effects, determine the interactions between the factors, and
allow the range of validity of the conclusions to be extended by the insertion of addi-
tional variables (Cox, 1992; Montgomery, 1991). In particular, fractionated-factorial
experiments allow a wide range of factors to be tested with small sample sizes (Box &
Hunter, 1961; Devor, Chang, & Sutherland, 1992). We finally urge researchers to
undertake research that focuses on EST as its primary goal rather than treating it as a
topic of secondary importance and making post hoc investigations and predictions
from project data that had another major agenda in mind. Unfortunately, many studies
that we reviewed on EST are of this type.
In sum, the rapid growth of global telecommunication networks, particularly the
Internet, has placed emphasis on EST as a surveying technique. EST is attractive
because it facilitates easy data management, location flexibility, and rapid transmis-
sion of the survey to all respondents across time and space. Yet, our review suggests
that it is too early to declare that EST has become a rival or a better technique than
major noncomputerized data collection techniques. Given current trends of rapidly
increasing e-mail availability, computer expertise, e-mail packages’compatibility, and
decreasing computer hardware and software cost, it is, however, conceivable that in
the near future, electronic surveying of many diverse populations will be possible.
Predicting this trend, several companies have recently introduced survey software
packages that work with e-mail systems to create, collect, and tabulate survey results.
In the long term, it seems that the most serious threats to EST will be legal ones and
potentially negative attitudes toward responding to a survey even in electronic format.
Aaker, D. A., Kumar, V., & Day, G. S. (1995). Marketing research. New York: John Wiley.
Albaum, G. (1987). Do source and anonymity affect mail survey results. Academy of Marketing
Science, 15, 74-81.
Anderson, S. E., & Gansender, B. M. (1995). Using electronic mail surveys and computer-
monitored data for studying computer-mediated communication systems. Social Science
Computer Review, 13, 33-46.
Ayidiya, S. A., & McClendon, J. M. (1990). Response effects in mail surveys. Public Opinion
Quarterly, 54, 229-247.
Bachmann, D., Elfrink, J., & Vazzana, G. (1996). Tracking the progress of e-mail vs. snail-mail.
Marketing Research, 8, 31-35.
111 ORGANIZATIONAL RESEARCH METHODS
Page 20
Baker, R. P. (1992). New technology in survey research: Computer-assisted personal interview-
ing (CAPI). Social Science Computer Review, 10, 145-156.
Batinic, B. (1997). How to make an Internet based survey. Advances in Statistical Software, 6,
125-132.
Beebe, T. J., Mika, T., Harrison, P. A., Anderson, R. E., & Fulkerson, J. A. (1997). Computer-
ized school surveys. Social Science Computer Review, 15, 159-169.
Bloom, P. N., Milne, G. R., & Adler, R. (1995). Avoiding misuse of new information technolo-
gies: Legal and societal considerations. Journal of Marketing, 58, 98-110.
Box, G. E., & Hunter, J. S. (1961). The 2k-p factorial design part I. Technometrics, 3, 311-351.
Chadwick, B. A., Bahr, H. M., & Albrecht, S. L. (1984). Social science research methods.
Englewood Cliffs, NJ: Prentice Hall.
Childers, T. L., Pride, W. M., & Ferrell, O. C. (1980). Areassessment of the effects of appeals on
response to mail surveys. Journal of Marketing Research, 17, 365-370.
Church, A. H. (1993). Estimating the effect of incentives on mail survey response rates: A
meta-analysis. Public Opinion Quarterly, 57, 62-79.
Churchill, G. A. (1995). Marketing research methodological foundations. New York: Dryden.
Clayton, P., Applebee, A., & Pascoe, C. (1996). E-mail surveys: Old problems with a new deliv-
ery medium. LASIE, 27, 30-39.
Comley, P. (1996). The use of the Internet as a data collection method [Online]. Available:
http://www.sga.co.uk/esomar.html.
Corman, S. R. (1990). Computerized vs. pencil and paper collection of network data. Social
Networks, 12, 375-384.
Couper, M. P., & Burt, G. (1994). Interviewer attitudes toward computer-assisted personal inter-
viewing (CAPI). Social Science Computer Review, 12, 38-54.
Couper, M. P., & Rowe, B. (1996). Evaluation of a computer-assisted self-interview component
in a computer-assisted personal interview survey. Public Opinion Quarterly, 60, 89-105.
Cox, D. R. (1992). Planning of experiments. New York: John Wiley.
Dacko, S. G. (1995). Data collection should not be manual labor. Marketing News, 29, 31-35.
Daft, R. L., & Lengel, R. H. (1984). Information richness: Anew approach to managerial behav-
ior and organizational design. Research in Organizational Behavior, 6, 191-233.
Daft, R. L., Lengel, R. H., & Trevino, L. K. (1987). Message equivocality, media selection,
and managerial performance: Implications for information systems. MIS Quarterly, 11,
355-366.
Devor, R. E., Chang, T., & Sutherland, J. W. (1992). Statistical quality design and control: Con-
temporary concepts and methods. New York: Macmillan.
Dillman, D. A. (1978). Mail and telephone surveys: The total design method. New York: John
Wiley.
Dillman, D. A. (1991). The design and administration of mail surveys. Annual Review Sociol-
ogy, 17, 225-249.
Dillon, W. R., Madden, T. J., & Firtle, N. H. (1987). Marketing research in a marketing environ-
ment. Homewood, IL: Irwin.
Dyson, E. (1994). If nothing on e-mail is private, just say so. Computerworld, 27, 22, 33.
Emery, V. (1995). How to grow your business on the internet. New York: Coriolis.
Erdos, P. L. (1983). Professional mail surveys. Malabar, FL: Krieger.
Fox, R. J., Crask, M. R., & Kim, J. (1988). Mail survey response rate: A meta analysis of se-
lected techniques for inducing response. Public Opinion Quarterly, 52, 229-247.
Gaiser, T. J. (1997). Conducting on-line focus groups. Social Science Computer Review, 15,
135-144.
Garton, L., & Wellman, B. (1995). Social impacts of electronic e-mail in organizations: A re-
view of the research literature. In B. R. Burleson (Ed.), Communication yearbook (Vol. 18,
pp. 434-452). Thousand Oaks, CA: Sage.
Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE 112
Page 21
Gates, R. H., & Jarboe, G. R. (1987). Changing trends in data acquisition for marketing research.
Journal of Data Collection, 27, 25-29.
Gilbert, G. H., Longmate, J., & Branch, L. G. (1992). Factors influencing mailed health surveys.
Public Health Reports, 107, 576-584.
Goree, C. T., & Marszalek, J. F., III. (1995). Electronic surveys: Ethical issues for researchers.
College Student Affairs Journal, 15, 75-79.
Gwyne, S. C., & Dickerson, J. F. (1997, April 21). Lost in the e-mail. Time, pp. 88-90.
Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1995). Multivariate data analysis
with readings. Englewood Cliffs, NJ: Prentice Hall.
Havice, M. J. (1990). Measuring non-responses and refusals to an electronic survey. Journalism
Quarterly, 67, 521-530.
Heberlin, T. A., & Baumgartner, R. (1978). Factors affecting response rate to mailed question-
naires: A quantitative analysis of the published literature. American Sociological Review,
3, 447-462.
Helliwell, J. (1986). Global standards promise fluency in electronic mail. PC Week, 3, 73-77.
Higgins, C. A., Dimnik, T. P., & Greenwood, H. P. (1987). The DISKQ survey method. Journal
of the Market Research Society, 29, 437-445.
Hunt, S. D., Sparkman, R. D., & Wilcox, J. (1982). The pretest in survey research: Issues and
preliminary findings. Journal of Marketing Research, 19, 269-273.
Jobber, D. (1986). Improving response rates in industrial mail surveys. Industrial Marketing
Management, 15, 183-195.
Jobber, D., & Sanderson, S. M. (1985). Evaluating the effectiveness of two variables on indus-
trial mail survey response rates. Industrial Marketing Management, 14, 119-121.
Johnston, J., & Walton, C. (1995). Reducing response effects for sensitive questions: A
computer-assisted self interview with audio. Social Science Computer Review, 13,
304-319.
Kaldenberg, D. O. (1994). Mail survey response patterns in a population of elderly: Does re-
sponse deteriorate with age? Public Opinion Quarterly, 58, 68-76.
Kalton, G. (1983). Compensating for missing survey data. Ann Arbor: University of Michigan
Press.
Kambil, A. (1995). Electronic commerce: Implications of the Internet for business practice and
strategy. Business Economics, 30, 27-32.
Kantor, J. (1991). The effects of computer administration and identification on the Job Descrip-
tive Index (JDI). Journal of Business & Psychology, 5, 309-323.
Karweit, N., & Meyers, E. D. (1983). Computers in survey research. In P. H. Rossi, J. D. Wright, &
A. B. Anderson (Eds.), Handbook of survey research (pp. 379-413). New York: Academic
Press.
Kerr, E. B., & Hiltz, S. R. (1982). Computer-mediated communication systems. New York:
Academy Press.
Kiesler, S. (1989). E-mail brings out the best and worst in group interaction. Infoworld, 4-9.
Kiesler, S., & Sproull, L. S. (1986). Response effects in the electronic survey. Public Opinion
Quarterly, 50, 402-413.
Kiesler, S., Zubrow, D., & Moses, A. M. (1985). Affect in computer-mediated communication:
An experiment in synchronous terminal-to-terminal discussion. Human-Computer Inter-
action, 1, 77-104.
Kittleson, M. J. (1995). An assessment of the response rate via the postal service and e-mail.
Health Values, 18, 27-29.
Kittleson, M. J. (1997). Determining effective follow-up of e-mail surveys. American Journal
of Health Behavior, 21, 193-196.
Kochmer, J., & Northwest. (1993). Northwestnet’s guide to our world Internet (4th ed.). Wash-
ington, DC: Northwest Academic Computing Consortium.
113 ORGANIZATIONAL RESEARCH METHODS
Page 22
Komsky, S. H. (1991). A profile of users of electronic mail in a university. Management Com-
munication Quarterly, 4, 310-340.
Lavrakas, P. C. (1996). To err is human. Marketing Research, 8, 30-36.
Leeuw, E. D., Hox, J. J., & Snijkers, G. (1995). The effect of computer-assisted interviewing on
data quality: A review. Journal of the Market Research Society, 37, 325-335.
Liefeld, J. P. (1988). Response effects in computer-administered questioning. Journal of Mar-
keting Research, 25, 405-409.
Linsky, A. S. (1975). Stimulating responses to mailed questionnaires: Areview. Public Opinion
Quarterly, 3, 82-101.
Lockhart, D. C., & Russo, J. R. (1996). Mail and telephone surveys in marketing research: A
perspective from the field. In R. P. Bagozzi (Ed.), Principles of marketing research (pp. 117-
161). Cambridge, UK: Blackwell.
Malhotra, N. K. (1993). Marketing research: An applied orientation. Englewood Cliffs, NJ:
Prentice Hall.
Markus, M. L. (1994). Electronic mail as the medium of managerial choice. Organization Sci-
ence, 5, 502-527.
Mehta, R., & Sivadas, E. (1995). Comparing response rates and response content in mail versus
electronic mail surveys. Journal of the Market Research Society, 37, 429-439.
Montgomery, D. C. (1991). Design and analysis of experiments. New York: John Wiley.
Mortensen, E. (1985). Electronic mail: An ever changing technology. The Office, 102, 101-103.
Neal, W. D. (1989). Computering and software in marketing research: The US perspective.
Journal of the Market Research Society, 31, 567-576.
O’Lear, S.R.M. (1996). Using electronic mail (e-mail) surveys for geographic research: Les-
sons from a survey of Russian environmentalists. Professional Geographer, 48, 209-217.
Oppermann, M. (1995). E-mail surveys: Potential and pitfalls. Marketing Research: A Maga-
zine of Management & Applications, 7, 29-33.
Parker, L. (1992). Collecting data the e-mail way. Training & Development, 46, 52-54.
Parker, T. (1995). How internet e-mail works. In M. A. Pike (Ed.), Using the Internet (pp. 283-322).
Indianapolis, IN: Que.
Paxson, M. C. (1995). Increasing survey response rates: Practical instructions from the total-
design method. Cornell Hotel & Restaurant Administration Quarterly, 36, 66-74.
Perl, Y., & Whitley, E. (1995, December). The environmental contribution of personal comput-
ers: A state of the art report. Computers and Society, pp. 6-8.
Rice, R. E. (1990). Computer-mediated communication system network data: Theoretical
concerns and empirical examples. International Journal of Man-Machine Studies, 32,
627-647.
Rossi, P. H., Wright, J. D., & Anderson, A. B. (1983). Sample surveys: History, current practice,
and future prospects. In P. H. Rossi, J. D. Wright, & A. B. Anderson (Eds.), Handbook of
survey research (pp. 1-20). New York: Academic Press.
Roth, P. L., & BeVier, C. A. (1998). Response rates in HRM/OB survey research: Norms and
correlates, 1990-1994. Journal of Management, 24, 97-118.
Salant, P., & Dillman, D. A. (1994). How to conduct your own survey. New York: John Wiley.
Saltzman, A. (1995). Improving response rates in disk-by-mail surveys. Marketing Research: A
Magazine of Management & Applications, 5, 32-39.
Schaefer, D. R., & Dillman, D. A. (1998). Development of standard e-mail methodology: Re-
sults of an experiment. Public Opinion Quarterly, 62, 378-386.
Schuldt, B. H., & Totten, J. W. (1994). Electronic mail vs. mail survey response rate. Marketing
Research: A Magazine of Management & Applications, 6, 36-44.
Sell, R. L. (1997). Research and the Internet: An e-mail survey of sexual orientation. American
Journal of Public Health, 2, 297.
Shannon, J. H., & Rosenthal, D. A. (1993). Electronic mail and privacy: Can the conflicts be re-
solved? Business Forum, 18, 31-42.
Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE 114
Page 23
Sproull, L. S. (1986). Using electronic mail for data collection in organizational research. Acad-
emy of Management Journal, 29, 159-169.
Stanton, J. M. (1998). An empirical assessment of data collection using the Internet. Personnel
Psychology, 51, 709-725.
Subramanian, A. K., McAfee, A. T., & Getzinger, J. P. (1997). Use of the World Wide Web for
multisite data collection. Academic Emergency Medicine, 4, 811-817.
Sudman, S. (1996). Sampling. In R. P. Bagozzi (Ed.), Principles of marketing research (pp. 73-
115). Cambridge, UK: Blackwell.
Swoboda, W. J., Muhlberger, N., Weitkunat, R., & Scheneeweib, S. (1997). Internet surveys by
direct mailing: An innovative way of collecting data. Social Science Computer Review, 15,
242-255.
Synodinos, N. E., & Brennan, J. M. (1988). Computer active interviewing in survey research.
Psychology & Marketing, 5, 117-137.
Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: The impact of data collec-
tion mode, question format, and question context. Public Opinion Quarterly, 60, 275-304.
Tse, A.C.B. (1998). Comparing the response rate, response speed and response quality of tow
methods of sending questionnaires: E-mail vs. mail. Journal of the Market Research Soci-
ety, 40, 353-361.
Tull, D. S., & Hawkins, D. I. (1993). Marketing research: Measurement & method. New York:
Macmillan.
Vazzana, G. S., & Bachmann, D. (1994). Fax attracts. Marketing Research: AMagazine of Man-
agement & Applications, 6, 19-25.
Veiga, J. F. (1984). Getting the mail questionnaire returned: Some practical research considera-
tions. Journal of Applied Psychology, 59, 217-238.
Walsh, J. P., Kiesler, S., Sproull, L. S., & Hesse, B. W. (1992). Self-selected and randomly se-
lected respondents in a computer network survey. Public Opinion Quarterly, 56, 241-244.
Watson, S. C. (1998). Aprimer in survey research. Journal of Continuing Higher Education, 46,
31-40.
Weisband, S., & Kiesler, S. (1996). Self disclosure on computer forms: Meta analysis and impli-
cations. Human Factors in Computing Systems Conference Proceedings, 3-10.
Yammarino, F. J., Skinner, S. J., & Childers, T. L. (1991). Understanding mail survey response
behavior: A meta-analysis. Public Opinion Quarterly, 55, 613-639.
Yu, J., & Cooper, H. (1983). Aquantitative review of research design effects on response rates to
questionnaires. Journal of Marketing Research, 20, 36-44.
Zikmund, W. G. (1994). Business research methods. New York: Harcourt Brace.
Zeki Simsek is a graduate student in the School of Business Administration, University of Connecticut. His
research interests include new data collection techniques, intraorganizational networks, and interorganiza-
tional relations.
John F. “Jack” Veiga (DBA Kent State) is the Airbus Industrie International Scholar and head of the Man-
agement Department, School of Business Administration, University of Connecticut. His current research
interests include technology acceptance, cross-cultural behavior, and top management teams.
115 ORGANIZATIONAL RESEARCH METHODS