7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
1/21
Negative Psychological Effects of SelectionMethods: Construct Formulation and an
Empirical Investigation into an Assessment Center
Neil Anderson*University of Amsterdam
Vicki GoltsiMediterranean College, Athens
This paper defines and formulates the construct of negative psychological effects (NPEs)of selection and assessment methods upon applicants. The results of an empirical studyinto NPEs in an operational assessment center are also reported. Applicants (n5107)completed measures at three timepoints: Time 1 (just before the AC), Time 2 (immediatelyafter the AC but before outcome decisions were known), and Time 3 (6 months after theAC). Both accepted and rejected candidates completed all three measures, which included
self-esteem, mental health, positive and negative affect, and career exploration behavior.No evidence of NPEs was found for rejected candidates despite significant between-groupdifferences on feedback reactions level items. Interestingly, well-being and positive affectdeclined slightly for successful candidates at subsequent measurement compared againstbaseline Time 1 norms. Reactions level outcomes did, however, differ significantlybetween accepted and rejected candidates, with the latter rating feedback dimensions farless favorably. These results are discussed in relation to future research into thepsychological impact of selection procedures upon candidates and the need for appliedpsychologists and HR practitioners to demonstrate that assessment methods do notexhibit NPEs upon applicants.
E xamination of applicant reactions to selectionmethods has been a rapidly burgeoning area ofresearch interest over recent years, whose findings have
implications for the practical design of assessment and
selection procedures by organizations. Several comprehen-
sive reviews of this research conducted in different
countries internationally have been published and are
now available (e.g., Anderson, Born, & Cunningham-
Snell, 2001; Chan & Schmitt, 2004; Imus & Ryan, 2005;
Ryan & Ployhart, 2000; Polyhart & Harold, 2004;
Schmitt & Chan, 1999). However, much of this researchhas concerned immediate-level reactions and preference
perceptions of applicants to different predictors, leaving
open to question the longer-term impacts and outcomes of
applicant exposure to specific selection methods (Bauer,
Maertz, Dolen, & Campion, 1998; Gilliland & Steiner,
2001). The unequivocal consensus across this body of
findings is that applicants express clear preferences for
job-relevant, transparent, and equitably administered
methods, but that this proliferation of studies has
failed to address centrally important questions of the
longer-term impact of selection methods upon candidate
decision making, attitudes to the organization, or indeed,
important issues of applicant psychological health and
well-being.
Reactions level research, while important and having
obvious implications for the design and conduct of
organizational selection procedure, can clearly only shedlight upon perceptual responses and preference evaluations
by applicants. As noted by Iles and Robertson (1997)
and Anderson (2005), this level of outcome research
can only present a picture of applicant responses that
relates to immediate-level reactive responses. The trigger
effects of these reactions to candidate decision making
in selection remain a pointedly under-researched area,
as do important questions over the impacts, both pragmatic
and psychological, of selection methods upon applicants.
The focus of the current paper is on the latter the negative
*Address for correspondence: Neil Anderson, University of Amster-
dam Business School, Roeterstraat 11, 1018 WB Amsterdam, The
Netherlands. E-mail: [email protected]
INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT VOLUME 14 NUMBER 3 SEPTEMBER 2006
236r 2006 The AuthorsJournal compilation r 2006 Blackwell Publishing Ltd, 9600 Garsington Road,
Oxford, OX4 2DQ, UK and 350 Main St, Malden, MA 02148, USA
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
2/21
(and positive) psychological effects upon applicants of
undergoing a managerial assessment center.
This paper has two aims: (i) to provide an operational
definition and conceptual formulation for the construct of
negative psychological effects (NPEs) caused by expo-
sure to selection methods, and (ii) to conduct an
exploratory, longitudinal study into the potential NPEs of
a managerial entry-level assessment center. We also aim to
clarify the diffuse and widening conceptual space coveredby studies into applicant reactions by arguing that our
understanding of these outcomes will only be improved by
definitional clarity of the outcome variables under scrutiny.
Finally, from a methodological point of view, the results
from the empirical study reported here present novel and
disconfirmatory findings that exposure to a multi-day
assessment center caused no discernible NPEs among
rejected candidates. Effect size statistics are reported,
showing both a notable lack of NPEs over time upon
unsuccessful candidates, but also an intriguing lack of
improvement in self-esteem, psychological well-being, and
positive affect among successful applicants.
NPEs and Positive Psychological Effects(PPEs) of Selection Methods
Toward a Working Definition of NPEs
The concept of NPEs was first proposed by Anderson
(2004). Although emphasizing the importance of research
into the possibility that selection methods may have an
unintentionally negative impact upon rejected candidates,
the construct was not originally formulated in detail or a
working definition proposed. Here, we rectify this by
proposing that NPEs are most appropriately defined asfollows:
Declines in applicant psychological well-being, general
mental health, or core self-esteem that are inveterate,
measureable, and statistically demonstrable, and that occur
as a result of exposure to rejection decisions, violations of
applicant rights, or unprofessional feedback given to
applicants by recruiters, at any stage during organizational
selection or promotion assessment procedures.
There are four main aspects of this definition. First, in
accordance with the wider literatures in IWO psychology
on mental health at work, stress, and job-related burnout,NPEs relate to longer-term declines in important aspects of
applicant mental health, psychological well-being, or core
self-esteem. Why these referent constructs? All are centrally
important aspects of the overall well-being of the indivi-
dual and can be argued to be fundamental concepts
underlying mental health (e.g., Judge & Bono, 2001; Hart
& Cooper, 2002). All are longstanding constructs of
research interest in applied psychology, with literally
thousands of primary studies and many meta-analyses
having been published into their antecedent causes and
consequent effects. In accordance with Judge and Bono
(2001), we therefore view NPEs as being at the interface of
applicant mental health and core self-esteem, with NPE
being a composite construct that encapsulates mental
health and self-esteem outcomes relevant to applicant
psychological well-being in its widest sense.
Second, NPEs relate to causal effects that are inveterate,
measureable, and statistically demonstrable. The Shorter
Oxford English Dictionary defines the term inveterate as . . .firmly established by long continuance; deep-rooted;
obstinate. Thus, we define NPEs as relating to longer-term
effects that are robust and deep-rooted, but of course not
necessarily either permanent in their duration or debilitating
in their effects sufficient to satisfy Diagnostic and statistical
manual of mental disorders-Fourth Edition (DSM-IV)
criteria for on-going personality disorders (American Psy-
chiatric Association, 1994). This acuity level is of course
rather debateable. That is, the level of impact needed to
satisfy the definition could be variously argued to be much
lower (e.g., a short term and minor drop in mood states or
positiveness), or much higher (e.g., acutely negative anddispositional changes that are relatively permanent). Rather,
in common with Fletcher (1991), Schuler and Fruhner
(1993), and Iles and Robertson (1997), we envisage NPEs
as being present for several weeks and months after receipt of
a negative selection decision. Thus, such declines in well-
being and/or core self-esteem need to be measureable, that is,
to register on widely accepted and psychometrically devel-
oped instruments, and to be statistically demonstrable. Both
of these requirements need to be fulfilled for NPEs against an
organizations selection procedure to be demonstrated prima
facie, even before other definitional criteria that we propose
can be evaluated (specifically, causality). It can also be stated
that NPEs need to be distinguishable from immediate levelreactions, preference ratings, and applicant reactions to
feedback (see Iles & Robertson, 1997; Robertson, Iles,
Gratton, & Sharpley, 1991; Ryan & Ployhart, 2000). The
latter are short-term applicant perceptual reactions to
selection methods; NPEs refer to longer-term psychological
impacts that result from being exposed to such methods.
Indeed, distinguishingthese two levelsof analysis in applicant
reactions research immediate-level reactions and longer-
term NPEs is a key contribution of the present paper.
Third, we propose that although NPEs are more likely,
as a result, of receiving reject outcome decisions at any
stage during recruitment and selection procedures, they canalso occur where the candidate is subsequently accepted
but has been treated in demonstrably unprofessional or
unambiguously abusive ways by recruiters (Derous, Born,
& De Witte, 2004). Here, it is possible that even accepted
candidates could suffer NPEs if the procedural treatment of
the organization has been so extreme, so biased, or so
personally abusive as to trigger such perceptions of
procedural or distributive injustice or deeper-rooted
psychological impacts (Gilliland & Steiner, 2001). Anec-
dotal evidence of poor treatment in selection situations
NPES OF ASSESSMENT CENTERS 237
r 2006 The AuthorsJournal compilationr Blackwell Publishing Ltd. 2006 Volume 14 Number 3 September 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
3/21
abounds, so we purposely include the possibility in our
operational definition that NPEs will not necessarily be
restricted to rejected applicants alone.
Finally, for NPEs to hold up as identifiable outcomes
from exposure to selection predictors, there is an onus of
proof of causality of exposure to the selection method itself
as the primary cause. Threats to causal inference need to be
examined and measured in that rejected applicants could
also have been exposed to other extraneous life events overthe time period of interest. Although this definition
acknowledges that NPEs can occur at any stage in
recruitment and selection, it is most likely that they will
emerge from methods that are of a longer duration, that
involve applicant preparation, that are job-relevant and
face valid, and that the results from which cannot easily be
externally attributed by applicants (Anderson, 2004; Chan
& Schmitt, 2004). Such effects are entirely plausible in
external selection procedures, but perhaps even more so in
internal assessment procedures for promotion where both
accepted and rejected applicants remain in the employ of
the organization (Fletcher, 1991). This raises the prospectthat NPEs can have knock-on effects to job performance,
both in the case of unsuccessful internal candidates who
remain in their existing job function and for successful
applicants who are given inappropriately negative feed-
back or who feel that their expectations of procedural
justice have in some way been violated. Thus, NPEs may
ultimately have considerable and quantifiable costs for an
organization in terms of poorer levels of job performance,
that is, NPEs will have a substantial and negative utility
value, whose exact costs to an organization may remain
unknown and unrecognized.
Toward a Working Definition of PositivePsychological Effects (PPEs)
Counter to these arguments, it is feasible to suggest that far
more positive effects might be observed where applicants
obtain successful outcome decisions or where candidates
receive positive feedback after taking part in a particular
selection method or procedure. Here, the opposite of NPEs
that can be hypothesized may occur, that is, PPEs. As a
direct, polemic opposite of NPEs, PPEs can be defined as
follows:
Increases in applicant psychological well-being, general
mental health, or core self-esteem that are inveterate,
measureable, and statistically demonstrable, and that occur
as a result of exposure to acceptance decisions, perceived
respecting of applicant rights, or complementary feedback
given to applicants by recruiters, at any stage during
organizational selection or promotion assessment proce-
dures.
Again, this definition supposes that such changes need
to be relatively enduring and measurable through typical
pre- and post-study designs. Moreover, it is likely that
perhaps such changes in psychological well-being and
mental health will be more commonly determined than
changes in core self-esteem as a dispositional, relatively
stable personality characteristic (e.g., Judge & Bono,
2001). Nevertheless, it is possible that where an applicant
is going through several selection procedures concurrently
(as is often the case), where applicants know others who
have been unsuccessful, and where they receive multiple
positive decisions and/or feedback comments on theircapabilities, that this results in enhanced core self-esteem at
least temporarily. The prospect for PPEs for successful
candidates is therefore noted and this tentative definition
advanced as an opposite set of outcome conditions to those
proposed for NPEs.
To summarize, these working definitions of NPEs and
PPEs are proposed to guide and provoke future research
into this neglected topic area. Both are intended to build
upon and extend earlier work by Anderson (2004) and to
provide a basic conceptual formulation for psychological
effect research and, importantly, to distinguish this level of
analysis from that of earlier research into applicantreactions and positiveness ratings of selection method
and feedback perceptions favorability.
Past Research into NPEs in Selection
Appendix A provides a summary of published studies into
applicant NPEs conducted since 1989 onwards. Studies
were identified from a comprehensive search and review of
the literature undertaken by the present authors. For the
sake of brevity, we used several rules of thumb to decide
whether or not a study should be included in this appendix.
Initially, the study must meet key aspects of our definitionof NPEs proposed above, that is, it must have included
outcome variables that relate to applicant mental health,
well-being, or core self-esteem. Studies that for instance
examined applicant reactions to selection methods alone
(e.g., Horvath, Ryan, & Stierwalt, 2000; Lievens, De
Corte, & Brysse, 2003) were not included in this table (for
such a comprehensive table covering reactions-level
studies, see Ryan & Ployhart, 2000). Furthermore, we
only included studies in this appendix if they examined
effects longitudinally during selection procedures, not just
immediate pre- and post-test effects of reactions to
cognitive ability tests and other measures (e.g., Macan,Avedon, Paese, & Smith, 1994). In this regard, past studies
have often measured key variables using such pre- and
post-method designs, but with notably few studies in-
corporating a longer-term outcome measurement time-
period some weeks or months following selection
decisions. Finally, we include studies relating to different
selection methods (not just assessment centers) in Appen-
dix A on the grounds that relatively little research has been
conducted across all predictors to date, and so a
comprehensive summary is warranted.
238 NEIL ANDERSON AND VICKI GOLTSI
International Journal of Selection and Assessmentr 2006 The Authors
Journal compilationr Blackwell Publishing Ltd. 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
4/21
Overviewing this existing body of research and counter
to what might be expected, there are relatively few studies
that have so far examined aspects of psychological impact
caused by selection procedures. This appendix summarizes
the findings of the fourteen published studies we located
over this 16-year period (19892004, inclusive), and covers
all predictor methods, not just assessment centers. Far less
research has examined NPEs then than might have been
anticipated, even in comparison with the growth of studiesinto applicant reactions over recent years (Salgado, 1999;
Salgado, Viswesvaran, & Ones, 2002). Despite the relative
paucity of empirical studies, more research has been
conducted very recently (five studies were published in
2004 alone: Derous, Born, & De Witte; Bauer, Truxillo,
Paronto, Campion, & Weekley; van Vianen, Taris,
Scholten, & Schinkel; Schmitt, Oswald, Kim, Gillespie,
& Ramsay; and, Schinkel, van Dierendonck, & Anderson),
and important findings emerge from this emergent stream
of research. A long-established focus of interest has been
the impact of cognitive ability and GMA testing upon
applicants (Gilliland, 1994; Lounsbury, Bobrow, & Jensen,1989; Ployhart & Ryan, 1997; Ployhart & Ryan, 1998;
Ployhart, Ryan, & Bennett, 1999; Schinkel et al., 2004;
Schmit & Ryan, 1992; Smither, Reilly, Millsap, Pearlman,
& Stoffey, 1993; van Vianen et al., 2004). Consistent race
differences in attitudes to GMA testing, links between
cognitive tests and favorable attitudes toward the organi-
zations selection procedure, recommendation and offer
acceptance intentions by applicants, and impacts upon self-
assessed performance, fairness, anxiety, and candidate
affective well-being have all been reported across these
studies. Notably fewer studies could be located that have
examined the psychological impacts of several other
popular selection methods, however, including structuredinterviews, personality inventories, work sample tests, or
situational judgment tests.
Assessment centers, despite their increasing popularity as
final-stage predictors for selection in several countries (e.g.,
Lievens & Klimoski, 2001), have similarly received scant
research attention into their potential for NPEs. Only two
peer-reviewed studies could be located that have examined
such longer-term psychological impacts upon actual job
applicants, each with directly contradictory findings (see also
Schuler & Fruhner, 1993). Robertson etal. (1991) conducted
a cross-sectional study into the impacts of three methods
biodata, situational interviews, and assessment centers atdiffering stages in the selection procedure. They found no
significant effects for assessment center experience or
outcome decision upon subsequent measures of applicant
self-esteem or psychological well-being. Fletcher (1991),
conversely, undertook theonly truly longitudinal study of AC
psychological impact, measuring applicant responses im-
mediately before, immediately after, and 6 months following
the AC itself. Significant declines in rejected applicants job
involvement and mood at work were reported, while
significant increases in successful applicants work ethic,
acquisitiveness, status aspiration, and job mastery were
found. Fletcher alsoreports significant maineffectdifferences
at Time 3 measures between successful and unsuccessful
applicants on measures of organizational commitment, need
for achievement, and psychological well-being. Note that this
AC was for an internal promotion procedure within a single
financial services organization where the unsuccessful
candidates continued to work alongside those who were
successful, and therefore, subsequently promoted. Despitethe laudible longitudinal element of this study design, the
sample was only 57 respondents,40 of whom were successful
and only 17 of whom were rejected for an accelerated
promotion program, an unusual ratio in comparison with
those normally found in assessment centers for selection.
Given that so few studies have been conducted into the
longer-term impact of ACs upon candidates, our under-
standing of these potential effects is at best inadequate. As
Fletcher (1991) correctly argues, exposure to a multi-day,
job-relevant, assessment procedure that is evaluated by
trained assessors who themselves are perceived as success-
ful organizational members, and where often quite detailedfeedback is given to candidates all serve as factors to
heighten the potential for NPEs against rejected applicants.
Likewise, we may hypothesize that successful completion
of such an interventive and psychologically demanding
method may well result in improvements in candidate
psychological well-being and self-esteem in having com-
peted successfully against many more candidates who
were subsequently rejected: such effects might be contrarily
termed Postitive Psychological Effects, or PPEs. Whether
this is the case or not, operational ACs for selection and
internal promotion present a critical case opportunity in
which to conduct research into such effects.
Method
Sample
A sample of 107 candidates at final stage assessment center
participated in the study. The candidates applied for a
highly prestigeous managerial-entry and graduate training
scheme within a leading multinational organization head-
quartered in the United Kingdom. The training scheme
lasted 3 years in total and included fixed-term postings to
various sectors of the business, all of which involvedperiodic attendance at trainee managerial courses and
leadership development events. Of this total of 107
participants, 53 (49.5%) were female and 54 (50.5%)
were male. This proportion of virtually equal males and
females at AC reflects the initial proportions of applicants
to this organizations selection procedure. Candidate age
ranged from 21 to 37, with a mean of 23.4 (SD52.6).
Fifty-seven percent of participants reported having past
experience of attending an assessment center, while the
modal figure for numbers of past ACs experienced was 2.
NPES OF ASSESSMENT CENTERS 239
r 2006 The AuthorsJournal compilationr Blackwell Publishing Ltd. 2006 Volume 14 Number 3 September 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
5/21
Of the 107 applicants, 28 were successful (26.2%) whereas
79 (73.8%) were rejected following completion of this AC.
Recruitment and Prescreening Procedure
Following advertisements in the national press, applicants
responded by standardized application form. Applicants
were pre-screened on the basis of university qualifications,
expected gradings, outside interests and positions ofresponsibility, and responses to open-ended questions as
to their reasons for applying. Short-listed applicants
subsequently underwent a 30-min structured interview by
telephone. Successful applicants following the structured
interview were invited to the 2-day assessment center.
Assessment Center
The assessment center took place over 2 days and involved
an intensive process of multi-rater observations of three
exercises targeting nine dimensions. Table 1 sets out the AC
exercise-dimension matrix in overview. In addition to thethree exercises (Analysis Report, Group Task, and Pre-
sentation), candidates gave verbal and numerical ability
tests and also completed an unstructured interview with
three assessors toward the end of the entire AC. The nine
dimensions evaluated were Analysis, Communication,
Planning and Organizing, Influencing Others, Team Work,
Leadership, Innovationand Change, Decision Making, and
Personal Effectiveness.
Procedure
A three-measurement timepoint, longitudinal design wasused. Time 1 questionnaires were administered immedi-
ately before the start of the AC while candidates were being
welcomed and briefed. Time 2 questionnaires were
completed immediately after the AC had finished. Note,
importantly, that outcome decisions from the AC were not
yet known by participants, this timing being essential to
remove any potential biasing effects of outcome decisions
upon their immediate reactions. Time 3 measures were
taken for both successful and unsuccessful candidates 6
months after the completion of the AC and once outcome
decisions were known by all participants. Successful
candidates were followed up at this stage within the
organization as all had commenced employment by thisstage. All 28 candidates to whom offers were made
accepted employment as this was a highly attractive
managerial training scheme. For rejected candidates, the
second author followed up responses using a postal
questionnaire to their home addresses.
Measures
In addition to biographical variables measured at Time 1
only (age, qualifications, ethnic minority status, and so
forth), measures were completed at all three timepoints
by participants. The questionnaire contained scales evalu-ating self-esteem, psychological well-being (the General
Health Questionnaire (GHQ) short-form version), and
positive and negative affect. Additionally, feedback
perceptions were evaluated only at Time 3 after the
organization had been able to give applicants formal
feedback on their performance, and career exploration
behavior was measured for unsuccessful and successful
applicants at Time 1 and Time 3 only. The details of
measures are as follows:
Self-Esteem. Rosenbergs (1989) standarized scale ofself-esteem was used. For the present sample, internal
consistencies (Cronbachs a) ranged from .77.88, and
testretest reliability was .82 (Time 1 to Time 2) and .85(Time 2 to Time 3).
Psychological Well-Being. The short-form versionGHQ (GHQ-12) was used to evaluate psyhological health
(Goldberg, 1992). Designed to detect non-psychotic
disorders in normal adult populations, this measure has
been widely used as a valid and reliable scale for measuring
key aspects of psychological health (Goldberg, 1992). The
present sample as ranged from .82 to .90; retest reliabilities
were above .70 between both measurement timepoints.
Note that as a measure of clinical disorder, higher scores on
the GHQ were originally scaled to indicate a higher
probability of disorder and thus indicate lower levels ofpsychological well-being. Scores were reverse coded in the
present study in order to simplify interpretation of
subsequent analyses such that higher scores indicate more
positive psychological well-being.
Positive and Negative Affect. The Positive and Nega-tive Affect Schedule (PANAS) was used (Watson, Clark, &
Tellegen, 1988). Consisting of 20 adjectives, this brief
schedule measures the two dimensions of positive and
negative affect. Ten adjectives describe negative moods
(distressed, upset, guilty, scared, hostile, irritable, ashamed,
Table 1. The assessment center matrix
Dimensions
Exercises
Analysis
report
Group
task Presentation
Analysis * *Communication * *
Planning and organizing * *
Influencing others * *
Team work *
Leadership *
Innovation and change * *
Decision making * *
Personal effectiveness * * *
Note: *Rated dimension by exercise.
240 NEIL ANDERSON AND VICKI GOLTSI
International Journal of Selection and Assessmentr 2006 The Authors
Journal compilationr Blackwell Publishing Ltd. 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
6/21
nervous, jittery, afraid), while 10 adjectives describe
positive moods (interested, excited, strong, enthusiastic,
proud, alert, inspired, determined, attentive, active).
Reliabilities (as) for positive affect (PA) and negative
affect (NA) were a5 .89 and .85, respectively. The measure
was sensitive to changes over time in the current
study: testretest for PA was only .68 (Time 1 to Time 2)
and .54 (Time 2 to Time 3), and for NA this was only .71
(Time 1 to Time 2) and .45 (Time 2 to Time 3), indicating
that there were changes in respondents self-rated affect
over time.
Feedback Perceptions. Feedback perceptions were
measured with an eight-item scale derived from Dodd(1977) at Time 3 only. Cronbachs a was .88.
Career Exploration Behavior. The Career ExplorationSurvey (CES) by Stumpf, Colarelli, and Hartman (1983)
was used. Five of the CES scales were used: (1) environ-
mental exploration (a5 .83, e.g., I went to various career
orientation programmes), (2) self-exploration (a5 .88,
e.g., I have been retrospective in thinking about my
career), (3) intended/systematic exploration (a5 .74, e.g.,
I tried specific work roles to see if I like them), (4)
amount of information obtained (a5 .79, e.g., I currently
have a moderate amount of information on how Ill fit into
various career paths), and (5) Focus (a5
.86, e.g., Howsure are you that you know the type of organization you
want to work for?).
Results
Pre-Analysis Checks
Initial comparisons of biographical variables between the
accepted and rejected candidate groups were carried out.
Independent sample t-tests at Time 1 revealed no sig-
nificant differences on either gender (t51.34, p5NS), or
marital status (t5 .36, p5NS), but a significant
difference favoring younger applicants in age (t5 3.46,
po.001). Comparisons on other biographical variables
(number of assessment centers previously attended, educa-
tion level, and degree type) revealed non-significant
differences between the accepted and rejected applicants,
indicating that these sub-groups were essentially equivalent
and were not prone to underlying differences in composi-
tion, which could have confounded differences found on
study measures of mental health and career exploration
behavior.
Descriptive Statistics and Intercorrelations
Table 2 reports descriptive statistics, intercorrelations, and
reliability coefficients for all study variables at Time 1.
Cronbachs as are all acceptable at above .80. Overviewing
this correlation matrix, note that self-esteem correlates
positively and significantly with psychological well-being
(r5.20, po.05) and with positive affect (r5.48, po.01),
but negatively and significantly with negative affect
(r5 .22, po.05). The highest correlations were found
between positive and negative affect (r5 .66, po.01),
and interestingly between positive affect and careerexploration behavior (r5.54, po.01).
Longitudinal Differences in Accepted Vs. RejectedCandidates
Table 3 reports means, standard deviations, t-test values,
and effect size statistics for the key outcome variables of
mental health and career exploration behavior. Cohens d
(Cohen, 1988) was used to examine sub-group differences
on these variables between the rejected (i.e., majority) and
Table 2. Descriptive statistics and intercorrelations for Time 1 study variables
Mean SD
Intercorrelations
1 2 3 4 5 6 7 8 9
1. Age 23.02 2.46
2. Gender 1.50 .50 .00
3. Marital status 1.05 .22 .40** .04
4. Self-esteem 16.94 3.64 .04 .10
.13 .885. Psychological well-being: GHQ-12 10.71 4.64 .10 .16 .10 .20* .84
6. Positive affect 36.15 5.10 .09 .04 .14 .48** .32** .89
7. Negative affect 17.12 4.00 .02 .11 .02 .22* .32** .66** .85
8. Career exploration behavior 77.13 12.99 .13 .03 .10 .16 .02 .54** .02 .82
9. AC outcome decision 1.74 .44 .23* .13 .03 .11 .18 .10 .09 .18
Notes: Values on the diagonal are Cronbachs a coefficients.
GHQ, General Health Questionnaire.*po.05,**po.01, two-tailed tests.
NPES OF ASSESSMENT CENTERS 241
r 2006 The AuthorsJournal compilationr Blackwell Publishing Ltd. 2006 Volume 14 Number 3 September 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
7/21
Ta
ble3
.Su
b-g
roup
differenc
es
inmen
talhea
lthover
time:
accep
tedvs
.re
jec
tedapp
lican
ts
Time1
Time2
Time3
Time1
Time3
Accepted
Rejected
Accepted
Rejected
A
ccepted
Rejected
t
SignificanceSEd
SD
ratio
d
t
SignificanceSEd
SD
ratio
d
Mean
SD
Mean
SD
Mean
SD
Mean
SD
Mean
SD
mean
SD
Self-esteem
16.2
8
3.2
5
17.18
3.7
6
16.3
2
3.8
7
16.4
1
3.9
1
16
.07
3.7
5
16.7
3
3.9
9
1.1
9
NS
.22
.86
.2
5
.7
8
NS
.22
.94
.1
7
Psychological
well-being
12.0
7
5.1
2
10.19
4.3
7
11.8
1
4.8
7
10.4
2
3.8
8
10
.86
6.7
8
10.4
5
5.6
3
1.7
2
NS
.22
1.1
7
1.4
1
.28
NS
.22
1.2
0
.0
7
Positive
affect
37.1
8
3.8
5
35.88
6.2
0
35.4
3
5.8
5
34.3
6
7.9
1
33
.39
7.2
0
32.3
0
7.9
5
1.2
8
NS
.22
.62
1.2
5
.67
NS
.22
.91
.1
4
Negative
affect
16.8
6
4.0
0
17.77
4.6
0
12.8
2
3.5
4
13.0
0
3.5
9
13
.89
5.1
7
14.8
8
6.0
7
.9
9
NS
.22
.87
.2
3
.8
3
NS
.22
.85
.1
7
Career
exploration
behavior
81.0
0
11.4
3
75.73
13.3
0
80.9
0
12.2
6
77.8
4
13.2
1
62
.17
12.8
6
68.2
4
15.6
0
2.0
0
N
S1
(p5
.051)
.22
.86
1.4
1
1.8
7
NS
.22
.82
.4
3
Notes:d5
differencebetween
rejectedandacceptedmeansinstandarddeviationunits(effectssize).d
valuescomputedbyexpressingthedifferencesbetweenthe
meansoftherejectedandacceptedgroupsinpooledstandarddeviationunits.d5
(meanfortherejecte
dgroupmeanfortheacceptedgro
up)/SDpooled.
Positived
valuesindicateacceptedapplicantsscorehigher,negativethatrejectedapplicantsscorehigher.SEd5
standarderrorofd(unequalN
valuesforgroups).
SDratio
of
rejectedgrouptoacceptedgroupstandarddeviations,valuesgreaterthan1.0
0indicatealargerrejectedgroupstandarddeviation.
p5
.051forthist-valueoft5
2.0
0,
indicatingthattheTime1d
ifferencebetweenacceptedandrejectedgroupswasmarginallynon-significant.
242 NEIL ANDERSON AND VICKI GOLTSI
International Journal of Selection and Assessmentr 2006 The Authors
Journal compilationr Blackwell Publishing Ltd. 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
8/21
accepted (i.e. minority) groups. According to Cohen, effect
sizes can theoretically range between positive and negative
infinity, although in personnel selection contexts it is rare
to find such extreme values (e.g., Anderson, Lievens, van
Dam, & Born, 2006; Ones & Anderson, 2002). As d in
effect indexes the standardized mean difference between
any two groups, it is particularly valuable when examining
differnces between sub-groups of varying sample sizes. Past
research into ethnic minority and gender differences inselection has made extensive use of such effect size
computations; here, we use Cohens d to compare
immediate and longer-term differences in psychological
well-being and career exploration behavior. Note that
according to Cohen (1988), effect sizesof zero to .20 can be
considered small, those around .50 are medium, and above
.80 are large in magnitude.
First, we consider differences in key variables at Time 1.
Two effect sizes approaching .50 were found. Accepted
candidates reported a higher base-line mean on psycholo-
gical well-being than that of subsequently rejected
candidates (means were 12.07 and 10.19, respectively,d5 .41). Subsequently accepted candidates also self-rated
more highly on career exploration behavior than
rejected applicants, suggesting that they were already more
actively involved in their job search (the means were 81.00
and 75.73, respectively, d5 .41). No other effect size
differences or t-test differences are evident in Table 3
between these two groups on base-line measures taken at
Time 1.
Second, differences between accepted and rejected
candidates are examined at Time 3, that is, 6 months
following this AC, and several interesting findings are
evident. Most importantly, no statistically significant
differences (according to t-test values) were found betweenaccepted and rejected candidates on self-esteem, psycholo-
gical well-being, positive or negative effect, or on the CES.
However, an effect size difference of d5 .43 was found
between accepted and rejected candidates on career
exploration behavior. Not surprisingly, this difference
was substantially higher for rejected applicants, perhaps
reflecting on-going job search behaviors for those still
active in applying for job vacancies elsewhere. A finding of
note here is that rejected applicants actually report a higher
mean (although also marginally higher SD value) on self-
esteem at Time 3 than accepted candidates (16.73 and
16.07, respectively), although this difference is non-significant (t5 .78, p5NS; d5 .17).
Third, receipt of a positive outcome decision did not
appear to improve values on self-esteem, positive affect, or
psychological well-being for accepted candidates. Time 1
to Time 3 mean ratings on self-esteem were 16.28
(SD53.25) and 16.07 (SD53.99), respectively; on
positive, affect values were 37.18 (SD53.85) and 33.39
(SD57.20); and on psychological well-being, 12.07
(SD55.12) and 10.86 (SD56.78), respectively. None of
these differences were significant, suggesting that there was
no longer-term PPEs from receipt of a successful select
decision at this AC.
Feedback Perception Differences
A final set of analyses were computed in order to examine
differences in feedback perceptions between the accepted
and rejected candidate groups. These are summarized in
Table 4. Large effect size differences in feedback percep-tions between the two groups are evident. For the eight-
item scale combined, d5 .86 (t54.26,po.001) on overall
feedback perceptions. Yet, the largest effect size difference
was found on the item Feedback information was
accurate: d51.08 (t56.52, po.001), strongly suggest-
ing that rejected candidates attributed unfavorable out-
come decisions to an inaccurate selection methodology.
Two other large effect size differences were found in
accordance with Cohens (1988) recommendations:
d5 .82 on Reference to good performance was made,
and d5 .93 on Feedback had an encouraging effect.
Note that feedback perceptions were also measured atTime 3 along with all other mental health scales and the
CES; for the former, we found several large effect size
differences between accepted and rejected candidates
(Table 4), but for the latter we found no sizeable or
statistically significant differences (Table 3).
Discussion
We believe that this paper makes two novel contributions
to the existing scant research into, and thus our under-
standing of, the psychological impact of assessment centers
upon applicants. First, we develop and expound anoperational definition of NPEs for all selection methods,
noting several facets of this definition and linking it to
recent research that has adopted the applicants perspective
to selection predictors. Second, although a relatively small-
scale primary study, we present the findings of a three-
timepoint longitudinal study into the NPEs and potential
PPEs of an operational assessment center in the United
Kingdom. Contrary to the findings of the Fletcher (1991)
study, which used identical measurement intervals to the
present investigation, we failed to find NPEs resulting from
this AC, most notably even though the effect size
differences between accepted and rejected candidates werelarge in reactions-level measures of applicant feedback
perceptions following this assessment center.
Non-Significant NPE Findings
Despite the fact that 6 months after the AC, and indeed in
most cases some months after receiving feedback on their
performance, rejected candidates reacted far more nega-
tively to the feedback reactions measure than did accepted
candidates (see Table 4), the overall feedback perception
NPES OF ASSESSMENT CENTERS 243
r 2006 The AuthorsJournal compilationr Blackwell Publishing Ltd. 2006 Volume 14 Number 3 September 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
9/21
effect size difference across all eight items combined was
large (d5
.86). It appears that rejected applicants attrib-uted this negative decision to a lack of accuracy in the
assessment process (d51.08, t56.52, po.001) in spite of
this being a 2-day, job-relevant assessment center. Detailed
feedback was made available to all candidates, regardless
of the outcome of the AC final decision. Again, it seems
that the wisdom of providing all applicants, especially
those who have been unsuccessful, with highly detailed
feedback is more questionable than was previously thought
by selection psychologists (Schinkel et al., 2004).
Undoubtedly, the key findings of this study are that in
contrast to these highly significant findings of feedback
effect differences, rejected applicants suffered no serious
declines in self-esteem, psychological well-being, positiveaffect, or indeed expounded career exploration behavior as
a result of being unsuccessful at this assessment center. This
is noteworthy in that this AC was for a highly prestigeous
organization that is perceived by applicants as being a
highly attractive employer. One explanation here is that the
selection ratios involved in being successful for this
organization were so competitive for applicants that their
perceptions of being unsuccessful could have been moder-
ated by this selection ratio. Various perceptual, psycholo-
gical, and attributional biases may therefore have allowed
unsuccessful candidates in this particular situation to
persuade themselves that in effect so many otherapplicants were also unsuccessful that I knew all along
that this was a long-shot. Counter to this explanation is
the fact that this was the final stage of the selection process
for the organization, that is, that applicants had success-
fully survived several stages of shortlisting and assessment
previously and this AC represented the final barrier to entry
to employment. Despite the job relevance of exercises at
this AC, trained assessors, and detailed feedback including
behavioral exemplars being given to rejected candidates,
they still construed to attribute this to inherent inaccuracies
and unjust elements in the method itself (see also Fletcher
& Kerslake, 1992).That unsuccessful applicants suffered no longer-term (or
at least after a 6-month period) NPEs or deleterious
impacts upon the career exploration behavior suggests
robust psychological barriers and perceptual screening
heuristics between their immediate-level reactions and
longer-term psychological impacts. This is an important
finding, although of course this is only a single study of a
single selection predictor method. It could also have been
the case that many of these rejected applicants had found
employment elsewhere during this period; unfortunately,
our follow-up measure did not include items to explore the
present employment status of those individuals. Further
research is called for to examine this issue. Yet, as arguedearlier in this paper, the AC is precisely the method that
might be expected to cause NPEs in rejected candidates. It
is noteworthy therefore that this study failed to find any
evidence of longer-term negative effects, and this strongly
suggests that future research in this area needs to
distinguish clearly between differences that are reactions-
level preference responses and longitudinal effects upon
applicant decision-making, psychological well-being, and
even career exploration behavior (Noe & Steffy, 1987).
Non-Significant PPE Findings
One further finding of relevance here is that we likewise
found no strong PPEs from being successful at this AC.
Note that self-esteem, psychological well-being, and
positive affect did not improve markedly either at Time 2
measures, that is, before the actual decision was known but
immediately following the AC procedure, or 6 months later
at Time 3. Rather, we found some evidence of declines in
psychological well-being for successful applicants 6
months after the AC and in most cases some months into
their job tenure. During the intervening period, most of
Table 4. Differences in feedback perceptions: accepted vs. rejected sub-groups
Feedback perception
Accepted Rejected
t-value SEd SDratio dMean (SD) Mean (SD)
Feedback provided 2.89 (.79) 2.28 (1.04) 3.18** .22 .76 .60
Reference to good performance 2.96 (.94) 2.11 (.99) 3.99*** .23 .95 .82
Reference to poor performance 2.57 (1.03) 2.24 (1.04) 1.43* .22 .99 .31
Feedback had an encouraging effect 3.03 (1.26) 1.88 (1.09) 4.25*** .23 1.16 .93Understanding of feedback information 3.29 (.90) 2.68 (1.15) 2.81** .22 .78 .54
Feedback information was accurate 3.59 (.75) 2.34 (1.10) 6.52*** .23 .68 1.08
Valuable information for personal development 3.21 (1.17) 2.16 (1.37) 3.87*** .23 .85 .76
Provision of developmental recommendations 2.07 (1.33) 2.07 (1.13) .01* .22 1.18 .00
Overall feedback perceptions scale 23.69 (5.81) 17.79 (6.58) 4.62*** .23 .88 .86
Notes: SEd5 standard error of d.*po.05,**po.01,***po.001.
244 NEIL ANDERSON AND VICKI GOLTSI
International Journal of Selection and Assessmentr 2006 The Authors
Journal compilationr Blackwell Publishing Ltd. 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
10/21
these applicants had already joined the organization and it
could have been that widely documented socialization
effects had eliminated any short-term increases in well-
being and core self-esteem (e.g., Saks, 1995; Saks &
Ashforth, 1997). Again, caution is warranted over the
interpretation of these findings as the sample size in the
present study is relatively small, so these findings should be
tempered with some caution. Future research is certainly
called for to examine both NPE and PPE effects inassessment centers and other selection methods; here, it is
appropriate to highlight that different time periods of
measurement also need to be included in future studies in
order to test the possibility that effects may be present but
might only remain manifest for some weeks rather than
months after receipt of the outcome decision.
Brief Comments over Future Directions for NPE andPPE Research
The present investigation contributes to the rather limitednumber of studies into the possible psychological effects,
either negative or positive, of selection methods upon
applicants. As noted earlier in this paper, it is surprising
that this area has attracted so little attention by psychol-
ogists active in employee selection research. As a con-
sequence, several major avenues for future research,
theoretical model-building, and empirical study are im-
mediately apparent. In the opening section of this paper,
two working definitions of NPEs and PPEs were postu-
lated; it is likely that other researchers will have rather
different opinions of these constructs and so may wish to
put forward somewhat differing formulations of these in
due course. Additional research would, it would at leastseem, most valuable to commence the mammoth task of
quantifying the direct effects and costs of NPEs upon
reduced job performance; that is, to investigate the utility-
related costs of NPEs to organizations who may well be
unaware of this possibility. Investigation of how such costs
can be moderated by the provision of accurate but tactful
feedback to unsuccessful internal candidates would be a
follow-up line for initial research. In addition to both of
these performance- and utility-related directions for future
studies, however, we would suggest that research into both
NPEs and PPEs as a psychologically related strategy is fully
warranted without the need to justify this theme of researchon what are essentially economic grounds. It is beholden
upon selection psychologists to investigate and quantify the
psychological impacts of the assessment methods used in
employee recruitment and selection settings, especially
where these may result in NPEs and outcomes. Further
research is clearly called for in this regard not just into the
NPEs of assessment centers but on other popular predictors
such as structured interviews, tests of general mental ability
and personality, job sample tests, and situational judgment
inventories, among others.
Conclusion
In comparison even with the growing body of research into
immediate-level applicant reactions, the potential for
predictor methods to provoke NPEs upon applicants has
remained a critically underresearched topic. Curiously, this
dark side of the moon, as Anderson (2004) refers to it,
has received notably little research attention by selection
psychologists despite health psychology and research intonegative psychological impacts of workload being vibrant
areas of study within IWO psychology more generally. This
paper represents an initial investigation into a sub-field that
first requires definitional clarity and conceptual distinction
from existing research efforts into immediate-level reac-
tions outcomes. To conclude, this study, and its principal
finding of a lack of longer-term NPEs upon applicants
caused by a final-stage assessment center, can only be cited
as one of the initial steps toward establishing this important
field of enquiry for further empirical investigation.
Acknowledgement
We wish to express our thanks to Sonja Schinkel for her
comments and suggestions on an earlier version of this
paper.
References
American Psychiatric Association (1994) Diagnostic and statisticalmanual of mental disorders, 4th Edn (DSM-IV). Washington,DC: APA.
Anderson, N. (2004) Editorial the dark side of the moon:
Applicant perspectives, negative psychological effects (NPEs),and candidate decision making in selection. International
Journal of Selection and Assessment, 12, 18.Anderson,N. (2005) Relationships between practice and research in
personnel selection: Does the left hand know what the right isdoing? In A. Evers, N. Anderson and O. Smit-Voskuyl (Eds),Handbook of selection (pp. 124). Oxford: Blackwell.
Anderson, N., Born, M. and Cunningham-Snell, N. (2001)Recruitment and selection: Applicant perspectives and out-comes. In N. Anderson, D.S. Ones, H. Sinangil and C.Viswesvaran (Eds), Handbook of industrial, work and organiza-tional psychology, Vol. 1: Personnel psychology (pp. 200218).London: Sage Publications Ltd.
Anderson, N., Lievens, F., van Dam, K. and Born, M. (2006) Aconstruct-driven investigation of gender differences in a leader-
ship-role assessment center. Journal of Applied Psychology, 9,219240.
Bauer, T.N., Maertz, C.P., Dolen, M.R. and Campion, M.A. (1998)Longitudinal assessment of applicant reactions to employmenttesting and test outcome feedback. Journal of Applied Psychol-ogy, 83, 892903.
Bauer, T.N., Truxillo, D.M, Paronto, M.E., Campion, M.A. andWeekley, J.A. (2004) Applicant reactions to different selectiontechnology: Face-to-face, interactive voice response, and com-puter-assisted telephone screening interviews. International
Journal of Selection and Assessment, 12(2), 135148.Chan, D. and Schmitt, N. (2004) An agenda for future research on
applicant reactions to selection procedures: A construct-oriented
NPES OF ASSESSMENT CENTERS 245
r 2006 The AuthorsJournal compilationr Blackwell Publishing Ltd. 2006 Volume 14 Number 3 September 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
11/21
approach. International Journal of Selection and Assessment,12, 923.
Cohen, J. (1988) Statistical power analysis for the behavioralsciences, 2nd Edn. Mahwah, NJ: Erlbaum.
Derous, E., Born, M. and De Witte, K. (2004) How applicants wantand expect to be treated: Applicants selection treatment beliefs
and the development of the social process questionnaire onselection. International Journal of Selection and Assessment, 12,99119.
Dodd, W.E. (1977) Attitudes toward assessment centre programs.
In J.L. Moses and W.C. Byham (Eds), Applying the assess-ment centre method (pp. 132146). Elmsford, NY: PergamonPress.
Fletcher, C. (1991) Candidates reactions to assessment centres andtheir outcomes: A longitudinal study. Journal of OccupationalPsychology, 64, 117127.
Fletcher, C. and Kerslake, C. (1992) The impact of assessment
centres and their outcomes on participants self-assessments.Human Relations, 45, 281289.
Gilliland, S.W. (1994) Effects of procedural and distributive justiceon reactions to a selection system. Journal of AppliedPsychology, 79, 117127.
Gilliland, S.W. and Steiner, D.D. (2001) Causes and consequencesof applicant perceptions of unfairness. In R. Cropanzano
(Ed.), Justice in the workplace (pp. 175195). Hillsdale, NJ:Erlbaum.
Goldberg, D. (1992) General health questionnaire (GHQ-12).Windsor, UK: NFER NELSON.
Hart, P.M. and Cooper, C.L. (2002) Occupational stress: Toward amore integrated framework. In N. Anderson, D.S. Ones, H.
Sinangil and C. Viswesvaran (Eds), Handbook of industrial andorganizational psychology, Vol. 2 (pp. 93114). Thousand Oaks,CA: Sage Publications.
Hausknecht, J.P., Day, D.V. and Thomas, S.C. (2004) Applicant
reactions to selection procedures: An updated model and meta-analysis. Personnel Psychology, 57, 639683.
Horvath, M., Ryan, A.M. and Stierwalt, S.L. (2000) The influenceof explanations for selection test use, outcome favorability, and
self-efficacy on test-taker perceptions. Organizational Behavior
and Human Decision Processes, 83, 310330.Iles, P. and Robertson, I. (1997) The impact of personnel selection
procedures on candidates. In N. Anderson and P. Herriot(Eds),International handbook of selection and assessment (pp. 543566). Chichester, UK: Wiley.
Imus, A.L. and Ryan, A.M. (2005) Relevance and rigor in research
on the applicants perspective: In persuit of pragmatic science. InA. Evers, N. Anderson and O. Voskuijl (Eds), Handbook ofSelection (pp. 280296). Oxford, UK: Blackwell.
Judge, T.A. and Bono, J.E. (2001) Relationship of core self-
evaluation traits self-esteem, generalized self-efficacy, locusof control, andemotional stability with jobsatisfaction andjobperformance: A meta-analysis. Journal of Applied Psychology,86, 8092.
Lievens, F. and Klimoski, R.J. (2001) Understanding the assessmentcenter process. Where are we now? In C.L. Cooper and I.T.Robertson (Eds), International review of industrial and organi-sational psychology, Vol. 16 (pp.245286). Chicester, UK: JohnWiley & Sons.
Lievens, F., De Corte, W. and Brysse, K. (2003) Applicantperceptions of selection procedures: The role of selection
information, belief in tests, and comparative anxiety. Interna-tional Journal of Selection and Assessment, 11, 6777.
Lounsbury, J.W., Bobrow, W. and Jensen, J.B. (1989) Attitudestoward employment testing: Scale development, correlates and
known-group validation. Professional Psychology: Researchand Practice, 20, 340349.
Macan, T.H., Avedon, M.J., Paese, M. and Smith, D.E. (1994) The
effects of applicants reactions to cognitive ability tests and an
assessment centre. Personnel Psychology, 47, 715738.Noe, R.A. and Steffy, B.D. (1987) The influence of individual
characteristics and assessment centre evaluation on career
exploration behavior and job involvement. Journal of Voca-
tional Behavior, 30, 187202.Ones, D.S. and Anderson, N. (2002) Gender and ethnic group
differences on personality scales in selection: Some British data.
Journal of Occupational and Organizational Psychology, 75,255276.
Ployhart, R.E. and Harold, C.M. (2004) The applicant attribution-
reaction theory (AART): An integrative theory of applicant
attributional processing. International Journal of Selection and
Assessment, 12, 8498.Ployhart, R.E. and Ryan, A.M. (1997) Toward an explanation of
applicant reactions: An examination of organizational justice
and attribution frameworks. Organizational Behavior and
Human Decision Processes, 72, 308335.Ployhart, R.E. and Ryan, A.M. (1998) Applicants reactions to the
fairness of selection procedures: The effects of positive rule
violations and time of measurement. Journal of Applied
Psychology, 83, 316.Ployhart, R.E., Ryan, A.M. and Bennett, M. (1999) Explanations
for selection decisions: Applicants reactions to informational
and sensitivity features of explanations. Journal of Applied
Psychology, 84, 87106.Robertson, I.T., Iles, P.A., Gratton, L. and Sharpley, D. (1991) The
impact of personnel selection and assessment methods on
candidates. Human Relations, 44, 963982.Rosenberg, M. (1989) The self-esteem scale. Windsor, UK: NFER
NELSON.Ryan, A.M. and Ployhart, R.E. (2000) Applicants perceptions of
selection procedures and decisions: A critical review and agenda
for the future. Journal of Management, 26, 565606.Ryan, A.M., Greguraas, G.J. and Ployhart, R.E. (1996) Perceived
job relatedness of physical ability testing for firefighters:
Exploring variations in reactions. Human Performance, 9,
219240.Saks, A.M. (1995) Longitudinal field investigations of the moderat-
ing and mediating effects of self-efficacy on the relationship
between training and newcomer adjustment. Journal of AppliedPsychology, 80, 211225.
Saks, A.M. and Ashforth, B.E. (1997) Organization socialization:
Makingsense of thepast andpresent as a prologue forthe future.
Journal of Vocational Behavior, 51, 234279.Salgado, J.F. (1999) Personnel selection methods. In C.L. Cooper
and I.T. Robertson (Eds), International review of industrial andorganisational psychology (pp. 154). Chichester, UK: JohnWiley & Sons.
Salgado, J.F., Viswesvaran, C. and Ones,D.S. (2002) Predictors used
for personnel selection: An overview of constructs, methods
and techniques. In N. Anderson, D.S. Ones, H. Sinangil and
C. Viswesvaran (Eds), Handbook of industrial, work and
organizational psychology, Vol. 1. London: Sage.Schinkel, S., van Dierendonck, D. and Anderson, N. (2004)
The impact of selection encounters on applicants: An experi-
mental study into feedback effects after a negative selection
decision. International Journal of Selection and Assessment, 12,197205.
Schmit, M.J. and Ryan, A.M. (1992) Test-taking dispositions:
A missing link? Journal of Applied Psychology, 77, 629637.Schmit, M.J. and Ryan, A.M. (1997) Applicant withdrawal: The
role of test-taking attitudes and racial differences. Personnel
Psychology, 50, 855876.
246 NEIL ANDERSON AND VICKI GOLTSI
International Journal of Selection and Assessmentr 2006 The Authors
Journal compilationr Blackwell Publishing Ltd. 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
12/21
Schmitt, N. and Chan, D. (1999)The statusof research on applicant
reactions to selection tests and its implications for managers.
International Journal of Management Research, 4561.Schmitt, N., Oswald, F.L., Kim, B.H., Gillespie, M.A. and Ramsay,
L.J. (2004) The impact of justice and self-serving bias explana-
tion of the perceived fairness of different types of selection tests.
International Journal of Selection and Assessment, 12, 160171.Schuler, H. and Fruhner, R. (1993) Effects of assessment center
participation on self-esteem and on evaluation of the selection
situation. In H. Schuler, C.J.L. Farr and M. Smith (Eds),Personnel selection and assessment: Individual and organiza-
tional perspectives (pp. 109124). Mahwah, NJ: Lawrence
Erlbaum.
Smither, J.W., Reilly, R.R., Millsap, R.E., Pearlman, K. and Stoffey,R.W. (1993) Applicant reactions to selection procedures.Personnel Psychology, 46, 4976.
Stumpf, S.A., Colarelli, S.M. and Hartman, K. (1983) Developmentof the career exploration survey (CES). Journal of VocationalBehaviour, 22, 191226.
van Vianen, A., Taris, R., Scholten, E. and Schinkel, S. (2004)Perceived fairness in personnel selection: Determinants andoutcomes in different stages of the assessment procedure.
International Journal of Selection and Assessment, 12, 149159.Watson, D., Clark, L.A. and Tellegen, A. (1988) Development andvalidation of brief measures of positive and negative affect: ThePANAS scales. Journal of Personality and Social Psychology, 5,10631070.
NPES OF ASSESSMENT CENTERS 247
r 2006 The AuthorsJournal compilationr Blackwell Publishing Ltd. 2006 Volume 14 Number 3 September 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
13/21
Ta
bleA1
.Chrono
log
ica
lsum
maryo
fresearc
hintopsyc
ho
log
ica
le
ffec
tso
fse
lec
tionme
tho
dsupon
app
lican
ts
Author(s)
Construc
ts
measure
d
Sample
Predicto
r
method
Measures
Measurem
ent
timing
Independent
variable(s)
Depende
nt
variable(
s)
Key
findings
Lounsburyetal.
(1989)
Attitudestoward
employmentte
sting
n5
546for
telephoneinterview
n5
112applicants
formanualwork
position
Testingingene
ral
Opinionsof
employmenttesting
Study1:
Study2:post-testor
post-feedbacktest
Sex,age,race,
occupation,
education
attainment,
experiencewith
testing
Attitudestotes
t
takingperceive
d
fairnessoftests
Instudy2Hispanics
andyounger
applicantshadmore
positiveattitudes
Significantlymore
favorableattitudes
wereobserved
towardemployment
testingwhenjob-
relationhadbeen
explainedand
feedbackbeengiven
thanifithadnot
Rejectedapplicants
hadmorenegative
attitudesthan
selectedorhold-
fileapplicants
Robertsonetal.
(1991)
Beliefsaboutthe
adequacyofse
lection
procedures,
perceivedcareer
impact
n5
170
Employeesofa
financialinstitution
Biodata,situational
interview,
assessmentcenter
Withdrawal
cognitions
Organizational
commitment
Psychologicalhealth
Self-esteem
Atspecificpoin
ts
duringthe
assessmentprocess
Select/reject
Withdrawal
cognitions
Organizational
commitment
Psychological
health
Perceivedcareer
impactwas
significantlygreater
forrejectedthanfor
acceptedcandidates
Acceptedand
rejectedcandidates
didnotdifferon
psychologicalhealth
orself-esteem
Candidatesinthe
assessmentcenter
phaseweremore
concernedwith
adequacythan
candidatesinthe
situationalinterview
phase
Acceptedcandidates
exhibitedhigher
organizational
commitmentand
believedmethods
AppendixA
248 NEIL ANDERSON AND VICKI GOLTSI
International Journal of Selection and Assessmentr 2006 The Authors
Journal compilationr Blackwell Publishing Ltd. 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
14/21
Ta
bleA1
.Con
tinue
d
Author(s)
Construc
ts
measure
d
Sample
Predicto
r
method
Measures
Measurem
ent
timing
Independent
variable(s)
Depende
nt
variable(
s)
Key
findings
weremoreadequate
thanrejected
candidates
Fletcher(1991)
Psychological
well-being
Needforachievement
Organizational
commitment
Jobinvolvement
n5
57
Highergradeclerical
staff
Assessmentce
nter
(groupdiscussion,
in-tray,objective
setting,psycho
metric
tests,self-repo
rt,
interview)
Self-esteematwork
anddepressedmood
atwork
Pre-test
Post-test(straight
afterassessme
nt)
Post-test(6months
later)
Select/reject,
assessmentcenter
participation
Psychological
well-being
Successful
candidateshad
significantlylower
scoresthan
unsuccessful
candidateson
depressedmoodat
work
Self-esteemincreases
fromT1toT2and
dropsatT3toalevel
lowerthanatT1
Unsuccessful
candidatesshoweda
significantlylower
levelofself-esteemat
T3
Smitheretal.(1993)Perceivedpred
ictive
validityandface
validity,job
relatedness,
proceduraland
distributivejus
tice
n15
154entry-level
andrecruiting/
employment
managers
n25
1895
applicantsforawide
rangeofcivilservice
jobs
Descriptionsofeight
cognitivetestsandsix
otherprocedures
(personality
inventory,
in-basket,
leaderlessgrou
p
discussion,
bio
data,
unstructured
interview,and
structuredinterview)
Jobrelatedness
(predictivevalidity
andfacevalidity)
organizational
attractiveness,
likelihoodof
improvement,
perceivedknowledge
ofresults,affect,
cognitiveability
Study1:post-test
Study2:post-test
andpost-feedb
ack
testafter1mo
nth
Study1:perceived
predictivevalidity,
facevalidityGPA,
andcognitivetest
performance
Study2:
organizational
attractiveness,
likelihoodof
improvement,
perceivedknowledge
ofresults,affect
Perceptionsofjob
relatedness,
proceduraljust
ice,
distributivejustice
andrecommen
dation
intentions
Perceptionsof
validitywerenot
enhancedthrough
facevalidity
manipulations
Perceivedvaliditydid
notinescapably
correspondwiththe
actualvalidityofthe
selectionprocedures
Applicantsreactions
werepositively
relatedto
organizational
attractiveness,
justice
perceptionsand
willingnessto
recommendthe
employertoothers
Higher-ability
applicantswerenot
morelikelyto
perceiveselection
NPES OF ASSESSMENT CENTERS 249
r 2006 The AuthorsJournal compilationr Blackwell Publishing Ltd. 2006 Volume 14 Number 3 September 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
15/21
Ta
bleA1
.Con
tinue
d
Author(s)
Construc
ts
measure
d
Sample
Predicto
r
method
Measures
Measurem
ent
timing
Independent
variable(s)
Depende
nt
variable(
s)
Key
findings
proceduresaslessjob
related
Proceduraljusticeas
wellas
recommendation
intentionswere
positivelyrelatedto
affectandlikelihood
ofimprovementand
perceivedpredictive
validity
Gilliland(1994)
Proceduralfairness,
distributivefairness
n5
270
undergraduate
studentsparticipating
inexchangefor
financialrewards
(whenselected)and/
orcredits(when
rejected)
Worksampletest,
cognitiveabilitytest,
overtintegritytest
Fairnessperceptions
Recommendation
intentions:
Self-efficacy
Performance
Post-feedback
test
Hiringexpectation
Perceivedfairness,
recommendation
intentions,self-
efficacy,performance
Selectedindividuals
perceivedgreater
fairnessinthe
selectionprocessand
outcomedecision
thanrejected
individuals,andthis
effectwasmore
pronouncedwhen
individualshadhigh
expectationsofbeing
hired
Rejectedapplicants
weremorelikelyto
recommendwhen
givenexplanationsfor
rejectdecision
Fairprocedureslead
tohigherself-efficacy
forselected
candidatesandlower
forrejected
Ryan,
Greguras,and
Ployhart(1996)
Self-efficacy
Job-relatedness,
experience,
consistencyof
test
administration
N5
81Fire
Firefightersattending
aweekofsessionsat
astatefireschool
3.7
%female
81.4
8%male
Sevenphysicalability
tests(PAT)
NA
Self-efficacy,
experience,current
fitnesslevel,
departmentfitness
climate
Perceived
job-relatedness
Self-efficacyfora
specificPATwas
positivelyrelatedto
theperceivedjob-
relatednessofthat
PAT
Individualswith
higherself-efficacy
sawPATs
250 NEIL ANDERSON AND VICKI GOLTSI
International Journal of Selection and Assessmentr 2006 The Authors
Journal compilationr Blackwell Publishing Ltd. 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
16/21
Ta
bleA1
.Con
tinue
d
Author(s)
Construc
ts
measure
d
Sample
Predicto
r
method
Measures
Measurem
ent
timing
Independent
variable(s)
Depende
nt
variable(
s)
Key
findings
PloyhartandRyan
(1997)
Processandou
tcome
fairness
n5
80297
applicantsto
graduateschoolin
psychology
GRE-scores,GPA,
researchandw
ork
experiences,re
search
interests,lettersof
recommendation,
personalstatem
ent
Organizationaljustice
Processfairness
Outcomefairness
Selectiondecision
Perceivedcausefor
decision
Self-efficacy
Self-esteem,
Self-assessed
performance
Recommendation
intention
Applicationintention
Acceptanceintention
Reapplication
intention
Expectancyoffuture
success
Pre-test(before
application),post-
test
(afteroffer)
Admissionstatus,
attributionsfor
decision
Recommendation
Application
Acceptanceand
reapplication
intentions
Self-efficacy
Self-esteem
Self-assessed
performance
Morefairlyperceived
proceduresleadto
morefavorable
intentionsandself-
assessed
performance.
Effectincreasedas
outcomefairness
increased
Applicantswho
perceivedprocedures
tobeunfairandwere
selectedreported
lowerself-efficacy
Self-servingbiasonly
occurredwhen
individualsperceived
fairprocedures
SchmitandRyan
(1997)
Test-takingattitudes,
withdrawal
cognitions
n5
3290Applicants
forthepostofpolice
officer,ofwhom
n5
618were
interviewedafter
withdrawal
Surveysand
Interviews
Test-takingattitudes
Pre-test
Anxiety
Motivation
Race
Withdrawal
Higherlevelsof
anxietyand
motivationin
applicantsleadto
lowerlikelihoodof
withdrawal
Caucasians
possessedslightly
highertest-taking
motivation,wereless
anxious,andbelieved
moreintheefficacyof
thetestscomparedto
African-Americans
Test-takingattitudes
onlypossessasmall
relationtowithdrawal
PloyhartandRyan
(1998)
Processandou
tcome
fairness
n5
283
undergraduate
studentsparticipating
inexchangefor
credits
CognitiveabilitytestFairness,
Intentions,Job
acceptance,
Future
experiment,
Reapplication,
Recommendation
Pre-testpost-te
st
post-feedback
questionnaire
Select/reject,
Administrative
consistency
Performance
expectations,job
acceptance
intentions,futu
re
experiment
intentions,
Positive
inconsistency
generallydoesnot
resultindifferent
perceptionsof
fairnesscompared
NPES OF ASSESSMENT CENTERS 251
r 2006 The AuthorsJournal compilationr Blackwell Publishing Ltd. 2006 Volume 14 Number 3 September 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
17/21
Ta
bleA1
.Con
tinue
d
Author(s)
Construc
ts
measure
d
Sample
Predicto
r
method
Measures
Measurem
ent
timing
Independent
variable(s)
Depende
nt
variable(
s)
Key
findings
Self-perceptions,
Withdrawal
reapplication
intentions,
recommendation
intentions,
withdrawal
withconsistency
Post-hirejob
acceptancewas
unaffectedbyprocess
fairness
Unfairlyperceived
proceduresleadto
negativeintentions.
Themostunfavorable
reactionsoccurred
withunfairly
perceivedprocedures
andafairoutcome
Pre-hiringintentions
andoutcomeare
centraltotheforming
ofreactionsbecause
oftheirinfluenceon
post-hireintentions
andfairness
PloyhartandRyan
(1998)
Processfairnes
s,
explanationadequacy
n15
156
undergraduate
studentsparticipated
inexchangefor
credits
n25
35entry-level
graduatestudents
justinformedonthe
decision
Study1:Scenarioof
cognitiveabilitytest,
andjobknowledge
teststaken
Study2:GRE,
GPA,
researchintere
st,
personalstatem
ent,
researchexperience,
workexperienc
e,
letterof
recommendation
Processfairness
Reportedself-
perceptions,
Organizational
perceptionsletter
Informationaland
sensitivity
explanation,select/
reject
Perceptionsof
processfairnesswere
enhancedby
providingpersonalor
procedural
information
Reportsofself-
perceptionsfor
rejectedparticipants
wereharmedby
providingpersonal
informationand
enhancedby
providingadiversity
justification.
Vice
versaforaccepted
applicants
Reportsonself-
perceptionswere
mostfavorablewhen
researchinterest,
workexperience,or
researchexperience
informationwas
provided
252 NEIL ANDERSON AND VICKI GOLTSI
International Journal of Selection and Assessmentr 2006 The Authors
Journal compilationr Blackwell Publishing Ltd. 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
18/21
Ta
bleA1
.Con
tinue
d
Author(s)
Construc
ts
measure
d
Sample
Predicto
r
method
Measures
Measurem
ent
timing
Independent
variable(s)
Depende
nt
variable(s)
Key
findings
Derousetal.(2004)Selectiontreatment
beliefs
n15
660senior-
yearstudentsat
campusrecruitment
event
n25
634applicants
fordifferentbranches
ofindustry
SocialProcess
Model
ofSelection(SPS
model)
Socialprocess
questionnaireon
selectionimportance
(SPQS-I),
Expectedrealization
(SPQS-E)!
only
theapplicantsfilled
outthisquestionnaire
Students:SPQS-I
questionnairea
fter
screeningprocedure
onthesameday
Applicants:
SPQS-IandSP
QS-E
afterinitialscreening
andbeforeactual
testing
Gender,workstatus,
educationallevel,
motivationtoapply,
priorselection
experience
Selectiontreatment
importance,
expectedselection
treatment
Supportforstability
ofsix-factormodel
(SPQS-I)
Workstatusdidnot
effectvaluationand
expectationof
treatment
Motivationandprior
selectionrelatedto
SPQS-Ebutnotto
SPQS-I
Educationrelatedto
SPQS-Ebutnotto
SPQS-I
Femalesvaluedall
treatmentfactors
higherthanmales(in
SPQS-I)
Bauer,Truxillo,
Paronto,
Weekley,
andCampion(2004)
Selectionprocedural
justice
n5
153students
Face-to-face
interviewscree
nings,
interactivevoic
e
response(IVR)
screenings,
computer-assis
ted
telephone-screening
interviews
(contentofscreening
isidenticalfor
3
screeningmeth
ods)
WonderlicPersonnel
test,
NEO(measures
conscientiousness),
selectionprocedural
justicescale(SPJS)
!
11dimensions
offairness
Time1:inclassroom,
questionnaires
to
measureindependent
variables
Time2:
IVRcondition:
studentshadtocall
forautomatic
screeningandtook
surveyonwebsite
TIcondition:
studentswerecalled
forscreeningandtook
surveyonwebsite
FTFcondition:
livescreening
interviewatcampus.
Afterscreening
taken
tocomputerforfinal
survey
Cognitiveability,
conscientiousness,
age,gender,work
experience
Processandou
tcome
fairness
IVRwasratedlower
oninterpersonal
treatment,two-way
communicationand
openness(IVRisnon-
interpersonal)
Nodifferenceson
otherprocedural
justicefactors
betweenIVRand
othermethodsof
screening
vanVianenetal.
(2004)
Fairnessinpersonnel
selection
n5
282applicants
fordifferentjobsin
different
organizations
Psychometrictests,
feedbackinterview
Testbeliefs,
perceivedfairnessof
theselection,
perceptionofthe
Time1:pre-testing
Time2:post-testing,
pre-feedback
Time3:post-
feedback
Opennessto
experiences,age,
education
Testbeliefs,
perceivedjob
relatedness,
perceived
performance,
Significanteffectof
previoustest
experiencesontest
beliefs
Opennessto
NPES OF ASSESSMENT CENTERS 253
r 2006 The AuthorsJournal compilationr Blackwell Publishing Ltd. 2006 Volume 14 Number 3 September 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
19/21
Ta
bleA1
.Con
tinue
d
Author(s)
Construc
ts
measure
d
Sample
Predicto
r
method
Measures
Measurem
ent
timing
Independent
variable(s)
Depende
nt
variable(s)
Key
findings
feedback-interview,
jobattractiveness
pre-andpost-
feedbackfairne
ss
measures,perc
eived
feedbacktreatm
ent,
feedbackconte
nt,
jobattractivene
ss
experienceinfluenced
perceptionsof
performance,
feedbackandfairness
Perceived
performanceand
perceivedjob
relatednesswere
independentlyrelated
topre-feedback
fairnessperceptions
Post-feedback
fairnessperceptions
werepositively
affectedbypre-
feedbackfairness
perceptions,
feedbackcontentand
perceivedfeedback
treatment.Itwasnot
relatedtojob
attractiveness
Perceivedfeedback
treatmentand
feedbackcontent
directlyaffectedjob
attractiveness
Schmittetal.(2004)
n5
644freshmen
students
Situational
JudgementInventory
(SJI)
Biographicalitems,
situationaljudgment
items,reactionitems
onperformance
beliefs,
test
relevance,
fairness
Reactionitems
directlyafterth
e
bio-dataand
SJIitems
Biographicalitems
andsituational
judgmentitems
in12performance
dimensions
(measuresprevious
experience,
interest,
motivation),ACT/SAT
test
Performancebeliefs
(absoluteand
comparative),
perceptionsoftest
relevance,
fairness
Relevanceand
fairnessaredirectly
influencedby
performancebeliefs
(consistentwithself-
servingbias)
Indirecteffectof
performancebeliefs
onfairnessmediated
byrelevance
perceptions
(consistentwith
justiceperspective)
Relevanceand
performanceinteract
intheirimpacton
fairnesswhen
comparative
254 NEIL ANDERSON AND VICKI GOLTSI
International Journal of Selection and Assessmentr 2006 The Authors
Journal compilationr Blackwell Publishing Ltd. 2006
7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC
20/21
Ta
bleA1
.Con
tinue
d
Author(s)
Construc
ts
measure
d
Sample
Predicto
r
method
Measures
Measurem
ent
timing
Independent
variable(s)
Depende
nt
variable(
s)
Key
findings
performanceas
opposedtoabsolute
performanceis
considered
Minoritiesreactless
favorablyto
standardizedtests
thanmajorities
Schinkeletal.
(2004)
Feedbackeffec
t
n5
119students
GMAtests
Affectivewell-being
scal