-
287287
The Journal of General Psychology, 2008, 135(3), 287300Copyright
2008 Heldref Publications
Science Versus the Stars: A Double-Blind Test of the Validity of
the NEO Five-Factor
Inventory and Computer-Generated Astrological Natal Charts
ALYSSA JAYNE WYMANSTUART VYSE
Connecticut College
ABSTRACT. The authors asked 52 college students (38 women, 14
men, M age = 19.3 years, SD = 1.3 years) to identify their
personality summaries by using a computer-generated astrological
natal chart when presented with 1 true summary and 1 bogus one.
Similarly, the authors asked participants to identify their true
personality profile from real and bogus summaries that the authors
derived from the NEO Five-Factor Inventory (NEO-FFI; P. T. Costa
Jr. & R. R. McCrae, 1985). Participants identified their real
NEO-FFI profiles at a greater-than-chance level but were unable to
identify their real astrological summaries. The authors observed a
P. T. Barnum effect in the accuracy ratings of both psychological
and astrological measures but did not find differences between the
odd-numbered (i.e., favorable) signs and the even-numbered (i.e.,
unfavorable) signs.
Keywords: astrology, double-blind test, five-factor model,
NEO-FFI, personal validation, personality
ASTROLOGERS NATAL CHARTS and psychologists personality profiles
share a common purposeto provide a description of the respondents
personal-ityand they are based on at least two common assumptions.
First, astrologers and personality psychologists assume that people
possess stable characteristics that, to varying degrees, determine
their behavior. This assumption gives person-ality assessment its
value. Second, both astrologers and psychologists assume that the
instruments they have developed for this purpose can measure these
traits. Despite these similarities, astrology and trait psychology
represent very differ-ent theories about the causes of personality.
Astrologers believe an individuals character is determined by the
arrangement of the planets and stars in relation
Address correspondence to Stuart Vyse, Department of Psychology,
Box 5621, Connecticut College, New London, CT 06320, USA;
[email protected] (e-mail).
-
288 The Journal of General Psychology
to the moment of that persons birth. As a result, the natal
charts that astrolo-gers construct are designed to identify the
relative locations of various celestial objects at the precise time
and place of birth. In contrast, many contemporary personality
psychologists place the source of personality 9 months earlierat
conceptionwhen the individuals genetic profile is determined
(Brody, 1994). The personality inventories that psychologists use
are designed to measure these traits by asking participants to
report on their own cognitions and behavior. Groups of specific
cognitions and behaviors are thought to be indicative of more
general underlying traits.
The similarities between these psychological and astrological
personality assessments have led to comparisons of their relative
validity. Carlson (1985) tested participants ability to recognize
their astrological charts and their Califor-nia Personality
Inventory (CPI) profiles. Presented with personality descriptions
based on their own astrological charts and two other personality
descriptions randomly selected from those of others, Carlsons
participants were unable to identify their astrological profiles at
greater-than-chance levels. Moreover, using the same procedure, he
found that participants were no more skilled at identify-ing their
CPI profiles.
A number of factors have been implicated in peoples acceptance
of personal-ity feedback, regardless of whether it is psychological
or astrological. Taylor and Brown (1988) suggested that many people
hold an unrealistically enhanced view of themselves, a
characteristic that may make it difficult for them to accurately
identify their own personality profiles. If, as Taylor and Brown
suggested, this self-enhancing phenomenon were widespread, it would
throw into question the usefulness of the personal validation
method of Carlsons (1985) test. Doubts about the accuracy of
self-knowledge are further supported by Sundbergs (1955) study
showing that participants were unable to discriminate between their
own profiles and bogus profiles derived from the Minnesota
Multiphasic Personality Inventory. However, other tests with the
CPI have shown that participants can identify their own personality
profiles at greater-than-chance levels (Greene, Harris, &
Macon, 1979; Word, 1996). Therefore, although personal validation
has not yet been successfully demonstrated for astrological charts,
a limited number of studies have shown that participants have
sufficient self-understanding to discriminate true CPI from bogus
CPI (e.g., Word) and 16 Personality Factors (16PF; Hampson,
Gilmour, & Harris, 1978) reports.
Several studies have shown that the P. T. Barnum effect (Meehl,
1956), a general tendency for individuals to accept vague and
ambiguous descriptions as typical of themselves, influences the
acceptance of both astrological (Glick, Got-tesman, & Jolton,
1989) and psychological (Dickson & Kelly, 1985; Forer, 1949;
Guastello & Rieke, 1990; ODell, 1972; Sundberg, 1955; Word,
1996) personal-ity feedback. Furthermore, there is evidence that,
for both kinds of assessment, positive feedback is more readily
accepted. For example, in the study by Greene et al. (1979),
participants who misidentified their CPI profiles were most
likely
-
to pick the more positive of the two profiles presented. Other
researchers have found similar results (e.g., Collins, Dmitruk,
& Ranney, 1977; Glick et al., 1989; Hamilton, 1995). In the
case of astrology, the odd-numbered sun signs (i.e., Aries, Gemini,
Leo, Libra, Sagittarius, Aquarius) are thought to be more favorable
in content than the even-numbered ones (i.e., Taurus, Cancer,
Virgo, Scorpio, Cap-ricorn, Aquarius), and some researchers have
found a difference in the degree of acceptance of charts for odd
sun signs versus even sun signs (Glick et al., 1989; Hamilton,
2001). However, a more recent large-scale study did not replicate
this effect (Wunder, 2003). Last, researchers have shown that prior
knowledge of an individuals astrological sun sign is associated
with a bias toward acceptance of astrological readings (Hamilton,
1995; Van Rooij, 1994).
We designed the present study as a replication of Carlsons
(1985) study with several substantive modifications. First, in the
intervening years, both astrology and personality psychologies have
changed. For astrology, recent decades have seen the development of
sophisticated computer programs capable of converting birth data
into detailed natal charts that include lengthy personality
descriptions. In the field of personality, the years since Carlsons
study have included the emer-gence of the Five-Factor Model (FFM),
or Big Five, as the dominant trait theory of personality (Costa
& McCrae, 1992b; Goldberg, 1993). One of our goals in the
present study was to update Carlsons work by using
computer-generated natal charts and the NEO Five-Factor Inventory
(NEO-FFI; Costa & McCrae, 1985).
Carlsons (1985) participants were not able to identify either
their astrologi-cal personality descriptions or their CPI profiles
given a three-choice task. In other tests with the CPI,
participants have successfully identified their own pro-files in
two- (Greene et al., 1979) and three-choice (Word, 1996) tasks;
however, no studies have reported successful identification of
astrological profiles. To maximize the likelihood of correct
identification, we used a simple two-choice task. We presented each
participant with one real astrological personality descrip-tion and
one bogus astrological personality description. Similarly,
identifications of the NEO-FFI involved one real profile and one
bogus profile. In addition, both to provide more detailed
information about participants accuracy judgments and to assess the
P. T. Barnum effect, sun-sign bias, and odd-even sun-sign effect,
we asked participants in the present study to make detailed
accuracy ratings of both the astrological charts and the
personality test results.
On the basis of previous research indicating that, at least in
some instances, participants were able to correctly identify their
CPI profiles (Greene et al., 1979; Word, 1996) and the presumed
influence of the simpler two-choice arrangement of the present
study, we hypothesized that participants would be able to correctly
identify their real NEO-FFI profiles. But because no previous study
had used a two-choice test with astrological profiles, we were
unable to make a hypothesis in this case. We also asked
participants to rate the accuracy of all four personality reports
(bogus and real astrological charts and NEO-FFI profiles) on a
9-point Likert-type scale ranging from 1 (This is a completely
inaccurate description of me)
Wyman & Vyse 289
-
290 The Journal of General Psychology
to 9 (This is a completely accurate description of me), and on
the basis of previous research we hypothesized that participants
would give the real NEO-FFI profile significantly higher accuracy
ratings than they would the bogus NEO-FFI profile. We did not make
a hypothesis regarding the accuracy ratings of the astrologi-cal
summaries. In addition, we hypothesized that participants ratings
of all the profiles, astrological and psychological as well as
bogus and real, would be rated somewhat favorably, consistent with
the P. T. Barnum effect, and that participants who knew their sun
signs would judge their real astrological readings to be more
accurate, consistent with the sun-sign bias. Last, due to the
conflicting results of prior studies, we made no hypothesis about
differences in the accuracy ratings of odd and even signs because
of the positive and negative personality descriptions associated
with them.
Method
Participants
Participants were 52 Connecticut College students (38 women, 14
men). The ages of the students were between 18 and 22 years (M age
= 19.3 years, SD = 1.3 years). We recruited participants from an
introductory psychology class and through fliers posted around
campus inviting any student to participate in a study of astrology
and psychological assessment. There were 14 psychology majors, 28
nonpsychology majors, and 10 participants who were undecided about
their major. Students from the introductory psychology class
received course credit, and all volunteers received their
personalized astrological natal charts as a reward for
participation.
Measures
NEO Five-Factor Inventory (NEO-FFI). The NEO-FFI is a 60-item
scale derived from the revised NEO Personality Inventory (Costa
& McCrae, 1985; 1992a) that measures five facets of
personality: Neuroticism (N), Extraversion (E), Open-ness to
experience (O), Agreeableness (A), and Conscientiousness (C).
Example items include, I try to be courteous to everyone I meet (A)
and I like to be where the action is (E), and all items are rated
on a 5-point Likert scale ranging from 1 (strongly disagree) to 5
(strongly agree). The NEO-FFI has shown stable Cronbachs alphas
ranging from .68 to .83 for domains of N, E, and O and alphas from
.63 to .79 for A and C domains.
Knowledge and beliefs questionnaire. The only other measure was
a brief question-naire that asked for demographic information and
assessed knowledge of astrology and belief in the validity of both
astrology and psychological personality tests. Two questions asked
participants to rate the accuracy of personality descriptions
derived from astrological charts and psychological tests on a
Likert-type scale ranging
-
from 1 (very accurate) to 7 (very inaccurate). Five questions
assessed astrological knowledge. The first three asked if the
participant knew his or her sun sign and, if so, to name it and
list three adjectives associated with people who have the sign. The
last two asked participants whether they knew and, if so, to name
their ris-ing or ascendant sign and their moon sign. This
questionnaire also asked for the participants date, location, and
time of birth; gender; and major.
Materials
Participants made their assessments on NEO-FFI summary feedback
sheets and astrological charts that were modified to allow for
accuracy ratings. The NEO-FFI summaries use one of three statements
to indicate whether the respon-dent is high, moderate, or low on
each of the five traits. For example, the high-N statement is,
sensitive, emotional, and prone to experience feelings that are
upsetting and the low-O statement is, down to earth, practical,
traditional, and pretty much set in your ways. We modified the
feedback sheets to allow partici-pant ratings of the accuracy of
each of the five traits, as well as a single global rating of the
report as a whole. These ratings were made on a 9-point Likert
scale ranging from 1 (This is a completely inaccurate description
of me) to 9 (This is a completely accurate description of me).
We generated astrological personality profiles by using Estoric
Technolo-gies Solar Fire Five v5.0.19 (Dawson & Johnson, 2000)
program. Although no conventional reliability or validity data are
available for this program, the National Council for Geocosmic
Research recommended it for research (National Center for Geocosmic
Research, 2004), and it was endorsed by Burk (2001), who said,
Solar Fire continues to be the cutting-edge astrology program
avail-able (p. 342). We edited the natal chart output so that all
references to the signs, planets, or houses were gone, to avoid
bias due to astrological knowledge. We also removed personality
descriptions derived from planetary aspects because astrological
charts contain different numbers of aspects and including them
would have resulted in real and bogus charts of different lengths.
The resulting charts included 29 one- to four-sentence personality
descriptions (e.g., You take pride in your home and family. You
like being the center of attention in your home envi-ronment; You
love spontaneity and new games. You also love new and creative
projects, often initiating them yourself).1 Each of the
astrological personality reports included the same 9-point accuracy
rating scale used for the NEO-FFI summaries. Using this scale, the
participants rated the accuracy of each of the 29 descriptions and,
in a separate item, the accuracy of the chart as a whole.
Procedure
Testing occurred in two sessions that were 3 weeks apart. When
participants volunteered for this study, we asked them to bring the
verified date, time, and
Wyman & Vyse 291
-
292 The Journal of General Psychology
location of their birth to the first session and encouraged them
to contact their parents to obtain accurate birth information. In
the first session, we gave partici-pants a packet containing the
consent form, NEO-FFI questionnaire, and knowl-edge and beliefs
questionnaire, respectively. We instructed participants to fill out
the three forms in the order that they were presented. We verbally
reminded participants to provide the date, location, and time of
their birth and that provid-ing inaccurate information would be a
violation of the honor code of the college. After they had
completed all forms and returned the packet, we told participants
that they would be contacted in approximately two weeks concerning
the date of the next session.
During the second session, an experimenter who was blind to the
hypothesis and had no knowledge of the real and bogus summaries
presented each participant with his or her specific packet
containing an instruction sheet and four personal-ity summaries, a
real natal chart, a bogus natal chart, a real NEO-FFI summary, and
a bogus NEO-FFI summary. We randomly selected the bogus
astrological and NEO-FFI summaries from other participants in the
study. We randomized the order of the summaries so that each
participant had an equal chance of being presented with each
summary in each ordinal position. We presented half of the
participants with the astrological summaries first and the other
half with the NEO-FFI summaries first. Each report was labeled as
either Psychological Report or Astrological Report.
Participants were asked to attend to everything in the packet in
the specific order that it was presented, first reading the
directions and then rating each state-ment of all four personality
summaries on the 9-point scale. In addition, partici-pants provided
a single overall accuracy rating for each summary, and we asked
them to identify which of the two NEO-FFI reports they believed was
their own and which of the two astrological summaries was their
own. Last, participants considered all four of the personality
reports and identified the one that they thought was the most
accurate description of their personality. On completion of all
items, participants returned the packet to the experimenter. In
return, the experimenter handed each participant a debriefing form
and the participants real astrological natal chart, which was
labeled with the participants name on its cover sheet and kept in a
separate box from the other experimental materials.
Results
Personal Validation of Personality Descriptions
As we hypothesized, participants were able to correctly identify
their NEO-FFI profiles at greater-than-chance levels, 2(1, N = 52)
= 17.31, p < .001 (78.8% correct identifications). In contrast,
only 46.2% of participants were able to iden-tify their real
astrological summaries, which was not significantly better than the
percentage of chance, 2(1, N = 52) = 0.31, p > .05.
-
Accuracy of Personality Reports
Participants rated the accuracy of psychological and
astrological personality descriptions in two ways. For each of the
five dimensions of the NEO-FFI and each of the 29 personality
descriptions of the astrological reports, participants made
individual ratings on a 9-point scale. In addition, they assessed
the accuracy of each of the four reports in a single overall rating
on the same 9-point scale. For the purposes of this analysis, we
summed the individual ratings of items on the reports and divided
each sum by 5 for the NEO-FFI reports and by 29 for the
astrological reports, resulting in mean ratings for each type of
report.
To assess the relative accuracy ratings of the four personality
reports, we conducted a 2 (gender) 4 (personality report)
mixed-design multivariate analy-sis of variance (MANOVA) with
repeated measures across the four personal-ity descriptions and
dependent variables being the overall ratings of accuracy and the
mean accuracy ratings of the individual personality descriptions.
We included gender as an independent variable because previous
researchers have shown that women have significantly higher levels
of belief in astrology (e.g., Wunder, 2003). In this case, neither
the main effect of gender nor the interaction of personality report
and gender was significant, but the main effect of personal-ity
report was significant, Wilkss = .489, F(6, 43) = 7.57, p <
.001, 2 = .51. Follow-up univariate analyses of variance revealed
that there were significant effects of personality report on both
overall ratings, F(3, 144) = 7.23, p < .001, 2 = .13, and mean
individual item ratings, F(3, 144) = 15.01, p < .001, 2 = .24.
The means and standard deviations for the mean-item and overall
ratings for each of the four personality summaries are shown in
Table 1. Tukey HSD tests revealed
Wyman & Vyse 293
TABLE 1. Means and Standard Deviations of Accuracy Ratings of
Real and Bogus NEO-Five Factor Inventory (NEO-FFI; P. T. Costa,
Jr., & R. R. McCrea, 1992a) Summaries and Real and Bogus
Astrological Charts (N = 50)
Measure Mean-item ratings Overall ratings
Real NEO-FFI M 7.19 7.23 SD 1.14 1.27Bogus NEO-FFI M 6.21 6.09
SD 1.19 1.53Real astrological M 6.18 6.17 SD 0.91 1.17Bogus
astrological M 6.05 6.48 SD 1.16 1.39
-
294 The Journal of General Psychology
that, for both dependent variables, the real NEO-FFI was rated
as significantly more accurate than the other personality reports
(p < .05) and that there were no significant differences in
accuracy ratings among the other three reports.
Last, when asked to choose which of the four personality reports
provided the most accurate description of themselves, 54.9% of
participants chose the real NEO-FFI, 19.6% chose the real
astrological report, 15.7% chose the bogus astrological report, and
only 9.8% chose the bogus NEO-FFI as the most accu-rate personality
summary. Participants selected real NEO-FFI significantly more
often than they did other summaries, 2(3, N = 51) = 25.31, p <
.001.
P. T. Barnum Effect
The hypothesis that both psychological and astrological
assessments would show the P. T. Barnum effect was supported. Each
of the four personality reports was submitted to a one-sample t
test for which the null hypothesis was that accuracy ratings in
Table 1 were not significantly different from 5 (This is right as
much as it is wrong). Both for overall ratings and for mean
individual item ratings, the t tests revealed that all four
personality reports were significantly dif-ferent from this neutral
rating. Thus, both bogus NEO-FFI and bogus astrological reports
showed evidence of the P. T. Barnum effect.
Sun-Sign Bias and Odd-Numbered Sign Bias
The hypothesis that participants prior knowledge of their sun
sign would bias them in favor of accepting the astrological
descriptions was partially sup-ported. To test this hypothesis, we
divided participants into two groupsthose who correctly reported
their sun sign and those who did not. Then we divided the
statements of the real astrological summaries into statements
related to the partic-ipants sun sign and statements unrelated to
the participants sun sign. Analysis of these items showed evidence
of sun-sign bias. Participants who knew their sun sign gave
significantly higher accuracy ratings to the sun-sign-related
statements than to the non-sun-sign statements, t(34) = 2.36, p
< .05, whereas participants who did not know their sun sign did
not, t(15) = 0.12, p > .05. Last, to determine whether the
sun-sign bias affected accuracy ratings of the real astrological
charts, we conducted a one-way MANOVA across these two groups on
the mean-item and overall ratings of the participants real
astrological charts. There were no significant differences in the
accuracy ratings of these two groups, Wilkss = .996, F(2, 49) =
0.09, p > .05.
To analyze whether there was a bias in favor of the odd-numbered
signs (Aries, Gemini, Leo, Libra, Sagittarius, Aquarius), which are
often considered more favorable than the even-numbered ones
(Taurus, Cancer, Virgo, Scorpio, Capricorn, Aquarius), we grouped
participants by odd and even signs and con-ducted a one-way MANOVA
across these groups with the overall and mean-item
-
accuracy ratings of both the real and bogus astrological charts
as the dependent variables. The analysis was not significant,
indicating that there was no bias pro-duced by the favorableness of
the participants sun sign, Wilkss = .991, F(4, 47) = 0.48, p >
.05.
Supplemental Exploratory Analyses
To further examine the results, we performed a number of
additional analy-ses. We constructed a correlation matrix to
examine the interrelationships of knowledge and belief in
astrology, belief in psychological measures, and accu-racy ratings
of the various personality measures (see Table 2). Belief in
astrology was not correlated with accuracy ratings for either the
real or bogus astrological summaries, but participants with greater
knowledge of astrology gave the bogus chart a significantly lower
total accuracy score. There was no correlation of knowledge or
belief with overall ratings of the astrological summaries.
Despite the negative relation between astrological knowledge and
accuracy ratings of the bogus astrological profile, a MANOVA
revealed that participants who correctly identified their real
astrological chart did not differ from those who did not do so in
their knowledge of or belief in astrology, Wilkss = .972, F(2, 49)
= 0.71, p > .05. Last, participants who knew their sun signs
were less likely to correctly identify their real astrological
profiles than those who did not know their sun signs (42% vs. 56%),
although a chi-square test showed that this differ-ence was not
statistically significant, 2(1, N = 52) = 0.95, p < .05.
In the case of psychological measures, participants who had
greater belief in psychologically based measures gave the bogus
NEO-FFI a higher overall rating; however, we did not see this
pattern in either the total rating of the bogus NEO-FFI or the
mean-item ratings of the real NEO-FFI. Similar to the results with
the astrological profile, those who correctly identified their real
NEO-FFI results did not differ significantly from those who did not
in their degree of belief in psychological measures of personality,
t(50) = 0.89, p > .05.
There were a number of positive correlations among the ratings
of the real and bogus astrological summaries and the real and bogus
NEO-FFI summaries, suggesting that some participants gave
consistently higher ratings to all of the reports than others.
These correlations were more often seen in the total ratings of the
astrological and psychological summaries than in the overall
ratings, perhaps because of the presumably greater reliability of
the total ratings in comparison with the single item of the overall
ratings.
Discussion
Consistent with previous research, we found that participants
were unable to identify their own astrological charts at a
greater-than-chance level, a result that extended the string of
failure using the personal validation technique. Unlike
Wyman & Vyse 295
-
296 The Journal of General Psychology
TA
BL
E 2
. Int
erco
rrel
atio
ns A
mon
g A
stro
logi
cal B
elie
f an
d K
now
ledg
e, B
elie
f in
the
Acc
urac
y of
Psy
chol
ogic
al P
erso
nalit
y M
easu
res,
and
Acc
urac
y R
atin
gs o
f R
eal a
nd B
ogus
Ast
rolo
gica
l and
NE
O-F
ive
Fac
tor
Inve
ntor
y (N
EO
-FF
I; P
. T. C
osta
, Jr.
, &
R. R
. McC
rea,
199
2a)
Sum
mar
ies
Var
iabl
e 1
2 3
4
5 6
7 8
9
10
11
1. A
stro
logi
cal b
elie
f
.2
5 .0
5 .0
5 .2
4 .2
7 .1
8 .0
3 .1
9 .2
4 .2
7 2
. Ast
rolo
gica
l kno
wle
dge
.
06
.10
.0
1 .0
3 .
27*
.14
.10
.10
.07
3. B
elie
f in
psy
chol
ogic
al m
easu
res
.01
.0
7 .0
4 .0
3 .
05
.01
.28*
.2
6 4
. Ove
rall
ratin
g of
rea
l ast
rolo
gica
l
s
umm
ary
.7
7***
.0
5 .2
2 .1
9 .2
2 .
02
.12
5. M
ean-
item
rat
ing
of r
eal a
stro
logi
cal
sum
mar
y
.1
3 .3
8**
.33*
.4
3**
.13
.34*
*
6. O
vera
ll ra
ting
of b
ogus
ast
rolo
gica
l
s
umm
ary
.4
6**
.20
.28*
.2
3 .3
6*
7. M
ean-
item
rat
ing
of b
ogus
a
stro
logi
cal s
umm
ary
.22
.33*
.2
2 .3
5*
8. O
vera
ll ra
ting
of r
eal N
EO
-FFI
s
umm
ary
.8
6***
.
06
.16
9. M
ean-
item
rat
ing
or r
eal N
EO
-FFI
s
umm
ary
.00
.29*
10. O
vera
ll ra
ting
of b
ogus
NE
O-F
FI
sum
mar
y
.81*
**
11. M
ean-
item
rat
ing
of b
ogus
NE
O-F
FI
sum
mar
y
* p