TESTING FOR HIERARCHICAL STRUCTURE AND PRIMING EFFECTS ...polisci.msu.edu/jacoby/research/values/tess/Jacoby, Prelim Report... · TESTING FOR HIERARCHICAL STRUCTURE AND PRIMING EFFECTS

TESTING FOR HIERARCHICAL STRUCTUREAND PRIMING EFFECTS AMONG

INDIVIDUAL VALUE CHOICES

William G. JacobyDepartment of Political Science

Michigan State University303 South Kedzie HallEast Lansing, MI 48824

[email protected]

A preliminary report on a study supported by the programfor Time-Sharing Experiments in the Social Sciences

February 2006

This paper is a preliminary report from a study of value choices within the American mass

public. Longstanding psychological theories have stressed that individual value preferences

are very stable, highly structured and broadly applicable to virtually all situational contexts.

But, recent research suggests that value choices may be relatively malleable and susceptible to

contextual effects. Furthermore, the apparent existence of value ambivalence raises questions

about citizens’ abilities to differentiate among values in the first place. At the present time, it

is not clear which of these two general perspectives provides the most accurate representation

of core values.

The program for Time-Sharing Experiments in the Social Sciences (TESS) provided sup-

port for a study that used some new items on an internet survey in order to to address the

important issues raised by these potentially conflicting perspectives. The empirical results

show that most people do make consistent pairwise choices among values. Furthermore, most

individuals’ sets of pairwise choices are fully transitive, meaning that they can be assembled

into hierarchical structures. Finally, value choices are largely impervious to priming effects.

SUBSTANTIVE BACKGROUND

For present purposes, “values” can be defined as an individual’s abstract, general concep-

tions about the desirable and undesirable end-states of human life ( 1973). As such, values

provide criteria for evaluating external stimuli and interacting with other elements of the

social environment. They effectively define what is ”good” and ”bad” in the world.

Human values have been the focus of an enormous amount of research across a variety

of disciplines, including political science (Kuklinski 2001; Feldman 2003), sociology (Spates

1983), economics (Katona 1975), marketing (Kamakura and Mazzon 1991), and philosophy

(Hansson 2001). However, the modern, empirical study of values really began in psychology,

particularly with the pioneering work of Milton Rokeach (1973; 1979). Subsequent research

has proceeded in a wide variety of directions (Seligman, Olson, Zanna 1996). But, there

are two general ideas drawn from this research tradition that deserve particular emphasis:

First, a critical distinguishing feature of values is their applicability across different contexts

(Schwartz and Bilsky 1987; Schwartz 1996). Values provide individuals with general evalu-

ative standards that can always be applied, regardless of the specific circumstances under

which the values are invoked. Second, people are rarely affected by single values, in isolation

from other values. Instead, hierarchically-organized value structures are the key to under-

standing human behavior (e.g., Schwartz and Bilsky 1987; Schwartz 1992; 1996; Verplanken

and Holland 2002).

There is a clear consensus in the psychological literature regarding the contextual sta-

bility and hierarchically structured nature of individual values. However, recent research–

particularly, in political science– has taken very different, and potentially conflicting, posi-

tions with respect to these value characteristics. For example, theories of value pluralism

(Tetlock 1986), issue framing (Nelson, Clawson, Oxley 1997; Grant and Rudolph 2003),

and campaign effects (McCann 1997) all hold that value choices are context-specific and

changeable in different circumstances. In a somewhat different vein, theories of value am-

bivalence (Feldman and Zaller 1992; Alvarez and Brehm 2003) and indifference (Maio and

Olson 1998) suggest that people may have trouble making and/or expressing reliable and

consistent choices between values in the first place.

The apparent variability and ambivalence in human values pose serious challenges for tra-

ditional psychological theories. If individual value preferences are changeable with contextual

factors, then the ”transsituational” (Schwartz 1996) nature of values is badly compromised

(Seligman and Katz 1996). Similarly, ambivalence about values challenges the existence of

transitive choices and, thereby, largely precludes the existence of individual value structures

(Maio and Olson 1998; Jacoby 2002). Thus, it is critically important to determine whether

the recent findings have merit, or whether the traditional theoretical understanding provides

the more accurate depiction of individual values.

The existing empirical evidence is ambiguous and largely indeterminate. For example,

studies that seem to reveal framing effects usually examine differences in reactions to single

values. In fact, preference orders across multiple values do not vary systematically across

2

issue frames (Nelson et al. 1997, p. 576). Similarly, most studies of ambivalence never test

directly individual willingness or ability to make choices between specific values; instead,

they examine indirect evidence based upon the supposed consequences of value ambivalence

(Feldman and Zaller 1992; Alvarez and Brehm 2002).

Finally, and perhaps most fundamentally, the very existence of hierarchical value struc-

tures has never been rigorously tested. Again, some studies simply focus on single values,

taken separately from other values (e.g., Feldman 1988). In other cases, researchers ask

subjects to rank-order a set of values (e.g., Rokeach 1973; Schwartz 1992). Both of these

approaches bypass any direct assessment of consistency or transitivity in value choices. The

one line of work that does attempt to construct preference orders from pairwise value choices

(Jacoby 2002; 2006) is based upon a very limited dataset which precludes evaluation of the

measurement error that undoubtedly exists in expressed statements of choices between values

(Miethe 1985; McCarty and Shrum 2000).

STUDY DESIGN AND DATA COLLECTION

The problems outlined in the previous section are addressed by using a new set of items,

administered as part of an internet survey that was supported by the program for Time-

Sharing Experiments in the Social Sciences. The project focuses on individual choices among

five important values: Liberty, economic security, equality, morality, and social order. These

particular values were selected because of their salience in American political culture and

their relevance to current policy controversies. The values were presented to the survey

respondents under varying experimental conditions aimed at priming particular values and

value conflicts. Respondents made choices among subsets of the values in a manner that

enables empirical tests of hierarchical value structuring. Such tests are impossible with more

traditional data collection strategies (i.e., rating single values or rank-ordering the full set of

values).

The overall plan of the study is laid out in Figure 1. Respondents were randomly assigned

to one of eight conditions. Seven of these conditions involve two survey questions that either

3

emphasize a specific value (five conditions) or the conflict between a pair of values (two

conditions). A control group was given two questions that do not mention values. The

questions used in each of the conditions are shown in Part I of the Appendix.

Next, all respondents were asked to place themselves along the liberal-conservative con-

tinuum using a relatively standard seven-point scale (Part II in the Appendix). This item

served as a distraction task for respondents in the seven experimental conditions. The re-

sponses to this question are also intrinsically important because ideology is one of the factors

that is hypothesized to affect value ambivalence (Feldman and Zaller 1992) and, hence, the

degree of hierarchical structuring among individual value choices.

The value choices were elicited next, using the method of triads (Coombs 1964; Weller

and Romney 1988). Respondents were first shown a screen that introduced, and provided

a brief definition for, each of the five values (Part IIIA in the Appendix). After that, they

were shown a series of ten screens. On each screen, a distinct combination of three values

(i.e., “a triad”) was listed. For each triad, respondents were asked to indicate which one of

the three values is most important, and which one of the three is least important. Given the

nature of the internet survey, respondents could make the selections very easily, by clicking

radio buttons with the computer mouse. The ten triads (i.e., all possible subsets of three

values from the five) are shown in Part IIIB of the Appendix. The order of the triads was

varied randomly, as was the order of the three values within each triad.

The internet survey was conducted by Knowledge Networks, Inc.; it is designated ”TESS

37 Jacoby.” Data collection was carried out from June 16 through June 21, 2005. A total of

1000 surveys were fielded, and 649 respondents completed the interview schedule. Thus, the

response rate was about 65 %.

CONSISTENCY AND TRANSITIVITY IN VALUE CHOICES

Let us begin by considering choices among pairs of values. The individuals’ responses

to each triad can be broken down into three pairwise choices. For example, assume a triad

4

containing values A, B, and C, with a respondent stating that A is most important and

C is least important. This, in turn, implies that A is more important than B, B is more

important than C, and A is more important than C.

In the full set of ten triads, each pair of values appears in three different triads. There-

fore, three replications of each respondent’s choice between each value pair are recovered.

This allows an assessment of consistency in pairwise choices (i.e., how many times does the

respondent make the same choice for each pair?) and permits the determination of that

person’s “dominant” choice (i.e., the value chosen two or three times) for each value pair.

Consistency in Pairwise Choices

Table 1 presents basic descriptive data on the pairwise value choices. Each row of the

table represents one of the ten distinct value pairs that can be derived from the set of five

values. Beginning at the left side of the table, the first four columns represent the number of

times the first-mentioned value in that row is chosen over the second-mentioned value, across

the three replications for that pair. The rightmost column within each row summarizes the

amount of consistency in choices involving that value pair; that is, the percentage of times

respondents chose one or the other value consistently across all three replications of that

particular value pair.

The entries in Table 1 show that the respondents are highly consistent in their value

choices. In every case, 70% or more make the same pairwise choice across all three replica-

tions. Stated somewhat differently, the percentage of respondents who exhibit some degree

of inconsistency ranges from a high of 29.58% (for choices between equality and morality)

or just under one-third of the sample, to a low of 19.57% (for choices between morality and

social order) or just under one-fifth of the sample

As a standard of comparison for the entries in Table 1, consider an admittedly unrealistic

situation in which everyone is indifferent between two values (due to ambivalence, failure to

recognize any potential conflict between the values, and so on). In that case, the probability

5

of choosing one value over the other on a single choice is 0.50. And, the probability of

making consistent choices across all three replications is only 0.25. In other words, if people

did not maintain real preferences between a given pair of values, then only one-fourth of the

sample should be exhibiting consistent choices across the three replications of each choice.

But, again, the empirical levels of consistency are much higher than this stringent standard.

This, in turn, suggests that most people really do have crystallized preferences across the

values examined in this study.

Variability in Value Importance

The replicated pairwise choices can be used to determine each person’s “dominant” choice

for each value pair; that is, the value that is chosen two or three times (i.e., a majority) across

the replications. These dominant pairwise choices can, in turn, be used to determine each

respondent’s beliefs about the relative importance of the separate values. This is accom-

plished by assigning each value a score (for each survey respondent), showing the number of

times that value is the dominant choice over other values in that person’s full set of choices

(Peffley, Knigge, Hurwitz 2001). As a simple example, consider three values: A, B, and C.

Assume that these are presented to respondents in triads (probably along with other values),

and that an arbitrary respondent has A dominating B (i.e., A was the selected over B on at

least two of the three choices), A also dominating C, and B dominating C. For this person,

A would receive a score of 2, since he/she has it dominating two other values (B and C); B

would be scored 1 (it only dominates C); and C would receive a 0, since it never dominates

any other values (even though it may have been chosen in one of the replicated pairwise

choices). With the Knowledge Networks data, there are five values, so the scores for each

value can range from zero to four.

Table 2 shows the distribution of importance scores for each of the four values. The most

striking feature of the table is the wide variability among the preferences. At the individual

level, there does not seem to be general agreement about which values are most (or least)

important. “Morality” received the largest number of high ( ”4”) scores but even so, only

6

about one-third of the respondents rated this value as more important than any other. And,

very few people rated “equality” or “social order” very highly; less than eight percent of the

respondents placed each of these values at the top of their rankings. At the other extreme,

”social order” received the largest number of zero scores. But, again, only about 30% of

the respondents ranked this value in last place. Overall, it is fair to say that each of the

four values shows up in every possible position within the value hierarchies of a substantial

number of people.

The dot plot shown in Figure 2 summarizes succinctly the aggregate distribution of

importance ratings for each value. Specifically, the horizontal position of the point plotted

within each row corresponds to the mean importance score for that value; the solid bar

around each point represents a 95% confidence interval. From the figure, it can be seen that

morality, economic security, and liberty are considered to be the most important values.

Their mean importance scores are 2.36, 2.32, and 2.29, respectively. However, as can be

discerned from the overlapping confidence intervals in Figure 2, the differences between

these means are not statistically significant. The remaining two values, equality and social

order, have significantly lower mean importance scores, at 1.58 and 1.45, respectively. Again,

the difference between these two means is not statistically significant.

At the aggregate level, Americans do not seem to differentiate fully among the different

values. Instead, public opinion sorts values into two distinct sets: One group that is relatively

more important, composed of morality, liberty, and economic security and a second group

that is usually viewed as less important, equality and social order.

Are Individual Value Choices Transitive?

For each survey respondent, the full set of dominant pairwise choices can be examined

for transitivity. If transitivity holds in a respondent’s value choices then, for any set of three

values (again, say A, B, and C), choices on two of the pairs imply the choice that will be

made on the third pair. So, if A is chosen over B, and B is chosen over C, then transitivity

implies that A is also chosen over C. If an individual’s choices among the values are fully

7

transitive (i.e., the transitivity relation holds for all possible subsets of three values), then

each of the values will receive a distinct score. Therefore, it is possible to assemble that

person’s pairwise choices into a complete rank-ordering of the values. Figure 3 provides

a simple example showing how the dominant pairwise pairwise choices are obtained from

a hypothetical individual’s responses to the triads, and how the dominant choices used to

construct an overall preference ordering for that person.

Intransitivity exists in a set of choices among three values whenever the third choice is

not implied by the other two choices. For example, A is chosen over B, B over C, and then C

is chosen over A. Here, each value is only selected over a single other value, so it is impossible

to construct a rank-ordering of the three; hence, the three choices form an intransitive set.

Thus, the relative prevalence of transitivity and intransitivity in the data constitutes a direct

empirical test for the existence of individual value hierarchies.

Table 3 provides the basic data on transitivity among individual value choices. The first

row of the table shows that only 12.48% of the respondents exhibited any intransitivity

among all of their dominant pairwise choices. Stated differently, nearly nine-tenths of the

respondents maintain fully-ordered (i.e., completely transitive) value preferences. The re-

maining rows in the table give the proportion of intransitive choices on all subsets of three

values. Parenthetically, note that some people are intransitive on more than one triad; hence

the proportions intransitive on the particular triads sum to more than the 12.48% figure given

above.

The information in Table 3 shows a very obvious pattern: On any given triad, only a

tiny minority of the respondents give intransitive choices among the values. The number of

intransitivities never rises above about three percent on any value triad, and it is generally

smaller than that. Clearly, the vast majority of the respondents have no difficulty providing

a complete ordering of the five values. This, in turn, provides strong confirmation for the

existence of fully-differentiated hierarchical value structures within the mass public.

8

CAN VALUE CHOICES BE PRIMED?

The experimental manipulation in this study randomly assigned survey respondents to

eight different conditions. Five of these conditions are each intended to prime one particular

value. Specifically, respondents were asked to indicate the degree to which the Bush and

Kerry presidential campaigns emphasized one of the values (See sections IA through IE in

the Appendix for question wording). Two other conditions emphasized value conflict, by

asking respondents to indicate the degree to which the presidential campaigns emphasized

liberty versus social order (see section IF in the Appendix) and liberty versus equality (see

section IG in the Appendix). Finally, the eighth condition did not prime any values; instead,

respondents were asked about partisanship in the presidential campaigns (Appendix section

IH). After the experimental manipulation, all respondents were asked to place themselves

along a seven-point liberal-conservative scale. Then, they completed the battery of value

choice items.

If values are the “transsituational” phenomena argued by Rokeach (1973), Schwartz

(1996) and others, then priming should have no discernible effect on pairwise value choices.

Alternatively, if feelings about values can be manipulated through environmental conditions

(e.g., Tetlock 1986; Nelson, Clawson, Oxley 1997; Maio and Olson 1998), then priming

particular values should enhance the degree to which those values are chosen when they are

juxtaposed against other values. The design of the current study enables multiple tests of

the priming hypothesis.

With five values, there are ten pairwise choices. For each pair, the dependent variable

is the probability that one value is the dominant choice over the other. It is important

to emphasize that our empirical measurement of these choices is particularly reliable, since

dominant choices result from the choices made over the three replications of each value pair

from the triad items. For each of the ten value pairs, there are two independent variables.

These are dummies, scored one if one of the two values in the pair was primed, and zero

otherwise. For example, assume that the dependent variable is the probability of choosing

9

equality over economic security. In that case, people who received the equality prime are

scored one on the first predictor, and people who received the economic security prime are

scored one on the other predictor. According to the priming hypothesis, the first predictor

should increase the probability of choosing equality, while the second predictor should de-

crease that probability. Note that respondents in either of the two value conflict conditions

are coded as having been primed for both values in the pair.

Table 4 contains the results from this analysis. Each row of the table contains the

maximum likelihood estimates for a logistic regression predicting a value choice using primes

for the two values included in that particular pair. Notice first that the equations do not fit

the data very well. All of the pseudo R-squared values are very small (i.e., all but one have

zeroes out to at least the second decimal place). Turning to the individual value primes,

a number of the coefficients have the wrong sign. That is, five of the coefficients for the

first prime are negative (they should be positive) and three of the coefficients for the second

prime are positive (they should be negative). Note, however, that only two of the coefficients

are statistically different from zero at the 0.05 level (directional test): The equality prime

increases the probability of choosing that value over economic security and also over social

order. In all of the remaining instances, exposure to a value prior to a pairwise choice

situation has no systematic impact on the probability of choosing that value over another

value.

Furthermore, even the two seemingly-significant coefficients in Table 4 should not be

taken too seriously. Since each of the predictors in each of the equations constitutes a

potential prime, there are actually 20 tests contained in the empirical results. The existence

of these multiple tests should be taken into account when assessing statistical significance.

The Bonferroni adjustment for performing n one-sided tests at the α level is α/n. Here, that

produces a probability value of 0.0025; in other words, the latter value is the probability level

that should be required to reject the null hypothesis at the 0.05 level, across 20 different

tests. This would, in turn, require a coefficient that is 2.81 times larger than its standard

10

error. Neither of the significant coefficients in Table 4 meet this more stringent (but more

valid) standard. Thus, there is virtually no reliable evidence in these results for priming

effects in value choices.

CONCLUSION

The TESS Program provided a unique opportunity for an empirical analysis of of individ-

ual value choices. The preliminary results described in this report lead to several conclusions–

all, admittedly, somewhat tentative at this point. First, the successful completion of the data

collection, itself, shows that the method of triads is a reasonable strategy for obtaining repli-

cated pairwise comparisons among values. This is important because it provides reliable

information on value choices. The latter can be used to test for individuals’ hierarchical

value structures. Without this capability, the existence of such rank-ordered structures must

remain an untested assumption.

Second, the data from the internet survey show that people do, generally, make consis-

tent choices among values. That, in turn, attests to the viability of values as a psychological

construct. If values were only “truisms,” without strong cognitive and affective underpin-

nings, then we would expect people to move frequently from one to the other when asked

to choose between them. But, that just did not happen with these data: The vast majority

of the survey respondents made the same choice across the three replications of each value

pair. So, this provides strong evidence against criticisms that have been leveled against the

values concept in the recent literature.

Third, the distribution of value preferences reveals both consensus and conflict in the

ways that Americans think about fundamental principles like liberty and equality. There

appears to be wide agreement that morality, liberty, and economic security comprise the most

important values, with equality and social order falling at somewhat less salient positions

in public esteem. But, these aggregate patterns emerge from enormous variability among

individual choices among the values. All five of these values are highly important for some

subsets of the public, and less so for others.

11

Fourth, individual value choices are usually transitive. This means that people really do

differentiate among values. And, that is consistent with the existence of hierarchical value

structures throughout the mass public.

Fifth, the data show no evidence of priming effects in value choices. Experimental ma-

nipulations intended to enhance the accessibility of one value relative to another simply had

no effect on the probability of choosing one value over the other. Such a finding is consistent

with the view that values are “transsituational” phenomena, or judgmental standards that

are relatively immune to external influences.

In conclusion, the initial evidence from this study vindicates psychological theories and

scholarly understandings that have been widely accepted, but seldom tested, over the years.

Of course, further analysis is necessary in order to deal with questions like the following:

What factors influence individual value choices? What causes the inconsistent and/or intran-

sitive value choices that, although relatively rare, do arise in the data? Are there identifiable

subsets of the public whose value choices can be affected by environmental factors? Of

course, I intend to address all of these questions in the immediate future. But, even at this

early point, it is clear that the TESS program’s support of this study has enabled important

insights regarding value choices in the American mass public.

12

REFERENCES

Alvarez, R. Michael and John Brehm. (2002) Hard Choices, Easy Answers. Princeton, NJ:Princeton University Press.

Coombs, Clyde H. (1964) A Theory of Data. New York: Wiley.

Feldman, Stanley. (2003) “Values, Ideology, and the Structure of Political Attitudes.” InOxford Handbook of Political Psychology. David O. Sears, Leonie Huddy, Robert Jervis(Editors). New York: Oxford University Press.

Feldman, Stanley and John Zaller. (1992) “The Political Culture of Ambivalence: Ide-ological Responses to the Welfare State.” American Journal of Political Science 36:268-307.

Grant, J. Tobin and Thomas J. Rudolph. (2003) “Value Conflict, Group Affect, and theIssue of Campaign Finance.” American Journal of Political Science 47: 453-469.

Hansson, Sven Ove. (2001) The Structure of Values and Norms. Cambridge, UK: Cam-bridge University Press.

Jacoby, William G. (2002) Core Values and Political Attitudes. In Barbara Norrander andClyde Wilcox (Editors), Understanding Public Opinion (Second Edition). Washington,DC: Congressional Quarterly.

Jacoby, William G. (2006) “Value Choices in the American Public.” American Journal ofPolitical Science. Forthcoming.

Kamakura, Wagner A. and Jose Alfonso Mazzon. (1991) “Value Segmentation: A Modelfor the Measurement of Values and Value Systems.” Journal of Consumer Research18: 208-218.

Katona, George. (1975) Psychological Economics. New York: Elsevier.

Kuklinski, James H. (2001) Introduction: Political Values.” In Citizens and Politics: Per-spectives from Political Psychology. James H. Kuklinski (Editor). Cambridge, UK:Cambridge University Press.

Maio, Gregory R. and James M. Olson. (1998) Values as Truisms: Evidence and Implica-tions.” Journal of Personality and Social Psychology 74: 294-311.

McCann, James A. (1997) Electoral Choices and Core Value Change: The 1992 PresidentialCampaign.” American Journal of Political Science 41: 564-583.

McCarty, John A. and L. J. Shrum. (2000) “Measurement of Personal Values in Research.”Public Opinion Quarterly 64: 271-298.

Miethe, Terance D. (1985) “The Validity and Reliability of Value Measurements.” Journalof Psychology 119: 441-453.

Nelson, Thomas E.; Rosalee A. Clawson; Zoe M. Oxley. (1997) “Media Framing of a CivilLiberties Conflict and Its Effect on Tolerance.” American Political Science Review 91:567-583.

Rokeach, Milton. (1973) The Nature of Human Values. New York: Free Press.

Rokeach, Milton. (1979) Understanding Human Values: Individual and Societal. New York:Free Press.

Schwartz, Shalom H. (1992) “Universals in the Content and Structure of Values: TheoreticalAdvances and Empirical Tests in 20 Countries.” In Mark P. Zanna (Editor), Advancesin Experimental Social Psychology. Orlando, FL: Academic Press.

Schwartz, Shalom H. (1996) “Value Priorities and Behavior: Applying a Theory of Inte-grated Value Systems.” In Clive Seligman, James M. Olson, Mark P. Zanna (Editors),he Psychology of Values: The Ontario Symposium, Volume 8. Mahwah, NJ: LawrenceErlbaum Associates.

Schwartz, Shalom H. and Wolfgang Bilsky. (1987) “Toward a Universal PsychologicalStructure of Human Values.” Journal of Personality and Social Psychology 53: 550-562.

Seligman, Clive and Albert N. Katz. (1996) “The Dynamics of Value Systems.” In ThePsychology of Values: The Ontario Symposium, Volume 8. Clive Seligman, James M.Olson, Mark P. Zanna (Editors). Mahwah, NJ: Lawrence Erlbaum Associates.

Seligman, Clive; James M. Olson; Mark P. Zanna. (1996) The Psychology of Values: TheOntario Symposium, Volume 8. Mahwah, NJ: Lawrence Erlbaum.

Spates, James L. (1983) “The Sociology of Values.” Annual Review of Sociology 9: 27-49.

Tetlock, Philip E. (1986) “A Value Pluralism Model of Ideological Reasoning.” Journal ofPersonality and Social Psychology 50: 819-827.

Verplanken, Bas and Rob. W. Holland. (2002) “Motivated Decision Making: Effects of Ac-tivism and Self-Centrality on Values in Choices and Behavior.” Journal of Personalityand Social Psychology 82: 434-447.

Weller, Susan C. and A. Kimball Romney. (1988) Systematic Data Collection. NewburyPark, CA: Sage.

14

APPENDIX

ITEM LIST FOR INTERNET SURVEY

I. Experimental Manipulation. The general objective is to encourage respondents to thinkabout values without explicitly invoking their own value choices. To this end, the items askrespondents to assess the degree to which values were apparent in the 2004 major partypresidential campaigns (this, of course, assumes that the survey would go into the field after theelection). There are eight different conditions total: Five conditions prime a single value; twoconditions prime conflict between two values; and one condition is a control which does notprime any values. In each condition, respondents are asked two questions. Note that the orderin which the candidates are mentioned will be varied randomly.

A. “Liberty” condition:

In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized the idea of liberty, or guaranteeing the widest freedom possible foreveryone to act and think as they consider most appropriate.

Please use the following scale to indicate how much you think the Bush campaignemphasized liberty. Just use the mouse to click on the appropriate scale value:

1 2 3 4 5 6 7

The Bush campaigndid not emphasize liberty at all

The Bush campaignplaced a great deal of

emphasis on liberty

How about the Kerry campaign? How much did the Kerry campaign emphasize liberty?

1 2 3 4 5 6 7

The Kerry campaigndid not emphasize liberty at all

The Kerry campaignplaced a great deal of

emphasis on liberty

16

B. “Equality” condition:

In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized the idea of equality, or making sure that everybody has the samechance to get ahead in life.

Please use the following scale to indicate how much you think the Bush campaignemphasized equality. Just use the mouse to click on the appropriate scale value:

1 2 3 4 5 6 7

The Bush campaigndid not emphasize equality at all

The Bush campaignplaced a great deal ofemphasis on equality

How about the Kerry campaign? How much did the Kerry campaign emphasize equality?

1 2 3 4 5 6 7

The Kerry campaigndid not emphasize equality at all

The Kerry campaignplaced a great deal ofemphasis on equality

17

C. “Economic Security” condition:

In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized the idea of economic security, or guaranteeing a steady job and adecent income to everyone.

Please use the following scale to indicate how much you think the Bush campaignemphasized economic security. Just use the mouse to click on the appropriate scale value:

1 2 3 4 5 6 7

The Bush campaigndid not emphasize economic security at all

The Bush campaignplaced a great deal

of emphasis oneconomic security

How about the Kerry campaign? How much did the Kerry campaign emphasize economicsecurity?

1 2 3 4 5 6 7

The Kerry campaigndid not emphasize economic securityat all

The Kerry campaignplaced a great deal

of emphasis oneconomic security

18

D. “Morality” condition:

In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized the idea of morality, or people living according to the rules thatconstitute decent human behavior.

Please use the following scale to indicate how much you think the Bush campaignemphasized morality. Just use the mouse to click on the appropriate scale value:

1 2 3 4 5 6 7

The Bush campaigndid not emphasize morality at all

The Bush campaignplaced a great deal ofemphasis on morality

How about the Kerry campaign? How much did the Kerry campaign emphasize morality?

1 2 3 4 5 6 7

The Kerry campaigndid not emphasize morality at all

The Kerry campaignplaced a great deal ofemphasis on morality

19

E. “Social Order” condition:

In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized the idea of social order, or being able to live in a safe and peacefulsociety where the laws are respected and enforced.

Please use the following scale to indicate how much you think the Bush campaignemphasized social order. Just use the mouse to click on the appropriate scale value:

1 2 3 4 5 6 7

The Bush campaigndid not emphasize social order at all

The Bush campaignplaced a great deal of

emphasis on social order

How about the Kerry campaign? How much did the Kerry campaign emphasize social order?

1 2 3 4 5 6 7

The Kerry campaigndid not emphasize social order at all

The Kerry campaignplaced a great deal of

emphasis on social order

20

F. “Liberty versus Social Order” condition (note that the order in which liberty and social orderare mentioned will be varied randomly across respondents in this condition):

In the recent presidential election campaign, the candidates and parties emphasized thedifferences between certain themes and ideas. We are interested in your feelings about thehow much the two candidates emphasized the idea of liberty (that is, guaranteeing thewidest freedom possible for everyone to act and think as they consider most appropriate)versus the idea of social order (that is, being able to live in a safe and peaceful societywhere the laws are respected and enforced).

Please use the following scale to indicate how much you think the Bush campaignemphasized liberty versus social order. Just use the mouse to click on the appropriate scalevalue:

1 2 3 4 5 6 7

The Bush campaignplaced much more emphasis on libertythan on social order

The Bush campaignemphasized liberty

and social orderjust about equally

The Bush campaignplaced much moreemphasis on social

order than on liberty

How about the Kerry campaign? How much did the Kerry campaign emphasize libertyversus social order?

1 2 3 4 5 6 7

The Kerry campaignplaced much more emphasis on libertythan on social order

The Kerry campaignemphasized liberty

and social orderjust about equally

The Kerry campaignplaced much moreemphasis on social

order than on liberty

21

F. “Liberty versus Equality” condition (note that the order in which liberty and equality arementioned will be varied randomly across respondents in this condition):

In the recent presidential election campaign, the candidates and parties emphasized thedifferences between certain themes and ideas. We are interested in your feelings about thehow much the two candidates emphasized the idea of liberty (that is, guaranteeing thewidest freedom possible for everyone to act and think as they consider most appropriate)versus the idea of equality (that is, making sure that everybody has the same chance to getahead in life).

Please use the following scale to indicate how much you think the Bush campaignemphasized liberty versus equality. Just use the mouse to click on the appropriate scalevalue:

1 2 3 4 5 6 7

The Bush campaignplaced much more emphasis on libertythan on equality

The Bush campaignemphasized libertyand equality just

about equally

The Bush campaignplaced much more

emphasis on equality than on liberty

How about the Kerry campaign? How much did the Kerry campaign emphasize libertyversus equality?

1 2 3 4 5 6 7

The Kerry campaignplaced much more emphasis on libertythan on equality

The Kerry campaignemphasized libertyand equality just

about equally

The Kerry campaignplaced much more

emphasis on equality than on liberty

22

E. Control Group, or “Political Party” condition:

In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized their ties to their own political parties.

Please use the following scale to indicate how much you think the Bush campaignemphasized its ties to the Republican party. Just use the mouse to click on the appropriatescale value:

1 2 3 4 5 6 7

The Bush campaigndid not emphasize the Republican party at all

The Bush campaignplaced a great dealof emphasis on the

Republican Party

How about the Kerry campaign? How much did the Kerry campaign emphasize theDemocratic party?

1 2 3 4 5 6 7

The Kerry campaigndid not emphasize the Democraticparty at all

The Kerry campaignplaced a great dealof emphasis on the

Democratic Party

23

II. Distraction Task. The idea here is to encourage respondents to think about something otherthan the value or values that were included in their version of the experimental manipulation.The latter should still remain accessible in short-term memory. But, this distraction is intendedinhibit the degree to which their own expressed value choices (obtained immediately afterward)are based directly upon their responses to the experimental manipulation items.

Many people use the labels “liberal,” “moderate,” or “conservative” to describe their ownpolitical views. Which one of the following labels best describes your own political ideas?Please use the mouse to click the check box next to the appropriate label:

Q Extremely conservative

Q Conservative

Q Sightly conservative

Q Moderate, middle of the road

Q Slightly liberal

Q Liberal

Q Extremely liberal

Q I don’t generally think ofmyself in these terms.

24

III. Value Triads. The objective here is to elicit respondents’ value preferences in a manner thatprovides repeated measurement of each pairwise value comparison and does not presuppose theexistence of a fully-ranked value hierarchy.

A. Instruction Screen:

We’d like to ask you about some things that are important for our society, such as liberty,equality, economic security, morality, and social order. First, here is what we mean by theseideas:

By LIBERTY we mean a guarantee of the widest freedom possible for everyone to actand think as they consider most appropriate.

By EQUALITY we mean narrowing the gap in wealth and power between the rich andthe poor.

By ECONOMIC SECURITY we mean the gurarantee of a steady job and a decentincome.

By MORALITY we mean people living according to the rules that constitute decenthuman behavior.

By SOCIAL ORDER we mean being able to live in an orderly and peaceful societywhere the laws are respected and enforced.

All five of these ideas are important, but sometimes we have to choose between what is moreimportant and what is less important. And, the specific choices we make sometimes dependupon the comparisons we have to make.

On the next few screens, we will show you these ideas in sets of three. For each set, pleaseuse the mouse to indicate the idea that you think is most important of the three, and also theidea that you think is least important of the three. In some cases, you might think all threeof the ideas are very important, but please try to indicate the ones you think are most andleast important if you had to choose between them.

If you absolutely cannot decide which ideas are most or least important, then just click thebox at the bottom of the screen, and move on to the next set of three ideas.

25

B. Value triads. Each set of three values will appear on a separate screen. Note that the orderof the triads, as well as the order of the three values within each triad, will be variedrandomly.

The MOSTIMPORTANTidea from these

three is:

The LEASTIMPORTANTidea from these

three is:

Q Q Liberty

Q Q Equality

Q Q Economic Security

The MOSTIMPORTANTidea from these

three is:

The LEASTIMPORTANTidea from these

three is:

Q Q Equality

Q Q Morality

Q Q Social Order

The MOSTIMPORTANTidea from these

three is:

The LEASTIMPORTANTidea from these

three is:

Q Q Economic Security

Q Q Morality

Q Q Social Order

26

The MOSTIMPORTANTidea from these

three is:

The LEASTIMPORTANTidea from these

three is:

Q Q Liberty

Q Q Economic Security

Q Q Morality

The MOSTIMPORTANTidea from these

three is:

The LEASTIMPORTANTidea from these

three is:

Q Q Liberty

Q Q Morality

Q Q Equality

The MOSTIMPORTANTidea from these

three is:

The LEASTIMPORTANTidea from these

three is:

Q Q Social Order

Q Q Equality

Q Q Economic Security

27

The MOSTIMPORTANTidea from these

three is:

The LEASTIMPORTANTidea from these

three is:

Q Q Equality

Q Q Social Order

Q Q Liberty

The MOSTIMPORTANTidea from these

three is:

The LEASTIMPORTANTidea from these

three is:

Q Q Morality

Q Q Economic Security

Q Q Equality

The MOSTIMPORTANTidea from these

three is:

The LEASTIMPORTANTidea from these

three is:

Q Q Economic Security

Q Q Social Order

Q Q Liberty

28

The MOSTIMPORTANTidea from these

three is:

The LEASTIMPORTANTidea from these

three is:

Q Q Liberty

Q Q Morality

Q Q Social Order

Table 1: Consistency in pairwise value choices.

Number of times the first value is chosen overthe second value across the three replications Percentage making

consistent choicesacross all three

replicationsZero One Two Three

Liberty versus equality 25.12% 11.56% 15.72% 47.61% 72.73%

Liberty versus economic security 38.67% 12.02% 12.33% 36.98% 75.65%

Liberty versus morality 41.91% 10.02% 10.63% 37.44% 79.35%

Liberty versus social order 21.57% 10.48% 11.40% 56.55% 78.12%

Equality versus economic security 52.39% 14.64% 10.02% 22.96% 75.35%

Equality versus morality 56.09 10.02% 19.57% 14.33% 70.42%

Equality versus social order 33.13% 12.79% 12.63% 41.45% 74.58%

Economic security versus morality 44.68% 7.24% 12.48% 35.59% 80.27%

Economic security versus social order 22.03% 11.71% 15.87% 50.39% 72.42%

Morality versus social order 25.27% 8.32% 11.25% 55.16% 80.43%

Note: In each row, the entries in the four leftmost columns indicate the percentage of respondents that selected thefirst-mentioned value zero, one, two, or three times across the three replications of the pairwise choice. Therightmost column gives the percentage of respondents that made fully consistent choices across all threereplications. Sample size is 649.

Table 2: The distribution of importance scores for each value.

Value

Liberty EqualityEconomicSecurity Morality

SocialOrder

ImportanceScore

Zero: 14.48% 23.88% 9.09% 16.18% 30.20%

One: 15.10% 27.43% 20.80% 15.41% 25.89%

Two: 20.96% 23.57% 22.65% 16.80% 19.72%

Three: 26.19% 17.41% 23.88% 19.11% 16.80%

Four: 23.27% 7.70% 23.57% 32.51% 7.40%

Note: Cell entries give the percentage of respondents with each importance rating for each value. The number ofobservations is 649.

Table 3: Transitivity in pairwise value choices.

Value triad

Percentage whose choices across valuepairs are intransitive

Any intransitive choices across all ten value triads 12.48%

Liberty, equality, economic security 2.47%

Liberty, equality, morality 3.39%

Liberty, equality, social order 1.85%

Liberty, economic security, morality 0.77%

Liberty, economic security, social order 2.16%

Liberty, morality, social order 1.08%

Equality, economic security, morality 2.16%

Equality, economic security, social order 2.77%

Equality, morality, social order 1.08%

Economic security, morality, social order 1.69%

Note: The entry in the first row shows the percentage of respondents with any intransitivity on any ofthe subsets of three values (i.e., triads). The entries in the remaining rows of the table give thepercentage of respondents whose choices on that specific triad are intransitive. All percentagesare based upon 649 observations.

Table 4: The impact of value priming on pairwise choices between values.

Maximum likelihood logistic regression coefficients (standard errors in parentheses)

Pseudo R2Value choice: Prime for firstvalue in pair

Prime for secondvalue in pair

Liberty over equality

-0.068(0.196)

-0.086(0.220)

0.000

Liberty over economic security

0.019(0.187)

0.093(0.242)

0.000

Liberty over morality

0.272(0.187)

-0.188(0.250)

0.004

Liberty over social order

-0.082(0.203)

-0.220(0.223)

0.001

Equality over economic security

0.466*(0.215)

0.404(0.249)

0.007

Equality over morality

0.340(0.215)

0.235(0.255)

0.004

Equality over social order

0.589*(0.220)

-0.159(0.209)

0.010

Economic security over morality

-0.308(0.242)

-0.014(0.244)

0.002

Economic security over social order

-0.053(0.254)

-0.154(0.216)

0.001

Morality over social order

-0.133(0.256)

-0.083(0.218)

0.000

Note: Table entries are maximum likelihood coefficients for a logistic regression equation predicting the valuechoice listed in the leftmost column. The number of observations is 649 in each equation. From the left, thefirst coefficient in each row is for a dummy variable indicating that the first-mentioned value is primed. Thesecond coefficient is for a dummy variable indicating that the second-mentioned value is primed. Accordingto the value priming hypothesis, the first coefficient should be positive, and the second coefficient should benegative. Coefficients marked with an asterisk are significantly different from zero at the 0.05 level in adirectional test.

EXPERIMENTAL MANIPULATION(Eight Conditions, Two Items in Each)

Five conditions, each priming one specific value

Two conditions, each priming a conflictbetween two specific values

One control group receives no value priming

DISTRACTION TASK(All Respondents, One Item)

Liberal-conservative self-placement

INSTRUCTIONS FOR TRIADS(All Respondents)

One screen, providing a one-sentence definition for eachvalue, and instructions for responding to the triad items

VALUE TRIADS(All Respondents, Ten Items)

Ten screens, each of which displays three values. Respondentsselect the most- and least-important value on each screen

Figure 1: Overall Structure of Proposed Study

Mean importance score

0 1 2 3 4

Social Order

Equality

Liberty

Economic Security

Morality

Figure 2: Mean value importance scores.

Note: Points in the dotplot represent mean importance scores (calculated across 649 internet surveyrespondents) for each value. The error bars around each point represent the 95% confidenceinterval for the mean.

Figure 3: A simple example showing how triads are used to obtain a full preference order (and,in so doing, test for transitivity). The example assumes only four values (A, B, C, andD) and follows a single, hypothetical, person’s choices.

A. Triads: All possible subsets of three values (i.e., “triads”) are presented to the respondent. For eachtriad, the most-preferred value is underlined and the least-preferred value is crossed out.

A B CA B DA C DB C D

B. Dominant Pairwise Choices: With four values, each value pair occurs in two different triads(e.g., A and B appear in the first two triads). For all six possible value pairs, examine the choicesmade between them.

B is chosen over A twice (in the first two triads), hence B dominates AC is chosen over A twice (in the first and third triads), hence C dominates AA is chosen over D twice (in the second and third triads), hence A dominates DB is chosen over C twice (in the first and fourth triads), hence B dominates CB is chosen over D twice (in the second and fourth triads), hence B dominates DC is chosen over D twice (in the third and fourth triads), hence C dominates D

Based upon the preceding results, the hypothetical respondent is fully consistent in his/her pairwisechoices (i.e., identical choices occur on both comparisons for each value pair).

C. Obtain the Preference Order from the Dominant Choices: Count the number of times eachvalue dominates another value. The resultant numeric scores give the preference order for therespondent.

A dominates one other value (D), so it is assigned a 1B dominates three other values (A, C, and D), so it is assigned a 3C dominates two values (A and D), so it is assigned a 2D dominates no other values, so it is assigned a 0

For this respondent, the preference ordering for the values (based upon the dominant pairwisechoices) is B (most preferred), C, A, D (least preferred).

If the scores assigned to the values are fully-ordered (as they are in this example), then all of thechoices are transitive and a complete preference ordering for the values can be constructed. If thereare any ties in the scores, then at least some of the pairwise choices are intransitive, and a full rankingof the values is impossible.