Hughes, S., \u0026 Barnes-Holmes, D., (2011). On the Formation and Persistence of Implicit Attitudes: New Evidence from the Implicit Relational Assessment Procedure (IRAP). The Psychological

$Page 1: Hughes, S., \u0026 Barnes-Holmes, D., (2011). On the Formation and Persistence of Implicit Attitudes: New Evidence from the Implicit Relational Assessment Procedure (IRAP). The Psychological$
The Psychological Record, 2011, 61, 391–410

This research is part of the doctoral dissertation of the first author, carried out under the

supervision of the second author, and was funded by a scholarship from the Irish Research

Council for Science, Engineering and Technology (IRCSET) to the first author. We would like to

thank two anonymous reviewers for their valuable comments on the work reported in this article.

Correspondence concerning this article should be addressed to Sean Hughes or Dermot

Barnes-Holmes, National University of Ireland Maynooth, County Kildare, Ireland. E-mail: sean.

[email protected] or [email protected].

On the FOrmatiOn and Persistence OF imPlicit attitudes: new evidence FrOm the imPlicit

relatiOnal assessment PrOcedure (iraP)

Sean Hughes and Dermot Barnes-Holmes

National University of Ireland Maynooth, Ireland

Research increasingly supports the Implicit Relational Assessment Pro-cedure (IRAP) as a measure capable of providing a sensitive index of preexisting implicit attitudes and cognitions. The current study consti-tutes the first attempt to determine if the IRAP is also sensitive to im-plicit attitudes engineered through either direct relational training or verbal instruction. Following attitude-induction training, participants completed an IRAP in addition to two self-report procedures designed to measure newly formed attitudes. Both implicit and explicit attitudes emerged and persisted in response to both relational training and ver-bal instruction. Furthermore, the IRAP data indicated significant im-plicit attitudes when participants both affirmed attitude-consistent and negated attitude-inconsistent relations. The findings are consistent with previous attitude-formation research, but the relational proper-ties of the IRAP raise specific conceptual issues pertaining to the na-ture of implicit attitudes themselves.Key words: implicit attitudes, attitude formation, Implicit Relational Assessment Procedure, IRAP, IAT, attitude change, propositional, associative

Traditionally, attitude research has relied heavily on the use of direct procedures such as questionnaires or interviews that require individuals to report their deliberate, carefully considered evaluative judgment or beliefs (termed explicit attitudes; Greenwald & Banaji, 1995; Nosek, 2007). More re-cently, researchers have focused on developing a new class of indirect pro-tocols, which purportedly capture so-called implicit or automatic attitudes. Numerous indirect methodologies have been advanced in this regard (e.g., Fazio, Jackson, Dunton, & Williams, 1995; Nosek & Banaji, 2001; De Houwer, 2003; Payne, Cheng, Govorun, & Stewart, 2005; Sriram & Greenwald, 2009),

392 HUGHES AND BARNES-HOLMES

including the most well-established latency-based procedure, the Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, 1998).

The IAT is a computerized dual-categorization procedure designed to provide a relativistic index of the strength of association between two pairs of concepts. For illustrative purposes, consider an IAT targeting im-plicit racial attitudes. During one task respondents are required to rapidly categorize names or pictures of white people with positive evaluative at-tributes and black people with negative evaluative attributes. In a second task, the associative categorization procedure is reversed (i.e., white people are paired with negative and black with positive evaluative descriptors). An oft-replicated finding indicates that performance by white participants is consistently more rapid and accurate on the former task than on the latter task, suggesting an implicit race bias favoring white people (e.g., Amodio & Devine, 2006; Nosek et al., 2007). Concomitantly, participants’ self-reported attitudes profess egalitarian sentiments toward members of other racial groups, despite their implicit anti-black/pro-white bias. This disparity be-tween self-reported endorsements of racial equality and IAT performance raises the possibility that implicit and explicit attitudes are systematically related to different types of behavior. That is, explicit attitudes are thought to best predict intentional, controlled, and elaborated behaviors, whereas implicit attitudes track spontaneous, immediate, and perhaps automatic re-sponses and judgments (e.g., Friese, Hofmann, & Schmitt, 2008; Hofmann, Gawronski, Gschwendner, Le, & Schmitt, 2005).

During the past decade, the IAT and alternative indirect methodologies have been deployed in the measurement of implicit attitudes and cognition, both within social psychology (e.g., Banaji, 2001; Fazio & Olson, 2003) and beyond its boundaries in health psychology (Teachman, Gapinski, Brownell, Rawlins, & Jeyaram, 2003), clinical psychology (Egloff & Schmukle, 2002), forensic psychology (Gray, Brown, & MacCulloch, 2005), and consumer psy-chology (Karpinski, Steinman, & Hilton, 2005; see Greenwald, Poehlman, Uhlmann, & Banaji, 2009, for a comprehensive overview). It is important to note, however, that many indirect procedures (including the IAT) are designed to target the association between two pairs of concept categories. Hence, effects provided by these procedures are often interpreted as offer-ing empirical support for the associative nature of implicit cognition. For example, some social–cognitive accounts of implicit attitudes qualify per-formance (“attitudes”) in terms of an underlying associative network, which competes with and/or interacts with a higher-order propositional reasoning process (e.g., Gawronski & Bodenhausen, 2006; Rydell & McConnell, 2006; Strack & Deutsch, 2004).

In contrast, a number of researchers from the functional contextual tradition posit that implicit cognition can be understood in purely behav-ioral terms without recourse to mediating mechanisms such as mental rep-resentations, mental processes, and/or memory traces (see Hughes, Barnes-Holmes, & De Houwer, 2011). One recently articulated account, termed the relational elaboration and coherence (REC) model, has been offered in this respect. The REC model stems from relational frame theory (RFT; Hayes, Barnes-Holmes, & Roche, 2001), a contemporary functional account of human language and cognition that conceptualizes verbal and higher cognitive activity as a product of arbitrarily applicable relational respond-ing. According to the REC model, the behavioral effects captured by both

393IRAP ATTITUDE FORMATION

self-report and indirect procedures are thought to reflect the operation of the same behavioral process (i.e., relational responding) operating under two broadly different conditions. On the one hand, brief and immediate relational responding provides the basis for what cognitively oriented researchers commonly term implicit attitudes. On the other hand, the ex-tended relational responding needed to produce a response that coheres with one or more other relational responses in the person’s behavioral repertoire provides the basis for so-called explicit attitudes1 (see Barnes-Holmes, Barnes-Holmes, Stewart, & Boles, 2010, for a detailed explanation of the REC model).

When implicit cognition is viewed as relational, rather than strictly associative, an alternative, nonassociative measure of implicit attitudes quickly becomes possible. One procedure that has recently been offered in this respect is the Implicit Relational Assessment Procedure (IRAP; Barnes-Holmes, Barnes-Holmes, Power, Hayden, Milne, & Stewart, 2006). Specifically, the IRAP aims to measure the probability of brief and immediate relational responses (BIRRs) that are made apparent when the behavioral system is put under pressure to respond quickly and accurately.

Similar to other response-latency methodologies, the IRAP requires par-ticipants to respond in ways that are either consistent or inconsistent with their prior learning histories by choosing between a pair of relational re-sponse options (e.g., “Similar” and “Opposite”) when presented with a label stimulus (e.g., “Pleasant”) and a target stimulus (e.g., “Flower”). The rationale behind the IRAP is that responding should be faster on consistent (e.g., Flower–Similar–Pleasant) than on inconsistent trials (e.g., Flower–Opposite–Pleasant) because brief and immediate relational responding will coordinate more often with consistent overt responding. The response time differential between consistent and inconsistent trials (defined as the IRAP effect) is as-sumed to provide a nonrelative index of the strength of the relation between the two targeted concepts.

To date, the IRAP effect has been replicated across a number of attitude domains, ranging from social stereotyping (Barnes-Holmes, Murphy, Barnes-Holmes, & Stewart, 2010; Drake et al., 2010; Power, Barnes-Holmes, Barnes-Holmes, & Stewart, 2009) to age-related attitudes (Cullen, Barnes-Holmes, Barnes-Holmes, & Stewart, 2009), implicit self-esteem (Vahey, Barnes-Holmes, Barnes-Holmes, & Stewart, 2009), and implicit sexual beliefs (Dawson, Barnes-Holmes, Gresswell, Hart, & Gore, 2009). Furthermore, the IRAP dem-onstrates levels of predictive validity comparable to the IAT (Barnes-Holmes, Murtagh, Barnes-Holmes, & Stewart, 2010; Barnes-Holmes, Waldron, Barnes-Holmes, & Stewart, 2009; Roddy, Stewart, & Barnes-Holmes, in press), as well as a resilience to faking strategies (McKenna, Barnes-Holmes, Barnes-Holmes, & Stewart, 2007).

It is important to note, however, that initial IRAP research has focused primarily on measuring preexperimentally established implicit attitudes. Although not denying the relevance and importance of this approach, a more complete understanding of implicit attitudes, at least from a functional perspective, requires the systematic analysis of the conditions under which

1 According to the REC model, attitudes are understood as the behavioral consequences

of observable environment–behavior interactions. Consequently, attitudes are not hypothetical

constructs that influence behavior and require a subsequent explanation. Attitudes are the

behaviors in question and can be understood in terms of behavioral principles.


implicit attitudes emerge in the first place. With this in mind, the current study sought to provide the first experimental analysis of implicit attitude formation as measured by the IRAP. In doing so we sought to investigate whether the IRAP can provide a sensitive measure not only of preestablished attitudes but also of those newly engineered within the laboratory. To pro-vide a context for such work we will first outline how the question of at-titude formation has been addressed to date.

implicit attitude Formation

Initial investigation into the origins of implicit attitudes assumed that they reflect highly stable responses shaped through long-term socialization ex-periences (e.g., Olson & Dunham, in press; Rudman, Phelan, & Heppen, 2007). Interestingly, however, recent work has established that implicit evaluations are in fact highly sensitive to new experiences (e.g., Castelli, Carraro, Gawronski, & Gava, 2010) and contextual moderation (Rydell & Gawronski, 2009) and are amenable to change (Prestwich, Perugini, Hurling, & Richetin, 2010). This re-search has typically relied on two methods to generate novel automatic evalu-ations. On the one hand, attitudes can be instigated through evaluative condi-tioning (EC) preparations in which an induced change in stimulus valence re-sults from repeated pairings of a neutral stimulus (CS) with either a positive or negatively valenced unconditioned stimulus (UCS; De Houwer, 2007; De Houwer, in press; Walther, Nagengast, & Trasselli, 2005). For example, Olson and Fazio (2001, 2002) developed a paradigm that sought to determine how readily im-plicit attitudes can emerge via EC within the experimental setting. During a learning phase, unknown arbitrary stimuli (e.g., names of Pokémon characters) were repeatedly paired with preexisting UCS stimuli (e.g., positive or negative pictures or words). When subsequently tested using an IAT, participants dem-onstrated an implicit preference for the Pokémon characters previously paired with positive stimuli over those paired with negative stimuli (for related find-ings see Dijksterhuis, 2004; Gibson, 2008; Mitchell, Anderson, & Lovibond, 2003; Walther, Ebert, & Meinerling, 2011).

On the other hand, the formation of implicit attitudes may reflect not only associative learning but also verbal instruction that specifies the evalu-ative properties of new attitude stimuli (referred to as propositional knowl-edge). For instance, Gregg, Seibt, and Banaji (2006) established positive or negative preferences toward two fictitious social groups (called “Niffites” and “Luupites”) by exposing participants to a detailed verbal “narrative” depict-ing one group as positive and the other as negative. Other participants were exposed to 120 pairings of the CSs (social group names) with UCSs (positive or negative attributes). When subsequently assessed for the acquisition of auto-matic preferences, participants in both conditions demonstrated similar and significant IAT effects. Moreover, verbal instruction was sufficient to general-ize evaluations of the original two groups (i.e., Niffites and Luupites) to two additional fictitious social groups (i.e., “Jebbians” and “Haasians”) when the former groups were described as equivalent to the latter (see also De Houwer, 2006; Ranganath & Nosek, 2008; Whitfield & Jordan, 2009).

Thus it appears that repeated contingency learning and verbally con-veyed information both constitute successful means for shaping implicit at-titudes. Critically, however, no research has attempted to ascertain whether the IRAP also provides a sensitive measure of experimentally induced


attitudes. Toward this end, we employed two attitude-induction manipula-tions, differentiated by the procedure used to train the relation between two arbitrary attitude objects (the nonsense words “Cug” and “Vek”) and either positively or negatively valenced words. One condition (relational training) involved a contingency learning procedure that sought to establish positive and negative evaluative functions for “Cug” and “Vek” through repeatedly reinforced relational responses. A second condition (verbal instruction) embedded those stimulus relations within a verbal narrative that depicted members of one of two fictitious social groups (defined as the Cugs or Veks) as characterized by positive or negative evaluative attributes. A third con-dition was also employed (combined training), which involved a sequential combination of verbal instruction and relational training. An IRAP was then administered to determine whether implicit preferences were successfully engineered, while three self-report procedures were used to evaluate the impact of these manipulations on explicit attitudes. Given that Gregg et al. (2006) found no significant differences across their induction conditions with the IAT, we refrained from making any specific predictions for the IRAP in this regard.

Finally, recent work has suggested that negating the meaning of evalu-ative stimuli is a verbal rule-based process outside the remit of automatic-ity (e.g., Deutsch, Gawronski, & Strack, 2006; Gawronski, Deutsch, Mbirkou, Seibt, & Strack, 2008; although see Deutsch, Kordts-Freudinger, Gawronski, & Strack, 2009). According to RFT, however, humans can learn to relate stim-uli in multiple different ways above and beyond simple associations (i.e., equivalence). For instance, individuals can affirm (Apple–Similar–Fruit) or negate (Pig–Opposite–Fruit) the relation between stimuli as well as respond to hierarchical, comparative, temporal, cause/effect, and deictic relations (see Hayes et al., 2001, for a detailed overview). These different patterns of relational responding, known as relational frames, are considered to be gen-eralized operant classes, and thus the various properties of different frames will vary contingent on the behavioral histories attached to each frame. For example, the relational frame of coordination, which provides the functional basis for affirmation, permeates almost all verbal behavior. That is, virtually every verbal utterance involves referring to some object or event. In contrast, the frame of distinction, which provides the functional basis for negation, occurs less often than coordination in natural language. Therefore, while there may be differences in response strengths or probabilities between the relational frames of coordination and distinction, RFT would predict that automatic negation effects may nevertheless be observed on the IRAP. Consequently, we sought not only to generate new implicit attitudes but also to determine whether IRAP performances were driven by both affirming relationally consistent information and negating relationally inconsistent information.

method

Participants

Sixty-four undergraduates were recruited from various departments at the National University of Ireland Maynooth and completed the current


study on a voluntary basis. Participants were randomly assigned to a receive one of three procedures to induce novel attitudes, either relational training, verbal instruction, or combined training (each described subsequently).

apparatus and materials

indirect procedure. Automatic group preference was investigated us-ing a computer-based IRAP. The IRAP allows for the on-screen presentation of standardized instructions, stimuli, and feedback, in addition to a quan-tification and measurement of response accuracy and latency. The current study employed two groups of target stimuli comprised of 12 words divided into two groups, six positively valenced and six negatively valenced nouns (based on Greenwald et al.’s, 1998, consistent and inconsistent categoriza-tion of positive and negative words). Two nonsense syllables functioned as label stimuli (“Cug” and “Vek”), with “Similar” and “Opposite” as response options.

Preexperimental direct procedure. Two of the attitude-induction condi-tions involved presenting participants with a graphic and elaborated narra-tive that established the nonsense syllables “Cug” and “Vek” as positively or negatively valenced stimuli. After reading the narrative, participants were asked to indicate the perceived valence of each of the nonsense syl-lables, and then to provide three adjectives for each syllable that were deemed to be broadly synonymous (e.g., Cug = “Horrible,” “Wicked,” “Nasty”; Vek = “Wonderful,” “Enlightened,” “Virtuous”). If any participant failed to respond in a manner that was consistent with the prior narrative, he or she was to be reexposed to the narrative and then subsequently retested, but this was never necessary.

Postexperimental direct procedure. Following the IRAP task, all partici-pants completed two hard-copy feeling thermometers numerically labeled from 0 ºC (cold) to 100 ºC (hot) in 10 ºC intervals designed to determine self-reported evaluations of the two nonsense syllables. Two semantic differ-ential scales were then completed, each comprised of a 7-point Likert scale ranging from 1 (Negative) on the left to 7 (Positive) on the right. These scales assessed participants’ evaluation of how positive or negative they believed “Cug” and “Vek” to be. Finally, to determine the perceived realism of their self-reported evaluations of the two nonsense words, participants rated how meaningful the induced attitudes were on a 7-point scale (1 = Not at all to 7 = Extremely).

Procedure

relational training. Participants in this condition completed a “training” IRAP that reinforced four interrelated stimulus relations involving two arbi-trary nonsense words ( “Cug” and “Vek”) with either six positively valenced (e.g., “Pleasant”) or six negatively valenced (e.g., “Nasty”) trait adjectives. That is, participants were trained to produce the following four relational re-sponses: Cug–Positive–Similar, Cug–Negative–Opposite, Vek–Positive–Opposite, and Vek–Negative–Similar. The relationship between the two labels (“Cug” and “Vek”) and the positive and negative words was counterbalanced across participants. Prior to commencing training, participants were provided with instructions detailing the general background to the research and were


given an illustration and explanation of the four trial types utilized in the task (see Figure 1). Participants subsequently confirmed that they fully un-derstood the instructions and clicked a “Yes” button on the screen to begin.

Figure 1. Examples of the four trial types used in the test phase of the IRAP program. The label words (“Cug” and “Vek”), target words (positive or negative exemplars), and relational response options (“Similar” and “Opposite”) are indicated in addition to the correct relational response for each trial type. The arrows and correct response option did not appear on the screen or in the instructions.

On each IRAP trial, four stimuli were presented concurrently. A label stimulus appeared at the top center location of the screen (either “Cug” or “Vek”), a single target word (e.g., “Peace” or “Hate”) was presented at the midcenter location, and the two response options, “Similar” or “Opposite,” appeared in the bottom left- and right-hand corners. The instruction “Press d for” was presented above the left response option and “Press k for” was presented above the right response option. The IRAP program required participants to choose one of the two response options on each trial by pressing the “d” key for left and the “k” key for right using the left and right index fingers, respectively. Selecting the response option deemed correct for a trial removed all four stimuli from the screen for a 400-ms intertrial interval, after which the next trial was presented. If the other (incorrect) option was chosen, a large red X appeared on screen directly below the target stimulus. The red X and all other stimuli remained on screen until the correct response option was chosen, after which the screen cleared and the program progressed to the 400-ms intertrial interval. The left–right locations of the two response options (“Similar” and “Opposite”) alternated quasirandomly across trials, constrained only in that they could not occupy the same positions across three successive trials. Participants


were instructed to pay close attention to the key assignments, as they would change unpredictably across trials.

The training IRAP presented the trials in blocks of 48. In order to com-plete the induction via relational training, participants were required to achieve a minimum accuracy of 95% and a maximum median response la-tency of 3,000 ms across a single block of 48 trials. A minimum of one and a maximum of eight training blocks were completed. Failure to meet the cri-terion for progression across eight successive training blocks resulted in the participant being debriefed and thanked and their data removed from the study; however, no data were removed on this basis. Completion of the train-ing phase resulted in an on-screen instruction to report to the experimenter, who subsequently commenced the IRAP proper.

verbal instruction. Participants in the verbal-instruction condition read an elaborated, graphic account detailing the meanings of the words “Cug” and “Vek” and their links to a fabricated “ancient language.” Half the partici-pants were informed that, according to this language, people described as “Cug” were “valued for their wisdom and generosity . . . and honored for their acts of goodness and caring” while those called “Vek” were “evil, horrific, disgusting, and depraved . . . people who committed acts of extreme cruelty and violence.” The other half received a narrative in which the orthographic distinction between “Cug” and “Vek” was reversed, such that “Cug” was neg-ative and “Vek” positive. Participants were then presented with a question-naire to determine if the induced attitudes towards “Cug” and “Vek” were consistent with the induction. Specifically, participants were asked to write three adjectives that they believed were synonymous with the two sample words. Upon successful completion, participants then proceeded to IRAP.

combined training. Participants in this condition were subject to a double- barreled preference induction, with exposure to the verbal instruc-tion and then exposure to the attitude induction via repeated relational training outlined above. The counterbalancing controls described previously were also used in this condition, and the assignment of positive and nega-tive valence to the “Cug” and “Vek” nonsense syllables remained consistent across the two induction phases. Following the two induction procedures, participants proceeded to the IRAP.

implicit relational assessment Procedure (iraP). Following each of the three attitude-induction procedures, participants completed an IRAP. Participants were briefed that during the IRAP they would sometimes be re-quired to respond in a manner that appeared to contradict the information learned in the relational training phase (or narrative) but doing so was part of the experiment. Participants were also informed after each block of trials that the feedback contingencies would be reversed for the following block.

The IRAP was similar to the training version except that it comprised a minimum of two practice blocks and a fixed set of six test blocks, and the feedback contingencies alternated across successive blocks (described in the next section). Each block consisted of the same 24 trials, composed of four different trial types (see Figure 1).

The mastery criteria for completing the practice phase and progress-ing to the test blocks was 80% accuracy and a median response latency of 3,000 ms or less across two successive practice blocks. A feedback screen was presented if a participant failed to achieve one or both mastery crite-ria for either practice block. The feedback stipulated the criteria needed to


complete the practice phase and presented the participant’s accuracy and latency scores for the previous two practice blocks. Failure to achieve the practice criteria resulted in presentation of another two practice blocks until the criteria were met or eight blocks were completed. If a participant failed to meet the criteria after eight blocks, he or she was debriefed and thanked and his or her involvement in the experiment was terminated (the data for two participants were removed on this basis). Successful completion of the practice phase caused the program to progress to the test blocks. The test blocks were presented in a similar format to the practice blocks but without the performance criteria for progression. However, a new message appeared before each test block informing participants that “This is a test—go fast, making a few errors is okay.” Although participants were exposed to six test blocks irrespective of their performance, the data for a participant were re-moved if he or she failed to maintain the practice criteria across the test blocks (the data for six participants were removed on this basis).2

experimental sequence and participant assignment. Two IRAP se-quences were utilized; the first assigned half of the participants to a consistent- relations- first (CF) IRAP sequence and the second to an inconsis-tent-relations-first (IF) sequence. The first practice block in the CF sequence reinforced responses that were relationally consistent with the relational-training or verbal-instruction procedures. The second practice block in the CF sequence reinforced responses that were inconsistent with the attitude-induction procedures. If subsequent practice blocks were presented, they alternated between CF and IF. The same alternating sequence was also used across all six test blocks. The IF sequence was similar to the CF se-quence, with the exception that participants moved through the IRAP in an inconsistent- to- consistent sequence.

Following the conclusion of all six test blocks, instructions appeared on screen stating that that part of the experiment had concluded. Thereafter, participants completed the direct attitude procedures (memory check and meaningfulness scale) and were debriefed and thanked. Failure to respond on the explicit procedures in a manner that was consistent with the induc-tion phase of the experiment resulted in data being discarded (the data for four participants were removed on this basis).3

results

data Preparation

The primary datum was response latency, defined as the time in mil-liseconds (ms) that elapsed from the onset of each IRAP trial to the first correct response emitted by the participant. To minimize contamination by individual differences associated with age, motor skills, and/or cognitive ability, the response latency data for each participant were transformed into D-IRAP scores using an adaptation of Greenwald, Nosek, and Banaji’s (2003)

2 Consistent with previous IRAP research, implementing an accuracy criterion ensured

that the IRAP was capturing the targeted behavior. Failure to control for excessive or spurious

errors would make difficult any interpretation of the resulting IRAP effects.

3 The explicit measures in the current study functioned largely as a manipulation check.

Specifically, we sought to ensure that participants could remember the trained or instructed

contingencies after completing the IRAP.


D algorithm (the details of this data transformation have been published in a number of previous articles; see, e.g., Barnes-Holmes, Waldron, et al., 2009, for a full explanation). Three mean overall D-IRAP scores were calculated, one for each IRAP test block pair, and computed such that a positive value indicated faster responding on consistent relative to inconsistent trials, that is, the formation of novel automatic evaluations in accordance with the attitude-induction procedure.4 Conversely, scores that approached zero in-dicated an equivalent performance across consistent and inconsistent trials and thus the absence of an implicit preference.

implicit attitudes

Persistence of the induced iraP effects. The three mean D-IRAP scores for each induction procedure are presented in Figure 2. The direction of the induced effects was consistent with the experimental manipulation in each case. However, while both relational training and combined training showed evidence of stability in both generating and maintaining implicit preferences across the IRAP, preferences engineered through verbal instruction attenu-ated by the end of the testing phase. The data also showed an overall attenu-ation trend across successive IRAP blocks in general.

Figure 2. Mean D-IRAP scores with standard error bars for the relational-training, verbal-instruction, and combined-training conditions. A positive D-IRAP score indicates the formation of an implicit attitude in accordance with the induction procedure. The zero point indicates the absence of an induced preference.

Verbal Instruction

Relational Training

Combined Training

.4

.35

.3

.25

.2

.15

.1

.05

0

Mea

n D-

IRAP

IRAP Test BlocksTest Blocks 1 & 2 Test Blocks 3 & 4 Test Blocks 5 & 6

When submitted to a 3 (relational, verbal, combined training) × 3 (D-IRAP Blocks 1, 2, 3) mixed repeated-measures ANOVA, D-IRAP scores re-vealed that the formation of implicit attitudes did not differ significantly

4 To control for the possible impact of the two orthogonally counterbalanced methods

factors (the valence of the nonsense word used and the order of IRAP test blocks), three separate

2 × 2 × 3 mixed repeated-measures ANOVAs were conducted, one for each induction condition. In

each case the ANOVA failed to yield any significant effect (all ps > .2) for order or valence and thus

they were removed from all subsequent analyses. Even when an overall 3 (induction condition) ×

3 (IRAP test blocks) × 2 (CF/IF IRAP sequence) ANOVA was computed, order and sequence did not

impact on the obtained IRAP effects (all ps > .1).


across attitude-induction conditions (p > .7), but there was a trend indicat-ing that the strength of the IRAP effects attenuated across successive IRAP blocks F(2, 49) = 2.2, p = .1, η

p2 = 0.04. The interaction between induction

condition and IRAP block was nonsignificant (p > .4). Three separate one-way between-participant ANOVAs, for each pair of test blocks, confirmed that the IRAP effects did not differ significantly between the induction conditions (all ps > .3). Nine separate one-sample t tests indicated that each of the IRAP effects was significantly different from zero (ps > .05), except for the IRAP effect for the third pair of test blocks in the verbal-instruction condition (p > .5). In sum, iteratively training or verbally instructing participants as to the evaluative properties of new attitude stimuli is sufficient to generate new implicit evaluations as measured by the IRAP.

affirmation and negation effects. In tackling the question of implicit attitude formation, we also sought to address whether newly generated at-titudes were driven by both affirming relationally consistent information and negating relationally inconsistent information. Such an analysis is af-forded by the methodology, given that two of the four IRAP trial types assess the affirmation of the newly established stimulus relations and the other two trial types assess the negation of stimulus relations. Therefore, to determine whether participants showed significant affirmation and negation effects for their newly formed attitudes, we calculated D-IRAP effects for the two affirmation trial types and for the two negation trial types, yielding a mean D-IRAP score for each.

A 2 (affirmation, negation) × 3 (induction condition) repeated-measures ANOVA showed a significant difference between affirmation and negation performance, F(2, 49) = 10.4, p = .002, η

p2 = .18. However, this effect did not

differ as a function of the attitude-induction procedure employed (p = .7). The two IRAP effects, collapsed across the three induction conditions, were then subjected to one-sample t tests, and these revealed that both affirma-tion effects, t(50) = 7.1, p = .001, and negation effects, t(50) = 2.2, p = .038, were independently significant (see Figure 3). Critically, therefore, partici-pants showed evidence of negation responding at the automatic level when measured by the IRAP.

Figure 3. Mean D-IRAP scores with standard error bars for the affirmation and negation trial types collapsed across the three induction conditions.

.3

.25

.2

.15

.1

.05

0NegationAffirmation

IRAP Trial Types

Mea

n D-

IRAP


explicit attitudes

As noted previously, data were discarded for those participants who failed to produce self-reported attitudes that were consistent with the in-duction procedures. Thus the self-reported evaluations of “Cug” and “Vek” were analyzed simply to determine if they differed across induction proce-dures. A 2 (word valence) × 3 (induction condition) mixed repeated-measures ANOVA revealed the expected main effect for valence, F(2, 48) = 379.1, p < .001, η

p2 = .89, but, critically, this effect was not qualified by an in-

teraction with induction condition (p > .3). Parallel analyses for the feeling thermometer yielded similar statistical conclusions. Specifically, a 2 (word valence) × 3 (induction condition) mixed repeated-measures ANOVA revealed the expected main effect for valence, F(2, 47) = 88.8, p < .001, unqualified by an interaction with induction condition (p > .7).

meaningfulness check. Seventy-one percent of participants reported the perceived realism of their self-reported preferences at or above the midpoint of the meaningfulness scale. This sample comprised 80% of participants in the relational-training condition (M = 5.1), 88% in the verbal-instruction condition (M = 5.3), and 88% in the combined-training condition (M = 5.3). Critically, these meaningfulness ratings did not differ significantly across the three induction conditions (p > .6).

discussion

The main goal of the present article was to determine the independent and additive effects of verbal rules and direct relational learning in engineering automatic and self-reported evaluations. In doing so, we were particularly interested in the antecedent conditions necessary for the formation of automatic evaluations toward unknown stimuli. Whereas a considerable amount of research has focused on the IRAP as a measure of well- established behavioral histories—the contingencies for which lie largely outside the experimental preparation—the current study is the first to determine whether the IRAP proves equally sensitive to verbal relations established within an experimental context.

Overall, analyses of the persistence in the IRAP effects indicated that the mode of attitude induction did not moderate the emergence of automatic evaluations when formed by either relational training or verbal instruction or by a combination of the two. Moreover, the relative persistence of newly formed evaluations did not vary when acquired through either induction procedure. However, it is worth noting that a trend in the data suggested that induced IRAP effects appeared to be less persistent in the verbal- instruction condition. Our findings thus advance the IRAP not only as a methodology capable of indexing relational histories of a preexperimental nature but also as a measure sensitive to laboratory-induced effects.

Furthermore, we sought not only to instigate automatic evaluations but also to determine whether they reflected both the affirmation and/or nega-tion of newly established stimulus relations. Analysis of the IRAP trial-type data demonstrated differential effects for the ability to affirm and negate re-lations, in that participants showed significantly stronger IRAP effects when affirming than when negating the relations. Critically, however, negation did contribute significantly to the observed IRAP effects, indicating that they


were not driven purely by affirmation. In addition, the finding that nega-tion was significantly smaller than affirmation on the IRAP is behaviorally consistent with the fact that the former relational frame occurs at a lower probability than the latter in natural language.5

In sum, our data corroborate earlier findings that implicit and explicit attitudes can be generated in the laboratory through minimal amounts of verbal information provided about fictitious social groups or repeated expo-sure to relational training (e.g., Gregg et al., 2006). Additionally, the current study extends earlier work by demonstrating that the IRAP functions in a broadly similar fashion to alternative procedures when targeting newly in-duced stimulus relations.

It is worth noting that the current study did not involve demonstrating a derived transformation of evaluative functions for the “Cug” and “Vek” stimuli (see Hayes et al., 2001). That is, the attitude-induction procedure in-volved reinforcing or instructing the stimulus relations directly, which were later “tested” using the IRAP proper. Consequently, future studies might at-tempt to extend the current work by determining if the basic effects could be replicated via a derived transformation of functions. As an aside, a recent study has provided evidence for a derived transfer of evaluative functions on an indirect measurement procedure (i.e., the IAT; O’Toole; Barnes-Holmes, & Smyth, 2007), and thus there is little reason to doubt that this could not be replicated using the IRAP.

implications for attitude-Formation research

The current work shines a light on a number of important theoretical and practical issues for research on implicit attitude formation. First, the findings may encourage attitude researchers to pay consideration not only to the emer-gence of automatic effects but also to their persistence (e.g., stability across time, resistance to change; Holland, Verplanken, & van Knippenberg, 2002; Krosnick & Petty, 1995; Petty, Tormala, & Rucker, 2004). Whereas a consider-able amount of research exploring attitude formation has focused on the ca-pacity of a procedure to establish new preferences or evaluations, such work often remains comparatively silent about the strength or persistence of those effects. This issue becomes particularly pronounced given that much research has qualified both the formation and persistence of automatic evaluations in terms of the single reversal between consistent and inconsistent tasks on the IAT (e.g., De Houwer, 2006; Olson & Fazio, 2001; McConnell, Rydell, Strain, & Mackie, 2008; Whitfield & Jordan, 2009). However, one oscillation between con-sistent and inconsistent tasks may afford only a brief test of the strength and relative duration of newly formed attitudes.

In contrast, the architecture of the IRAP subjects automatic evaluations to not one but at least six repeated alternations between competing rela-tional contingencies. It is possible, therefore, that the IRAP’s repeated rever-sals may function to undermine weakly established attitudes and as such afford a more informative analysis of the relational strength than a single IAT. Put simply, we suggest that strong attitudes may be those that persist

5 It should be noted that although automatic affirmation effects may often be stronger

than negation effects due to the differential frequencies of each in natural language, the REC

model would predict that the magnitude of a negation effect would increase with an appropriate

behavioral history (e.g., extensive “overtraining” on a negation relation).


across multiple reversals between relational contingencies (as observed when preexisiting attitudes are measured by the IRAP). Conversely, weak attitudes could be defined as those that fail to persist and rapidly attenu-ate to nonsignificance when faced with inconsistent IRAP blocks. Extending this point, the degree to which an IAT effect would be maintained across an equivalent number of relational reversals, relative to an IRAP effect, re-mains to be seen. Therefore, the current work highlights not only the need to consider whether an automatic evaluation is generated as a result of a procedural manipulation but also to assess the duration of that effect either across time or in the face of multiple contingency reversals. We offer the IRAP as one potential procedure in this latter respect.

A second issue raised by the current data relates to the fact that research-ers have typically emphasized associative conditioning procedures as the method of choice for establishing new implicit attitudes toward known (Olson & Fazio, 2006) or fictitious (Gregg et al., 2006) social groups, individuals (Rydell, McConnell, Mackie, & Strain, 2006; Whitfield & Jordan, 2009), and non-sense words (Mitchell, Anderson, & Lovibond, 2003). While the specific imple-mentation of the procedure varies across studies, associative methodologies are distinguished by the pairing of a stimulus with another liked or disliked stimulus to instigate a change in liking of that stimulus. Critically, this work has frequently operated on the assumption that implicit attitudes resulting from associative procedures arise due to the theoretical process of association formation (e.g., Gawronski & Bodenhausen, 2006; Rydell & McConnell, 2006). Notwithstanding the plausibility of the assumption, evidence indicates that implicit effects do not always emerge exclusively from associative learning procedures (De Houwer, 2006). Furthermore, there is no a priori reason that implicit effects should emerge as a function of the process of association for-mation (see Hughes et al., 2011, for a detailed discussion). This point is cru-cial given that the current work constituted an instance of relational rather than associative learning. That is, instead of pairing a stimulus repeatedly with either a liked or a disliked stimulus, our relational training procedure presented a nonsense word with both positive and negative terms an equal number of times, and participants had to respond to these with one of two relational terms. If automatic effects are uniquely a product of associative procedures (i.e., stimulus pairings), and by implication associative processes—rather than the relational responses that participants had to learn to emit in the current study—the relational training procedure should have been unsuccess-ful in establishing automatic evaluations. This was clearly not the case.

Extending this point, implicit attitude effects are often interpreted as the product of automatic excitation or inhibition of links between mental representations in memory (see Gawronski & Bodenhausen, 2006). Negating the meaning of a stimulus, on the other hand, requires validating the asso-ciation and thus is assumed to be a propositional, rule-based process beyond the remit of automaticity. Therefore, if implicit effects are inherently asso-ciative, no evidence for negation should be observed at the automatic level. Once again this was not the case; the IRAP performances reflected both sig-nificant affirmation and negation effects, indicating that participants were not simply learning associations but instead responding in accordance with the new stimulus relations.

On balance, perhaps an associative account of the current findings is possible. For example, if an associative network includes dissociations—with


some “nodes” repelling other nodes as well as attracting others—such ef-fects could be modeled associatively. That is, responding “Similar” would imply associations whereas responding “Opposite” would imply dissocia-tions. The concept of dissociations, however, is not found in traditional as-sociative accounts. Rather, associative models typically conceptualize learn-ing in terms of excitatory and inhibitory links between nodes in a network (Gawronski & Bodenhausen, 2006), and thus mental representations are ei-ther associated (through excitation) or they are not (through inhibition). In other words, inhibitory links may reduce excitation, but they do not generate a “repelling” effect between nodes in the network. Furthermore, even if the concept of repelling was permitted, implicit effects involving comparative relations have been reported with the IRAP (e.g., Power et al., 2009), and thus it seems likely that associative accounts will need to explain a range of rela-tional effects rather than just affirmation and negation. If in doing so, asso-ciative links are seen as taking on the properties of relations, it will become difficult to distinguish an associative from a relational account.

Another associative account of the current findings might be possible, however. For example, Gawronski et al. (2008) argued that automatic nega-tion effects occur in situations where the overall meaning of the negated stimulus is stored as an independent unit in associative memory. Arguably, this occurs when the negation has been practiced extensively, as in the com-mon phrase no problem, or implies a clear opposite (e.g., not rich or not ac-tive). Although this interpretation might be applied to the participants in the current study who were exposed to extensive relational training (i.e., prac-tice), it is difficult to see how it could explain the negation effects observed for the participants who received only verbal instruction. Specifically, the instruction simply stated that one social group was good and the other bad, and thus the negation relations were not even specified, let alone trained.

Taken as a whole, the current data illustrate that one cannot simply assume that implicit attitudes are always associative or that indirect pro-cedures always tap associations. Rather, our findings lend support to the notion that implicit attitudes, and their induction, may be conceptualized as involving relational (or propositional) processes6 rather than strict associa-tions. The IRAP reflects a methodology that not only is capable of indexing laboratory-engineered relational histories but also is sensitive to the persis-tence of such effects.

references

AMODIO, D. M., & DEVINE, P. G. (2006). Stereotyping and evaluation in implicit race bias: Evidence for independent constructs and unique effects on behavior. Journal of Personality and Social Psychology, 91(4), 652–661.

6 Whereas propositions and relational frames may reflect different ways of talking about

the same psychological phenomena (e.g., attitudes), it is important to note that they represent

conceptually distinct and separate levels of analysis (see De Houwer, Gawronski, & Barnes-

Holmes, 2010). Specifically, propositions reflect the cognitive (mental) process of assigning a

truth value to a specific relation between mental representations (e.g., Mitchell, De Houwer, &

Lovibond, 2009). In contrast, relational frames refer to overarching operant behaviors made

possible through a history of appropriate exemplars typically provided by interaction with the

verbal community.


BANAJI, M. R. (2001). Implicit attitudes can be measured. In H. L. Roediger III, J. S. Nairne, I. Neath, & A. Surprenant (Eds.), The nature of remembering: Essays in honor of Robert G. Crowder (pp. 117–150). Washington, DC: American Psychological Association.

BARNES-HOLMES, D., BARNES-HOLMES, Y., POWER, P., HAYDEN, E., MILNE, R., & STEWART, I. (2006). Do you really know what you believe? Developing the Implicit Relational Assessment Procedure (IRAP) as a direct measure of implicit beliefs. The Irish Psychologist, 32, 169–177.

BARNES-HOLMES, D., WALDRON, D., BARNES-HOLMES, Y., & STEWART, I. (2009). Testing the validity of the Implicit Relational Assessment Procedure (IRAP) and the Implicit Association Test (IAT): Measuring attitudes toward Dublin and country life in Ireland. The Psychological Record, 59, 389–406.

BARNES-HOLMES, D., MURPHY, A., BARNES-HOLMES, Y., & STEWART, I. (2010). The Implicit Relational Assessment Procedure (IRAP): Exploring the impact of private versus public contexts and the response latency criterion on pro-white and anti-black stereotyping among white Irish individuals. The Psychological Record, 60, 57–66.

BARNES-HOLMES, D., BARNES-HOLMES, Y., STEWART, I., & BOLES, S. (2010). A sketch of the Implicit Relational Assessment Procedure (IRAP) and the relational elaboration and coherence (REC) model. The Psychological Record, 60, 527–542.

BARNES-HOLMES, D., MURTAGH, L., BARNES-HOLMES, Y., & STEWART, I. (2010). Using the Implicit Association Test and the Implicit Relational Assessment Procedure to measure attitudes towards meat and vegetables in vegetarians and meat-eaters. The Psychological Record, 60, 287–306.

CASTELLI, L., CARRARO, L., GAWRONSKI, B., & GAVA, K. (2010). On the determinants of implicit evaluations: When the present weighs more than the past. Journal of Experimental Social Psychology, 46, 186–191.

CULLEN, C., BARNES-HOLMES, D., BARNES-HOLMES, Y., & STEWART, I. (2009). The Implicit Relational Assessment Procedure (IRAP) and the malleability of ageist attitudes. The Psychological Record, 59, 591–620.

DAWSON, D. L., BARNES-HOLMES, D., GRESSWELL, D. M., HART, A. J. P., & GORE, N. J. (2009). Assessing the implicit beliefs of sexual offenders using the Implicit Relational Assessment Procedure: A first study. Sexual Abuse: A Journal of Research and Treatment, 21, 57–75.

DE HOUWER, J. (2003). The extrinsic affective Simon Task. Experimental Psychology, 50, 77–85.

DE HOUWER, J. (2006). Using the Implicit Association Test does not rule out an impact of conscious propositional knowledge on evaluative conditioning. Learning and Motivation, 37, 176–187.

DE HOUWER, J. (2007). A conceptual and theoretical analysis of evaluative conditioning. The Spanish Journal of Psychology, 10, 230–241.

DE HOUWER, J. (in press). Evaluative conditioning: A review of procedure knowledge and mental process theories. In T. R. Schachtman & S. Reilly (Eds.), Applications of learning and conditioning. Oxford, UK: Oxford University Press.

DE HOUWER, J., GAWRONSKI, B., & BARNES-HOLMES, D. (2010). A behavioral-cognitive framework for attitude research. Unpublished manuscript.

DEUTSCH, R., GAWRONSKI, B., & STRACK, F. (2006). At the boundaries of automaticity: Negation as reflective operation. Journal of Personality and Social Psychology, 91, 385–405.


DEUTSCH, R., KORDTS-FREUDINGER, R., GAWRONSKI, B., & STRACK, F. (2009). Fast and fragile: A new look at the automaticity of negation processing. Experimental Psychology, 56(6), 434–446.

DIJKSTERHUIS, A. (2004). I like myself but I don’t know why: Enhancing implicit self-esteem by subliminal evaluative conditioning. Journal of Personality and Social Psychology, 86, 345–355.

DRAKE, C. E., KELLUM, K. K., WILSON, K. G., LUOMA, J. B., WEINSTEIN, J. H., & ADAMS, C. H. (2010). Examining the Implicit Relational Assessment Procedure: Four preliminary studies. The Psychological Record, 60, 81–100.

EGLOFF, B., & SCHMUKLE, S. C. (2002). Predictive validity of an Implicit Association Test for assessing anxiety. Journal of Personality and Social Psychology, 83, 1441–1455.

FAzIO, R. H., JACKSON, J. R., DUNTON, B. C., & WILLIAMS, C. J. (1995). Variability in automatic activation as an unobtrusive measure of racial attitudes: A bona fide pipeline? Journal of Personality and Social Psychology, 69, 1013–1027.

FAzIO, R. H., & OLSON, M. A. (2003). Implicit measures in social cognition research: Their meaning and use. Annual Review of Psychology, 54, 297–327.

FRIESE, M., HOFMANN, W., & SCHMITT, M. (2008). When and why do implicit measures predict behaviour? Empirical evidence for the moderating role of opportunity, motivation, and process reliance. European Review of Social Psychology, 19, 285–338.

GAWRONSKI, B., & BODENHAUSEN, G. V. (2006). Associative and propositional processes in evaluation: An integrative review of implicit and explicit attitude change. Psychological Bulletin, 132, 692–731.

GAWRONSKI, B., DEUTSCH, R., MBIRKOU, S., SEIBT, B., & STRACK, F. (2008). When “just say no” is not enough: Affirmation versus negation training and the reduction of automatic stereotype activation. Journal of Experimental Social Psychology, 44, 370–377.

GIBSON, B. (2008). Can evaluative conditioning change attitudes toward mature brands? New evidence from the Implicit Association Test. Journal of Consumer Research, 35, 178–188.

GRAY, N. S., BROWN, A. S., & MACCULLOCH, M. J. (2005). An implicit test of the associations between children and sex in pedophiles. Journal of Abnormal Psychology, 114, 304–308.

GREENWALD, A. G., & BANAJI, M. R. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review, 102, 4–27.

GREENWALD, A. G., MCGHEE, D. E., & SCHWARTz, J. L. K. (1998). Measuring individual differences in implicit cognition: The Implicit Association Test. Journal of Personality and Social Psychology, 74, 1464–1480.

GREENWALD, A. G., NOSEK, B. A., & BANAJI, M. R. (2003).Understanding and using the Implicit Association Test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85, 197–216.

GREENWALD, A. G., POEHLMAN, T. A., UHLMANN, E., & BANAJI, M. R. (2009). Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. Journal of Personality and Social Psychology, 97, 17–41.

GREGG, A. P., SEIBT, B., & BANAJI, M. R. (2006). Easier done than undone: Asymmetry in the malleability of implicit preferences. Journal of Personality and Social Psychology, 90, 1–20.


HAYES, S. C., BARNES-HOLMES, D., & ROCHE, B. (2001). Relational frame theory: A post-Skinnerian account of human language and cognition. New York: Plenum.

HOFMANN, W., GAWRONSKI, B., GSCHWENDNER, T., LE, H., & SCHMITT, M. (2005). A meta-analysis on the correlation between the Implicit Association Test and explicit self report measure. Personality and Social Psychology Bulletin, 31, 1369–1385.

HOLLAND, R. W., VERPLANKEN, B., & VAN KNIPPENBERG, A. (2002). On the nature of attitude-behavior relations: The strong guide, the weak follow. European Journal of Social Psychology, 32, 869–876.

HUGHES, S., BARNES-HOLMES, D., & DE HOUWER, J. (2011). The dominance of associative theorizing in implicit attitude research: Propositional and behavioral alternatives. The Psychological Record, 61, 465–498.

KARPINSKI, A., STEINMAN, R. B., & HILTON, J. L. (2005). Attitude importance as a moderator of the relationship between implicit and explicit attitude measures. Personality and Social Psychology Bulletin, 31, 949–962.

KROSNICK, J. A., & PETTY, R. E. (1995). Attitude strength: An overview. In R. E. Petty and J. A. Krosnick (Eds.), Attitude strength: Antecedents and consequences. Hillsdale, NJ: Erlbaum and Associates.

MCCONNELL, A. R., RYDELL, R. J., STRAIN, L. M., & MACKIE, D. M. (2008). Forming implicit and explicit attitudes toward individuals: Social group association cues. Journal of Personality and Social Psychology, 94, 792–807.

MCKENNA, I. M., BARNES-HOLMES, D., BARNES-HOLMES, Y., & STEWART, I. (2007). Testing the fake-ability of the Implicit Relational Assessment Procedure (IRAP): The first study. International Journal of Psychology and Psychological Therapy, 7, 253–268.

MITCHELL, C. J., DE HOUWER, J., & LOVIBOND, P. F. (2009). The propositional nature of human associative learning. Behavioral and Brain Sciences, 32, 183–198.

MITCHELL, C. J., ANDERSON, N. E., & LOVIBOND, P. F. (2003). Measuring evaluative conditioning using the Implicit Association Test. Learning and Motivation, 34, 203–217.

NOSEK, B., & BANAJI, M. R. (2001). The go/no-go association task. Social Cognition, 19, 625–666.

NOSEK, B. A., SMYTH, F. L., HANSEN, J. J., DEVOS, T., LINDNER, N. M., RANGANATH, K. A., SMITH, C. T., OLSON, K. R., CHUGH, D., GREENWALD, A. G., & BANAJI, M. R. (2007). Pervasiveness and correlates of implicit attitudes and stereotypes. European Review of Social Psychology, 18, 36–88.

NOSEK, B. A. (2007). Implicit–explicit relations. Current Directions in Psychological Science, 16, 65–69.

OLSON, M. A., & FAzIO, R. H. (2001). Implicit attitude formation through classical conditioning. Psychological Science, 12, 413–417.

OLSON, M. A., & FAzIO, R. H. (2002). Implicit acquisition and manifestation of classically conditioned attitudes. Social Cognition, 20, 89–104.

OLSON, M. A., & FAzIO, R. H. (2006). Reducing automatically activated racial prejudice through implicit evaluative conditioning. Personality and Social Psychology Bulletin, 32, 421–433.

OLSON, K. R., & DUNHAM, Y. (in press).The development of implicit social cognition. In B. Gawronski & B. K. Payne (Eds.), Handbook of implicit social cognition: Measurement, theory, and applications. New York: Guilford Press.


O’TOOLE, C., BARNES-HOLMES, D., & SMYTH, S. (2007). A derived transfer of function and the Implicit Association Test. Journal of Experimental Analysis of Behavior, 88, 263–283.

PAYNE, B. K., CHENG, C. M., GOVORUN, O., & STEWART, B. (2005). An inkblot for attitudes: Affective misattribution as implicit measurement. Journal of Personality and Social Psychology, 89, 277–293.

PETTY, R. E., TORMALA, z. L., & RUCKER, D. D. (2004). Resisting persuasion by counterarguing: An attitude strength perspective. In J. T. Jost, M. R. Banaji, & D. A. Prentice (Eds.), Perspectivism in social psychology: The yin and yang of scientific progress (pp. 37–51). Washington, DC: American Psychological Association.

POWER, P. M., BARNES-HOLMES, D., BARNES-HOLMES, Y., & STEWART, I. (2009). The Implicit Relational Assessment Procedure (IRAP) as a measure of implicit relative preferences: A first study. The Psychological Record, 59, 621–640.

PRESTWICH, A., PERUGINI, M., HURLING, B., & RICHETIN, J. (2010). Using the self to change implicit attitudes. European Journal of Social Psychology, 40, 61–71.

RANGANATH, K. A., & NOSEK, B. A. (2008). Implicit attitude generalization occurs immediately, explicit attitude generalization takes time. Psychological Science, 19, 249–254.

RANGANATH, K. A., & NOSEK, B. A. (2009). Creating distinct implicit and explicit attitudes with an illusory correlation paradigm. Unpublished manuscript.

RODDY, S., STEWART, I., & BARNES-HOLMES, D. (in press). Anti-fat, pro-slim, or both? Using two reaction time based measures to assess implicit attitudes to the slim and overweight. Journal of Health Psychology.

RUDMAN, L. A., PHELAN, J. E., & HEPPEN, J. (2007). Developmental sources of implicit attitudes. Personality and Social Psychology Bulletin, 33(12), 1700–1713.

RYDELL, R. J., MCCONNELL, A. R., MACKIE, D. M., & STRAIN, L. M. (2006). Of two minds: Forming and changing valence inconsistent implicit and explicit attitudes. Psychological Science, 17, 954–958.

RYDELL, R. J., & MCCONNELL, A. R. (2006). Understanding implicit and explicit attitude change: A systems of reasoning analysis. Journal of Personality and Social Psychology, 91, 995–1008.

RYDELL, R. J., & GAWRONSKI, B. (2009). I like you, I like you not: Understanding the formation of context-dependent automatic attitudes. Cognition and Emotion, 23, 1118–1152.

SRIRAM, N., & GREENWALD, A. G. (2009). The Brief Implicit Association Test. Experimental Psychology, 56, 283–294.

STRACK, F., & DEUTSCH, R. (2004). Reflective and impulsive determinants of social behavior. Personality and Social Psychology Review, 8, 220–247.

TEACHMAN, B. A., GAPINSKI, K. D., BROWNELL, K. D., RAWLINS, M., & JEYARAM, S. (2003). Demonstrations of implicit anti-fat bias: The impact of providing causal information and evoking empathy. Health Psychology, 22, 68–78.

VAHEY, N. A., BARNES-HOLMES, D., BARNES-HOLMES, Y., & STEWART, I. (2009). A first test of the Implicit Relational Assessment Procedure (IRAP) as a measure of self-esteem: Irish prisoner groups and university students. The Psychological Record, 59, 371–388.


WALTHER, E., EBERT, I., & MEINERLING, K. (2011). Does cue competition reduce conditioned liking of brands and products? Psychology and Marketing, 28(5), 520–538.

WALTHER, E., NAGENGAST, B., & TRASSELLI, C. (2005). Evaluative conditioning in social psychology: Facts and speculations. Cognition and Emotion, 19, 175–196.

WHITFIELD, M., & JORDAN, C. H. (2009). Mutual influences of explicit and implicit attitudes. Journal of Experimental Social Psychology, 45(4), 748–759.

Hughes, S., \u0026 Barnes-Holmes, D., (2011). On the Formation and Persistence of Implicit Attitudes: New Evidence from the Implicit Relational Assessment Procedure (IRAP). The Psychological

Documents