Trait and state disgust: an experimental investigation of disgust and avoidance in colorectal cancer decision scenarios

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

1

Trait and state disgust: An experimental investigation of disgust and avoidance in colorectal

cancer decision scenarios

Lisa M. Reynolds

Department of Psychological Medicine, The University of Auckland

Sarah A. McCambridge

Department of Psychological Medicine, The University of Auckland

Ian P. Bissett

Department of Surgery, The University of Auckland

&

Nathan S. Consedine, Ph.D.

Department of Psychological Medicine, The University of Auckland

Reynolds, L. M., McCambridge, S. A., Bissett, I. P., & Consedine, N. S. (in press). Trait and state disgust: An experimental investigation of disgust and avoidance in colorectal cancer

decision scenarios. Health Psychology.

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

2

Abstract

Objective: To evaluate whether trait and experimentally-manipulated state disgust

independently and/or interactively predict immediate and anticipated avoidance in decision

scenarios related to colorectal cancer (CRC).

Methods: Eighty participants, aged 18 to 66 years, completed questionnaires assessing trait

disgust prior to a laboratory session. Participants were gender block randomised to disgust or

control conditions before completing tasks assessing immediate avoidance of a CRC disgust

elicitor (stoma bag) and anticipated avoidance in hypothetical CRC scenarios.

Results: Manipulation checks confirmed the elicitation of disgust in the experimental condition.

Persons in the experimental condition were more likely to exhibit immediate avoidance

behaviors in response to a commonly used bowel disease device (stoma bag), while trait

disgust predicted time to touch the device. Trait disgust also moderated the influence of state

disgust on anticipated avoidance, namely delay in help seeking for bowel symptoms and

predicted rating disgusting side effects as more deterring to adherence.

Conclusions: The current report suggests the importance of examining disgust in CRC

contexts and provides the first empirical demonstration that state and trait aspects of disgust

may interactively operate to deter certain types of decisions. It thus furthers understanding of

emotions and avoidance in a health context that has had surprisingly little focus to date.

Keywords: disgust, avoidance, emotion, decision-making, colorectal-cancer

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

3

Introduction

Contemporary medical systems are placing an increasing emphasis on open

communication and patient-led decision making. Whilst medical decisions can involve

relatively simple choices about whether to seek care for symptoms or participate in screening,

they can also reflect multi-faceted treatment options with uncertain outcomes. Patients are

increasingly asked to make decisions in personal health contexts where the stakes are

enormous. Emerging research indicates a role for emotions in health-related decisions

(Anderson, 2003; Broadstock & Michie, 2000) with studies suggesting that aversive emotions

predict avoidance of recommended screenings and treatments (Consedine, Ladwig, Reddig, &

Broadbent, 2011; Consedine, Magai, Krivoshekova, Ryzewicz, & Neugut, 2004). Anticipated

pain or embarrassment can promote avoidance of screening tests (Magai, Consedine, Neugut,

& Hershman, 2007) and fear of aversive side effects can influence the decision to forego

recommended cancer treatment (Carey & Burish, 1988). The colorectal cancer (CRC) context

generates numerous aversive stimuli; decisions in response to these can have serious

implications, and as such, generate particular challenges to patient decision-making.

Disgust and decision-making

The literature examining emotions in health decision-making has primarily focussed on

fear and embarrassment (Consedine & Moscowitz, 2007), and essentially overlooked other

health-relevant emotions such as disgust. This oversight is surprising given disgust’s particular

relevance to health (Curtis, Aunger, & Rabie, 2004). In theory, disgust functions to minimize or

prevent exposure to health risks, such as those posed by violations of the body, exposure to

bodily products (e.g., feces), and contamination threats (Leshner, Bolls, & Thomas, 2009).

Several qualitative and cross-sectional studies implicate disgust as a barrier to decisions to

attend cancer screening (Chapple, Ziebland, Hewitson, & McPherson, 2008; Deutekom et al.,

2010; O'Sullivan & Orbell, 2004; Worthley et al., 2006), and a retrospective analysis of case

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

4

notes reported ‘repugnance’ as a barrier to opting for stoma surgery (Akute, 2000). While a

single study has shown that state disgust predicts greater intention to use condoms (Tybur,

Bryan, Magnan, & Caldwell Hooper, 2011), experimental studies investigating disgust and

decision-making in health contexts are scant. To advance previous research, the current report

makes three specific contributions: examining disgust in the context of CRC decisions;

considering both immediate and anticipated avoidance; and evaluating the roles of both trait

disgust and state disgust.

First, the current report evaluates the causal impact of disgust on decisions in a new

health domain (CRC) in which the emotion is highly relevant (Reynolds, Consedine, Pizarro, &

Bissett, 2013). Multiple disgust elicitors are found in the CRC trajectory including screening

tests requiring the self-collection of stool samples, examinations involving the insertion of

instruments into the anus, symptoms such as constipation or diarrhoea, aversive treatment

effects, and the potential need for a colostomy (Reynolds, Consedine, Pizarro, et al., 2013).

Behavioral avoidance in CRC is well-established with uptake rates failing to reach

recommended levels (Worthley et al., 2006), delays in seeking medical help for bowel

symptoms (Cockburn, Paul, Tzelepis, McElduff, & Byles, 2003), and withdrawal from

treatment (Simmonds, 2000). In the CRC context, the stakes of avoidance are high (O’Connell,

Maggard, & Ko, 2004). Non-participation in bowel screening predicts greater risk of late stage

presentation and worse prognoses (Doubeni et al., 2013; Morris et al., 2012), while the

avoidance of treatments predicts disease progression and poorer survival (Gill & Goldberg,

2004; Ragnhammar, Hafstrom, Nygren, Glimelius, & SBU-Group., 2001). Given that the core

behavioral response of disgust involves avoidance or withdrawal, disgust may influence CRC

decisions. Experimentally testing this possibility is the broadest aim of the current work.

Second, the report examines the possibility that disgust may be differentially relevant to

immediate versus anticipated avoidance. Many CRC related decisions pertain to situations that

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

5

are yet to occur – perhaps implying that current feelings may be more or less relevant. Prior

work suggests that the anticipation of aversive emotion may play a complex role in health

behaviors, influencing both ‘good’ health behaviors i.e., fear of later regret influencing the

decision to take part in screening (Sandberg & Conner, 2009), as well as decisions that are

deleterious (Consedine et al., 2004). Anticipated negative emotion predicts avoidance in

women with elevated breast cancer risk (Sussner et al., 2009) and cancer fear predicts greater

information avoidance (Miles, Voorwinden, Chapman, & Wardle, 2008). However, most

experimental work has tested links between disgust and immediate avoidance, suggesting that

disgust sensitive individuals behave avoidantly when presented with elicitors such as a used

comb or a filled bedpan (Olatunji et al., 2009; Tsao & McKay, 2004). However, it is unclear

whether disgust may also impact decision-making regarding future events. The current report

advances prior work by examining whether state disgust impacts both immediate behavioral

avoidance as well as avoidant health-relevant decisions about future (anticipated) events.

Disgust sensitivity and state disgust

Finally, the report extends existing research by testing the possibility that trait disgust

(i.e., the stable tendency to experience disgust) and state disgust (i.e., current emotional

experience) are differentially relevant to decisions across immediate versus anticipated

timeframes. Whilst timeframes may influence decisions (above) so too may stable individual

differences in emotional proclivities. Persons vary in the tendency to feel disgust, known as

either trait disgust or dispositional disgust sensitivity (Rozin, Haidt, McCauley, Dunlop, &

Ashmore, 1999), with women generally scoring higher in disgust sensitivity (Druschel &

Sherman, 1999; Haidt, McCauley, & Rozin, 1994). Trait disgust has been linked to avoidant

responding and moral judgements (Deacon & Olatunji, 2007; Inbar, Pizarro, Knobe, & Bloom,

2009) and it may be that these differences in trait disgust also influence CRC decision-making.

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

6

However, it is unclear how state and trait disgust might each be relevant and/or whether

they might interact in predicting immediate versus anticipated avoidance in CRC contexts. One

possibility is that decisions among persons with greater dispositional sensitivity may be more

strongly impacted by state disgust than less sensitive persons such that when they experience

disgust, decisions become more avoidant. Related studies have demonstrated interactions

between dispositional fear and state fear, with fearful people making more pessimistic

judgements about future events (Lerner & Keltner, 2000) and trait and state emotion appear to

differentially impact the cognitive processes that inform decision-making (MacLeod &

Rutherford, 1992). In furthering understanding of disgust’s relevance to health decision-

making, the current report specifically tests whether trait disgust moderates the impact of state

disgust.

The current report

The context of CRC contains many disgust elicitors, and as such, disgust-generated

avoidance seemed likely (Reynolds, Consedine, Pizarro, et al., 2013). Thus, the aims in this

experiment were to investigate the influence of state and trait disgust in this context as follows:

1. To test whether trait disgust and state disgust would independently predict immediate and

anticipated avoidance related to CRC scenarios.

2. To test whether trait disgust would moderate the relationship between state disgust and

avoidance, with individuals higher in trait sensitivity being more avoidant when disgusted.

Methods

Procedure

Following ethics approval, obtained by the University of Auckland Human Participants

Ethics Committee, data were collected between November 2011 and February 2012. People

fluent in English and aged over 18 were invited to participate in a study on ‘Emotions,

Decision-Making and Health’ through campus posters, flyers and emails. No incentive for

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

7

participation was offered. Those interested in taking part were emailed an Internet link to a

baseline questionnaire that included questions about demographics, health status and a measure

of trait disgust. After completing the questionnaire, to control for gender differences in trait

disgust (Haidt et al., 1994), men and women were randomized in separate blocks of four to

either the control or experimental condition, using a Microsoft Excel generated randomisation

list. Participants were then invited to attend a 20-minute laboratory session. In the experimental

condition an olfactant to induce state disgust (see below) was sprayed in the laboratory prior to

participant arrival.

In the laboratory, participants were given an introduction to the study and informed that

the session was not taped or recorded, but that the researcher would be timing different phases

and taking notes. Participants were encouraged to take as long as they needed to complete each

task. Participants then completed a state emotion measure, followed by several tasks designed

to assess behavioral reactions and decisions regarding CRC symptoms, screening, drug

treatment options and side effects. The measure of state emotion was then repeated at the end of

the session. The following pages focus on the tasks specifically related to immediate and

anticipated avoidance, with aspects of other tasks published elsewhere (Reynolds, Consedine,

& McCambridge, 2013). Procedures in both the control and experimental conditions were

standardised so that factors such as the greeting by the researcher, description of tasks, response

options to questions and room set-up were the same for all participants. Given the importance

of participants being blind to the emotions and behaviors of interest, at the completion of the

session, participants were asked to keep their experience of the study confidential and not to

share details of the experiment with others. To maximise blindness among other potential

participants, those who took part were not de-briefed, however were offered the opportunity to

have a summary of results sent to them on completion of analysis.

Disgust Induction

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

8

In order to induce state disgust in the experimental condition an established olfactant

(Schnall, Haidt, Clore, & Jordan, 2008) was used; a commercially available odor reminiscent of

feces (Liquid AssTM). Three sprays of this ‘objectionable’ but non-hazardous odor (Liquid

Assetts Novelties LLC, 2005) were delivered into a rubbish bin liner hidden from participant

view. Due to the presence of odor, the researcher was not blind to condition. Three of the forty

participants in the experimental condition commented on the smell in the room.

Participants

Of the 88 participants who completed the baseline questionnaire, one person was unable

to attend the follow-up session at the laboratory due to access problems, and four could not be

contacted, leaving 83 participants who participated in the follow-up session. Given this report’s

focus on emotion in decision-making, and induced disgust via olfaction, three participants were

excluded from the final analysis – one who reported a recent event impacting on their current

emotional state and two who wore noticeably strong cologne potentially interfering with the

disgust induction. These exclusion criteria had not been foreseen and were determined after the

laboratory session but prior to data analysis. This left a sample of 80, with 40 participants in

each condition, meeting the pre-determined sample size needed to provide sufficient power

(80%) to detect an effect size of 0.30 based on prior work (Schnall et al., 2008). Ages ranged

from 18 to 66 years (median = 22 years) with a slight majority of females (58.8%, n = 47).

None of the demographic characteristics varied significantly between the experimental and

control condition (see Table 1). Health status was investigated by asking participants whether

they had ever been diagnosed with a number of health conditions. Participants were generally

healthy with a small percentage reporting previous diagnoses: heart condition (3.8%, n = 3);

bowel condition (5.0%, n = 4); compromised immune function (2.5%, n = 2); other serious

health problem (2.5%, n = 2); and importantly, no participants had a previous cancer diagnosis.

There were no differences in health status by experimental condition.

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

9

- - - - - - - - - - - - - - - - - - - - - - - - - - -

Insert Table 1 about here

- - - - - - - - - - - - - - - - - - - - - - - - - -

Measures

Trait disgust. The Disgust Scale-Revised (DS-R) (van Overveld, de Jong, Peters, &

Schouten, 2011) is a measure of trait disgust and was collected in the baseline questionnaire.

Participants rate 27 items on a scale of 0 to 4 on how disgusting they find each experience, and

how much they agree with a number of behavioral responses to established elicitors. It can be

divided into three subscales, animal-reminder, contamination and core (food related) disgust.

Mean scores for all items provides a total score, and mean scores for each of the subscales are

calculated, with higher scores indicating greater trait disgust. DS-R has been used in numerous

studies and has good internal consistency for the total score (α =.88) and also for core (α =.78)

and animal-reminder (α =.78) subscales, however reliability for contamination has been found

to be low (α =.54) (van Overveld et al., 2011). In this study, animal-reminder disgust (DS-

animal) was used as the operationalization of trait disgust, as it relates to elicitors such as

bodily excretions and death that provide reminders of our animal nature (Olatunji, Haidt,

McKay, & David, 2008) and is likely the most applicable to the CRC context. The DS-Animal

subscale consists of eight items, with one reverse coded item. In the current study, the DS-

Animal subscale had adequate internal consistency (α = .80). Median splits on the DS-Animal

were used to categorize participants as low or high in trait disgust.

State disgust. To assess state disgust, participants completed the state Differential

Emotions Scale (DES) (Izard, Libero, Putnam, & Haynes, 1993) at two time points; when they

first entered the laboratory (Time 1) and again at the completion of the experiment (Time 2).

The DES is a 30-item scale containing three items for each of ten emotions. Using a 1 to 5

scale, participants rate the extent to which they are currently experiencing each emotion. Mean

scores are calculated for all items (to provide an overall score), and for each of the subscales to

provide a score for individual emotions, with higher scores indicating greater state emotion.

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

10

Because disgust and fear promote avoidance and are likely in health contexts, these emotions

were specifically analyzed. However, all emotions were assessed to maximize participant

blindness. In previous research these subscales have shown adequate reliability: fear α =.83

to .88; disgust α =.56 to .77 (Izard et al., 1993; Youngstrom & Green, 2003). Because the DES

omits items specifically assessing embarrassment, three items were added from the

Susceptibility to Embarrassment Scale (SES) using the same scale and scoring as the DES

(Kelly & Jones, 1997). In this study, the DES subscales were reliable (disgust α=.79, fear

α=.72) as were the three embarrassment items (α=.82).

Assessment of Avoidant Decision-Making

Immediate behavioral avoidance: Participants were shown a stoma bag designed to

look previously used; the bag was crumpled, a non-hazardous brown spread (MarmiteTM) was

very lightly applied around the inside edges, and a sticker with a name crossed out placed on

the outside. They were told: “This is a stoma bag used to collect feces. It is sterile. Feel free to

inspect it more closely”. Behavioral responses were recorded including whether the bag was

touched and, if so, the delay (seconds) to do so. Established avoidant behavioral responses to

disgust (Angyal, 1941) were observed and coded by the research assistant (1 if they occurred

and 0 if not). These included whether participants averted their eyes, touched their nose, folded

their arms or leaned backwards. A yes = 1 and no = 0 code was given to each of these

behaviors and a total score between 0 and 4 calculated. Time has been used as a measure of

avoidance in prior disgust experimentation (Woody & Tolin, 2002), however coding of

established behavioral responses to disgust does not appear to have been used in previous

experimental health work.

Anticipated behavioral avoidance: Anticipated avoidance was assessed by presenting

participants with a number of hypothetical decision-making scenarios related to CRC screening

and treatment. These included:

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

11

1. Medical assessment delay: Participants were asked to imagine they had been feeling

unwell with bowel symptoms for the past 5 days including diarrhoea and a dull stomach

pain and possibly blood in their stool. They were instructed that if they made a doctor’s

appointment they would need to collect a fecal sample to take to the clinic, and asked to

tick one of two boxes that gave the option of ‘call and make an appointment today’ or

‘wait another few days’. (NB for reviewer: FOBT reference was removed here).

2. Side effect aversions: Participants were asked to imagine they had cancer and required

medication to prolong their life. They were asked to identify four from a choice of 12

common side effects that would most deter their taking a life-extending cancer treatment:

4 ‘neutral’ side effects (e.g., blurry vision); 4 classified as ‘disgusting’ (e.g., genital

discharge); and 4 ‘embarrassing’ (e.g., moderate weight gain). A team of health

professionals, psychologists and researchers, including clinicians experienced in cancer

treatment contexts, generated these items based on normative chemotherapy side effects.

Classification of items as ‘disgusting’, ‘embarrassing’ or ‘neutral’ was based on theory;

disgust elicitors involved bodily product and excretions, whereas embarrassment elicitors

had a social or appearance related element that would have been difficult to hide. The

four most deterrent items for each participant were coded as 1, with all other items coded

as 0. Scores for each category of neutral, disgusting and embarrassing were then

calculated with possible scores ranging from 0 to 4. The score for disgust was used for

analyses in this report.

Analytic strategy

Analyses began by assessing whether disgust was successfully induced using a 2

(disgust v. control condition) x 2 (start v. end of session) MANOVA. Next, because prior

studies have found reliably greater trait disgust among women (Haidt et al., 1994), sex

differences in trait disgust were checked using an independent t-test analysis to test for

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

12

differences at baseline, and a 2 (gender) x 2 (disgust v. control condition) ANOVA was run on

state disgust scores to test whether state disgust scores differed for men and women across

conditions. Next, the relationship between study measures was investigated using Pearson

correlations (for parametric variables) and Spearman’s rho correlations (for non-parametric

variables) before assessing the primary research questions. Analyses then proceeded in two

major phases. First, the relationship between trait disgust, state disgust and immediate

avoidance (i.e., responses to the stoma bag) was assessed using ANOVA analyses with 2 (low

v. high trait disgust) x 2 (disgust v. control condition). Second, relationships between trait

disgust, state disgust and anticipated avoidance (i.e., the decision to delay medical assessment

and side effect aversions) were tested using logistic regression to measure the categorical

decision whether to delay seeking medical help, and a 2 (low v. high trait disgust) x 2 (disgust

v. control condition) ANOVA to investigate side effect aversions.

Results

Disgust manipulation

A significant difference in state disgust was found across conditions, Wilks’ Λ = .90,

F(2,77)=4.40, p=.016, ηp2 =.10 (see Figure 1); the experimental group reported more disgust

than the control group at both the beginning, F(1,78)=5.65, p=.020, ηp2 =.07, and end of the

session, F(1,78)=7.75, p=.007, ηp2 =.09. The absence of a difference between state disgust at

the beginning and end of the session, showed that disgust was sustained throughout the

experiment, Wilks’ Λ = 1.00, F(1, 78)=.11, p=.739, ηp2 =.001. Importantly, in terms of

eliminating other possible affective bases for avoidance, the induction was specific to disgust,

as embarrassment, F(2,77)=.12, p=.884, ηp2 =.003 and fear, F(2,77)=1.20, p=.308, ηp2 =.03,

did not vary as a function of condition.

- - - - - - - - - - - - - - - - - - - - - - - - - - - Insert Figure 1 about here

- - - - - - - - - - - - - - - - - - - - - - - - - -

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

13

Given that participants in both control and experimental conditions were presented with

numerous stimuli designed to elicit disgust over and above the olfactant (e.g., both groups were

presented with an apparently ‘used’ stoma bag), the subsequent analysis of state disgust

concentrates on DES disgust scores at the start of the laboratory session i.e., before these other

factors had been introduced to participants.

Gender, trait disgust and state disgust

The scores of trait disgust did not differ significantly in men (M=1.54, SD=.91) from

women (M=1.71, SD=.86), t(78)=.862, p=.391, in this study. Similarly, state disgust scores

did not differ for men and women across conditions with no main effect on state disgust for

gender, F(1,76)=.04, p=.847), nor was there an interaction effect between gender and

condition, F(1,76)=.41, p=.523 (men M=3.76, SD=1.80; women M=3.80, SD=1.80). Despite

the trait and state scores of men and women being in the expected direction (i.e., higher for

women), in the absence of significant gender differences for either trait disgust or state disgust,

sex was not controlled for in subsequent analyses.

Correlations between Study Measures

Correlations between study measures are presented in Table 2. Perhaps unexpectedly,

trait disgust was not associated with state disgust, nor was it associated with the decision to

delay seeking medical assistance. As expected, however, trait disgust was correlated with

observed avoidance behaviors and the time taken to touch the stoma bag. State disgust was not

associated with any of the other measures.

- - - - - - - - - - - - - - - - - - - - - - - - - -

Insert Table 2 about here

- - - - - - - - - - - - - - - - - - - - - - - - - - -

Immediate behavioral avoidance, trait disgust and state disgust

To analyse whether manipulated (state) and trait disgust influenced immediate avoidant

behaviors, responses to the presentation of a stoma bag were assessed. In total, 62 of 80

participants touched the stoma with no difference in the proportion who touched the bag by

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

14

condition, χ2 (1, N = 80) = .00, p=1.000. Next, possible differences in observed avoidance

behaviors during the stoma exercise were tested. Persons in the disgust condition exhibited

more avoidance behaviors such as gaze aversion, folding arms, leaning backwards, and

touching their nose, F(1,76)=4.06, p=.048, ηp2 =.05, and there was a non-significant trend in

the predicted direction for trait disgust, F(1,76)=3.68, p=.059, ηp2 =.05; persons with higher

trait disgust showed more avoidant behaviors (see Figure 2). There was, however, no

interaction between trait disgust and condition on immediate avoidance, F(1,76)=.28, p=.596.

- - - - - - - - - - - - - - - - - - - - - - - - - - -

Insert Figure 2 about here

- - - - - - - - - - - - - - - - - - - - - - - - - - -

Next, for the portion of the sample that touched the stoma bag, differences in the time

taken to touch the bag were examined. While there was no effect for condition, F(1,58)=.38,

p=.539, there was an effect for trait disgust, F(1,58)=4.50, p=.038, ηp2 =.07, with people high

in trait disgust taking longer to touch the bag (see Figure 3). No interaction between trait

disgust and condition was found (F(1,58)=.75, p=.390). Thus, high trait disgust was associated

with greater immediate avoidance as indexed by a greater delay in time to touch the stoma bag.

- - - - - - - - - - - - - - - - - - - - - - - - - - -

Insert Figure 3 about here

- - - - - - - - - - - - - - - - - - - - - - - - - - -

Anticipated avoidance, trait disgust and state disgust

To investigate hypotheses regarding the effects of state and trait disgust on anticipated

avoidance, decisions on two hypothetical scenarios were assessed: 1) whether to seek treatment

when confronted with bowel symptoms; and 2) ranking disgusting side effects as deterring to

taking cancer drugs. To test whether people chose to delay a medical consultation in the

presence of bowel symptoms, a logistic regression was run with trait disgust and condition

entered at the first step, and the interaction between these variables entered in the next step. The

final model was significant, χ2 (3, N=80) = 11.95, p=.008 and is presented in Table 3.

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

15

Examination of the plot suggests delay was more likely in the disgust condition among those

with high versus low trait disgust (see Figure 4).

- - - - - - - - - - - - - - - - - - - - - - - - - - -

Insert Table 3 about here

- - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - Insert Figure 4 about here

- - - - - - - - - - - - - - - - - - - - - - - - - - - Finally, analysis of the scenario asking participants to rank the side effects that would

most deter them from taking cancer treatment drugs was investigated. There was a main effect

for trait disgust with those high in trait disgust more likely to rate the disgusting side effects as

deterring them from taking cancer medication, F(1,76)=5.18, p=.026, ηp2 =.06 (see Figure 5).

There was no effect for condition (F(1,76)=.002, p=.965), and no interaction effect

(F(1,76)=.42, p=.519) in this model.

- - - - - - - - - - - - - - - - - - - - - - - - - - -

Insert Figure 5 about here

- - - - - - - - - - - - - - - - - - - - - - - - - - -

Discussion

The current report tested whether trait disgust or induced state disgust were related to

immediate and anticipated avoidance behaviors in CRC contexts, and whether trait disgust

moderated the relationship between state disgust and avoidance. Disgust was successfully

induced in the disgust condition and, as expected, both greater trait disgust and experimental

condition were associated with greater immediate avoidance, and trait disgust was associated

with greater deterrence to ‘disgusting’ side effects. However, trait disgust also moderated the

influence of the disgust condition in decisions regarding anticipated future delay, perhaps

suggesting that more disgust sensitive individuals are prone to avoidance when disgusted, at

least regarding a possible future exposure. Below, these findings are discussed in light of

related work on emotions, disgust, and decision-making, issues of state versus trait and

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

16

immediate versus anticipated avoidance, and clinical implications regarding avoidance in CRC

contexts and directions for future research are suggested.

Disgust and avoidance in CRC contexts

The first contribution of this work lies in advancing knowledge regarding links

between state and trait disgust and both immediate and anticipated avoidance behaviors in

CRC contexts. Consistent with recent work (Borg, de Jong, & Mazza, 2012; Deacon &

Olatunji, 2007; Tsao & McKay, 2004; Woody & Tolin, 2002), trait disgust predicted

immediate avoidance of CRC-relevant stimuli. Although there was no effect on whether an

apparently used stoma bag (a commonly used bowel disease treatment device) was touched,

greater trait disgust predicted greater delay before touching the bag and there was a non-

significant trend for increased observed avoidance behaviors. Prior works assessing trait

disgust are necessarily vulnerable to the third variable problem and it is important to recall that

immediate avoidance was also predicted by state disgust; although why state disgust only

predicted observed avoidance behaviors, but not the time taken to touch the bag is difficult to

interpret and warrants further study. These findings suggest that disgust may be causally

implicated in at least some immediate forms of health-relevant avoidance.

More broadly, although prior works have cross-sectionally linked disgust to outcomes

in domains including organ donation intention (O'Carroll, Foster, McGeechan, Sandford, &

Ferguson, 2011), bowel screening (Chapple et al., 2008; O'Sullivan & Orbell, 2004) and

adaptation to colostomy (Smith, Loewenstein, Rozin, Sherriff, & Ubel, 2007), this report

represents the first test of a possible link between disgust and avoidant health decision making.

CRC contexts contain many disgust elicitors (Reynolds, Consedine, Pizarro, et al., 2013) and

while avoidance is understandable in light of disgust’s evolved functions, it may be

unhelpful, in many, if not most, health decision contexts if avoidance of screenings and

treatment are a consequence. Late presentation and/or treatment delay or withdrawal for

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

17

example, bode ill for prognoses (O’Connell et al., 2004) and these data suggest that disgust

may act as an affective substrate for avoidant decision-making in CRC disease.

Conversely, the applicability of earlier experimental studies to health is limited insofar

as there is no immediately evident “cost” to avoidance in laboratory contexts and such

avoidance may simply reflect disgust’s health promoting functions in forestalling exposure to

potential contaminants. However, the current design enabled us to assess whether trait and state

disgust predicted CRC-related avoidance in both immediate as well as in future health

decisions. For future health scenarios, trait disgust predicted potential avoidance; namely,

reporting the disgusting side effects of a possible treatment as more deterring, while state

disgust only predicted future avoidance (delay) in concert with the trait. Although this may

indicate that disgust is less causally relevant to future decisions, it may also be that people

place a greater decisional “weight” on dispositional sensitivities when the consequences of a

decision are broader or longer lasting. Designs in which disgust is elicited and temporal or

life-impacting health decisions are then systematically varied are an obvious next step in this

regard.

Finally, the current report provides the first empirical demonstration that state and trait

aspects of disgust may interactively operate to deter certain types of decisions. Although

interactions were not evident in the immediate avoidance tasks, and neither trait nor state

disgust independently predicted delay seeking medical assistance for bowel symptoms,

analyses in this report suggest that people with greater trait disgust are sometimes more likely

to delay, at least when they are actually disgusted; persons in the control condition were no

more or less likely to delay as a function of their trait disgust sensitivity.

Thus, a third contribution lies in demonstrating that disgust-related avoidant behavior

and decision-making may be interactively influenced by state and trait disgust. This finding is

consistent with emerging work showing that individuals with greater trait disgust exhibit an

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

18

enhanced ability to detect subtle deviations from the color white, a task modeled to represent

the capacity to perceive ‘impurities’ (Sherman, Haidt, & Clore, 2012). Similarly, a

moderating effect for state disgust was seen in a recent report which found that, when

disgusted, people who were fearful of contamination rated their perceived threat of becoming

ill as greater than when not disgusted (Verwoerd, de Jong, Wessel, & van Hout, 2013).

Interpretatively, it has been suggested that dispositional tendencies to specific classes

of aversive emotion can activate automatic processing styles that guide subsequent perception

and judgment (Lerner & Keltner, 2000). Alternately, the impact of trait and state emotion

may vary depending on whether automatic (non-conscious) or strategic (consciously

mediated) processing occur (MacLeod & Rutherford, 1992). In this experiment, immediate

avoidance may reflect unconscious automatic reactions while delay in care-seeking for

symptoms may have activated more considered or deliberative processes. Thus, like anxious

individuals primed to respond to threatening stimuli, it may be that when appraising a

potentially disgust eliciting health contact, disgust sensitive individuals are primed to notice

and respond avoidantly when disgusted. Further investigation into the underlying mechanisms

would be valuable and could explain how dispositional tendencies moderate the role of state

disgust on immediate avoidance and anticipated avoidance.

Clinical Implications

This report establishes the importance of acknowledging disgust in CRC contexts.

Whilst the study sample were healthy volunteers, these findings may translate to clinical

populations, where disgust might equally create avoidant decision-making among those

experiencing or in contact with CRC – including patients, care givers and health professionals

alike. Understanding how disgust might influence detrimental avoidance behaviors is a first

step to better management of unhelpful withdrawal, delay and avoidance in this context.

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

19

Secondly, this report builds on prior work by suggesting that anticipated disgust might

help explain some avoidant health decision-making. Although the design does not enable us

to be sure that it was the disgust nature of the scenarios that deterred behavior, such a process

seems likely. Anticipated emotion is relevant clinically because it impacts health-related

behavior (Chapman & Coups, 2006; O'Carroll et al., 2011; van Overveld, de Jong, & Peters,

2010; Woody & Tolin, 2002) and is often worse than that which is ultimately experienced;

for example persons undergoing colorectal procedures report that the event was better than

anticipated (Von Wagner et al., 2009). Work in the area of affective forecasting shows that

people typically overestimate the affective impact of illness and disability and underestimate

their ability to adapt (Ubel, Loewenstein, Schwarz, & Smith, 2005) which can have serious

implications in cancer decision making. Disgust is likely no different and vulnerable persons

may avoid screening or delay treatment because of ‘inflated’ or erroneous expectancies

regarding how they will feel. Conversely, individuals may also habituate to disgust elicitors

over time (Rozin, 2008). Continued investigation into the role of anticipated disgust in CRC

contexts would provide data to guide communications aimed at encouraging participation in

health promoting behavior.

Finally, this report describes a potentially interactive role for trait and state disgust in

avoidant decision-making. If, as the current findings suggest, state disgust influences the

decision-making of some people but not others, targeted messaging and/or interventions

aimed at disgust sensitive individuals more susceptible to making detrimental decisions may

be warranted. Early identification of those most vulnerable using either standardized

measures of disgust sensitivity (such as DS-R, van Overveld et al, 2011) or simply asking

people how they feel about such matters, combined with messages that acknowledge and

normalize disgust responses and/or provide coping strategies could potentially alleviate

avoidant decision-making.

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

20

Limitations and concluding remarks

While the current report represents a useful addition to understanding how trait and

state disgust influence avoidant decisions in immediate and anticipated CRC contexts, there are

limitations worth noting. Firstly, associations between disgust and avoidance were relatively

small and the moderating influence of trait disgust only evidenced in one scenario, implying

other factors are also involved. The use of dichotomized measures have also been criticised

(MacCallum, Zhang, Preacher, & Rucker, 2002), and whilst alternative analyses in this

experiment produced near identical results, an ANOVA-based approach was maintained due

to greater accessibility and interpretation of findings. These findings are also limited by a

small sample size, limited ethnicity and age – any of which may have implications for

generalizability. Whilst the experimental design was built on previous cross-sectional and

qualitative work, it was conducted with young, essentially healthy volunteers who are unlikely

to have experienced stool sampling or bowel cancer tests. Prior experience was not assessed

hence the extent to which such factors may have impacted results cannot be determined.

Contrary to prior work, no gender differences in disgust were found (although trends were in

the expected direction); sample size is a likely explanation. Further, facility restraints meant

that avoidance behaviors were coded by the experimenter within the room, who, due to the

nature of the olfactory induction, was not blind to condition.

It is unclear how the processes documented in this report might operate among those

making real life decisions about going to their doctor with bowel symptoms, attending bowel

screening, or proceeding with CRC treatment. Recent longitudinal work investigating real-life

adaptation to colostomies suggests estimations of future adaptation are inaccurate (Peeters,

Smith, Loewenstein, & Ubel, 2012). This inability to predict future affect has important

implications for decision-making in health contexts. Designs that assess trait and state disgust

in people making real-life decisions about CRC screening and treatment could provide

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

21

important clinical insights in an area where avoidance is common but in which there has been

little research to date.

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

22

References

Akute, O. O. (2000). Colorectal carcinoma in Ibadan, Nigeria: A 20-year survey - 1971 to

1990. Hepato-Gastroenterology, 47(33), 709-713.

Anderson, C. J. (2003). The Psychology of Doing Nothing: Forms of Decision Avoidance

Result From Reason and Emotion. Psychological Bulletin January, 129(1), 139-167.

Angyal, A. (1941). Disgust and related aversions. Journal of Abnormal and Social Psychology,

36, 393-412.

Borg, C., de Jong, P. J., & Mazza, M. (2012). Feelings of Disgust and Disgust-Induced

Avoidance Weaken following Induced Sexual Arousal in Women [Article]. PLoS ONE,

7(9), 1-8. doi: 10.1371/journal.pone.0044111

Broadstock, M., & Michie, S. (2000). Processes of patient decision making: Theoretical and

methodological issues. Psychology & Health, 15(2), 191-204.

Carey, M. P., & Burish, T. G. (1988). Etiology and treatment of the psychological side effects

associated with cancer chemotherapy: A critical review and discussion. Psychological

Bulletin, 104(3), 307.

Chapman, G. B., & Coups, E. J. (2006). Emotions and preventive behavior: Worry, regret, and

influenza vaccination. Health Psychology, 25(1), 82-90.

Chapple, A., Ziebland, S., Hewitson, P., & McPherson, A. (2008). What affects the uptake of

screening for bowel cancer using a faecal occult blood test (FOBt): A qualitative study.

Social Science and Medicine, 66(12), 2425-2435.

Cockburn, J., Paul, C., Tzelepis, F., McElduff, P., & Byles, J. (2003). Delay in seeking advice

for symptoms that potentially indicate bowel cancer. American Journal of Health

Behavior, 27(4), 401-407.

Consedine, N. S., Ladwig, I., Reddig, M. K., & Broadbent, E. A. (2011). The many faeces of

colorectal cancer screening embarrassment: Preliminary psychometric development and

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

23

links to screening outcome. British Journal of Health Psychology, 16(3), 559-579. doi:

10.1348/135910710X530942

Consedine, N. S., Magai, C., Krivoshekova, Y. S., Ryzewicz, L., & Neugut, A. I. (2004). Fear,

anxiety, worry, and breast cancer screening behavior: A critical review. Cancer

Epidemiology, Biomarkers and Prevention, 13(4), 501-510.

Consedine, N. S., & Moscowitz, J. T. (2007). The role of discrete emotions in health outcomes:

a critical review. Journal of Applied and Preventive Psychology, 12, 59-75.

Curtis, V., Aunger, R., & Rabie, T. (2004). Evidence that disgust evolved to protect from risk

of disease. Proceedings of the Royal Society Biological Sciences: Series B, 271, S131-

S133.

Deacon, B., & Olatunji, B. O. (2007). Specificity of disgust sensitivity in the prediction of

behavioral avoidance in contamination fear. Behaviour Research and Therapy, 45(9),

2110-2120. doi: 10.1016/j.brat.2007.03.008

Deutekom, M., Van Rossum, L. G. M., Van Rijn, A. F., Laheij, R. J. F., Fockens, P., Bossuyt,

P. M. M., . . . Jansen, J. B. M. J. (2010). Comparison of guaiac and immunological fecal

occult blood tests in colorectal cancer screening: The patient perspective. Scandinavian

Journal of Gastroenterology, 45(11), 1345-1349.

Doubeni, C. A., Weinmann, S., Adams, K., Kamineni, A., Buist, D. M., Ash, A. S., . . . Weiss,

N. S. (2013). Screening colonoscopy and risk for incident late-stage colorectal cancer

diagnosis in average-risk adults [Article]. Annals of Internal Medicine, 158(5), 312-320.

Druschel, B. A., & Sherman, M. F. (1999). Disgust sensitivity as a function of the Big Five and

gender. Personality and Individual Differences, 26(4), 739-748. doi:

http://dx.doi.org/10.1016/S0191-8869(98)00196-2

Gill, S., & Goldberg, R. M. (2004). First-line treatment strategies to improve survival in

patients with advanced colorectal cancer [Review]. Drugs, 64(1), 27-44.

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

24

Haidt, J., McCauley, C., & Rozin, P. (1994). Individual differences in sensitivity to disgust: A

scale sampling seven domains of disgust elicitors. Personality and Individual

Differences, 16, 701-713.

Inbar, Y., Pizarro, D. A., Knobe, J., & Bloom, P. (2009). Disgust sensitivity predicts intuitive

disapproval of gays. Emotion, 9(3), 435-439. doi: 10.1037/a0015960

Izard, C. E., Libero, D. Z., Putnam, P., & Haynes, O. M. (1993). Stability of emotion

experiences and their relations to traits of personality. Journal of Personality & Social

Psychology 64(5), 847-860.

Kelly, K. M., & Jones, W. H. (1997). Assessment of dispositional embarrassability. Anxiety,

Stress & Coping, 10(4), 307-333.

Lerner, J. S., & Keltner, D. (2000). Beyond valence: Toward a model of emotion-specific

influences on judgement and choice. Cognition & Emotion, 14(4), 473-493.

Leshner, G., Bolls, P., & Thomas, E. (2009). Scare 'em or disgust 'em: The effects of graphic

health promotion messages. Health Communication, 24, 447-458.

Liquid Assetts Novelties LLC. (2005). Liquid Ass [Material Safety Data Sheet]. Retrieved

from http://www.liquidass.com/MSDS.html.

MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of

dichotomization of quantitative variables. Psychological Methods, 7, 19-40.

MacLeod, C., & Rutherford, E. M. (1992). Anxiety and the selective processing of emotional

information: Mediating roles of awareness, trait and state variables, and personal

relevance of stimulus materials. Behaviour Research and Therapy, 30(5), 479-491. doi:

10.1016/0005-7967(92)90032-C,

Magai, C., Consedine, N. S., Neugut, A. I., & Hershman, D. L. (2007). Common psychosocial

factors underlying breast cancer screening and breast cancer treatment adherence: a

conceptual review and synthesis. Journal of Women's Health, 16(1), 11-23.

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

25

Miles, A., Voorwinden, S., Chapman, S., & Wardle, J. (2008). Psychologic predictors of cancer

information avoidance among older adults: the role of cancer fear and fatalism. Cancer

epidemiology biomarkers & prevention, 17(8), 1872-1879.

Morris, E. J. A., Whitehouse, L. E., Farrell, T., Nickerson, C., Thomas, J. D., Quirke, P., . . .

Patnick, J. (2012). A retrospective observational study examining the characteristics and

outcomes of tumours diagnosed within and without of the English NHS Bowel Cancer

Screening Programme. British Journal of Cancer, 107(5), 757-764. doi:

10.1038/bjc.2012.331

O'Carroll, R. E., Foster, C., McGeechan, G., Sandford, K., & Ferguson, E. (2011). The "ick"

factor, anticipated regret, and willingness to become an organ donor. Health

Psychology, 30(2), 236-245. doi: 10.1037/a0022379

O'Sullivan, I., & Orbell, S. (2004). Self-sampling in screening to reduce mortality from

colorectal cancer: A qualitative exploration of the decision to complete a faecal occult

blood test (FOBT). Journal of Medical Screening, 11(1), 16-22.

O’Connell, J. B., Maggard, M. A., & Ko, C. Y. (2004). Colon cancer survival rates with the

new American Joint Committee on Cancer sixth edition staging. Journal of the National

Cancer Institute, 96(19), 1420-1425.

Olatunji, B. O., Haidt, J., McKay, D., & David, B. (2008). Core, animal reminder, and

contamination disgust: Three kinds of disgust with distinct personality, behavioral,

physiological, and clinical correlates. Journal of Research in Personality, 42(5), 1243-

1259. doi: 10.1016/j.jrp.2008.03.009

Olatunji, B. O., Wolitzky-Taylor, K. B., Ciesielski, B. G., Armstrong, T., Etzel, E. N., & David,

B. (2009). Fear and disgust processing during repeated exposure to threat-relevant

stimuli in spider phobia. Behaviour Research and Therapy, 47(8), 671-679.

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

26

Peeters, Y., Smith, D., Loewenstein, G., & Ubel, P. (2012). After Adversity Strikes:

Predictions, Recollections and Reality Among People Experiencing the Onset of

Adverse Circumstances [Article]. Journal of Happiness Studies, 13(4), 589-600. doi:

10.1007/s10902-011-9281-7

Ragnhammar, P., Hafstrom, L., Nygren, P., Glimelius, B., & SBU-Group. (2001). A systematic

overview of chemotherapy effects in colorectal cancer [Review]. Acta Oncologica,

40(2-3), 282-308.

Reynolds, L. M., Consedine, N. S., & McCambridge, S. A. (2013). Mindfulness and disgust in

colorectal cancer scenarios: Non-judging and non-reacting components predict

avoidance when it makes sense. Mindfulness. doi: 10.1007/s12671-013-0200-3

Reynolds, L. M., Consedine, N. S., Pizarro, D. A., & Bissett, I. P. (2013). Disgust and

behavioral avoidance in colorectal cancer screening and treatment: A systematic review

and research agenda. Cancer Nursing, 36(2), 122-130. doi:

10.1097/NCC.0b013e31826a4b1b

Rozin, P. (2008). Hedonic "adaptation": Specific habituation to disgust/death elicitors as a

result of dissecting a cadaver. Judgement and Decision Making, 3(2), 191-194.

Rozin, P., Haidt, J., McCauley, C. R., Dunlop, L., & Ashmore, M. (1999). Individual

differences in disgust sensitivity: Comparisons and evaluations of paper-and-pencil

versus behavioral measures. Journal of Research in Personality, 33(3), 330-351.

Sandberg, T., & Conner, M. (2009). A mere measurement effect for anticipated regret: Impacts

on cervical screening attendance. British Journal of Social Psychology, 48(2), 221-236.

Schnall, S., Haidt, J., Clore, G. L., & Jordan, A. H. (2008). Disgust as embodied moral

judgment. Personality and Social Psychology Bulletin, 34(8), 1096. doi:

10.1177/0146167208317771

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

27

Sherman, G. D., Haidt, J., & Clore, G. L. (2012). The faintest speck of dirt: Disgust enhances

the detection of impurity. Psychological Science, 23(12), 1506-1514. doi:

10.1177/0956797612445318

Simmonds, P. (2000). Palliative chemotherapy for advanced colorectal cancer: systematic

review and meta-analysis. BMJ: British Medical Journal, 321(7260), 531.

Smith, D. M., Loewenstein, G., Rozin, P., Sherriff, R. L., & Ubel, P. A. (2007). Sensitivity to

disgust, stigma, and adjustment to life with a colostomy. Journal of Research in

Personality, 41(4), 787-803.

Sussner, K. M., Thompson, H. S., Jandorf, L., Edwards, T. A., Forman, A., Brown, K., . . .

Valdimarsdottir, H. B. (2009). The influence of acculturation and breast cancer-specific

distress on perceived barriers to genetic testing for breast cancer among women of

African descent. Psycho-Oncology, 18(9), 945-955.

Tsao, S. D., & McKay, D. (2004). Behavioral avoidance tests and disgust in contamination

fears: Distinctions from trait anxiety. Behaviour Research and Therapy, 42(2), 207-216.

Tybur, J. M., Bryan, A. D., Magnan, R. E., & Caldwell Hooper, A. E. (2011). Smells like safe

sex: Olfactory pathogen primes increase intentions to use condoms. Psychological

Science, 22(4), 478-480. doi: 10.1177/0956797611400096

Ubel, P. A., Loewenstein, G., Schwarz, N., & Smith, D. (2005). Misimagining the

unimaginable: The disability paradox and health care decision making. Health

Psychology, 24(4), S57-S62. doi: 10.1037/0278-6133.24.4.S57

van Overveld, M., de Jong, P. J., & Peters, M. L. (2010). The Disgust Propensity and

Sensitivity Scale - Revised: Its predictive value for avoidance behavior. Personality and

Individual Differences, 49(7), 706-711. doi: 10.1016/j.paid.2010.06.008

RUNNING HEAD: TRAIT AND STATE DISGUST IN DECISION-MAKING

28

van Overveld, M., de Jong, P. J., Peters, M. L., & Schouten, E. (2011). The Disgust Scale-R: A

valid and reliable index to investigate separate disgust domains? Personality and

Individual Differences, 51(3), 325-330. doi: 10.1016/j.paid.2011.03.023

Verwoerd, J., de Jong, P. J., Wessel, I., & van Hout, W. J. (2013). “If I feel disgusted, I must be

getting ill”: Emotional reasoning in the context of contamination fear. Behaviour

Research and Therapy, 51(3), 122-127. doi: http://dx.doi.org/10.1016/j.brat.2012.11.005

Von Wagner, C., Knight, K., Halligan, S., Atkin, W., Lilford, R., Morton, D., & Wardle, J.

(2009). Patient experiences of colonoscopy, barium enema and CT colonography: A

qualitative study. British Journal of Radiology, 82(973), 13-19. doi:

10.1259/bjr/61732956

Woody, S. R., & Tolin, D. F. (2002). The relationship between disgust sensitivity and avoidant

behavior: Studies of clinical and nonclinical samples. Journal of Anxiety Disorders,

16(5), 543-559. doi: 10.1016/s0887-6185(02)00173-1

Worthley, D. L., Cole, S. R., Esterman, A., Mehaffey, S., Roosa, N. M., Smith, A., . . . Young,

G. P. (2006). Screening for colorectal cancer by faecal occult blood test: Why people

choose to refuse. Internal Medicine Journal, 36(9), 607-610. doi: 10.1111/j.1445-

5994.2006.01155.x

Youngstrom, E. A., & Green, K. W. (2003). Reliability generalization of self-report of

emotions when using the Differential Emotions Scale. Educational and Psychological

Measurement, 63(2), 279-295.