A formal model of new reinforcement sensitivity theory (RST) Alan Pickering Department of Psychology a.pickering@gold.ac.uk.

A formal model of “new” reinforcement sensitivity

theory (RST)

Alan PickeringDepartment of Psychology

a.pickering@gold.ac.uk

Overview• Review:

Old RSTNew RSTPast theoretical models of system interactions

• Present outline of a formal model of interactions in “new” RST

• Conclusions

OLD RSTBehavioural Activation System = BASBehavioural Inhibition System = BIS

SYSTEM RESPONDS TO

OUTPUTS TRAIT

BAS ConditionedReward

Approach + Arousal

Imp (Ext)

BIS ConditionedPunishment

Inhibition + Arousal

Anxiety(N)

NEW RSTFlight/Fight/Freeze System = FFFS

SYSTEM RESPONDS TO

OUTPUTS TRAIT

BAS Reward Approach + Arousal

Imp (Ext)

FFFS Punishment Flight/Fight/Freezing

???

BIS Goal Conflict Inhibition + Arousal

Anxiety(N)

Interactions

• Dynamic interactions between activated systems

• E.g., mutually inhibitory above• But NOT necessarily statistical

interactions

System 1 System 2

Input Input

Interactions in Old RST 1Gray & Smith (1969). In Gilbert and Sutherland

(Eds) Animal Discrimination Learning. London: Academic Press.

Interactions in Old RST 2Pickering (1997). European Psychologist, 2,

139-163.

Interactions in New RST 1McNaughton & Corr (2004). Neuro-

science and Biobehavioural Reviews, 28, 285-305.

Corr (2004). Neuroscience and Biobehavioural Reviews, 28, 317-332.

Interactions in New RST 2

System Interactions and Joint Subsystems RST

• SimilaritiesBoth emphasise joint actions of systems with independent sensitivities

• DifferencesJoint subsystems is an additive account whereas system interactions are typically nonlinear and may cause statistical interactions

Separable Subsystems• Response to

Reward (S+) solely controlled by BAS/IMP etc

• A single main effect

Joint Subsystems• Response to

reward (S+) reflects both BAS/IMP and BIS/ANX

• Two main effects (but no interaction)

A Simple Model of New RST• Has dynamically interacting systems• Has 3 key sensitivity parameters

wA BAS sensitivity

wF FFFS sensitivity

wI BIS sensitivity

• Has two key parameters concerning strengths of input stimuliSR reward stimulus strength

SF fear stimulus strength

independent

Two System ModelSF

SR

FFFS BAS

System Outputs

wF wA

inhibitory

excitatory

Three System ModelSF

SR

FFFS BAS

FFFS Output

wF wA

BIS

BAS Output

wI

AND

inhibitory

excitatory

Simulation 1: No BISwA = 0.5; SR=0.5; SF=0.5/0.9

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 0.2 0.4 0.6 0.8 1

FFFS Sensitivity

Act

ivat

ion

FFFSActivation0.5

BASActivation0.5

FFFSActivation0.9

BASActivation0.9

Simulation 2: With BISwA = 0.5; SR=0.5; SF=0.5/0.9; plus wI = 0.5

-0.2

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

FFFS Sensitivity

Act

ivat

ion

FFFSActivation0.5

BASActivation0.5

FFFSActivation0.9

BASActivation0.9

Simulation 2: BIS ActivationwA = 0.5; SR=0.5; SF=0.5/0.9; plus wI = 0.5

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 0.2 0.4 0.6 0.8 1

FFFS Sensitivity

Act

ivat

ion

BISActivation0.5

BISActivation0.9

Simulation 3: Varying SF & SR

SF + SR = 1; wA = wF = 0.5; plus no BIS / wI = 0.5

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

Strength of Fear Stimulus

Ac

tiv

ati

on

FFFSActivation

BASActivation

FFFSActivationwith BISBASActivationwith BISBISActivation

Simulation 4: Simulating self-reported trait values

• How might self-report trait values map onto the 3 underlying sensitivities in the model?

• Assume trait (e.g., anxiety) is a reflection of one system (e.g., BIS)

• Assume people do not have direct awareness of their sensitivity values

• Start with simplest possible model

Simulation 4: Further Assumptions

Assume …• for a given situation, that each system

output level corresponds to the level of an emotional state

• that a self-reported trait reflects the average memory of a specific emotional state across a large no. of situations

• that the situations for each simulated person differ randomly in SR and SF

Simulation 4: Simplifications

• Only relevant features of situation are SR and SF

• 200 random situations for each person• Perfect recall of mean system outputs

across al 200 situations• 100 simulated subjects with

sensitivitites drawn independently from normal distribution (m=0.5; s.d.=0.15)

Simulation 4: ExperiencesFor simulated subject #1

Simulation 4: SensitivitieswI for 100 simulated subjects

m=0.49, sd=0.14

BIS Sensitivity

.85

.80

.75

.70

.65

.60

.55

.50

.45

.40

.35

.30

.25

.20

Fre

qu

en

cy

20

15

10

5

0

Simulation 4: Results“Trait” Correlations (N=100)

BAS FFFS BIS

FFFS -0.53

BIS 0.40 0.33

Simulation 4: ResultsRegression predicting self-reported BAS from 3 sensitivitiesR2 = 0.89

Coefficientsa

.179 .015 11.631 .000

.431 .018 .835 24.049 .000

-.206 .016 -.442 -12.755 .000

-.077 .017 -.157 -4.515 .000

(Constant)

WA

WF

WI

Model1

B Std. Error

UnstandardizedCoefficients

Beta

StandardizedCoefficients

t Sig.

Dependent Variable: MBASOUTa.

Simulation 4: ResultsRegression predicting self-reported FFFS from 3 sensitivitiesR2 = 0.82

Coefficientsa

.486 .021 23.280 .000

-.036 .024 -.064 -1.466 .146

.455 .022 .908 20.761 .000

.021 .023 .039 .888 .377

(Constant)

WA

WF

WI

Model1

B Std. Error

UnstandardizedCoefficients

Beta

StandardizedCoefficients

t Sig.

Dependent Variable: MFFFSOUTa.

Simulation 4: ResultsRegression predicting self-reported BIS from 3 sensitivitiesR2 = 0.85

Coefficientsa

-.053 .020 -2.605 .011

.464 .024 .772 19.677 .000

.148 .021 .274 7.005 .000

.278 .022 .487 12.396 .000

(Constant)

WA

WF

WI

Model1

B Std. Error

UnstandardizedCoefficients

Beta

StandardizedCoefficients

t Sig.

Dependent Variable: MBISOUTa.

Conclusions & ChallengesCONCLUSIONS1. New RST produces at least as

complex a pattern of possible effects as old RST

2. Current models seem to predict that the “BIS-related” personality trait may be strongly influenced by sensitivities of all 3 systems

Conclusions & ChallengesCHALLENGES1. To see if the conclusions generalise to

all model variants, including ones with more realistic assumptions

2. Are there any variants which produce a radically different pattern of predictions?

3. To apply the model to task data to see if it can predict patterns of results