The partial reinforcement extinction effect (PREE) Frode Svartdal University of Tromsø Oct. 2013
Apr 01, 2015
The partial reinforcement extinction effect (PREE)
Frode SvartdalUniversity of Tromsø
Oct. 2013
Extinction: Basics Extinction is defined in terms of a
reinforcement process Extinction contingencies
The stimulus (SR or US) is discontinued The learning contingency is discontinued
Extinction process The conditioned response is reduced
(strength, frequency, etc.) Relearning, … not forgetting
Catania, 1984)
Extinction: BasicsOperant conditioning
Extinction: BasicsClassical conditioning
Factors affecting the extinction rate In general: Fast acquisition / high rate of
responding fast extinction Amount of reward
High fast extinction Variability
Stimulus Response Reinforcement
Some forms of learning do not extinguish (easily)
Evaluative conditioning (e.g., Diaz, Ruiz, & Beyens, 2005)
= high ext. persistence
Factors affecting the extinction rate
Partial Reinforcement Extinction Effect
Partial (Intermittent) Reinforcement (PRF) increased extinction response
Continuous Reinforcement (CRF) reduced extinction persistence
First demonstrations
Operant conditioning;free operant; rats;Skinner (1938)
Classical conditioning;blink response; students;Humphreys (1939)
100%
50%
Ferster & Culbertson, 1975
Free operant
PRF
CRF
Free operant
Compared to CRF:
PRF • higher asymptotes• more persistent responding under extinction
EXTINCTION
Rats, maze running speed under extinction (Weinstock, 1954)
CRF
PRF (30%)
Classical conditioning (rats): PREE
25%
50%
100%
Extinction
PRF response rateLOWER than CRF response rate
15%
Classical conditioning; eyelid; human subjects(Svartdal & Flaten, in prep.)
Operant conditioning; humans;Svartdal, 2003, Exp. 4
Conclusions (… preliminary)
PREE is a very robust outcome Measures & species
Bar pressing, rats Maze running, rats Pecking, pigeons Blink reflex, humans, rabbits …
Contingency Operant/instrumental
Discrete trial Free operant
Classical
But…
How general is the PREE? Reversed PREE observed under some
conditions Generalized PREE observed under some
conditions Alternative methods of analysis
Nevin (1988): ”PREE is an artefact because of wrong method of analzing extinction performance”
Response unit issue PREE or not dependig on how the response
is defined (Mowrer & Jones, 1945!
Reversed PREE
What happens if the subject is exposed to a mixture of PRF and CRF contingencies?
Reversed PREE
Pavlik & Carlton, 1965: Rats; bar pressing, free operant
Gr. 1: Single contingency; CRF Gr. 2: Single contingency; PRF Gr. 3: Two signalled schedules
alternated for the same subjects; CRF + PRF
Reversed PREEConventionalPREE
Reversed PREE
ReversedPREE
Reversed PREE
Pavlik & Carlton (1965): Single reinforcement schedules (CRF vs. PRF) in
between-groups experiments PREE Two schedules (CRF vs. PRF) for the same subjects
Reversed PREE
Other research Reversed PREE observed Generalized PREE (overall increased persistence,
but no difference between conditions) Conventional PREE rarely if ever observed in within-
subjects manipulations of CRF - PRF
PREE as a generalization: Ecological validity
If applied to a situation with a very specific schecule for a specific behavior PREE
Example:
Single mother – child is begging for toys only from mom
If applied to various situations with mixed contingencies Reversed PREEGeneralized PREE
Example:
Mother and father – child begs for toys from both
Response unit issue
Free operant responding: What is the response unit?
Mowrer & Jones,1945:
What should be counted as the response unit - single responses or the unit of responses required for reinforcement?
Free-operant Intermittent reinforcemet, e.g., FR4
Response unit
FR4
Reinforced responses
PREE Total responses
Total responses / reinforcement ratio
Reversed PREE
Nevin: PREE is an artefact
PREE: Alternative analyses
Nevin, 1988: Behavioral momentum• ”RPREE” is the rule – the response is
stronger following CRF• in free-operant responding (but not in discrete-
trial experiments) • following extended training
• Extinction performance• Traditional measure: Number of responses • Nevin: Slope of the extinction curve
PREE
RPREE
SHORT LONG
Absolute numberof responses
Relative to initial ext response level
Nevin, 1988
PREE vs. RPREE – important variables
Dependent measure No. of responses vs. relative change
Type of situations Free operant vs. discrete trial
Complexity of situation One vs. more schedules (e.g., multiple
schedule) Design
Between groups vs. within subjects
PREE typically observed
Measure Number of responses
Situation Discrete trial
Schedule Single
Design Between-groups manipulation of reinforcer rate
Other CRF schedule must be 100%
PREE: My interests
Interaction PREE & Reversed PREE Cognition (verbalization) related to
behavioral PREE
The experimental situation
”Computer responses”presented
Left, right
Subject responsesrecorded
Left, right
The experimental situation
Task Complete a four-response chain of responses started by
the computer E.g.: Computer: L R Subject: R L
Instructed task: Identify and apply the functional rule(s) ”Obtain as many correct answers as you can.”
Rules (depending on experiment) ”Repeat computer sequence” ”Reverse computer sequence”
Feedback (visual, autitory) for correct answer; nothing happens if answer is incorrect
The experimental situation
Manipulations (between groups and/or within groups)
RuleReverse (typically used)Repeat
ContingencyCRF (100%)PRF (20-60%)
The experimental situation
Reward rate manipulated Between groups Within subjects (multiple schedule)
Discrete trial situation; fixed number of trials 180 acquisition trials 40 extinction trials
Conventinal PREE; operant responding; students; Svartdal, 2003, Exp. 4
Reversed & conventional PREE; operant responding; students; Svartdal, 2000
Reversed PREE Purpose: Explore the relationship
between PREE and RPREE PREE vs. RPREE: Contradiction or
compatible effects? Method
Independent groups: PRF and CRF Within: CRF and PRF
Svartdal, 2000 ctd.
Multiple schedule, alternating Group 40/40
Half trials (signalled): 40% Half trials (signalled): 40%
Group 80/80 Half trials (signalled): 80% Half trials (signalled): 80%
Group 80/40 Half trials (signalled): 80% Half trials (signalled): 40%
PRF
”CRF”
”CRF” + PRF
PREE80%40%
* No. of responses: RPREE* Relative change: No difference
Svartdal, 2000 ctd.
Relationship between schedule components
Simplest assumption: Modulation between component schedules: 60% + context = 60% reference 60% + context = 100% reduced
persistence 60% + context = 20% increaced
persistence
Performance of a 60% schedule depending onother schedule = 100%, 60%, or 20%Svartdal, 2000
Svartdal, F. (2000). Persistence during extinction: Conventional and Reversed PREE under multiple schedules. Learning and Motivation, 31, 21-40.
Cognition in PREE
• Currently: Strong cognitive arguments to interpret conditioning in terms of cognition
• Classical conditioning: Lovibond & Shanks, 2002• Operant conditioning: Shanks & St John, 1994• Implicit learning doubted: Shanks, 2005• Extinction: Lovibond, 2004
• Basic argument:
CONTINGENCY CONSCIOUS APPREHENSION BEHAVIORAL CHANGE
CONTINGENCY CONSCIOUS APPREHENSION NO BEHAVIORAL CHANGE
• Large number of studies supporting this assumption
Cognition in PREE
So, since the behvioral PREE is very robust, a ”cognitive PREE” must be easy to measure
Basic prosedure: Behavioral acquisition under
100% vs. 60% reinforcer rate Measurement of verbalized PREE
Cognition in PREE
Prediction of persistence:”How likely is it that you will continueresponding if reward no longer appears?”
Several experiments havedemonstrated no sensitivityto learning history inpredictions
T rl1 T rl3 T rl5 T rl7 T rl9 T rl11 T rl13 T rl15 T rl17 T Ext0 ,0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
1,0
Re
spo
nse
s (pro
po
rtion
of p
ossib
le)
100/100 60/60
3 extinction trials;immediate behavioralsensitivity
No differencein predictions
Svartdal & Silvera, in prep.
Cognition in PREE
Retrospective judgments:”How many responses did you emit afterreward no longer appeared?”
Subjects are very accurate in descrbing their own behavior, including their own extinction persistence
Cognition in PREE
Svartdal, F. (2003). Extinction after partial reinforcement: Predicted vs. judged persistence. Scandinavian Journal of Psychology, 44, 55-64.
Meta-cognitive PREE?
We all have long experience with various contingencies
Maybe a ”meta-cognition” evolves: Uncertain outcomes Persist Certain outcomes Quit
Meta-cognitive PREE?
Scenarioes presented to subjects, manipulation Reliable outcome vs. Unreliable outcome
Persistence judgments of behavior
Meta-cognitive PREE?
Naive students: No effect ofoutcome manipulation
Meta-cognitive PREE?
Psychology students(have read about PREE)
Naive students
Meta-cognitive PREE?
Svartdal, F. (2000). Persistence during extinction: Are judgments of persistence affected by contingency information? Scandinavian Journal of Psychology, 41, 315-328.
PREE: Theory
Mowrer & Jones: Diskriminasjonshypo- tesen PRF:
Læringbetingelsene ekstinksjonsbetingelsene Generalisering til ekstinksjon
CRF: Læringbetingelsene # ekstinksjonsbetingelsene Liten generalisering til ekstinksjon
PREE: Theory
Amsel: Frustrasjonshypotesen PRF:
Forventning om belønning frustrasjon når belønning uteblir
Frustrasjons-cues assosieres med læringssituasjonen
Under ekstinksjon: Frustrasjon pga uteblitt belønning
Læringssituasjonen ekstinksjonssituasjonen
CRF: Frustrasjon oppstår ikke under læring Læringssituasjonen # ekstinksjonssituasjonen
PREE: Theory
Capaldi: Sequential hypothesis PRF:
Ikke-belønnede trials blir signal på at belønning snart vil følge: … N N N R N N N R …
Dvs.: Det opparbeides en forventning om belønning når belønning uteblir
Under ekstinksjon: Mange responser pga forventning om belønning
CRF: Ingen erfaring med uteblitt belønning under
læring Under ekstinksjon: Få responser
PREE: Theory
Status: Diskriminasjonshypotesen står svakt Amsels hypotese står rimelig sterkt Capaldis hypotese står ganske sterkt Nevins modell: Ingen hypotese i vanlig
forstand Discrete-trial-situasjonen
Capaldi og Amsel dominerende Fri-operant-situasjonen
Svak teoretisk forståelse