One reason decision-making in pigeons (and humans)
Stephen E. G. Lea,
Catriona M. E. Ryan, Catherine Bryant, Louise Millar, Lisa A. Leaver and Andy J. Wills
School of Psychology
International Association for Research in Economic Psychology and
Society for the Advancement of Behavioral Economics
Rome Conference 2008
6th September 2008 IAREP/SABE Rome meeting 2
With thanks to:• European Union
Framework Programme 6 grant no. 516542 (NEST), “From Associations to Rules in the Development of Concepts” (FAR)
• Tom Zentall, Gerd Gigerenzer, and colleagues elsewhere in the FAR network for discussion
6th September 2008 IAREP/SABE Rome meeting 3
Background• Gigerenzer et al argue that using a single dimension is
often a more efficient way of dealing with a multidimensional problem than an “optimal” regression equation approach: “one reason decision making”
• Non-humans should need to use such cognitive short-cuts even more than humans do (e.g. Bateson has applied Gigerenzer’s approach to mate choice)
• But in categorization experiments, control by a single dimension when many are available is often taken as a sign of the use of a rule (available only to humans) rather than associative learning (available to all vertebrates)
• Some pigeon categorization data suggest that control by a minority of dimensions may be normal for non-humans
6th September 2008 IAREP/SABE Rome meeting 4
Research planQuestions:
Could birds’ “failure” to use all the information available in categorization represent an adaptive, efficient strategy?
How does birds’ performance differ from that of humans, who can presumably use rules?
Experiments: Vary cue salience and validity between dimensions, and compare pigeons with humans, under closely similar procedures
6th September 2008 IAREP/SABE Rome meeting 5
Testing apparatus
touchscreen
feeders
6th September 2008 IAREP/SABE Rome meeting 6
Discrimination procedurePigeons• Go-left/Go-right discrimination between two white-on-
black multi dimensional stimuli, with dimensions spatially separated;
• Food reward at separate feeders for left and right responses (differential outcomes)
• Choice key responses reinforced on FI schedule, forcing 10-12s exposure to stimulus on each trial
• Pecks to wrong choice key have no scheduled consequences – so reward available on all trials
• 80 trials per session, trials start with observing key peck
Students• Reward by feedback stimulus linked to pay
6th September 2008 IAREP/SABE Rome meeting 7
Experiments 1a, 1b, 2: Stimuli• Four dimensions, with elements arranged in a
square• All dimensions 100% valid• Only the perfect exemplars were used in
training, so all dimensions were redundant with one another
chequer
lozenge
A B
bar
doughnut
6th September 2008 IAREP/SABE Rome meeting 8
Conditions• Phases 1 & 2: Early pecks to the choice key do not bring
reinforcement forward in time – Prototype: Entire stimulus appears immediately at start of
trial
– Wait to Reveal: Elements of stimulus appear cumulatively, one at a time in random order, at 2.5s intervals, and first peck to either choice key stops any more elements appearing
– Test: Modified stimulus (e.g. anomalous elements included) used, and reinforcement given regardless of choice made
• Phase 3: Early pecks to the choice key result in (almost) immediate reinforcement if correct (not reported here)
• Phase 6: Bird can choose which dimensions to reveal.
6th September 2008 IAREP/SABE Rome meeting 9
Prototype A stimulus display with choice keys
6th September 2008 IAREP/SABE Rome meeting 10
• 3 birds trained A-left, B-right, 3 with the reverse
• All birds receive 2 sessions of Prototype training, 1 of Reveal training, and so on for 40 sessions
• One bird slow to learn so given an extra 4 session cycles
• Analyses of final performance based on last 5 3-session cycles
• Performance assessed by first choice key pecked – used to score ‘hit’ if on correct side or ‘miss’ if on incorrect side.
Expt 1a: Design
6th September 2008 IAREP/SABE Rome meeting 11
Wait to Reveal: No consistent tendency to postpone choice until particular elements have been revealed, either between or within birds
Phase 1: Stimulus information available at time of first choice peck (final 5 reveal sessions)
0
0.25
0.5
0.75
1
bb cm gy hk ma wv
bird
prob
abili
ty o
f av
aila
bilit
y
bar
doughnut
lozenge
chequer
6th September 2008 IAREP/SABE Rome meeting 12
However, stimulus elements differ, fairly consistently across birds, in their impact on
response accuracy in Wait to Reveal sessions
Impact of stimulus dimensions on response accuracy, final 5 reveal sessions of Phase 1
-0.5
0
0.5
1
1.5
2
bb cm gy hk ma wv
bird
log
imp
rove
me
nt
in o
dd
s ra
tio f
or
corr
ect
re
spo
nse
bar
doughnut
lozenge
chequers
sessioncycle
*
*
*
* *
*
*
**
*
*
*
*P<0.05 for improvement in accuracy if this dimension is present
6th September 2008 IAREP/SABE Rome meeting 13
Phase 2: “One-away” trials
• In training, all dimensions co-vary exactly • Test trials use stimuli in which one dimension gives
anomalous information relative to the other 3• Reinforcement regardless of response• Can assess how many test trial responses (out of
max 48) consistent with sorting by:– Family Resemblance (or Overall Similarity):
response based on majority of dimensions– Each Single Dimension (UniDimensional sorts)
6th September 2008 IAREP/SABE Rome meeting 14
One-away trial results Trials/48 predicted by:
Bird FR UD
Bar
UD
Donut
UD
Lozenge
UD
Chex
bb 38 32 38 28 26cm 36 32 26 24 38gy 36 26 36 32 26hk 36 25 45 23 25ma 32 28 38 24 26wv 40 30 38 30 30
Agreement between birds as to which single dimension predicts best: Kendall W = 0.47, P=0.03 (FR not part of this analysis)
6th September 2008 IAREP/SABE Rome meeting 15
Linking Phase 1 and Phase 2
Impact of stimulus dimensions on response accuracy, final 5 reveal sessions of Phase 1
-0.5
0
0.5
1
1.5
2
bb cm gy hk ma wv
bird
log
im
pro
ve
me
nt
in o
dd
s r
atio
for
co
rre
ct
resp
on
se
bar
doughnut
lozenge
chequers
sessioncycle
*
*
*
**
*
*
*
*
*
*
*
*P<0.05 for improvement in accuracy if this dimension is present
Phase 2
Phase 1
1-away trialsPredicted by :
UD Donut
UD Donut
UD Donut
UD Donut
UD Chex
FR
6th September 2008 IAREP/SABE Rome meeting 16
Conclusions from Expt 1a• Substantial control by multiple features• Control in conflict tests (1-away trials)
appears to be by same features that have most influence on accuracy in incomplete stimulus tests (Wait to Reveal conditions in acquisition)
• But still no evidence of selective exposure to most useful dimensions
6th September 2008 IAREP/SABE Rome meeting 17
Expt 1b: Design• Pigeon stage 1 (Prototype and Wait to Reveal
trials) and Stage 2 (Conflict / 1-away trials) combined into 1 human session.
• Stage 1 – Wait to Reveal :– cycles of 2 prototype trials followed by 1 reveal trial. – 40 trials per segment. Criterion – 80% correct. Up to 3
segments to reach criterion
• Stage 2 – 1-aways :– 60 prototype trials and 20 1-away test trials
• 10 humans trained and tested A-left, B-right, 10 with the reverse; 2 failed to reach criterion
6th September 2008 IAREP/SABE Rome meeting 18
Stimulus Information available when choice made phase 1
0
10
20
30
40
50
60
70
80
90
100
H2 H6 H7 H9subjects
% of reveal trials
Bar
Donut
Lozenge
Chex
Phase 2 Strategy :
UD sort on BAR
UD sort on BAR
UD sort on CHEX
UD sort on CHEX
Some humans do what we’d expect
6th September 2008 IAREP/SABE Rome meeting 19
Stimulus information available when choice made - phase 1
0
10
20
30
40
50
60
70
80
90
100
H3 H5 H8 H10 H11 H13subjects
% of reveal trials
Bar
Donut
Lozenge
Chex
Phase 2 Strategy
FR sort UD sort
BAR
FR sortUD sort
Lozenge
UD sort
Lozenge
UD sort
Donut
Some don’t
6th September 2008 IAREP/SABE Rome meeting 20
Humans’ sorting strategies
• Of the 14 UD humans, 4 appear to use a ‘Gigerenzer’ strategy across the session. These 4 people focused strongly on 1 dimension, and they tended make a choice decision when this element was revealed, and not before.
• The other 10 humans who did do a clean uni-dimensional sort in phase 2 didn’t wait selectively for that dimension in phase 1 – in some sense similar to the pigeons!
UD Bar
UD Donut
UD Lozenge
UD Chex
Total UD
Total FR
Total
5 1 5 3 14 4 /18
Sort strategy used in phase 2 :
6th September 2008 IAREP/SABE Rome meeting 21
Experiment 2 – “Choose to Reveal” method
• Pigeons only• At start of trial, bird saw 4 white discs in place of
the 4 dimensions of the stimulus• When the bird pecked a disc it disappeared and
was replaced by the appropriate dimension• The bird could thus choose the order in which the
dimensions were revealed• Procedure otherwise as in Wait to Reveal • Each bird had 20 sessions
1
2 3
4
6th September 2008 IAREP/SABE Rome meeting 22
Expt 2: Dimensions chosen first
• All birds consistently tended to choose certain dimensions first
Dimensions chosen first in last 5 sessions of choose-reveal condition
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
bb cm gy hk wv mean
bird
prop
ortio
n tim
es d
imen
sion
cho
sen
bar used
donut used
lozenge used
chex used
6th September 2008 IAREP/SABE Rome meeting 23
Experiment 2: Salience
• As in Phase 1, there are consistent differences in the impact of the availability of different dimensions on response accuracy
Phase 6: Impact of stimulus dimensions on response accuracy
-1
-0.5
0
0.5
1
1.5
2
2.5
bb cm gy hk wv
bird
log
im
pro
vem
ent
in o
dd
s ra
tio
n f
or
corr
ect
resp
on
se
bar
doughnut
lozenge
chequers
*
*
**
* *
* *
*
** *
*
6th September 2008 IAREP/SABE Rome meeting 24
Relating dimension choice with
dimension salience
• 3 birds (bb,gy and wv) selectively chose dimensions which significantly improved their accuracy,
• The remaining 2 birds chose dimensions which did not improve their accuracy. The overall performance of these birds correspondingly declined
Dimensions chosen first in last 5 sessions of choose-reveal condition
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
bb cm gy hk wv mean
bird
pro
po
rtio
n t
imes
dim
ensi
on
ch
ose
n
bar used
donut used
lozenge used
chex used
Phase 6: Impact of stimulus dimensions on response accuracy
-1
-0.5
0
0.5
1
1.5
2
2.5
bb cm gy hk wv
bird
log
im
pro
vem
ent
in o
dd
s ra
tio
n f
or
corr
ect
resp
on
se
bar
doughnut
lozenge
chequers
*
*
**
* *
* *
*
** *
*
6th September 2008 IAREP/SABE Rome meeting 25
Conclusions from Expt 2
• Birds do consistently choose one dimension in preference to the others….
• …and this is sometimes, but not always, the one most useful to them.
• There is thus some evidence that birds will select the most useful information if they can control the order in which the stimulus dimensions are revealed, but not if they have to wait for it to be revealed passively.
6th September 2008 IAREP/SABE Rome meeting 26
Conclusions from Expts 1a, 1b & 2
• Pigeons did not show 1-reason decision making in the Gigerenzer sense:– Some cues did control behaviour more than
others, despite their being redundant with one another, as in previous experiments
– But the birds did not selectively delay their choice response until the most useful information was available
– But when they could choose what information to see first, some of the birds did choose the most useful information
• Some humans do show selective waiting for the information they are actually using...
• ...but others behave quite like pigeons
6th September 2008 IAREP/SABE Rome meeting 27
Experiment 3: Manipulating dimension validity
• So far all the dimensions have been 100% valid – so all equally usable (though in practice pigeons do not use them equally)
• What happens if we assign different predictive values to each dimension?
• Specifically, if only one of the dimensions reliably indicates the correct response, will either humans or birds detect it?
6th September 2008 IAREP/SABE Rome meeting 28
Experiment 3a, 3b Design
• Used 3 dimensions instead of 4 (discarded lozenge, because little evidence of use),
• Each dimension had a different predictive value:
• One was 100% predictive, i.e. its presence always signalled the correct side for the bird to respond,
• One was 75% predictive, i.e. the bird was reinforced for responding on one side on 75% of trials and the other on the remaining 25%,
• One had no predictive value, i.e. the bird was reinforced for responding on one side in 50% of trials and the other for the remaining 50%.
6th September 2008 IAREP/SABE Rome meeting 29
Experiment 3a Design detailsPhase 1 :• Birds trained as in phase 1 of the previous
experiment (in cycles where 2 sessions using the whole stimulus were followed by one Wait to Reveal session)
• 6 naïve birds were used, in 3 groups of 2 birds (differing by assignment of dimensions to validities), one bird in each pair trained A-left, the other B-left.
• Each bird given a total of 55 sessions (18 cycles + 1 Prototype to finish)
Phase 2:• Choose to Reveal conditions as in Experiment 2
6th September 2008 IAREP/SABE Rome meeting 30
stimulus information available at time of first choice peck
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
mb sf tt sn rg fc
bird
pro
bab
ilit
y o
f av
aila
bil
ity
bar
doughnut
chequer
Bar 100%Donut 75%Chex 50%
Bar 50%Donut 100%Chex 75%
Bar 75%Donut 50%Chex 100%
Wait to Reveal: Still no evidence of waiting for most valuable information
6th September 2008 IAREP/SABE Rome meeting 31
Predictive value of dimensions not always learned, especially if they conflicted with
‘attractiveness’ of dimension
impact of stimulus dimensions on response accuracy
-0.5
0
0.5
1
1.5
2
mb sf tt sn rg fc
bird
log
im
pro
ve
me
nt
in o
dd
s r
ati
o f
or
co
rre
ct
res
po
ns
e
bar
doughnut
checks
session cycle
bar 100%donut 75%chex 50%
bar 50%donut 100%chex 75%
bar 75%donut 50%chex 100%
6th September 2008 IAREP/SABE Rome meeting 32
Choose to Reveal (Phase 2): First zone pecked
1
2 3
FAR 5.2: dimensions chosen in free choice (all sessions)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
mb sf tt sn rg fc
bird
pro
po
rtio
n t
imes
ch
ose
n
bar used
donut used
chex used
bar 100%donut 75%chex 50%
bar 50%donut 100%chex 75%
bar 75%donut 50%chex 100%
6th September 2008 IAREP/SABE Rome meeting 33
Impact of dimensions on accuracyEven if they did not choose to peck the 100% dimension first,
“chex” and “donut” birds’ accuracy was strongly affected by it
Impact of stimulus dimensions on response accuracy in free choice
-1
-0.5
0
0.5
1
1.5
2
mb sf tt sn rg fc
bird
log
im
pro
vem
ent
in o
dd
s ra
tio
n f
or
corr
ect
resp
on
se
bar
doughnut
checks
bar 100%donut 75%chex 50%
bar 50%donut 100%chex 75%
bar 75%donut 50%chex 100%
*
*
*
*
*
**
*
*
6th September 2008 IAREP/SABE Rome meeting 34
Comparing the pigeon
experiments
Expt 1a: No weighting of dimensions (all at 100%)
• Tendency across birds for Donut to have most impact on accuracy
Expt 3a: Dimensions weighted
• Accuracy for Birds in Donut and Chex groups impacted by relevant (100%) dimension
impact of stimulus dimensions on response accuracy
-0.5
0
0.5
1
1.5
2
mb sf tt sn rg fc
bird
log
im
pro
vem
ent
in o
dd
s ra
tio
fo
r
corr
ect
resp
on
se
bar
doughnut
checks
session cycle
bar 100%donut 75%chex 50%
bar 50%donut 100%chex 75%
bar 75%donut 50%chex 100%
Impact of stimulus dimensions on response accuracy, final 5 reveal sessions of Phase 1
-0.5
0
0.5
1
1.5
2
bb cm gy hk ma wv
bird
log
impr
ovem
ent
in o
dds
ratio
for
cor
rect
re
spon
sebar
doughnut
lozenge
chequers
session cycle
*
*
*
* *
*
*
*
*
*
*
*
6th September 2008 IAREP/SABE Rome meeting 35
Expt 3a conclusions
If one dimension is made 100% predictive, pigeons will:
• Come under control of it, provided that less valid dimensions are not too salient – but the process is slow
• Choose to expose themselves to it (3/6 birds)
But they will not:• wait selectively for it to be revealed passively
Now back again to the humans…
6th September 2008 IAREP/SABE Rome meeting 36
Methods – Expt 3b• Pigeons’ Phase 1 (Prototype and Wait to Reveal) and
Phase 2 (Choose to Reveal) combined in one session.• Phase 1 :
– cycles of 2 Prototype trials then 1 Wait to Reveal trial.– 48 trials per block – had to reach criterion of 80%
• Phase 2 : – stimulus elements obscured by white discs in every trial. Tap to
remove the disc and reveal the element.– 48 trials, also had to maintain 80% criterion
• 30 humans tested, split between 3 groups (each group had a different dimension 100% predictive)
• Results shown only for those humans (20/30) who reached criterion
6th September 2008 IAREP/SABE Rome meeting 37
Phase 1 Human – information available when choice made
3/6 (and two weak cases)
1/6 5/8Humans who had a clear tendency to wait for 100% dimension :
= 9/20
Subjects grouped by 100% valid dimension:
0
10
20
30
40
50
60
70
80
90
100
2 4 7 8 16 19 6 12 13 9 11 18 1 3 5 14 10 15 17 20
subject
% r
ev
ea
l tri
als
bar
donu
chex
6th September 2008 IAREP/SABE Rome meeting 38
Phase 2 - Humans – first zone chosen
0
10
20
30
40
50
60
70
80
90
100
2 4 7 8 16 19 6 12 13 9 11 18 1 3 5 14 10 15 17 20
subject
% t
rials
bar
donut
chex
1
2 3
Humans who had a clear tendency to choose for 100% dimension first:3/6 6/8 = 15/20
OR – had a strategy of hitting 1…2…3
Humans who had a clear tendency to choose for 100% dimension first:6/6
6th September 2008 IAREP/SABE Rome meeting 39
If one dimension is made 100% predictive, humans will:• Mostly come under control of it – but a surprising
number (1/3 of those tested) of humans failed to get to criterion in Phase 1. Some got ‘caught’ by the 75% predictive dimension. Others were just ‘dazed and confused’.
• Remaining conclusions apply to those who got to 80% criterion, who must have picked up on the 100% dimension (75% cue does not give enough info)
o In most but not all cases choose to expose themselves to it first (15/20 of those who reached criterion)
o In some cases wait selectively for it to be revealed passively (only 9/20)
Expt 3b conclusions
6th September 2008 IAREP/SABE Rome meeting 40
• Humans are not like pigeons, but they are more like pigeons than you might think
• Do humans or pigeons use rules in this sort of task? To answer this we need to know what a rule would look like for a non-verbal animal: we suggest a rule should be transferable between situations
• Neither pigeons nor humans are very good at transferring “rules” that govern sorting into waiting situations – no pigeons and fewer than half the humans did this
• But both are better at using their sorting “rules” to choose what information to see
• Question for further research: How much of this could be predicted by an associative model?
Finally: Associations and Rules
Thanks!Questions?
6th September 2008 IAREP/SABE Rome meeting 42
Expt A Pigeons, Acquisition
Condition: F(1,5)=17.32**
Condition x Session cycle F(12,59)=2.27*
• Relatively slow learning: 17-38 sessions before 80% accuracy achieved
• Once learning is evident, performance consistently worse on Reveal sessions than Prototype sessions
correct first choice as a function of session cycle and training condition
0
0.25
0.5
0.75
1
1 2 3 4 5 6 7 8 9 10 11 12 13
session cycle
prop
ortio
n co
rrec
t
Prototype
Reveal
6th September 2008 IAREP/SABE Rome meeting 43
Number of elements revealed (max 4) increases gradually across sessions, from
<2 in early sessions to a mean of ~2.5
Session cycle F(12,59.3)=4.77**
Number of elements revealed
0
1
2
3
4
1 4 7 10 13
Session cycle
Mea
n el
emen
ts
reve
aled
/tria
l
mean andrange
6th September 2008 IAREP/SABE Rome meeting 44
Trials with more reveals give more accurate performance
Note that performance when 4 elements are revealed is as good as in prototype conditions (when all 4 are available throughout)
Phase 1: Accuracy as a function of number of elements revealed (final 5 reveal sessions)
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4 Proto
Number of elements
Pro
ba
bili
ty o
f co
rre
ct r
esp
on
se
6th September 2008 IAREP/SABE Rome meeting 45
Allowing early choice to bring early reward
No. of elements revealed in Phase 3 (immediate reward on correct choice)
0
1
2
3
4
1 4 7 10Session cycle
Mea
n el
em
ents
re
veal
ed/t
rial
Phase 3 mean andrangePhase 1 cycles 4-13
6th September 2008 IAREP/SABE Rome meeting 46
Additional Phase 3 results
• Reduction in accuracy under Reveal conditions compared with Phase 1.
• Accuracy for low numbers of elements greater than Phase 1
• Still no selective exposure to more useful information
• How else to look at this? Up to now, the birds had no control over the order of presentation of each dimension in Reveal sessions. What happens if we give them this control?
6th September 2008 IAREP/SABE Rome meeting 47
Results: Acquisition• Even slower than in Expt A. • Again, all tended to perform worse on reveal sessions (this was
very marked in one bird)
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
session cycle
mea
n %
co
rrec
t
proto
reveal
Expt A acquisition
Expt B acquisition