Read Montague Baylor College of Medicine Houston, TX Reward Processing and Social Exchange.

Read MontagueBaylor College of Medicine

Houston, TXwww.hnl.bcm.tmc.edu

Reward Processing and Social Exchange

I love you

Let’s do lunch

BA

A BAB

Natural visual statistics

Specific algorithms evolved to solve efficiently the array of problems of early

vision

Recharge or die

The real world imposes another difficult requirement on all mobile organisms

Natural reward-harvesting statistics?

genericwoodlandcreature

Models of reinforcement learning can provide insight

What computations should we expect?

Reward harvesting is an economic problem involving both valuation and choice

Sustain goal

Select goal

Pursue goal

Goal-directed choice

Actual experience Counterfactual experience‘what could have been’

guidancesignals

Midbrain dopamine neurons

burst

pause

R timenaive

R timeAfter learning

timeAfter learning(catch trial)

Pause, burst, and ‘no change’ responses represent reward prediction errors – ongoing emission of information

ERROR SIGNAL = current reward + next prediction - current prediction

Can we detect a reward prediction error signal in a human subject?

Neural correlates of these error signals have been observed in conditioning and decision

tasks

Passive conditioning tasks

Instrumental conditioning tasks

Sequential decision tasks

Social and economic exchange tasks

Midbrain dopamine neurons

burst and pause responses encode reward prediction errors

ERROR SIGNAL = current reward + next prediction - current prediction

Can these systems be re-deployed for abstractly defined rewards (ideas)?

“I want to solve Fermat’s last theorem”

What about the something more abstract like the expression and

repayment of trust?

Trust

Modeling

Must involve risk (uncertainty)

Harvesting returns from another agent

TRUST

I love you

Let’s do lunch

BA

A BAB

Simplifying and quantifying Trust(Berg et al., 1995; Weigelt and Camerer, 1988)

Trust is the amount of money a sender sends to a receiver without external enforcement.

Agent 2Agent 1

I

R

X 3

Investor Trustee

$20

A dynamic version of the Trust game (10 rounds)

investperiod

8 s 8 s10 s 10 s 10 s8 s8 s4 s4 s

cue to invest

repaymentrevealed toboth brains

Gave

13

Kept

29

investmentrevealed toboth brains

Kept Gave

6 14

repayperiod

cue to repay

delayperiod

delayperiod

delayperiod

delayperiod

investment phase repayment phase

Totals

2919

totalsrevealed toboth brains

inter-rounddelay period

investperiod

8 s 8 s10 s 10 s 10 s8 s8 s4 s4 s

cue to invest

repaymentrevealed toboth brains

Gave

13

Kept

29

Gave

13

Kept

29

investmentrevealed toboth brains

Kept Gave

6 14

Kept Gave

6 14

repayperiod

cue to repay

delayperiod

delayperiod

delayperiod

delayperiod

investment phase repayment phase

Totals

2919

Totals

2919

totalsrevealed toboth brains

inter-rounddelay period

Structure of a round

What is the behavioral signal that most strongly

influences changes in trust (money sent) ?

Agent 2Agent 1

I

R

Reciprocity = TIT-FOR-TAT

Reciprocity = TIT-FOR-TAT

mon

ey s

en

tto

part

ner

+Benevolent

signal

-Malevolent

signal

Neutralsignal

Questions about brain response

Responses that differentiate benevolent from neutral?

Responses that differentiate malevolent from neutral?

Responses that differentiate benevolent from malevolent?

Surprise response

*

time (sec)-8 0 10

-0.2

0

0.2

Trustee Brain:

‘intention to increase trust’ shifts with reputation building

Reputation develops

Trustee will increase trust on next move

Trustee will decreasetrust on next move

submit reveal*

increases or decreasesin future trust by trustee

-0.2

0

0.2

time (sec)

Signal is nowanticipating the

outcome

Temporal shift resembles value transfer in reward learning experiments

burst

pause

R timenaive

R timeAfter learning

timeAfter learning(catch trial)

-1

-.9

-.8

-.7

-.6

-.5

-.4

-.3

-.2

-.1

0

-1 -.8 -.6 -.4 -.2 0 .2 .4 .6 .8 1

r = .00TR

US

TE

E

decr

ease

s re

paym

en

t

Trustee decreases No information

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-1 -.8 -.6 -.4 -.2 0 .2 .4 .6 .8 1

r = .27

Change in next investment

TR

US

TE

E in

crease

s re

paym

en

t

Trustee decreases positive correlation

Why should intentions to increase repayment burn more energy?

Change in next investment

1. Social trust is about modeling others - always some underlying currency.

2. Re-deploy reward-harvesting machinery that we share with all vertebrates (abstractions gain reward status)

3. Use to probe pathologies (addicted state, autism spectrum disorder, borderline…)

Trust?

Baylor College of MedicinePearl Chiu

Amin KayaliBrooks King-Casas

Terry LohrenzSam McClure (Princeton)

Read MontagueDamon Tomlin

CaltechCedric Anen

Colin CamererSteve Quartz

UCLPeter Dayan

Nathaniel Daw

Salk InstituteTerry Sejnowski

Princeton UniversityJon Cohen www.hnl.bcm.tmc.edu/trust

Emory UniversityGreg Berns

University of AlabamaLaura KlingerMark Klinger

Families of autistic subjects

Funding SourcesNIDANIMH

Dana FoundationKane Family Foundation

Angel Williamson Imaging CenterInstitute for Advanced Study, Princeton NJ

Collaborators

rounds

.1

.2

.3

.4

3 4 5 6 7 8 9 10

fractionof

highlyaccurateguesses

by trustee

Investorevents

Trusteeevents

How do we know a reputation is forming?

8 s 10 s4 s. . .

investperiod

cue to invest

investmentrevealed toboth brains

delayperiod

guessperiod

cue to guess

Investment phase

synchronized onsubmission time of both

partners

Trustee

What is Fair?

i

i

i

Split the profit

Split ‘loaf’Split the total 10 + i each

i

Investor Trustee

20-i

collective ownership? common goods?