Cooperation, Reputation & Gossiping

Introduction Indirect Reciprocity Gossip Results

Cooperation, Reputation & Gossiping

V.A. Traag1, P. Van Dooren1, Y.E. Nesterov2

1ICTEAMUniversite Catholique de Louvain

2COREUniversite Catholique de Louvain

15 April 2011


Outline

1 Introduction

2 Indirect Reciprocity

3 Gossip

4 Results


Cooperation

Cooperation (and defection)

• Organizations (also Wikipedia, open source software, . . . )◮ Why do people contribute?

• Worker ants in colonies◮ Why do workers help without individual benefit?

• Prudent parasites in hosts◮ Why do parasites not replicate faster?

• Human body◮ Why do cells not replicate faster?

Central question

If defecting (not cooperating) is a real option, why (and how) hascooperation evolved?


Formal cooperation (and defection)

Prisoner’s Dilemma

• The game knows two options, donating or not donating.

• Donate at a cost c > 0 to benefit someone else with benefitb > c .

• Agents are paired, and play a round of donating or not.

• Cooperators C donate, defectors D do not donate.

This can be summarized in the payoff matrix

A =

(C D

C b − c −c

D b 0

)

Defectors dominate

Whatever strategy you encounter (C or D), always better to defect.


Dynamical View

• Natural to model game dynamics in an evolutionary context.

• Survival of the fittest (fitness = payoff).

Definition (Replicator equation)

Population with i = 1, . . . , n different mixed strategies pi

xi Relative abundance (frequency)

p =∑

i pixi Average strategy

fi = p⊺

i Ap Expected payoff

f = p⊺Ap Average payoff

Evolution of the population given by

xi = xi (fi − f ) = xi ((pi − p)⊺Ap).


Overview

What are possibly mechanisms to get cooperation?Payoff matrix

A =

(C D

C b − c −c

D b 0

)

Mechanisms

• Kin selection (r > cb)

Cooperate because offspring benefits of your cooperation. Basisof ‘selfish gene’, or ‘inclusive fitness’.

• Direct reciprocity (w > cb)

Cooperate because of possible future payoffs.

• Indirect reciprocity (q > cb)

Cooperate because someone else may cooperate with you in thefuture.

Nowak. Science (2006) vol. 314 (5805) pp. 1560-1563


Some strategies

Example (Always)

Defect/cooperate on all rounds

Other CDDDDCC

AllD DDDDDDD

AllC CCCCCCCC

Example (Tit-for-tat)

Start cooperating, then repeatopponent.

Other CDDDDCC

TFT CCDDDDC

Example (Win-Stay, Lose-Shift)

Change strategy if losing, keep itotherwise.

Other CDDDDCC

WSLS CCDCDCC

Example (Generous Tit-for-tat)

As TFT, but cooperates afterdefection with probability p.

Other CDDDDCC

GTFT CCDDCDC


Insufficient explanation

Why is kin selection and reciprocity not sufficient?

Insufficient explanation

• Humans cooperate also with non-kin.

• Humans cooperate in non-iterative situations.

Indirect reciprocity

• Cooperate if cooperated with others in the past.

• Brings reputation into play.

• How to respond to reputation?

• How to determine new reputation?

Nowak and Sigmund. Nature (2005) vol. 437 (7063) pp. 1291-1298


Image score

Definition (Image score, reputation)

• Integer status −5 ≤ Si ≤ 5 known to all.

• If cooperate increase (with 1).

• If defect decrease (with 1).

Definition (Discriminator Strategy)

• Cooperative threshold −5 ≤ kj ≤ 6.

• If status Si ≥ kj cooperate, otherwise defect.

• Strategy kj = −5 corresponds to AllC.

• Strategy kj = 6 corresponds to AllD.

Nowak and Sigmund. Nature (1998) vol. 393 (6685) pp. 573-577.


Other reputation dynamics

Morals

• Defecting a defector, good or bad?

• What action should be regarded as good?

• When to cooperate, when to defect?

GG GB BG BB

C ∗ ∗ ∗ ∗

D ∗ ∗ ∗ ∗

∗ ∗ ∗ ∗

Reputation of donor and recipientAction of donor

New reputation can beeither Good or Bad

Action can be eitherCooperate or Defect


Some reputation dynamics

GG GB BG BB

C G G G G

D B B B BImage scoring

C G G G G

D B G B BStanding

C G B G B

D B G B BJudging

C G B G B

D B B B BShunning


Leading eight

Best strategies

• In total 2, 048 different possible strategies.

• There are 8 strategies (leading eight) that perform best (highestpayoff, and ESS).

GG GB BG BB

C G ∗ G ∗

D B G B ∗

C D C ×

Maintainance of cooperation

Mark defectors

Punish defectors

Forgive defectors

Apologize

Ohtsuki and Iwasa. Journal of Theoretical Biology (2004) vol. 231 (1) pp. 107-120


Subjective reputation

Subjective reputation

• Unrealistic that everybody knows the reputation of everybody.

• Introduce a subjective (private) reputation.

• ‘Observe’ only a few interactions.

Observing

• Probability q of observing an interaction.

• Cooperation declines with lower q.

• Diverging reputations cause further errors.

• Good may defect bad, but not all agree on who’s bad.


Synchronize reputations

Synchronizing reputations

• Spread local information to synchronize reputations.

• Players ‘gossip’ about each other to share information.

• Start gossip, spread gossip and how to interpret gossip?

Lying, cheating and defecting

• Possibly ‘false’ gossips spread.

• Spread rumours unconditionally allows liars to invade.

• Liars cannot invade conditional rumour spreaders.


Proposed model

Basics

1 Each agent has a reputation of the other: Rij(m)

2 Everybody plays and cooperates/defects based on localreputation

3 Everybody gossips the result of the interaction

4 New reputation Rij(m + 1) based on:◮ Own observation (∆Iij (m)),◮ Gossip (∆Sij (k ,m)).

Decision to cooperate

The decision to cooperate αij(m) =

{0 if Rij(m) < 01 if Rij(m) ≥ 0


Gossiping

Consider all neighbours k when updating the reputation Rij

i j

k

The link tobe updated.

Does i ‘like’ k?

Will k gossip to i?

What actionhas j takento k?


Social strategy

G B

C G B

D B G

Reputation of k , or αik(m).

Action of j , or αjk(m)

Action is considered aseither Good or Bad

Social strategy

• Cooperation vs. good agent and defection vs. bad agent is good

• Change in reputation due to gossiping with neighbour k

∆Sij(k ,m) = αki (m)(2αik(m)− 1)(2αjk(m)− 1)


Individual strategy

C D

C + −

D − +

Action of j , or αji (m).

Action of j , or αij(m)

Action is considered aseither Good or Bad

Individual strategy

• +1 for ‘good’ actions, −1 for ‘bad’ actions to reputation

• We currently study WSLS-like: Consider CC and DD as good.

∆Iij(m) = (2αij(m)− 1)(2αji (m)− 1)


Reputation dynamics

Combine individual & social strategies

Combine with social influence parameter 0 ≤ λ ≤ 1

∆Rij(m) = (1− λ)

Individual strategy︷︸︸︷

(2αij(m)− 1)(2αji (m)− 1)+

λ1

n − 2

∑

k 6=i ,j

αki (m)(2αik(m)− 1)(2αjk(m)− 1)

︸︷︷︸

Social strategy

Reputation dynamics

Rij(m + 1) = Rij(m) + ∆Rij(m)


Cooperative fixed points

Fixed point

• For which networks do we have αij(m + 1) = αij(m)?

• Good reputation remains good, bad reputation remains bad

Undirected case

• If αij(m) = αji (m), fixed points are groups

• Cooperate within groups, defect between groups

• Implies it is (weakly) social balanced

• Can have q groups if

λ >q

q + 1

More social influence may lead to more fragmented cooperation.


Evolutionary dynamics

Four different regimes (Cooperate with prob p on first round)

p < 1/2 p > 1/2

λ < 1/2 Individualistic prejudiced

• Defect vs. cooperators

• Cycles of cooperation vs.defectors

Individualistic trusting

• Cooperate vs.cooperators

• Cycles of cooperationvs. defectors

λ > 1/2 Social prejudiced

• Cooperate vs. cooperators(except second round)

• Defect vs. defectors(except second round)

Social trusting

• Cooperate vs.cooperators

• Defect vs. defectors


Phase portraits Individual

C D

Gossipers

Individual PrejudicedC D

Gossipers

Individual Trusting

In ‘friendly’ environment, being individually prejudiced pays off.


Phase portraits Social

C D

Gossipers

Social PrejudicedC D

Gossipers

Social Trusting

In ‘hostile’ environment, being socially trusting pays off.


Conclusions

Proposed model

• Proposed model for gossiping and reputation dynamics

• Interesting possible cooperative network structure

• Evolutionary stable for some parameter range

• More socially oriented strategy could have developed fromindividual strategy

Shortcomings

• Actual convergence to fixed point not investigated

• Characterize directed fixed points

• Evolutionary dynamics investigated in limit of large n

• Interact all-to-all unrealistic, e.g. restrict to graph

• Gossip perhaps passed on further (cascades of gossip)


Thank you for your attention.

Questions?