Evolution & Economics No. 4

1

Evolution & EconomicsEvolution & EconomicsNo. 4No. 4

2

Evolutionary Stability in Repeated Evolutionary Stability in Repeated Games Played by Finite AutomataGames Played by Finite Automata

K. Binmore & L. SamuelsonK. Binmore & L. SamuelsonJ.E.T. 1991J.E.T. 1991

AutomataAutomata

3

Finite Automata playing the Prisoners’ DilemmaFinite Automata playing the Prisoners’ Dilemma

C DD

C,DC

Grim

C DD

DC

Tit For Tat (TFT)

C

D CD

CC

Tat For Tit (TAFT)

D

D CD

CC

TweedledumD

states (& actions)

transitions

4

Automata playing the Prisoners’ DilemmaAutomata playing the Prisoners’ Dilemma

D CD

C,DC

CA

D CC,D

C

TweedledeeD

D

C,D

D

C

C,D

C

5

• Two Automata playing together, eventually follow a cycle

(handshake) • The payoff is the limit of the means.• The cost of an automaton is the number of his states.• The cost enters the payoffs lexicographically.

lim

Let be automata, and let be the payoff at stage

of the repeated game.

The payoff in the repeated game is the limit of the means:

The no

i

T -1

ij=0T

a,b p a,b i

1π a,b = p a,bT

..of states of an automaton is denoted by: a a

6

C D

C 2 , 2 -1 , 3

D 3 , -1 0 , 0

The Structure of Nash Equilibrium in Repeated Games with Finite AutomataDilip Abreu & Ariel Rubinstein

Econometrica,1988

In Abreu Rubinstein, The fitness satisfies

iff:

and

if

>

th

e .n

U a,b

U a,c U b,c

π a,c π b,c

π a,c =

i)

π b,

ii) c a b

7

C D

C 2 , 2 -1 , 3

D 3 , -1 0 , 0

The Structure of Nash Equilibrium in Repeated Games with Finite AutomataDilip Abreu & Ariel Rubinstein

Econometrica,1988

(0,0)

(3,-1)

(-1,3)

(2,2)

N.E. of repeated Game

N.E in Repeated Games with Finite Automata(Abreu Rubinstein)

8

An automaton is an ESS, if for all automata

and

if then

a

.

nd

if and then

a b a :

π a,a π b,a

π a,a = π b,a π a,b π b,b

π a,a = π b,a π a,b = π b,b

i)

ii)

i ) ai bi

Binmore Samuelson:

9

For any game , if automaton is an ESS,

it has a single state.

G a Lemma :

If is then any ESS must have a single state.

If has more than one action, let it have an ESS

with more than one state.

G 1x1, G

a

Proof :

Let begin with the action a x. x ?x

ay

Let be identical to except that

it acts differently when it

observes at the start.

b a,

y

x?

x

ay

If then: x?

x

by

10

For any game , if automaton is an ESS,

it has a single state.

G a Lemma :

If is then any ESS must have a single state.

If has more than one action, let it have an ESS

with more than one state.

G 1x1, G

a

Proof :

Let begin with the action a x. x ?x

ay

Let be identical to except that

it acts differently when it

observes at the start.

b a,

y

If then: x?

x

by

x?

x

ay

11

For any game , if automaton is an ESS,

it has a single state.

G a Lemma :

If is then any ESS must have a single state.

If has more than one action, let it have an ESS

with more than one state.

G 1x1, G

a

Proof :

Let begin with the action a x. x ?x

ay

Let be identical to except that

it acts differently when it

observes at the start.

b a,

y

can invade the population

hence cannot be an ESS.

b a, a Q.E.D.

12

For any game , if automaton is an ESS,

it has a single state.

G a Lemma :

For the the two singletons are: P.D.

D

C,D

D

C

C,D

C

C is not an ESS, it can be invaded by D.

D is not an ESS, it can be invaded by Tit For Tat.

C DD

DC

Tit For Tat (TFT)

C

The has no ESS. P.D.

13

A Modified ESS. MESS.Definition :

An automaton is an ESS, if for all automata

and

if then

and

if and

. then

i)

ii)

a b a :

π a,a π b,a

π a,a = π b,a π a,b π b,b

π a,a = π b,a π a,b = π b,b i) ai bi

An automaton is an , if for all automata

and

if then

and

if and t

MESS

.hen

a b a :

π a,a π b,a

π a,a = π b,a π a,b π b,b

π a,a = π b,a π a,b = π b,b

i)

ii)

i ) ai bi

14

If is a MESS then it uses all its states

when it plays against itself.

aLemma :

C

Assume that does not use its state when playing

against itself. onstruct an automaton identical to

except that state has been deleted.

a s b, a

s

Proof :

There is no difference between when playing

t .

or

Bu

a,b a b. a < b

Hence such cannot be an ESS.a

Q.E.D.

15

If is a MESS then it uses all its states

when it plays against itself.

aLemma :

C

Assume that does not use its state when playing

against itself. onstruct an automaton identical to

except that state has been deleted.

a s b, a

s

Proof :

There is no difference between when playing

t .

or

Bu

a,b a b. a < b

Hence such cannot be an ESS.a Q.E.D.

In the P.D. Tit For Tat and Grim are not MESS(they do not use one state against themselves)

C DD

C,DC

Grim

C DD

DC

Tit For Tat (TFT)

C

16

= .

An automaton for this game is a pair:

The complexity of

, .

.

1 2

1 2 2 1

1 2+

a = a a

1a,b = π a ,b + π a ,b2a : a a a

π

For a general, possibly non symmetric game G.

Define the symmetrized version of G: G # #. A player is player 1 with probability 0.5 and player 2 with probability 0.5

The previous lemmas apply to (a1,a2)

1. An ESS has a single state │a1│=│a2│=1

2. If (a1,a2) is a MESS it uses all its states when playing against itself,

i.e. a1,a2 use all their states when playing against the other.

17

max

Consider the highest payoff that can be attained in

actions of

This can be achieved by the automaton playing against itself.,

1 2

1 2

1 1 2 2 1 2s ,s

* *1 2

s ,s G.

G.1 G s ,s + G s ,s2

a* = a ,a

a utilitarian automaton.a* -

If an automaton in is a MESS then it is utilitarian.# # .GLemma :

Let be a non utilitarian MESS.

There exists a mutant which earns a higher payoff.

a b

Proof :

The mutant has the following properties:

starts with an action different to that of .i i

bb a1.

If the initial action of the opponent is not that of then it must

be and continues by imitating the utilitarian automaton

.*i

-i

-i i

a

b b a

2.

,

If the initial action of the opponent is that of then fools it to

'believe' that it plays itself, and so obtains the payoff

.

-i i

i i -i

a b

G a a

3.

18

starts with an action different to that of . i ib a1.

If the initial action of the opponent is not that of then it must

be and continues by imitating the utilitarian automaton

.*i

-i

-i i

a

b b a

2.

,

If the initial action of the opponent is that of then fools it to

'believe' that it plays itself, and so obtains the payoff

.

-i i

i i -i

a b

G a a

3.

Properties are easy to obtain.

Property is less obvious. It cannot be done with .

But goes through all its states when playin . g -i ia a

1,2

3 Grim

If initial action made move to the state If is reached when

plays who is then in state , then mimics in stat

's

e i -i -i

i i i

b a q. q a

a q* b a q.

b,a a,a a,bmimics so gets the same against as agai s n tb a, a b a.

but b,b a*,a* a,a = a,b b,b a,b

cannot be a MESS.a

Q.E.D.

19

To prove existence of a MESS is like in repeated game (Folk Theorem)

If is the minimax of , then any payoff above it can be supported a MESS.m G

In the

No 'nice' automaton can be a MESS: It can be invaded by , which is shorter.CPrisoners' Dilemma :

20

D CD

CC

Tat For Tit (TAFT)

D

is a MESS :Tat For Tit

It can be invaded by:

D CD

C,DC

CA

D CC,D

C

ACD

C DD

CC

CCD

C DD

C

CDC,D

D CC,D

C,D

AA

21

D CD

CC

Tat For Tit (TAFT)

D

is a MESS :Tat For Tit

It can be invaded by:

D CC,D

C

ACD

22

D CD

CC

Tat For Tit (TAFT)

D

is a MESS :Tat For Tit

It can be invaded by:

D CD

C,DC

CA

23

D CD

CC

Tat For Tit (TAFT)

D

is a MESS :Tat For Tit

It can be invaded by:

C DD

CC

CCD

24

D CD

CC

Tat For Tit (TAFT)

D

is a MESS :Tat For Tit

No other (longer and more sophisticated) automaton can invade.

Any exploitation of TAFT (playing D against his C) makes TAFT play D,

so the average of these two periods is (3+0)/2 = 1.5 < 2, the average of cooperating.

C D

C 2 , 2 -1 , 3

D 3 , -1 0 , 0

25

Polymorphic MESS

A population consisting of: D CD

CC

Tat For Tit (TAFT)

D

C DD

CC

CCD

C DD

C

CDC,D

can be invaded only by:

D CD

C,DC

CA

D CC,D

C

ACD

If AC invaded, it does not do well against CD

D C D C …….C D C D …….

26

Polymorphic MESS

A population consisting of: D CD

CC

Tat For Tit (TAFT)

D

C DD

CC

CCD

C DD

C

CDC,D

can be invaded only by

CAD CD

C,DC

CA

D CC,D

C,D

AA

If AA invaded, it does not do well against CC

D C C C C…….C D D D D…….

27

Polymorphic MESS

A population consisting of: D CD

CC

Tat For Tit (TAFT)

D

C DD

CC

CCD

C DD

C

CDC,D

can be invaded only by

CAD CD

C,DC

CAbut if CA invaded then a sophisticated

automaton S can invade and exploit CA .S starts with C. if it saw C it continues with C

forever (the opponent must be CD or CC ).

If it saw D, it plays D again, if the other then

plays D it must be TAFT. S plays another D

and then C forever.

If, however, after 2x D, the other played C, then it

must be CA, and S should play D forever.

28

Polymorphic MESS

A population consisting of: D CD

CC

Tat For Tit (TAFT)

D

C DD

CC

CCD

C DD

C

CDC,D

can be invaded only by

CAD CD

C,DC

CA

When S invades, CA will vanish,

and then S which is a complex automaton will die out.