Repeated Games: The Prisoner's Dilemmachristosaioannou.com/Repeated Games The Prisoners Dilemma...In the Prisoner’s Dilemma, both players have an incentive to cheat, and everyone

Repeated Games: ThePrisoner’s Dilemma

Examples of Prisoner’s Dilemmas

Christos A. Ioannou2/20

• In the Prisoner’s Dilemma,• both players have an incentive to

cheat, and• everyone is better off if no one

cheats.

• Consider firms contemplating whetherto advertise,

• or individuals contemplating whether touse steroids,

• or firms contemplating whether topollute.

Can cooperation emerge without external enforcement?

Cam

el

Malboro

Don

’tA

dver

tise

Don’t Advertise

4,4 1,7-c

7-c,1 4-c,4-c

Sos

a

McGwire

Cle

anS

tero

ids

Clean Steroids

4,4 1,7-c

7-c,1 4-c,4-c

Cou

ntry

1

Country 2

Don

’tP

ollu

te

Don’t Pollute

4,4 1-c,7-c

7-c,1-c 4-2c,4-2c

cheats.

Cam

el

Malboro

Don

’tA

dver

tise

Don’t Advertise

4,4 1,7-c

7-c,1 4-c,4-c

Sos

a

McGwire

Cle

anS

tero

ids

Clean Steroids

4,4 1,7-c

7-c,1 4-c,4-c

Cou

ntry

1

Country 2

Don

’tP

ollu

te

Don’t Pollute

4,4 1-c,7-c

7-c,1-c 4-2c,4-2c

cheats.

Cam

el

Malboro

Don

’tA

dver

tise

Don’t Advertise

4,4 1,7-c

7-c,1 4-c,4-c

Sos

a

McGwire

Cle

anS

tero

ids

Clean Steroids

4,4 1,7-c

7-c,1 4-c,4-c

Cou

ntry

1

Country 2

Don

’tP

ollu

te

Don’t Pollute

4,4 1-c,7-c

7-c,1-c 4-2c,4-2c

cheats.

Cam

el

Malboro

Don

’tA

dver

tise

Don’t Advertise

4,4 1,7-c

7-c,1 4-c,4-c

Sos

a

McGwire

Cle

anS

tero

ids

Clean Steroids

4,4 1,7-c

7-c,1 4-c,4-c

Cou

ntry

1

Country 2

Don

’tP

ollu

te

Don’t Pollute

4,4 1-c,7-c

7-c,1-c 4-2c,4-2c

cheats.

Cam

el

Malboro

Don

’tA

dver

tise

Don’t Advertise

4,4 1,7-c

7-c,1 4-c,4-c

Sos

a

McGwire

Cle

anS

tero

ids

Clean Steroids

4,4 1,7-c

7-c,1 4-c,4-c

Cou

ntry

1

Country 2

Don

’tP

ollu

te

Don’t Pollute

4,4 1-c,7-c

7-c,1-c 4-2c,4-2c

cheats.

Cam

el

Malboro

Don

’tA

dver

tise

Don’t Advertise

4,4 1,7-c

7-c,1 4-c,4-c

Sos

a

McGwire

Cle

anS

tero

ids

Clean Steroids

4,4 1,7-c

7-c,1 4-c,4-c

Cou

ntry

1

Country 2

Don

’tP

ollu

te

Don’t Pollute

4,4 1-c,7-c

7-c,1-c 4-2c,4-2c

cheats.

Cam

el

Malboro

Don

’tA

dver

tise

Don’t Advertise

4,4 1,7-c

7-c,1 4-c,4-c

Sos

a

McGwire

Cle

anS

tero

ids

Clean Steroids

4,4 1,7-c

7-c,1 4-c,4-c

Cou

ntry

1

Country 2

Don

’tP

ollu

te

Don’t Pollute

4,4 1-c,7-c

7-c,1-c 4-2c,4-2c

cheats.

Cam

el

Malboro

Don

’tA

dver

tise

Don’t Advertise

4,4 1,7-c

7-c,1 4-c,4-c

Sos

a

McGwire

Cle

anS

tero

ids

Clean Steroids

4,4 1,7-c

7-c,1 4-c,4-c

Cou

ntry

1

Country 2

Don

’tP

ollu

te

Don’t Pollute

4,4 1-c,7-c

7-c,1-c 4-2c,4-2c

cheats.

Cam

el

Malboro

Don

’tA

dver

tise

Don’t Advertise

4,4 1,7-c

7-c,1 4-c,4-c

Sos

a

McGwire

Cle

anS

tero

ids

Clean Steroids

4,4 1,7-c

7-c,1 4-c,4-c

Cou

ntry

1

Country 2

Don

’tP

ollu

te

Don’t Pollute

4,4 1-c,7-c

7-c,1-c 4-2c,4-2c

Repeated Game Example

• The unique Nash equilibrium is (D,D).

• Consider the strategy Grim-Trigger (GT) where,if the other player chooses D once, then, youplay D forever, otherwise you play C.

• What should player 2 do if player 1 plays GT?

C D

C

D

2 2 0 3

3 0 1 1

C D

C

D

2 2 0 3

3 0 1 1

C D

C

D

2 2 0 3

3 0 1 1

C D

C

D

2 2 0 3

3 0 1 1

C D

C

D

2 2 0 3

3 0 1 1

C D

C

D

2 2 0 3

3 0 1 1

Discounting

• Players discount their payoffs with discount factorδ ∈ (0, 1).

• This says, that I care more about my payoff in period 1than in period 1,000.

• Let(a1, a2, . . . , aT

)be the choices for T periods; the

player’s discounted payoff is then,

ui(a1)+δui

(a2)+δ2ui

(a3)+· · ·+δT−1ui

(aT)=

T∑t=1

δt−1ui(at).

Discounting

• Players discount their payoffs with discount factorδ ∈ (0, 1).• This says, that I care more about my payoff in period 1

than in period 1,000.

• Let(a1, a2, . . . , aT

ui(a1)+δui

(a2)+δ2ui

(a3)+· · ·+δT−1ui

(aT)=

T∑t=1

δt−1ui(at).

Discounting

• Players discount their payoffs with discount factorδ ∈ (0, 1).• This says, that I care more about my payoff in period 1

than in period 1,000.

• Let(a1, a2, . . . , aT

ui(a1)+δui

(a2)+δ2ui

(a3)+· · ·+δT−1ui

(aT)=

T∑t=1

δt−1ui(at).

Normalized Discounted Payoffs

• What is the sum of2 + 2δ + 2δ2 + 2δ3 + · · ·

• The normalized discounted payoffs for action sequence(a1, a2, . . . , aT

)is

Ui(a1, a2, . . . , aT

)= (1− δ)

T∑t=1

δt−1ui(at).

)is

Ui(a1, a2, . . . , aT

)= (1− δ)

T∑t=1

δt−1ui(at).

)is

Ui(a1, a2, . . . , aT

)= (1− δ)

T∑t=1

δt−1ui(at).

Repeated GameDefinitionLet G be a strategic game. Denote the set of players by Nand the set of actions and payoff function of each player i byAi and ui, respectively. The T -period repeated game of Gfor discount factor δ is the extensive game with perfectinformation and simultaneous moves in which

• the set of players is N ,• the set of terminal histories is the set of sequences(

a1, a2, . . . , aT)

of action profiles in G,

• the player function assigns all players to all histories,

• the set of actions for player i after any history is Ai, and

• each player i evaluates terminal history according tonormalized discounted payoff Ui

(a1, a2, . . . , aT

).

• the set of players is N ,

• the set of terminal histories is the set of sequences(a1, a2, . . . , aT

)of action profiles in G,

(a1, a2, . . . , aT

).

a1, a2, . . . , aT)

(a1, a2, . . . , aT

).

a1, a2, . . . , aT)

(a1, a2, . . . , aT

).

a1, a2, . . . , aT)

(a1, a2, . . . , aT

).

a1, a2, . . . , aT)

(a1, a2, . . . , aT

).

Finitely Repeated Prisoner’s Dilemma

• Consider the Prisoner’s Dilemma gamefor T = 2.

• How many terminal histories arethere?

• How many non-terminal histories arethere?

• How many strategies are there?• What are the the Nash equilibria?• What are the subgame Perfect Nash

equilibria?

Finitely Repeated Prisoner’s Dilemma

• Consider the Prisoner’s Dilemma gamefor T > 2.

• How many terminal histories arethere?

• How many non-terminal histories arethere?

• How many strategies are there?• What are the the Nash equilibria?• What are the subgame Perfect Nash

equilibria?

Strategies in the Infinitely-Repeated

Games• We need to specify an action after every history.

• Recall the Grim-Trigger strategy; thus,

si(a1, . . . , at

)=

C h = ∅C if (a1j , . . . , a

tj) = (C, . . . , C)

D otherwise.

• It is often useful to represent strategies as a finiteautomaton; that is,

C DStart

C

D

Any

Games• We need to specify an action after every history.• Recall the Grim-Trigger strategy; thus,

si(a1, . . . , at

)=

C h = ∅C if (a1j , . . . , a

tj) = (C, . . . , C)

D otherwise.

C DStart

C

D

Any

si(a1, . . . , at

)=

C h = ∅C if (a1j , . . . , a

tj) = (C, . . . , C)

D otherwise.

C DStart

C

D

Any

si(a1, . . . , at

)=

C h = ∅C if (a1j , . . . , a

tj) = (C, . . . , C)

D otherwise.

C DStart

C

D

Any

Games (Cont.)• Next, we look at a limited punishment strategy.

C D D DStart

C

D Any Any

Any

• Now, consider the popular Tit-For-Tat.

C DStart

C

D

C

D

C D D DStart

C

D Any Any

Any

C DStart

C

D

C

D

C D D DStart

C

D Any Any

Any

C DStart

C

D

C

D

C D D DStart

C

D Any Any

Any

C DStart

C

D

C

D

Examples of Tit-for-Tat

High Low

High

Low

20 20 0 35

35 0 10 10

• The low price guarantee scheme works as follows.

1 I set a high price in the first period,

2 if you lower your price, then, I will lower my price too,

3 otherwise, if you keep your price high, I will keep my price high.

High Low

High

Low

20 20 0 35

35 0 10 10

High Low

High

Low

20 20 0 35

35 0 10 10

High Low

High

Low

20 20 0 35

35 0 10 10

High Low

High

Low

20 20 0 35

35 0 10 10

High Low

High

Low

20 20 0 35

35 0 10 10

Feasible Payoffs

C D

C

D

2 2 0 3

3 0 1 1

What payoffs are possibleas a Nash equilibrium?

Feasible Payoffs

C D

C

D

2 2 0 3

3 0 1 1

What payoffs are possibleas a Nash equilibrium?

Minmax Payoff

DefinitionPlayer i’s minmax payoff in a strategic game is

ui = mina−i∈A−i

(maxai∈Ai

ui (ai, a−i)

).

• Alternatively, we have

(BRi (a−i)) .

• If player 2 tries to punish player 1 forever, this is the bestplayer 1 can do.

Minmax Payoff

(maxai∈Ai

ui (ai, a−i)

).

(BRi (a−i)) .

Minmax Payoff

(maxai∈Ai

ui (ai, a−i)

).

(BRi (a−i)) .

Minmax Payoff

(maxai∈Ai

ui (ai, a−i)

).

(BRi (a−i)) .

Example

• What would happen in this game?

• In this game:

• Action A is strictly dominant for Player 1.

• Action C is strictly dominant for Player 2.

• Players will repeatedly play (A,C).

C D

A

B

3,3

1,2

2,1

0,0

Example

• In this game:

C D

A

B

3,3

1,2

2,1

0,0

Example

• In this game:

C D

A

B

3,3

1,2

2,1

0,0

Example

• In this game:

C D

A

B

3,3

1,2

2,1

0,0

Example

• In this game:

C D

A

B

3,3

1,2

2,1

0,0

Example (Cont.)

1 2 3

1

2

3

Player 1 Payoff

Pla

yer

2P

ayoff

(3, 3)

(2, 1)

(1, 2)

(0, 0)

Feasible Payoffs

Individually Rational Payoffs

Folk Theorem Payoffs

Experimental Data

Example (Cont.)

1 2 3

1

2

3

Player 1 Payoff

Pla

yer

2P

ayoff

(3, 3)

(2, 1)

(1, 2)

(0, 0)

Feasible Payoffs

Experimental Data

Example (Cont.)

1 2 3

1

2

3

Player 1 Payoff

Pla

yer

2P

ayoff

(3, 3)

(2, 1)

(1, 2)

(0, 0)

Feasible Payoffs

Experimental Data

Example (Cont.)

1 2 3

1

2

3

Player 1 Payoff

Pla

yer

2P

ayoff

(3, 3)

(2, 1)

(1, 2)

(0, 0)

Feasible Payoffs

Experimental Data

Example (Cont.)

1 2 3

1

2

3

Player 1 Payoff

Pla

yer

2P

ayoff

(3, 3)

(2, 1)

(1, 2)

(0, 0)

Feasible Payoffs

Experimental Data

Minmax examples

C D

C

D

2 2 0 3

3 0 1 1

C D

C

D

3 3 1 4

4 1 0 0

C D

C

D

1 1 2 4

4 2 1 1

Folk Theorem

C D

C

D

2 2 0 3

3 0 1 1

Folk Theorem

Any payoff (v1, v2) where both v1 > u1 and v2 > u2,can be supported as a Nash equilibrium if players aresufficiently patient.

Folk Theorem

C D

C

D

2 2 0 3

3 0 1 1

Folk Theorem

C D

C

D

2 2 0 3

3 0 1 1

Folk Theorem

Folk Theorem (Cont.)

C D

C

D

3 3 1 4

4 1 0 0

Folk Theorem (Cont.)

C D

C

D

1 1 2 4

4 2 1 1

Folk Theorem (Example)

C C D

D

Start

DD

D

C C

Any

C

C C C

D

Start

DD

C

C C

Any

D

Folk Theorem (Example)

C C D

D

Start

DD

D

C C

Any

C

C C C

D

Start

DD

C

C C

Any

D

Repeated Games: The Prisoner's Dilemmachristosaioannou.com/Repeated Games The Prisoners Dilemma...In the Prisoner’s Dilemma, both players have an incentive to cheat, and everyone

Documents