Infinitely Repeated Gameshrtdmrt2/Teaching/GT_2015_19/L12.pdfAn Infinitely Repeated Prisoner’s Dilemma An Infinitely Repeated Prisoner’s Dilemma We will analyze the game using

Infinitely Repeated Games

Carlos Hurtado

Department of EconomicsUniversity of Illinois at Urbana-Champaign

[email protected]

Jun 12th, 2015

C. Hurtado (UIUC - Economics) Game Theory

mailto:[email protected]

On the Agenda

1 Infinitely Repeated Games

2 How Players evaluate payoffs in infinitely repeated games?

3 An Infinitely Repeated Prisoner’s Dilemma

4 A General Analysis

C. Hurtado (UIUC - Economics) Game Theory


On the Agenda





C. Hurtado (UIUC - Economics) Game Theory 1 / 19



If a game has a unique Nash equilibrium, then its finite repetition has a uniqueSPNE (Exercise).Our intuition, however, is that long-term relationships may be fundamentallydifferent from one-shot meetings.Infinitely repeated games also model a long-term relationship in which the playersdo not know a priori when they will stop repeating the game: there is nopre-ordained number of repetitions.Recall the terminology: The game that is being repeated is the stage game.The stages of the game are t = 0, 1, 2, .... An infinitely repeated game is alsosometimes called a supergame.


How Players evaluate payoffs in infinitely repeated games?

On the Agenda







How Players evaluate payoffs in infinitely repeated games?A player receives an infinite number of payoffs in the game corresponding to theinfinite number of plays of the stage game.We need a way to calculate a finite payoff from this infinite stream of payoffs inorder that a player can compare his strategies in the infinitely repeated game.The most widely used approach is discounted payoffs:

- Let ρit denote the payoff that player i receives in the t-th stage of the game.- Player i evaluates an infinite sequence of payoffs as a sum of discounted values.

∞∑t=0

δti ρit

- This will be a finite number as long as (|ρit |)∞t=0 is bounded above.- This discounted sum is typically modified by putting (1 âĹŠ i ) in front (for reasons

that will be clear in a moment).

(1− δi)

∞∑t=0

δti ρit

- This is a renormalization of utility that doesn’t change player i ’s ranking of anytwo infinite sequences of payoffs.




We all know that:

an − bn = (a − b)(

an−1 + an−2b + an−3b2 + . . .+ abn−2 + bn−1)= (a − b)

n−1∑j = 0

an−j−1bj

Set a = x and b = 1:

xn − 1 = (x − 1)(

xn−1 + xn−2 + xn−3 + . . .+ a + 1)

= (x − 1)n−1∑j = 0

x j

Hence:n−1∑j = 0

x j =xn − 1x − 1 =

1− xn

1− x




Note that, if |x | < 1:lim

n→∞xn = 0

Then:

limn→∞

n−1∑j = 0

x j =

∞∑j = 0

x j =1

1− x

In particular, the (1− δi) insures that player i evaluates the sequence in which hereceives a constant c in each period as c:

(1− δi)

∞∑t=0

δti c = c(1− δi)

11− δi

= c




An alternative approach is limiting average payoffs:

limn→∞

∑n−1t=0 ρit

n

The existence of this limit is sometimes a problem.The advantage of this formula, however, is that it is easy to calculate the limitingpayoff if the sequence of payoffs eventually reaches some constant payoff.


An Infinitely Repeated Prisoner’s Dilemma

On the Agenda








We will analyze the game using discounted payoffs. Consider the following versionof the prisoner’s dilemma:

1/2 c ncc 2,2 -3,3

nc 3,-3 -2,-2Here, c refers to ”cooperate” while nc refers to ”don’t cooperate”.A theorem that we stated in the beginning of the class implies that there is aunique SPNE in the finite repetition of this game, namely nc, nc in each and everystage.This remains an SPNE outcome of the infinitely repeated game:

player 1 : nc in every stageplayer 2 : nc in every stage

Given the other player’s strategy, playing nc maximizes player i’s payoff in eachstage of the game and hence maximizes his discounted payoff (and also his averagepayoff, if that is how he’s calculating his return in the infinite game).




This isn’t a very interesting equilibrium. Why bother with infinite repetition if thisis all that we can come up with?In particular, we ask ”Can the players sustain (c, c) as the outcome in each andevery stage of the game as a noncooperative equilibrium?”Consider the following strategy as played by both players:

1. Play c to start the game and as long as both players play c.2. If any player ever chooses nc, then switch to nc for the rest of the game.

This is a trigger strategy in the sense that bad behavior (i.e., playing nc) by eitherplayer triggers the punishment of playing nc in the remainder of the game.It is sometimes also called a ”grim” trigger strategy to emphasize how unforgivingit is: if either player ever chooses nc, then player i will punish his opponent forever.




Does the use of this trigger strategy define an SPNE?Playing c in any stage does not maximize a player’s payoff in that stage (nc is thebest response within a stage).Suppose player i starts with this strategy and considers deviating in stage k toreceive a payoff of 3 instead of 2.Thereafter, his opponent chooses nc, and so he will also choose nc in theremainder of the game.The use of trigger strategies therefore defines a Nash equilibrium if and only if theequilibrium payoff of 2 in each stage is at least as large as the payoff fromdeviating to nc in stage k and ever thereafter:

(1− δi)

∞∑t=0

δti · 2 ≥ (1− δi)

[k−1∑t=0

δti · 2 + δk

i · 3 +

∞∑t=k+1

δti · (−2)

]




We have:

(1− δi)

∞∑t=0

δti · 2 ≥ (1− δi)

[k−1∑t=0

δti · 2 + δk

i · 3 +

∞∑t=k+1

δti · (−2)

]∞∑

t=k

δti · 2 ≥ δk

i · 3 +

∞∑t=k+1

δti · (−2)

∞∑t=0

δti · 2 ≥ 3 + δi

∞∑t=0

δti · (−2)

21− δi

≥ 3− 2δi

1− δi5δi ≥ 1

δi ≥ 15




Deviating from the trigger strategy produces a one-time bonus of changing one’sstage payoff from 2 to 3. The cost, however, is a lower payoff ever after.We see that the one-time bonus is worthwhile for player i only if his discountfactor is low (δi < 1/5), so that he doesn’t put much weight upon the low payoffsin the future.When each δi ≥ 1/5, do the trigger strategies define a subgame perfect Nashequilibrium (in addition to being a Nash equilibrium)?The answer is Yes!A subgame of the infinitely repeated game is determined by a history, or a finitesequence of plays of the game.There are two kinds of histories to consider:

1. If each player chose c in each stage of the history, then the trigger strategiesremain in effect and define a Nash equilibrium in the subgame.

2. If some player has chosen nc in the history, then the two players use thestrategies (nc,nc) in every stage.




Whichever of the two kinds of history we have, the strategies define a Nashequilibrium in the subgame. The trigger strategies therefore define a subgameperfect Nash equilibrium whenever they define a Nash equilibrium.Recall the fundamental importance of the Prisoner’s Dilemma: it illustrates quitesimply the contrast between self-interested behavior and mutually beneficialbehavior.The play of (nc, nc) instead of (c, c) represents the cost of noncooperativebehavior in comparison to what the two players can achieve if they instead wereable to cooperate.What we’ve shown is that that the cooperative outcome can be sustained as anoncooperative equilibrium in a long-term relationship provided that the playerscare enough about future payoffs!


A General Analysis

On the Agenda






A General Analysis

A General Analysis

We let (s1, s2) denote a Nash equilibrium of the stage game with correspondingpayoffs (π1, π2).Suppose that the choice of strategies (s∗1 , s∗2 ) would produce the payoffs (π∗1 , π

∗2 ).

whereπ∗i > πi ∀i

The strategies (s∗1 , s∗2 ) would therefore produce a better outcome for each player.The strategies (s∗1 , s∗2 ) are not a Nash equilibrium. When player −i chooses s∗−i ,the maximal payoff that player i can achieve by changing his strategy away froms∗−i is di > π∗i .Note that we are assuming that di > π∗i > πi

Can trigger strategies sustain the use of the strategies (s∗1 , s∗2 ) in each and everystage of the game?The trigger strategy here for each player i is:

1. Play s∗i to start the game and as long as both players play (s∗1 , s∗2 ).2. If any player ever deviates from the pair (s∗1 , s∗2 ), then switch to si for the

rest of the game.


A General Analysis

A General Analysis

We will calculate a lower bound on δi that is sufficient to insure that player i willnot deviate from s∗i .Suppose player i deviates from s∗i in stage k. We make two observations:

1. Player −i switches to s−i in each stage t > k. Player i’s best response is tochoose si in each stage after the k-th stage (recall our assumption that(s1, s2) is a Nash equilibrium).

2. The maximal payoff that player i can gain in the k-th stage is di (byassumption).


A General Analysis

A General Analysis

The following inequality is therefore necessary and sufficient for player i to preferhis trigger strategy to the deviation that we are considering:

(1− δi)

∞∑t=0

δti · π∗i ≥ (1− δi)

[k−1∑t=0

δti · π∗i + δk

i · di +

∞∑t=k+1

δti · πi

]∞∑

t=k

δti · π∗i ≥ δk

i · di +

∞∑t=k+1

δti · πi

∞∑t=0

δti · π∗i ≥ di + δi

∞∑t=0

δti · πi

π∗i1− δi

≥ di +πiδi

1− δi

π∗i ≥ di(1− δi) + δiπi

δi ≥ di − π∗idi − π


A General Analysis

A General Analysis

As in the previous example, we have obtained a lower bound on δi that is sufficientto insure that player i will not deviate from his trigger strategy given that theother player uses his trigger strategy.Several observations:

1. The analysis focuses on a single player at a time and exclusively on hispayoffs. The bound thus extends immediately to stage games with n > 2players. The assumption that there are two players has no role in the aboveanalysis.

2. Notice that any player who deviates from the ”better” strategies (s∗1 , s∗2 )triggers the switch by both players to the Nash equilibrium strategies (s1, s2).This is unfair in the sense that both players suffer from the bad behavior ofone of the two players (it is part of the definition of equilibrium).

3. If di ≤ π∗i , then player i has no incentive to deviate from s∗i (he doesn’t evenget a one-stage ”bonus” from ending the play of (s∗1 , s∗2 ) for the rest of thegame). We thus don’t have to worry about player i’s willingness to stick tohis trigger strategy regardless of the value of his discount factor.


Exercises

On the Agenda






Exercises

Exercises

Consider the following stage game:1\2 L C RT 1,-1 2,1 1,0M 3,4 0,1 -3,2B 4,-5 -1,3 1,1

a. Find the unique pure strategy Nash equilibrium.b. Write down a trigger strategy where the outcome of the game is (M, L).c. Find a lower bound on δi that is sufficient to insure that player i will not deviate

from his trigger strategy given that the other player uses his trigger strategy.


Infinitely Repeated Gameshrtdmrt2/Teaching/GT_2015_19/L12.pdfAn Infinitely Repeated Prisoner’s Dilemma An Infinitely Repeated Prisoner’s Dilemma We will analyze the game using

Documents