Essential Microeconomics -1- © John Riley January 8, 2014 GAMES WITH A HISTORY Finitely repeated games 2 Commitment? 5 Equilibrium threats 13 Sequential move games 16 Sub-game perfect equilibrium 27 One-stage deviation principle 30
Essential Microeconomics -1-
© John Riley January 8, 2014
GAMES WITH A HISTORY
Finitely repeated games 2
Commitment? 5
Equilibrium threats 13
Sequential move games 16
Sub-game perfect equilibrium 27
One-stage deviation principle 30
Essential Microeconomics -2-
© John Riley January 8, 2014
FINITELY REPEATED GAMES
Stage Game: Simultaneous move game played in each of T rounds or “stages”.
Strategy space in the t-th stage game: 1
...t
IS S S
Strategy profile 1( ,..., )Ts s s 1 ... TS S S
**
Essential Microeconomics -3-
© John Riley January 8, 2014
FINITELY REPEATED GAMES
Stage Game: Simultaneous move game played in each of T rounds or “stages”.
Strategy space in the t-th stage game: 1
...t
IS S S
Strategy profile 1( ,..., )Ts s s 1 ... TS S S
Payoffs: The stage t payoff to player i J is ( )t
iu s .
Future payoffs are discounted at the rate . Therefore if the strategy profile is sS , the payoff to
player i is
1
1
( ) ( )T
t t
i it
U s u s
where t t
iis S S
I
*
Essential Microeconomics -4-
© John Riley January 8, 2014
FINITELY REPEATED GAMES
Stage Game: Simultaneous move game played in each of T rounds or “stages”.
Strategy space in the t-th stage game: 1
...t
IS S S
Strategy profile 1( ,..., )Ts s s 1 ... TS S S
Payoffs: The stage t payoff to player i J is ( )t
iu s .
Future payoffs are discounted at the rate . Therefore if the strategy profile is sS , the payoff to
player i is
1
1
( ) ( )T
t t
i it
U s u s
where t t
iis S S
I
History
Player i can base her strategy in stage t on the prior actions of players, that is, on the history t
ih of the
game. Initially we shall assume that players observe the prior actions of all their opponents * .
Then 1 1( ,..., )t t
ih a a .
*A player who is randomizing could also make his mixed strategy public by allowing a third party to monitor his randomization device. For the cases we shall
consider, the equilibrium strategies are pure strategies so observing past actions is the same as observing past strategies.
Essential Microeconomics -5-
© John Riley January 8, 2014
Commitment?
Consider this simple stage game.
Regardless of player 2’s strategy, player 1’s
best response is U. Thus her NE best response is U.
And given that player 1 chooses U,
player 2’s best response is L. Thus
the unique NE is of the stage game is * ( , )s U L .
Player 2
L R
Player 1
U 4,4 2,2
D 3,5 1,1
Essential Microeconomics -6-
© John Riley January 8, 2014
Two stage game
Next suppose that the stage game is played twice.
For simplicity assume no discounting.
Stage 1 strategies:
1 1 1
1 2( , )s s s
Stage 2 strategies
In stage 2 the history of the game is 2 1h s so the strategy of player i is
2 2 2 1( ) ( )i is h s s .
It is readily confirmed that mutual best responses are to ignore history and repeat the unique NE
strategy of the stage game.
That is, the following strategy profile is an NE.
1 * 2 2 *, ( )s s s h s where * ( , )s U L
Player 2
L R
Player 1
U 4,4 2,2
D 3,5 1,1
Essential Microeconomics -7-
© John Riley January 8, 2014
We now show that the following strategy profile is also a NE.
Player 1: 1 1 1
1 2, ( )s D s h L .
Player 2: 1
2s L , 1
2 1
2 1
, if ( , )( )
, if ( , )
L h D Ls h
R h D L
Player 2 would like player 1 to choose D in the first stage so that his payoff is 5. He achieves this with
the threat to play R in the second stage if player 1 does not play D in the first stage.
Player 2
L R
Player 1
U 4,4 2,2
D 3,5 1,1
Essential Microeconomics -8-
© John Riley January 8, 2014
Player 1: 1 1 2
1 2, ( )s D s h U .
Player 2 1
2s L , 2
2 2
2 2
, if ( , )( )
, if ( , )
L h D Ls h
R h D L
Best response by player 2:
In the first stage player 2’s best response is L. Then 2 1 ( , )h s D L and so 2 2
2 ( )s h L . Player 1’s best
response in the second stage is therefore U
Best response by player 1:
If player 1 chooses U in the first stage then 1 ( , )s U L so player 2 chooses R in the second stage.
Hence player 1’s best response in the second stage is U.
Player 1’s total payoff is therefore 4+2 = 6.
If player 1 chooses D in the first stage then 1 ( , )s D L so player 2 chooses L in the second stage.
Hence player 1’s best response in the second stage is U. Her total payoff is therefore 3+4 =7.
Thus 1 1 2
1 2, ( )s D s h U is indeed a best response.
Player 2
L R
Player 1
U 4,4 2,2
D 3,5 1,1
Essential Microeconomics -9-
© John Riley January 8, 2014
In most economic games this kind of outcome seems distinctly unlikely. Crucially it requires that
player 2 can commit to her strategy. For if not player 1 can reason as follows.
“When stage 2 is reached player 2’s strictly dominant strategy is L so she will never follow through on
her threat. Thus player 2’s threat to play R is not credible.”
Equilibrium in the final stage without commitment
Let 1 2 2, ( )s s h be a NE of the two stage game.
In the absence of commitment, the equilibrium strategy in the second stage 2 2( )s h must also be a NE
of the stage game.
In the example the stage game has a unique NE * ( , )s U L therefore 2 2( ) ( , )s h U L
Thus the NE strategy of the two stage game is 1 2 2 1 *{ , ( )} { , }s s h s s .
But 1 2 2 1 *( ) ( ) ( ( )) ( ) ( )i i i i iU s u s u s h u s u s
The last term is a constant so the payoff in the two stage game is the payoff in the one stage game plus
a constant.
Essential Microeconomics -10-
© John Riley January 8, 2014
Adding a constant to the stage game payoffs does not change the payoff to deviating.
Since *s is the unique equilibrium of the stage game it follows that 1 *s s .
Proposition 9.2-1: Nash equilibria of a finitely repeated game
Suppose that , 1,...,ts t T is a NE of the stage game. Then (i) the strategy profile 1( ,..., )Ts s s is a
NE of the repeated game and (ii) in the absence of commitment, if *s is the unique NE of the stage
game then * *( ,..., )s s is the unique NE of the repeated game.
Proof of (i):
1 1 1( , ) ( ) ... ( ( ), ) ... ( ( ), )t t t T T T
i i i i i i i i i iU s s u s u s h s u s h s
Since ( , )t t
i is s is an equilibrium of the stage game ( ( ), ) ( , )t t t t
i i i i i iu s h s u s s .
Since this holds for all t,
1 1 1( , ) ( ) ... ( ) ... ( )t t T T
i i i i i iU s s u s u s u s
Essential Microeconomics -11-
© John Riley January 8, 2014
Proof of (ii):
Let 1( ,..., ( ))T Ts s s h be an NE of the finitely repeated game.
1 1 1( ) ( ) ... ( ( ), ) ... ( ( ), ( ))t t t T T T T
i i i i i i i iU s u s u s h s u s h s h
We apply a simple backwards induction argument.
In the absence of commitment ( )T Ts h must be an equilibrium of the stage game played in the T-th
stage. By hypothesis the unique NE of the stage game is *s so *( )T Ts h s .
Therefore
1 1 1 * *( ) ( ) ... ( ( ), ( )) ... ( , )t t t t T
i i i i i i i iU s u s u s h s h u s s
The last term is a constant. Thus the T stage game reduces to a T-1 stage game. In the absence of
commitment 1 1( )T Ts h must be an equilibrium of the stage game played in stage T-1. By hypothesis
the unique NE of the stage game is *s so 1 1 *( )T Ts h s .
Applying this argument inductively, it follows that *( ) , 1,...,t ts h s t T
Essential Microeconomics -12-
© John Riley January 8, 2014
Repeated games with multiple equilibria of the stage game
Example: Two-stage repeated partnership game
player i’s strategy : 1 2 2( , ( ))i i i i i i
s s s h S S .
player i’s payoff: 1 1 2 2( , ) ( , ) ( , )i i i i i i i i i
U s s u s s u s s where 2 2
1( , ( ))t
i i i i is s s h S S
Class Exercise:
(a) What are the NE strategies of the stage game?
(b) What are some NE strategies of the
repeated game?
Player 2
2 1a
2 2a 2 3a
1 1a 5,5 11,4 17,-9
Player 1 1 2a 4,11 16,16 28,9
1 3a -9,17 9,28 27,27
2 1a 2 2a 2 3a
1 1a 5,5 11,4 17,-9
Player 1 1 2a 4,11 16,16 28,9
1 3a -9,17 9,28 27,27
Essential Microeconomics -13-
© John Riley January 8, 2014
Equilibrium threats
We now argue that, for any sufficiently high discount factor, if the stage game has multiple
equilibria with different equilibrium payoffs, a player can use the threat to play the bad NE in the later
periods period to induce a more favorable behavior in the first period.
Consider the two-stage partnership game.
**
Player 2
2 1a
2 2a 2 3a
1 1a 5,5 11,4 17,-9
Player 1 1 2a 4,11 16,16 28,9
1 3a -9,17 9,28 27,27
2 1a 2 2a 2 3a
1 1a 5,5 11,4 17,-9
Player 1 1 2a 4,11 16,16 28,9
1 3a -9,17 9,28 27,27
Essential Microeconomics -14-
© John Riley January 8, 2014
Equilibrium threats
We now argue that, for any sufficiently high discount factor, if the stage game has multiple
equilibria with different equilibrium payoffs, a player can use the threat to play the bad NE in the later
periods period to induce a more favorable behavior in the first period.
Consider the two-stage partnership game.
Let ( )t t
is h be the strategy of player i in period t
Consider the following strategy:
1 3i
s , 2 2 2( ) 2 if (3,3)i
s h h , 2 2 2( ) 1 if (3,3)i
s h h .
Class discussion:
Is ( , )s s a NE strategy profile.
*
Player 2
2 1a
2 2a 2 3a
1 1a 5,5 11,4 17,-9
Player 1 1 2a 4,11 16,16 28,9
1 3a -9,17 9,28 27,27
2 1a 2 2a 2 3a
1 1a 5,5 11,4 17,-9
Player 1 1 2a 4,11 16,16 28,9
1 3a -9,17 9,28 27,27
Essential Microeconomics -15-
© John Riley January 8, 2014
Equilibrium threats
We now argue that, for any sufficiently high discount factor, if the stage game has multiple
equilibria with different equilibrium payoffs, a player can use the threat to play the bad NE in the later
periods period to induce a more favorable behavior in the first period.
Consider the two-stage partnership game.
Let ( )t t
is h be the strategy of player i in period t
Consider the following strategy:
1 3i
s , 2 2 2( ) 2 if (3,3)i
s h h , 2 2 2( ) 1 if (3,3)i
s h h .
Argue that ( , )s s is a NE strategy profile.
Note that with more than 2 periods, the threat to switch thereafter to action 1 is an even stronger
deterrent.
Player 2
2 1a
2 2a 2 3a
1 1a 5,5 11,4 17,-9
Player 1 1 2a 4,11 16,16 28,9
1 3a -9,17 9,28 27,27
2 1a 2 2a 2 3a
1 1a 5,5 11,4 17,-9
Player 1 1 2a 4,11 16,16 28,9
1 3a -9,17 9,28 27,27
Essential Microeconomics -16-
© John Riley January 8, 2014
Sequential move games
We now consider games in
which players move sequentially.
As an example, consider the
three-player penny matching
game. But now players move
one at a time.
*
Fig. 9.2-1: Game tree
(0,2,1)
(2,2,2)
(0,0,0)
(1,0,4)
(-1,3,3)
(0,0,0)
(0,0,0)
(0,0,0) T
H
H
H
T
T H
T
T
H
H
H T
T 3
3
2
3
2
3
1
Essential Microeconomics -17-
© John Riley January 8, 2014
Sequential move games
We now consider games in
which players move sequentially.
As an example, consider the
three-player penny matching
game. But now players move
one at a time.
Backwards induction
The key to solving sequential move
games is to begin by examining the
final stage of the game and then to
work backwards.
Arrows in the top figure indicate branches that are best responses for player 3.
Replace the third node by the best response payoffs….continue…
Fig. 9.2-1: Game tree
(0,2,1)
(2,2,2)
(0,0,0)
(1,0,4)
(-1,3,3)
(0,0,0)
(0,0,0)
(0,0,0) T
H
H
H
T
T H
T
T
H
H
H T
T 3
3
2
3
2
3
1
Fig. 9.2-2: Backwards induction
(2,2,2)
(4,0,1)
(0,0,0)
(0,2,1 )
H T 1 2
H
T
H
T
2
Essential Microeconomics -18-
© John Riley January 8, 2014
Language of a sequential move game
Nodes: At each node of the tree one player takes an action
******
2
T
B
(2,2)
(24,1)
(8,1)
(0,3 )
T
1
T
B
B
2 2
2
1
Essential Microeconomics -19-
© John Riley January 8, 2014
Language of a sequential move game
Nodes: At each node of the tree one player takes an action.
Branches: Each action is depicted as a branch in the tree
*****
2
T
B
(2,2)
(24,1)
(8,1)
(0,3 )
T
1
T
B
B
2 2
2
1
Essential Microeconomics -20-
© John Riley January 8, 2014
Language of a sequential move game
Nodes: At each node of the tree one player takes an action.
Branches: Each action is depicted as a branch in the tree
Initial node: Starting point of the game is called the initial node.
Convention: The player making the choice at the initial node is called player 1.
Other players are labeled according to the order of their first moves
****
2
T
B
(2,2)
(24,1)
(8,1)
(0,3 )
T
1
T
B
B
2 2
2
1
Essential Microeconomics -21-
© John Riley January 8, 2014
Language of a sequential move game
Nodes: At each node of the tree one player takes an action.
Branches: Each action is depicted as a branch in the tree
Initial node: Starting point of the game is called the initial node.
Convention: The player making the choice at the initial node is called player 1.
Other players are labeled according to the order of their first moves
Terminal node: The end of each branch from the last decision node. This is a possible outcome of
the game. Associated with each outcome is a vector of payoffs ( )u a .
***
2
T
B
(2,2)
(24,1)
(8,1)
(0,3 )
T
1
T
B
B
2 2
2
1
Essential Microeconomics -22-
© John Riley January 8, 2014
Language of a sequential move game
Nodes: At each node of the tree one player takes an action.
Branches: Each action is depicted as a branch in the tree
Initial node: Starting point of the game is called the initial node.
Convention: The player making the choice at the initial node is called player 1.
Other players are labeled according to the order of their first moves
Terminal node: The end of each branch from the last decision node. This is a possible outcome of
the game. Associated with each outcome is a vector of payoffs ( )u a .
Action profile: List of actions leading from the initial node to any terminal node.
2
T
B
(2,2)
(24,1)
(8,1)
(0,3 )
T
1
T
B
B
2 2
2
1
Essential Microeconomics -24-
© John Riley January 8, 2014
Language of a sequential move game
Nodes: At each node of the tree one player takes an action.
Branches: Each action is depicted as a branch in the tree
Initial node: Starting point of the game is called the initial node.
Convention: The player making the choice at the initial node is called player 1.
Other players are labeled according to the order of their first moves
Terminal node: The end of each branch from the last decision node. This is a possible outcome of
the game. Associated with each outcome is a vector of payoffs ( )u a .
Action profile: List of actions leading from the initial node to any terminal node.
Extensive form: The depiction of the game as a tree is called the extensive form representation of
the game or more simply the “extensive form” of the game.
*
2
T
B
(2,2)
(24,1)
(8,1)
(0,3 )
T
1
T
B
B
2 2
2
1
Essential Microeconomics -25-
© John Riley January 8, 2014
Language of a sequential move game
Nodes: At each node of the tree one player takes an action.
Branches: Each action is depicted as a branch in the tree
Initial node: Starting point of the game is called the initial node.
Convention: The player making the choice at the initial node is called player 1.
Other players are labeled according to the order of their first moves
Terminal node: The end of each branch from the last decision node. This is a possible outcome of
the game. Associated with each outcome is a vector of payoffs ( )u a .
Action profile: List of actions leading from the initial node to any terminal node.
Extensive form: The depiction of the game as a tree is called the extensive form representation of
the game or more simply the “extensive form” of the game.
NOTE: All nodes except the initial node are connected to a single prior node and at least
two successor nodes. This is what gives the graph its tree structure.
2
T
B
(2,2)
(24,1)
(8,1)
(0,3 )
T
1
T
B
B
2 2
2
1
Essential Microeconomics -26-
© John Riley January 8, 2014
Sub-games
As we now show, there can be NE of sequential move games that are highly implausible. We present
an example and then show that by introducing a modest “refinement” of the NE, the silly equilibrium
is eliminated. In the example, an Incumbent firm faces a potential Entrant. The game tree is depicted
in Fig. 9.2-3. Player 1 is the Entrant. Player 2 is the Incumbent.
Claims: (i) There are two NE. (ii) Only one is plausible.
*
Out
Enter
Fight
Share
(0,6)
(-2,1)
(3,3)
1
2
Fig. 9.2-3: Entry Game with sub-game
Essential Microeconomics -27-
© John Riley January 8, 2014
Sub-games
As we now show, there can be NE of sequential move games that are highly implausible. We present
an example and then show that by introducing a modest “refinement” of the NE, the silly equilibrium
is eliminated. In the example, an Incumbent firm faces a potential Entrant. The game tree is depicted
in Fig. 9.2-3. Player 1 is the Entrant. Player 2 is the Incumbent.
Claims: (i) There are two NE. (ii) Only one is plausible.
Note that the part of the tree inside the dotted rectangle
looks just like a game. This has all the requirements of a game;
an initial node and terminal nodes.
Out
Enter
Fight
Share
(0,6)
(-2,1)
(3,3)
1
2
Fig. 9.2-3: Entry Game with sub-game
Essential Microeconomics -28-
© John Riley January 8, 2014
Definition: Sub-game
Any branch of a game tree that begins with a single node is a sub-game.
Sub-game Perfect Equilibrium (SPE)
Consider the sub-game inside the dashed rectangle. We can
strengthen our definition of strategic equilibrium by requiring
that the NE strategies must also be equilibrium strategies of
each sub-game.
*
Out
Enter
Fight
Share
(0,6)
(-2,1)
(3,3)
1
2
Fig. 9.2-3: Entry Game with sub-game
Essential Microeconomics -29-
© John Riley January 8, 2014
Definition: Sub-game
Any branch of a game tree that begins with a single node is a sub-game.
Sub-game Perfect Equilibrium (SPE)
Consider the sub-game inside the dashed rectangle. We can
strengthen our definition of strategic equilibrium by requiring
that the NE strategies must also be equilibrium strategies of
each sub-game.
Definition: Sub-game perfect equilibrium
A NE strategy profile of a sequential move game is sub-game perfect if the strategy profile is also a
NE strategy of each of the sub-games.
Out
Enter
Fight
Share
(0,6)
(-2,1)
(3,3)
1
2
Fig. 9.2-3: Entry Game with sub-game
Essential Microeconomics -30-
© John Riley January 8, 2014
For our example, the unique equilibrium of the single sub-game
is for player 2 to choose Share. We can then lop off this part of
the tree and replace it by the NE payoffs of the sub-game.
Player 1’s unique SPE strategy is therefore to choose Enter.
1
Out
Enter
(0,6)
(3,3)
Fig. 9.2-4: Entry Game with payoffs from sub-game
1 1
Essential Microeconomics -31-
© John Riley January 8, 2014
The one-stage deviation principle
There is an easy way to check whether a strategy profile is sub-game perfect. It is enough to consider
one-stage deviations by players at each decision node.
Proposition 9.2-2: One-stage deviation principle
In a T stage sequential move game, suppose that for the strategy profile 1 2 2( , ( ),.., ( ))T Ts s s h s h there
is no one stage deviation by a player that raises that player’s payoff. Then the strategy profile is sub-
game perfect.
Essential Microeconomics -32-
© John Riley January 8, 2014
Proof: The proof follows by backwards induction.
1 2 2( , ( ),.., ( ))T Ts s s h s h NE strategy profile.
1 2 2( , ( ),.., ( ))T Ts s s h s h some other strategy profile.
1 1 1( ) ( ,...., ( ), ( ),..., ( ))T Ts s s h s h s h strategy profile that agrees with s for t and with
fors t
By hypothesis there is no one-stage deviation from s that benefits any player. So there is no one stage
deviation from ( )s for any stage t sub-game where t .
We consider deviations by player i. Suppose that stage a is the last stage at which, for some history,
player i deviates. Then the strategy profile s agrees with s for 1t a so that ( 1)s s a . We now
argue that player i must be at least as well off under strategy profile ( )s a as ( 1)s a .
Essential Microeconomics -33-
© John Riley January 8, 2014
Consider the stage a sub-game. By hypothesis, there is no one stage deviation from ( )s a that benefits
player i. But the only difference between ( )s a and ( 1)s a is the strategy at stage a . Therefore
( ( )) ( ( 1)) ( )i i i
U s a U s a U s . (0.0-1)
*
Essential Microeconomics -34-
© John Riley January 8, 2014
Consider the stage a sub-game. By hypothesis, there is no one stage deviation from ( )s a that benefits
player i. But the only difference between ( )s a and ( 1)s a is the strategy at stage a . Therefore
( ( )) ( ( 1)) ( )i i i
U s a U s a U s . (*)
Let stage b be the second last stage at which player i deviates. Consider the sub-game at stage b . Note
that ( 1) ( )s b s a since there are no deviations between the two stages. Arguing as before ( 1)s b is
a one stage deviation from ( )s b . Therefore ( ( )) ( ( 1))i i
U s b U s b . Combining this result with (*) it
follows that ( ( )) ( )i i
U s b U s . Repeating this argument for every stage in which player i deviates it
follows that ( ) ( )i i
U s U s .
Q.E.D.
Essential Microeconomics -35-
© John Riley January 8, 2014
While we have considered a sequential move game, the argument is almost identical for a finitely
repeated game. We therefore have the following corollary.
Corollary 9.2-3: One-stage deviation principle in finitely repeated games
In a finitely repeated game suppose that for the strategy profile 1 2 2( , ( ),.., ( ))T Ts s s h s h there is no one
stage deviation by a player that raises that player’s payoff. Then the strategy profile is sub-game
perfect.