3.2.2. Dynamic Games of complete information: Backward Induction and Subgame perfection - Repeated Games -
Jan 03, 2016
3.2.2. Dynamic Games of complete information: Backward Induction and Subgame perfection
- Repeated Games -
Strategic Behavior in Business and Econ
Outline
3.1. What is a Game ?3.1.1. The elements of a Game3.1.2 The Rules of the Game: Example3.1.3. Examples of Game Situations3.1.4 Types of Games
3.2. Solution Concepts3.2.1. Static Games of complete information: Dominant
Strategies and Nash Equilibrium in pure and mixed strategies
3.2.2. Dynamic Games of complete information: Backward Induction and Subgame perfection
Strategic Behavior in Business and Econ
Repeated Games
A Repeated Game is a special case of a dynamic (sequentialmoves) game that consists of a (usually) static game beingplayed several times, one after the other
The game that is repeated is called the “stage game”
The (stage) game can be played a given number of times(known to all the players) or an indefinite number of times.
Strategic Behavior in Business and Econ
Repeated Games
Thus, we can have a:
Finitely Repeated Game.When the stage game is player a number T of
rounds (1, 2, 3, . . ., T). T is known to all the players
Infinitely Repeated GameWhen either
After each round the game continues to the next round with probability p and ends with probability (1 – p)
The game is played forever but at each round the value of the payoffs decreases by a factor of “p”
Strategic Behavior in Business and Econ
Finitely Repeated Games
Recall the Prisoners' DilemmaA “generic” version of the game is represented in the tableBelow
C D
C 3,3 0 ,5
D 5 ,0 1, 1
Where,C stands for “cooperate”D stands for “defect”
Strategic Behavior in Business and Econ
Finitely Repeated Games
We saw that both players defecting is the unique equilibriumof the game. In fact, D is a Dominant Strategy for each player
C D
C 3,3 0 ,5
D 5 ,0 1, 1
The apparently paradoxical behavior is that, although bothPlayers would mutually benefit from Cooperation, selfInterests leads to the worse outcome by Defecting
Strategic Behavior in Business and Econ
Finitely Repeated Games
We saw that both players defecting is the unique equilibriumof the game. In fact, D is a Dominant Strategy for each player
C D
C 3,3 0 ,5
D 5 ,0 1, 1
Repeating the game opens interesting possibilitiesTo “punish” egoistic (defect) behaviorsTo “reward” the right (cooperative) behavior
Strategic Behavior in Business and Econ
Finitely Repeated Games
We saw that both players defecting is the unique equilibriumof the game. In fact, D is a Dominant Strategy for each player
C D
C 3,3 0 ,5
D 5 ,0 1, 1
Examples: “Stick and Carrot Strategies” (Trigger Strategies)1. I will start with cooperation, and will mimic your behavior
afterwards2. I will start with cooperation and will keep doing so unless
you defect. In such case I will defect forever
Strategic Behavior in Business and Econ
The 2-times Repeated Prisoners' Dilemma
The tree representation of the 2-times Repeated Prisoners'Dilemma is shown in the next slide:
Notice:The “dotted” lines representing the simultaneous choice in
each stage of the gameThe payoffs at the end of the game correspond to the
sum of the payoffs in each stage Try to imagine the tree in a 3-times Repeated Prisoners'
Dilemma
Strategic Behavior in Business and Econ
C
C
C
C
C
C
C
C
C
C
C
C
C
C
CD
D
D
D
D
D
D
D
DD
D
D
D
D
D
6, 6
3, 8
8, 3
4, 43, 8
0, 105, 5
1, 68, 3
5, 510, 0
6, 14, 4
1, 66, 1
2, 2
Stage 1 Stage 2
Strategic Behavior in Business and Econ
The 2-times Repeated Prisoners' Dilemma
The game must be solved by Backward Induction using theSubgame Perfection technique (since there are “linked” nodesthat indicate that the game is of Imperfect Information)
Notice that his is always the case when we repeat a static game
We must, therefore, “solve” each of the 4 “subgames” in the second stage of the game and then move backwards
Strategic Behavior in Business and Econ
C
C
C
C
C
C
C
C
C
C
C
C
C
C
CD
D
D
D
D
D
D
D
DD
D
D
D
D
D
6, 6
3, 8
8, 3
4, 43, 8
0, 105, 5
1, 68, 3
5, 510, 0
6, 14, 4
1, 66, 1
2, 2
Stage 1 Stage 2
Subgame 1
Subgame 2
Subgame 3
Subgame 4
Strategic Behavior in Business and Econ
Subgame 1
C D
C 6, 6 3, 8
D 8, 3 4 , 4
C D
C 3, 8 0 , 10
D 5, 5 1, 6
C D
C 4, 4 1, 6
D 6, 1 2, 2
C D
C 8, 3 5 , 5
D 10 , 0 6, 1
Subgame 2
Subgame 4Subgame 3
Strategic Behavior in Business and Econ
The 2-times Repeated Prisoners' Dilemma
Notice that the solution in each subgame is always the same:
Player 1: DefectPlayer 2: Defect
And, again, Defect is a Dominant Strategy for each playerin each subgame
This is not a coincidence (as we will see shortly)
Thus, proceeding backwards in the tree we get . . .
Strategic Behavior in Business and Econ
C
C
C
C
C
C
C
C
C
C
C
C
C
C
CD
D
D
D
D
D
D
D
DD
D
D
D
D
D
6, 6
3, 8
8, 3
4, 43, 8
0, 105, 5
1, 68, 3
5, 510, 0
6, 14, 4
1, 66, 1
2, 2
Stage 1 Stage 2
Subgame 1
Subgame 2
Subgame 3
Subgame 4
Strategic Behavior in Business and Econ
C
C
CD
D
D
4, 4
1, 6
6, 1
2, 2
Stage 1 Stage 2
Again, what remains afterwe move backwards in thethree is another “simultaneousmove” game, the one that corresponds to the firststage of the game.
We must “solve” in looking atthe table representation
Strategic Behavior in Business and Econ
C
C
CD
D
D
4, 4
1, 6
6, 1
2, 2
Stage 1 Stage 2
C D
C 4, 4 1, 6
D 6, 1 2, 2
Strategic Behavior in Business and Econ
C
C
CD
D
D
4, 4
1, 6
6, 1
2, 2
Stage 1 Stage 2
Thus, knowing what will bethe outcome in the secondstage of the game . . .
C D
C 4, 4 1, 6
D 6, 1 2, 2
Strategic Behavior in Business and Econ
C
C
CD
D
D
4, 4
1, 6
6, 1
2, 2
Stage 1 Stage 2
Both players will also defectin the first round. (It's againa Dominant Strategy !)
C D
C 4, 4 1, 6
D 6, 1 2, 2
Strategic Behavior in Business and Econ
The T-times Repeated Prisoners' Dilemma
Stage 1 (D, D)
T=1
Stage 1 (D, D)
T=2
Stage 2 (D, D)
Stage 1 (D, D)
T=3
Stage 2 (D, D)
Stage 3 (D, D)
Stage 1 (D, D)
Any T
Stage 2 (D, D)
Stage 3 (D, D)
Stage T (D, D)· · · ·
Strategic Behavior in Business and Econ
The T-times Repeated Prisoners' Dilemma
No matter how many times the Prisoners' Dilemma is repeated, theequilibrium is always the same: Defect in every round.Why don't punishments (rewards) work ?
Intuition:At the last repetition, since the players know that there will not
be a “new chance” (no punishment-reward is possible), the best thing to do is to Defect
Knowing that, in the next-to-last round players know that in the next round the opponent will not cooperate. Then, why should I cooperate today if tomorrow my opponent is going to defect ? Again, the best thing to do is to Defect
We can apply this argument “backwards” to conclude that the best thing to do is to Defect all the time.
Strategic Behavior in Business and Econ
Finitely Repeated Games: General Facts
Any Finitely Repeated Game that consists of the repetition of a stage game that is a static game produces a “Dynamic Game of Imperfect Information
The tree representing such game is (usually) very large It should be solved by Backward Induction The following statements are always true in such games:
If the stage game has a unique Nash Equilibrium, then the Finitely Repeated Game has unique Subgame Perfect Equilibrium consisting of the repetition of that Nash Equilibrium in every round of the game
If the stage game has more than one Nash Equilibria, then any “reasonable” combination of those equilibria is a Subgame Perfect Equilibrium of the Finitely Repeated Game
Strategic Behavior in Business and Econ
Infinitely Repeated Games
We have seen that both players defecting is the unique subgame perfect equilibrium of the Finitely RepeatedPrisoners' Dilemma
C D
C 3,3 0 ,5
D 5 ,0 1, 1
Trigger Strategies do not lead to cooperation1. I will start with cooperation, and will mimic your behavior
afterwards (Tit-for-Tat)2. I will start with cooperation and will keep doing so unless
you defect. In such case I will defect forever (Grimm Trigger)
Strategic Behavior in Business and Econ
There are two possible interpretations of games that are repeated but not a fixed number of rounds
After each round the game continues to the next round with probability p and ends with probability (1 – p)
Example: Two firms compete day after day, but there is certain probability that one of them goesbankrupt and then the game is over
The game is played forever (an indefinite number of times) but at each round the value of the payoffs decreases by “p”
Example: Two people negotiate with offers and counteroffers over an item. As time goes by, the item loses
value. The game is over when they reach an agreement
Infinitely Repeated Games
Strategic Behavior in Business and Econ
The two different interpretations of a Infinitely Repeated Game are technically equivalent.
Since there is no “last round”, there is no possibility of thinking backwards. This opens real opportunities to achieve cooperation in the Prisoners' Dilemma !
In general, Infinitely Repeated Games are very complex
Infinitely Repeated Games
Strategic Behavior in Business and Econ
Let x be any positive number (for instance, money) and p any positive number smaller than 1 (for instance, a probability).Then,
x·p + x·p2 + x·p3 + x·p4 + · · · = x
x·p2 + x·p3 + x·p4 + x·p5· · · = x
and so on ...
Mathematical aside (infinite sums)
p
(1 - p)
p2
(1 - p)
Strategic Behavior in Business and Econ
The Infinitely Repeated Prisoners' DilemmaHow cooperation can be sustained in equilibrium ?
Payoff Computation: Imagine that Player 2 plays Grimm TriggerWhat is the (expected) payoff for Player 1 if after each round thegame continues with probability p (and ends with probability (1-p))
If Player 1 plays “cooperate” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·
3 3·p + 0·(1-p) 3·p2 + 0·(1-p2) · · · · = 3 +3p + 3p2 + 3p3 + · · ·
• If Player 1 plays “defect” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·
5 1·p + 0·(1-p) 1·p2 + 0·(1-p2) · · · · = 5 + 1p + 1p2 + 1p3 + · · ·
Strategic Behavior in Business and Econ
The Infinitely Repeated Prisoners' DilemmaHow cooperation can be sustained in equilibrium ?
Payoff Computation: Imagine that Player 2 plays Grimm TriggerWhat is the payoff for Player 1 if after each round thevalue of the money decreases by a factor of p (for instance, if themoney decreases a 10% then p=0.9)
Playing “cooperate” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·
3 3·p 3·p2 · · · · = 3 + 3p + 3p2 + · · ·3 3·(0.9) 3·(0.9)·(0.9) = 3 + 3·(0.9) + 3·(0.9)2 + · · . = 3 + 2.7 * 2.43 + · · ·
• Playing “defect” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·
5 1·p 1·p2 · · · · = 5 + 1p + 1p2 + · · ·5 1·(0.9) 1·(0.9)·(0.9) = 5 + 1·(0.9) + 1·(0.9)2 + · ·
= 5 + 0.9 + 0.81 + · · ··
Strategic Behavior in Business and Econ
The Infinitely Repeated Prisoners' DilemmaHow cooperation can be sustained in equilibrium ?
Imagine that Player 2 plays Grimm TriggerWhat is the best for Player 1, Cooperate or Defect ?
Expected Payoff from “cooperate” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·
3 3·p + 0·(1-p) 3·p2 + 0·(1-p2) · · · · = 3 + 3p + 3p2 + · · ·
• Expected Payoff from “defect” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·
5 1·p + 0·(1-p) 1·p2 + 0·(1-p2) · · · · = 5 + 1p + 1p2 + · · ·
Strategic Behavior in Business and Econ
The Infinitely Repeated Prisoners' DilemmaHow cooperation can be sustained in equilibrium ?
Imagine that Player 2 plays Grimm TriggerWhat is the best for Player 1, Cooperate or Defect ?
Expected Payoff from “cooperate” all the time
E(Cooperate) = 3 + 3p + 3p2 + 3p3 + · · · = 3 + 3
• Expected Payoff from “defect” all the time
E(Defect) = 5 + 1p + 1p2 + 1p3 + · · · = 5 + 1
p
(1 - p)
p
(1 - p)
Strategic Behavior in Business and Econ
The Infinitely Repeated Prisoners' DilemmaHow cooperation can be sustained in equilibrium ?
Imagine that Player 2 plays Grimm Trigger What is the best for Player 1, Cooperate or Defect ?
E(Cooperate) = 3 + 3
• E(Defect) = 5 + 1
Cooperate will be better if E(Cooperate) > E(Defect), that is, if
3 + 3 > 5 + 1 p > ½
p
(1 - p)
p
(1 - p)
p
(1 - p)
p
(1 - p)
Strategic Behavior in Business and Econ
The Infinitely Repeated Prisoners' DilemmaHow cooperation can be sustained in equilibrium ?
Thus, cooperation can be sustained in equilibrium in the Infinitely Repeated Prisoners' Dilemma thanks to Trigger Strategies
Depends on “p” With Tit-for-Tat it is also possible to sustain cooperation, but
then p > 2/3 But, “cooperation” is not the unique equilibrium. There are equilibria
with “defection” as well
Strategic Behavior in Business and Econ
Infinitely Repeated Games: General Facts
Any Infinitely Repeated Game that consists of the repetition of a stage game that is a static game produces a “Dynamic Game of Imperfect Information”
The tree representing such game is (usually) very large It can not be solved by Backward Induction The following statement is always true in such games:
No matter how many equilibria the stage game has, any “reasonable” combination of strategies can be a Subgame Perfect Equilibrium in such games (Folk Theorem)
Strategic Behavior in Business and Econ
C D
C 3,3 0 ,5
D 5 ,0 1, 1
Infinitely Repeated Games: General Facts
What does “reasonable” mean ?
Notice that by playing Defect all the time any player can guarantee himself a payoff of at least 1 per each round. Thus, any “reasonable” outcome of the game should pay each playerat least 1 per round
Strategic Behavior in Business and Econ
Summary
Any Repeated Game that consists of the repetition of a stage game that is a static game produces a “Dynamic Game of Imperfect Information”
The tree representing such game is very large If the game is Finitely Repeated, it must be solved by
Backward Induction If the game is Infinitely Repeated, it can not be solved by
Backward Induction
Strategic Behavior in Business and Econ
The following statements are always true in Finitely Repeated Games:: If the stage game has a unique Nash Equilibrium, then the
Finitely Repeated Game has unique Subgame Perfect Equilibrium consisting of the repetition of that Nash Equilibrium in every round of the game
If the stage game has more than one Nash Equilibria, then any “reasonable” combination of those equilibria is a Subgame Perfect Equilibrium of the Finitely Repeated Game
The following statement is always true in Infinitely Repeated Games: No matter how many equilibria the stage game has, any “reasonable” combination of strategies can be a Subgame Perfect Equilibrium in such games (Folk Theorem)
Summary
Strategic Behavior in Business and Econ
Axelrod's Simulation
R. Axelrod, The Evolution of Cooperation
Prisoner’s Dilemma repeated 200 times
Economists submitted strategies
Pairs of strategies competed
Winner: Tit-for-Tat
Reasons:
Forgiving, Nice, Clear
Strategic Behavior in Business and Econ
Not necessarily tit-for-tat
Doesn’t always work
Don’t be envious
Don’t be the first to cheat
Reciprocate opponent’s behavior
Cooperation and defection
Don’t be too clever
To be credible, incorporate a clear policy of punishment
Lessons from Axelrod’s Simulation