Simple search methods for finding a Nash equilibrium Ryan Porter, Eugene Nudelman, and Yoav Shoham Games and Economic Behavior, Vol. 63, Issue 2. pp. 642-661,

Simple search methods for finding a Nash equilibriumRyan Porter, Eugene Nudelman, and Yoav Shoham

Games and Economic Behavior, Vol. 63, Issue 2. pp. 642-661, 2004

Georgia Kastidou David R. Cheriton School of Computer Science

University of Waterloo

Outline

Problem Contribution Algorithm for 2-player game

Experimental Results Algorithm for n-player game

Experimental Results Conclusions Future Work Take Home Message

Problem we will tackle… Consider a n-player normal-form game G=N, (Ai), (ui) where:

N=[1,..n] is the set of players Ai=[ai1,…,aimi] is the set of actions available to player i ui:A1x…xAnR is the utility function for each player Each player selects a mixed strategy from the set of available strategies:

“Support” of a mixed strategy pi is the set of all actions ai in Ai such that pi(ai)>0. x=(x1,..,xn): xi is the size of agent’s i support

The expected utility for player i for a strategy profile p=(p1,...,pn) is:

where:

Problem: Design an algorithm which can find a Nash Equilibrium for a normal form game A strategy profile p* in P is a Nash Equilibrium (NE) if:

ii Aa

iiiii apApP 1)(|]1,0[:

Aa

iii auappu )()()(

Ni

iii apap )()(

),(),(:, ***iiiiiiii ppupauAaNi

Development of Algorithms Two extremes:

1st: Aims to design a low complexity algorithm Gain deep insight in the structure of the problem Design highly specialized algorithms

2nd: Aims to design more simple algorithms with an “acceptable” complexity Identify shallow heuristics Hoping that given the increasing computing power they will be sufficient. focus on more “common” problems

Which one is better? Although the first is more interesting, in practice in a number of cases the

second is preferred either because is simpler or because it outperforms the first. Note: There are cases of optimal algorithms that have never been implemented because they are too

complicate.

Background/Related Work

Nash equilibrium is an important concept in game theory “little is known about the problem of computing a sample

NE in a normal-form game”

Normal form game is guaranteed to have at least one NE

It does not fall into a standard complexity class (Papadimitriou, 2001) it cannot be cast as a decision problem.

Background/Related Work 2-player game:

Lemke-Howson, (1964) Dickhaut and Kaplan (1991)

Enumerates all possible pairs of supports for 2-play game For each pair it solves a feasibility program

n-player game: Simplicial Subdivision (van der Laan et al., 1987)

Approximates a fixed point of a function which is defined on a simplotope

Govindan and Wilson (2003) first perturbs a game to one that has a known equilibrium, and then traces the solution back to the original game as the magnitude of the

perturbation approaches zero.

Background/Related Work

GAMUT: a “recently” (2004) introduced computational testbed for game theory

“Run the GAMUT:A Comprehensive Approach to Evaluating Game-Theoretic Algorithms” by E. Nudelman et al.

What the author propose and what’s their contribution?

Propose: heuristic-based algorithms for 2-player games and for

n-player games explore the space of support profiles using a backtracking procedure

to instantiate the support for each player separately. test using a variety of different distributions

Use of GAMUT (computational testbed for game theory)

Contribution: in a big number of cases the proposed algorithms

outperform the algorithm of Lemke-Howson on 2-player games and the Siplicial Subdivision on n-player games.

The Proposed Algorithms explore “support” profiles:

pure strategies played with nonzero probability

use backtracking procedures

are biased towards simple solutions preference for small supports based on the observation that a number of games in

the past proved to have at least one simple solution. e.g. for n = 2, the probability that there exists a NE consistent with

a particular support profile varies inversely with the size of the supports, and is zero for unbalanced support profiles.

Proposed Algorithm for 2-players game

Proposed Algorithm for 2-players game

υi : expected utility of agent i in an equilibrium

Proposed Algorithm for 2-players game

The first two classes of constraints require that: each player must be indifferent between

all actions within his/her support, and must not strictly prefer an action outside

of his/her support.

Experimental results The authors consider games from a number of different

distributions

D18: most common one D5, D6, and D7 are also important distributions

Experimental ResultsAlgorithms for 2-player games

Experiment-Setup: 2-player, 300-action games drawn from 24 of GAMUT’s 2- player

distributions. executed on 100 games drawn from each distribution.

First diagram: compares the unconditional median running times of the algorithms, might reflect the fact that there is a greater than 50% chance that the

distribution will generate a game with a pure

Second diagram: Compares the percentage of instances solved

Third diagram: the average running time conditional on solving an instance

Experimental ResultsAlgorithms for 2-player games

Compares the unconditional median running times of the algorithms.

(“Might reflect the fact that there is a greater than 50% chance that the distribution will generate a game with a pure”)

Experimental ResultsAlgorithms for 2-player games

Compares the percentage of instances solved

Experimental ResultsAlgorithms for 2-player games

Compares the average running time conditional on solving an instance

(unconditional average running time)

Experimental ResultAlgorithms for 2-player games

Compare the scaling behavior as the number of actions increases(unconditional average running time)

Experimental ResultAlgorithms for 2-player games

Covariance Games neither of the algorithms solved any of the games in another

“Covariance Game” distribution in which ρ =−0.9,

Proposed Algorithm for n-players games

Uses a general backtracking algorithm to solve a constraint satisfaction problem (CSP) for each support size profile

The variables in each CSP are: the supports Si , and the domain of each Si is the set of supports of size xi.

Constraints: no agent plays a conditionally dominated action.

Proposed Algorithm for n-players games

IRSDS: Input a domain for each player’s support.

For each agent whose support has been instantiated the domain contains only that instantiated support,

For each other agent i it contains all supports of size xi that were not eliminated in a previous call to this procedure.

On each pass of the repeat-until loop, every action found in at least one support in a player’s domain is checked for

conditional domination. If a domain becomes empty after the removal of a conditionally dominated

action, the current instantiations of the Recursive-Backtracking are inconsistent, and IRSDS

returns failure.

IRSDS repeats until it either returns failure or iterates through all actions of all players without finding a dominated action.

Proposed Algorithm for n-players games

IRSDS

R-B

IRSDS

R-B

IRSDS

R-B

IRSDS

R-B

…

…

IRSDS

R-B

R-B: Recursive Backtracking

IRSDS: Iterated Removal of Strictly Dominated Strategies

Failed

IRSDS

RBT

Algorithm 2

Failed

R-B

IRSDS

R-B

For all x=(x1,..xn) sorted in increasing order first by:

and then by:

i

ix

)(max , jiji xx

i

ix

Experimental Results: n-player games

Experiment-Setup: 6-player, 5-action games drawn from 22 of GAMUT’s n-player distributions.

15,625 outcomes and 93,750 payoffs executed on 100 games drawn from each distribution.

First diagram: compares the unconditional median running times of the algorithms, might reflect the fact that there is a greater than 50% chance that the

distribution will generate a game with a pure

Second diagram: Compares the percentage of instances solved

Third diagram: the average running time conditional on solving an instance

Experimental Results: n-player games

Compares the unconditional median running times of the algorithms

Compares the percentage of instances solved

Experimental Results: n-player games

Compares the average running time conditional on solving an instance

Experimental Results: n-player games

Compare the scaling behavior: number of players constant at 6 number of actions varies.

(unconditional average running time)

Compare the scaling behavior: number of players varies, number of actions constant 5.

(unconditional average running time)

Experimental Results Percentage of Pure Strategy NE

(2-player game) n-player game

Experimentals Results Average measure of support balanced

2-player, 300-action games 6-player, 5-action games

Conclusions

Propose algorithms that use backtracking approaches to search the space of support profiles, favoring supports that are small and balanced.

Both algorithms outperform the current state of the art.

The most difficult games “Covariance Game” model, as the covariance approaches

its minimal value hard because authors found that:

as the covariance decreases, the number of equilibria decreases, and

the equilibria that do exist are more likely to have support sizes near one half of the number of actions

Future Work

Employ more sophisticated CSP techniques

Explore local search, in which the state space is the set of all possible supports, and the available moves are to add or delete an action from the support of a player

Study the games that are generated by the Covariance Game distribution

Take home message

Studying the results of complicated problems can lead to observations that although might not provide ideas to find optimal solutions can provide insights on how to improve current approaches.

The selection of the tests and the parameters that will be examined very important. Not only because they can show that your algorithm is

working… E.g. “Covariance Game” model might proved a good starting

point for game theoretic algorithms