The Computational Complexity of Finding a Nash Equilibrium Edith Elkind, U. of Warwick.

The Computational Complexityof Finding a Nash Equilibrium

Edith Elkind, U. of Warwick

Based On…• Reducibility Among Equilibrium Problems

(Goldberg, Papadimitriou): Aug 2005 • The Complexity of Computing a Nash Equilibrium

(Goldberg, Daskalakis, Papadimitriou): Sep 2005

• 3-NASH is PPAD-Complete (Chen, Deng): Nov 2005

• Three-Player Games Are Hard (Daskalakis, Papadimitriou): Nov 2005

• Settling the Complexity of 2-Player Nash-Equilibrium (Chen, Deng): Dec 2005

Normal Form Games

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

• finite set of players {1, …, n}

• each player has k actions

(pure strategies): 1, …, k

• payoffs of the ith player: Pi: {1, …, k}n → R

Nash Equilibrium

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

• Nash equilibrium: a strategy profile such that

noone wants to deviate given other players’ strategies, i.e., each player’s strategy is a best response to others’ strategies:– (0, 0) and (1, 1) are both NE

Pure vs. Mixed Strategies

1 -1

-1 1

-1 1

1 -1

Row player:

Column player:

H

T

H T H T

H

T

• NE in pure strategies may not exist!– “matching pennies”

• Mixed strategy: a probability distribution over actions– 50% tail, 50% head

Existence of NE

• Theorem (Nash 1951): any game in normal form has an equilibrium in mixed strategies

$1 000 000 question:

how to find one?

Finding mixed NE in 2 x 2 Games

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

Suppose R plays 1 w.p. r

EP(C) from playing 0: (1-r)*1 EP(C) from playing 1: r*3

1-r = 3r iff r = ¼

Suppose C plays 1 w.p. c

EP(R) from playing 0: (1-c)*2EP(R) from playing 1: c*1

(1-c)*2 = c iff c = 2/3

NE: r=1/4, c=2/3

2 players, k actions

• Representation: two k x k matrices• Checking for pure NE: easy

– at most k2 of them

• Checking for mixed NE:– all straightforward methods are exptime– Lemke-Howson algorithm is exptime, too

(previous talk)

• For 2 players all NE are rational– but not for 3 and more players…

n players, 2 actions

• Representation: payoffs to each player for every action profile (vector in {0, 1}n): n2n numbers

• graphical games:– players are associated with the vertices of a graph;– each player’s payoff depends on his own action and

actions of his neighbors– n players, max degree d => n2d+1 numbers

TU

V

W t=0, u=0, v=0, w=0: 12t=1, u=0, v=0, w=0: 31 ….t=1, u=1, v=1, w=1: -6

W’s payoffs(16 cases):

Algorithms for NE in Graphical Games

• Bounded-degree trees:– Exp-time algorithm/poly-time approximation

algorithm to find all NE (Kearns, Littmann, Singh, UAI 2001)

– ??? poly-time algorithm to find a single NE (Kearns, Littmann, Singh, NIPS’2001):

• shown to be incorrect in E., Goldberg, Goldberg, ACM EC’06

• Graphs of max degree 2: – poly-time algorithm (EGG’06)

Is Finding NE NP-hard?

• Reminder: a problem P is NP-hard if you can reduce 3-SAT to it:– “yes”-instance 3-SAT → “yes”-instance of P– “no”-instance 3-SAT → “no”-instance of P

• Problem: each instance of NASH is a “yes”-instance!– every game has a NE– need complexity theory for search problems

• Side note: pure Nash for n players, NE of total value > K are NP-hard

Reducibility Among Search Problems

• S associates x in X with a solution set S(x)• Total search problem: for any x, S(x) is not empty

S: X Y

T: X’ Y’

If T is easy, so is S• S is reducible to T if:

– f, g easy to compute– g(T(f(x))) is in S(x)

f g

Equivalences: GP’05r-player game G NE of G

deg 3 graphical game G’ NE of G’

f g

d2-player game G’ NE of G’

deg d graphical game G NE of G

f g

• Color the graph of GG: d(u,v) ≤ 2 color(u) ≠ color(v)

• Each color is a player of G• RED chooses a red vertex in GG

and an action for that vertex in GG

• payoff=payoff1+payoff2

– payoff1: BLUE tries to guess which vertex RED chose; RED pays a penalty if BLUE guesses correctly

– payoff2: if all neighbors of a chosen vertex are also chosen, it gets same payoff as in GG, else 0

d-Graphical Game GG → d2-Player Game G

r-Player Game G → 3-Graphical Game GG

• Si: space of pure strategies of player i

• S- i = S1 * … Si-1*Si+1 *.. * Sr

• xij: the probability that ith player uses jth strategy

• xs: x1s1 * x2

s2 … * xrsr (for s in S-i)

• uijs: utility of the ith player when he plays j and others play

according to s

-p

NE: 0 ≤ xij ≤ 1

j xij =1

s in S uijsxs > s in S ui

j’sxs implies xij’ = 0-p

r-Player Game G → 3-Graphical Game GG

• Vertex Vij for any pair (player=i, action=j)

• Want: Pr[Vij plays 1] = Pr [i plays j in G]=xij

• Idea: graphical games can do math!– Enforce constraints from the previous slide…

v1 v2 v3

u

Set payoffs to u, v3 so that p[v3]=p[v1] * p[v2]

Need gadgets for +, *, c, =, min, max, …

Equivalences: GP’05r-player game G NE of G

deg 3 graphical game G’ NE of G’

f g

d2-player game G’ NE of G’

deg d graphical game G NE of G

f g

Combining Reductions: GP’05

r-player game G NE of G

9-player game G’ NE of G’

f g

Finding NE in a 4-player game is as hard as

finding NE in a r-player game for any constant r

X4

Completeness Results?

• Can we prove that any total search problem is reducible to r-NASH?

• Not really: the class T of all total search problems is a semantic class– not known how to find complete problems for these

• Want to pick a large subclass S of T s.t.– S includes some natural problems– there are problems that are complete for S– in particular, r-NASH is complete for S

• Input: Boolean circuits S (Successor), P (Predecessor):– n inputs, n outputs– S(0n) ≠ 0n, P(0n) = 0n

• Output: x ≠ 0n s.t. – S(P(x)) ≠ x or P(S(x)) ≠ x

Intuition: G=(V, E): – V = n; – E = {(x,y) | y=S(x), x=P(y)}

END OF THE LINE

00000

01011

11001

01011

PPAD

• PPAD: Polynomial Parity Argument, Directed version

• PPAD is the class of all search problems that are reducible to END OF THE LINE

search problem solution

circuits S, T “end of the line”f g

r-NASH is in PPAD

• Proof on Nash’s theorem:– existence of NE reduces to Brouwer’s fixpoint

theorem– Brouwer’s fixpoint theorem reduces to

Sperner’s lemma– Sperner’s lemma is proven by a parity

argument (similar to END OF THE LINE)

• Reduction of r-NASH to END OF THE LINE can be extracted from these proofs (Papadimitriou 94)

Brouwer’s Fixpoint Theorem

• Brouwer’s Theorem: Any continuous mapping from the simplex to itself has a fixpoint.

• Nash Brouwer proof sketch:– set of all strategy profiles → simplex– mapping: (s1, …, sn) → (s1+1, …, sn+n),

where i is a shift in the direction of best response to (s1, …, si-1, si+1, …, sn)

– NE is a point where noone wants to deviate, i.e., a fixpoint

• Proper coloring:– vertices on BC are not blue– vertices on AC are not green– vertices on AB are not yellow

• Sperner’s Lemma: there exists a trichromatic triangle

• Brouwer’s theorem Sperner’s Lemma:– x is blue if the grad(F) at x points away from A, etc.– trichromatic triangle “has no direction” – repeat at increased resolution…

Sperner’s Lemma

A

B

C

Opposite Direction: 3D-BROUWER

• Input:– 3D unit cube divided into 23n cubelets– cijk is the center of Kijk

– (cijk)=cijk+ijk, ijk is in {0, 1, 2, 3}, where• 1=(, 0, 0), 2=(0, , 0)• 3=(0, 0, ), 0=(-, -, -)

– circuit C: {0, 1} 3n → {0, 1, 2, 3} selects ijk

• Output: – a panchromatic cubelet, i.e., one that has all

of 0, 1, 2, 3 among its 8 neighbors

3D-BROUWER is PPAD-complete

• Papadimitriou (1994) shows that a more complicated version of 3D-BROUWER is PPAD-complete

• This version was proven hard in DGP’05

• Reduction from END OF THE LINE– embed the line L into 3d cube– “protect” L from color 0 using three other colors– color the rest of inner cubelets with 0

r-NASH vs 3D BROUWER

• Existence of NE follows from Brouwer’s fixpoint theorem

• NE are special cases of Brouwer’s fixpoints– just how special?

• Can any fixpoint be represented as a NE of a game?

• DGP’05: YES! 4-NASH is PPAD complete

• Proof: – 4-NASH deg 3 Graphical Nash– graphical games can compute fixpoints

4-NASH to 3-NASH

• Daskalakis, Papadimitriou: modify arithmetic gadgets so that the graph is 3-colorable

• Chen, Deng: same gadgets, but allow for small error

2-NASH

• Chen, Deng: – avoid graphical games– reduce directly from 3D-BROUWER to

2-NASH using arithmetic gadgets similar to graphical game gadgets

• Game over?

Graphical Games: Open Problems

• Degree:– deg 3 PPAD-complete (DGP’05b)– deg 2 polynomial time solvable (EGG’06)

• Pathwidth:– paths: poly-time– pathwidth 1: maybe algorithm from EGG’06 still works– pathwidth 2: any KLS-style algo is exptime (EGG’06)– pathwidth > r, r constant: PPAD-complete (EGG’06)

• Finding NE on trees?