Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work.

Graphical Models forStrategic and Economic

Reasoning

Michael KearnsComputer and Information Science

University of Pennsylvania

BNAIC 2003

Joint work with: Sham Kakade, John Langford,Michael Littman, Luis Ortiz, Satinder Singh

Probabilistic Reasoning

• Need to model a complex, multivariate distribution• Dimensionality is high --- cannot write in “tabular” form• Examples: joint distributions of alarms and earthquakes,

diseases and symptoms, words and documents

• The world is not arbitrary:– Not all variables (directly) influence each other– True for both causal and stochastic influences– Many probabilistic independences hold– Interaction has (network) structure– Should ease modeling and inference

• The answer: graphical models for probabilistic reasoning

Structure in Probabilistic Interaction

[Frey&MacKay 98]

[Horvitz 93]

Engineered “Natural”

International Trade

[Krempel&Pleumper]

Embargoes, free trade, technology, geography…

Corporate Partnerships

[Krebs]

Internet Connectivity

[CAIDA]

Structure in Social and Economic Analysis

• Trade agreements and restrictions• Social relationships between business people• Reporting and organizational structure in a firm• Regulatory restrictions on Wall Street• Shared influences within an industry or sector• Geographical dispersion of consumers• Structural universals (Social Network Theory)

Goal: Replicate the power of graphical modelsfor problems of strategic reasoning.

• Strategic Reasoning:– Variables are players in a game, organizations, firms, countries…– Interactions characterized by self-interest, not probability– Foundations: game theory and mathematical economics

Outline

• Graphical Games and the NashProp Algorithm– [K., Littman & Singh UAI01]; [Ortiz & K. NIPS02]

• Correlated Equilibria, Graphical Games, and Markov Networks– [Kakade, K. & Langford EC03]

• Arrow-Debreu and Graphical Economics– [Kakade, K. & Ortiz 03]

Graphical Games and NashProp

Basics of Game Theory

• Have players 1,…n (think of n as large)• Each has actions 1,…,k (think of k as small)• Action chosen by player i is a_i • Vector a is population joint action• Player i receives payoff M_i(a)• (Note: M_i(a) has size exponential in n!)

• (Nash) equilibrium: – Choice of mixed strategies for each player– No player has a unilateral incentive to deviate– Mixed strategy: product distribution over a

• Exists for any game; may be many

Graphical Models for Game Theory

• Undirected graph G capturing local (strategic) interactions• Each player represented by a vertex• N_i(G) : neighbors of i in G (includes i)• Assume: Payoffs expressible as M_i(a’), where a’ over only N_i(G)• Graphical game: (G,{M’_i})• Compact representation of game; analogous to graph + CPTs• Exponential in max degree (<< # of players)• As with Bayes nets, look for special structure for efficient inference• Related models: [Koller & Milch 01] [La Mura 00]

2

4

3

5

8

7

6

1

The NashProp Algorithm

• Message-passing, tables of “conditional” Nash equilibria• Approximate (all NE) and exact (one NE) versions, efficient for

trees • NashProp: generalization to arbitrary topology (belief prop)• Junction tree and cutset generalizations [Vickrey & Koller 02]

U1 U2 U3

W

V

T(w,v) = 1 <--> an “upstream” Nash where V = v given W = w <--> u: T(v,u_i) = 1 for all i, and v is a best response to u,w

• Table dimensions are probability of playing 0• Black shows T(v,u) = 1• Ms want to match, Os to unmatch• Relative value modulated by parent values• =0.01, = 0.05

Experimental Performance

number of players

com

pu

tati

on

tim

e

Correlated Equilibria, Graphical Games and Markov Networks

The Problems with Nash

• Technical:– Difficult to compute (even in 2-player, multi-action

case)

• Conceptual:– Strictly competitive– No ability to cooperate, form coalitions, or bargain– Can lead to suboptimal collective behavior

• Fully cooperative game theory:– Somewhat of a mathematical mess

• Alternative: correlated equilibria

Correlated Equilibria in Games[Aumann 74]

• Recall Nash equilibrium is a product distribution P(a)• Suffices to guarantee existence of equilibrium• Now let P(a) be an arbitrary distribution over joint

actions• Third party draws a from P and gives a_i to player i • P(a) is a correlated equilibrium:

– Conditioned on everyone else playing P(a|a_i), playing a_i is optimal– No unilateral incentive to deviate, but now actions are correlated– Reduces to Nash for product distributions

• Alternative interpretation: shared randomness• Everyday example: traffic signal

Advantages of CE

• Technical:– Easier to compute: linear feasibility formulation– Efficient for 2-player, multi-action case

• Conceptual:– Correlated actions a fact of the real world– Allows “cooperation via correlation”– Modeling of shared exogenous influences– Enlarged solution space: all mixtures of NE, and more– New (non-Nash) outcomes emerge, often natural ones– Avoid quagmire of full cooperation and coalitions– Natural convergence notion for “greedy” learning

• But how do we represent an arbitrary CE?– First, only seek to find CE up to (expected) payoff

equivalence– Second, look to graphical models for probabilistic reasoning!

Graphical Games and Markov Networks

• Let G be the graph of a graphical game (strategic structure)• Consider the Markov network MN(G):

– Form cliques of the local neighborhoods of G

– Introduce potential function c on each clique c

– Joint distribution P(a) = (1/Z) c c(a) • Theorem: For any game with graph G, and any CE of this game,

there is a CE with the same payoffs that can be represented in MN(G)

• Preservation of locality• Direct link between strategic and probabilistic reasoning in CE• Computation: In trees (e.g.), can compute a CE efficiently

– Parsimonious LP formulation

From Micro to Macro:Arrow-Debreu and Graphical

Economics

Arrow-Debreu Economics• Both a generalization (continuous vector actions) and

specialization (form of payoffs) of game theory• Have k goods available for consumption• Players are:

– Insatiable consumers with utilities for amounts of goods– “Price player” (invisible hand) setting market prices for goods

• Liquidity emerges from sale of initial endowments– Alternative model: labor and firms

• At equilibrium (consumption plans and prices):– Each consumer maximizing utility given budget constraint– Market clearing: supply equals demand for all goods– May also allow supply to exceed demand at 0 price (free disposal)

• ADE always exists• Very little known computationally

– [Devanur, Papadimitriou, Saberi, Vazirani 02]: linear utility case

Graphical Economics

• Again wish to capture structure, now in multi-economy interaction

• Represent each economy by a vertex in a network– “Economies” could be represent individuals or sovereign nations– From international relations to social connections

• Same goods available in each economy, but permit local prices• Interpretation:

– Allowed to shop for best prices in neighborhood– Utility determined only by good amounts, not their sources

• Stronger than ADE: graphical equilibrium– Consumers still maximize utility under budget constraints– Local clearance in all goods (domestic supply = incoming demand)

2

4

3

5

8

7

6

1

Graphical Economics: Results

• Graphical equilibria always exist (under ADE condition analogues)– does not follow from AD due to zero endowments of foreign goods– appeal to Debreu’s quasi-rationality: zero wealth may ignore zero

prices– Wealth Propagation Lemma: spread of capital on connected graph– relative gridding of prices and consumption plans

• ADProp algorithm:– computes controlled approximation to graphical equilibrium– message-passing on conditional prices and inbound/outbound demands– efficient for tree topologies and smooth utilities

2

4

3

5

8

7

6

1

Conclusion• Use of game-theoretic and economic models rising

– Evolutionary biology– Behavioral game theory and economics NYT 6/17– Neuroeconomics– Computer Science– Electronic Commerce

• Many of these uses are raising– Computational issues– Representational issues

• Well-developed theory of graphical models for GT/econ– Structure of interaction between individuals and organizations

• What about structure in– Utilities, actions, repeated interaction, learning, states,…

[email protected]/~mkearns