Software Agents in Economic Environments Robert S. Gazzale Ph.D. Candidate, Department of Economics Jeffrey MacKie Mason Professor, Dept. of Economics.

Software Agents in Economic Environments

Robert S. GazzalePh.D. Candidate, Department of Economics

Jeffrey MacKie MasonProfessor, Dept. of Economics & School of Information

CARAT 2002/2003

April 9, 2003

April 9, 2003 CARAT: Gazzale & MacKie Mason

Funding: Past & Present

Many thanks to:

CARAT

NSF

IBM


Collaborators Chris Brooks Computer Science, University of San Francisco

Yan Chen School of Information

Rajarshi Das IBM Institute for Advanced Commerce

Ed Durfee AI Lab, EECS

Jeff Kephart IBM Institute for Advanced Commerce


A Model of Economic Modeling

Environment

Outcomes



Environment

Outcomes



Environment

Outcomes


A Model of Economic Modeling: Alternative View

Environment

Outcome


A Model of Economic Modeling:Which Mapping?

Environment

Outcomes


Equilibrium: The Mapping from Environment to Outcome

One Agent Environment Optimal action

Non-cooperative Games Nash Equilibrium

Given what everybody else is doing, no one agent can change strategy to do better.


John Nash?


Problem: Finding Equilibria

Is it solvable? If so, will agents find it? Bounded Rationality (Herbert Simon)

Cognition is not free. Satisfice rather than optimize?


Problem: Out of equilibrium matters!

Particularly if agents are boundedly rational Do we get to equilibrium? What happens on path to equilibrium?


More Problems with “Equilibrium”

Which Equilibrium? If there are many equilibria, which is

going to happen when?


Why Software Agents?

Useful in alleviating equilibrium issues. Cheap. Present/Future of software agents in real markets Particularly where equilibrium not

solvable.


Application 1: Convergence to Equilibrium

Nash

Equilibrium

Theory: Supermodular (SPM) games played by learning agents converge to Nash Equilibrium



Nash

Equilibrium

Theory: Supermodular (SPM) games played by learning agents converge to Nash Equilibrium

Trust me, you don’t need to know what this is!



Nash

Equilibrium

Theory makes no predictions if NOTSupermodular.



Nash

Equilibrium

Is more supermodular “better”?



Nash

Equilibrium

Answers to these questions important in designing markets!


Convergence to Equilibrium: Human Experiment

Methodology Design game where parameter

controls whether or not game is SPM Laboratory experiments with human

subjects playing for real money!


Convergence to Equilibrium: Human Experiment

Methodology Design game where parameter

controls whether or not game is SPM Laboratory experiments with human

subjects playing for real money!


Convergence to Equilibrium: Human Experiment Results

Problem: Dynamics not complete with human subject experiments (60 rounds)



Efficiency of Outcome:Experimental Data

40%

50%

60%

70%

80%

90%

100%

0 10 20 30 40 50 60 Round

Fracti

on

of

Maxim

um

Welf

are

a20b00a20b18a20b20a20b40a10b20




40%

50%

60%

70%

80%

90%

100%

0 10 20 30 40 50 60 Round

Fracti

on

of

Maxim

um

Welf

are

a20b00a20b18a20b20a20b40a10b20

Will this treatment catch-up?




40%

50%

60%

70%

80%

90%

100%

0 10 20 30 40 50 60 Round

Fracti

on

of

Maxim

um

Welf

are

a20b00a20b18a20b20a20b40a10b20

Will this treatment catch-up?

Will any of these pull ahead?


Software agents to complete dynamics

Methodology Select various learning models Endow agents with these models Calibrate models with actual data Compare calibrated learning models Endow pool of agents with best model

and let run!


Software agents to complete dynamics

Use of computation power For each learning model and treatment

For each set of parameters (1100 sets) 12 agents play in each iteration for 60 rounds 1500 iterations of game

8,910,000,000 “decisions” in <6 hours! Select Parameters that most-closely fit

data. For best learning model

12 agents each iteration for 1000 rounds 1500 iterations of game


Simulation ResultsSimulated Results

70%

75%

80%

85%

90%

95%

100%

50 150 250 350 450 550 650 750 Round

Fra

cti

on

of

Maxim

um

Welf

are

a20b00a20b18a20b20a20b40a10b20

“Never” does catch up!

Pulls ahead for a short while!


Application 2: Agents Solving Difficult Problems

Many problems without analytical solution Natural domain for use of computer science methods to find optimum Many are “hill-climbing” methods

Economics needs to inform these solutions


A not so hard problem for an agent . . .

No matter where we start, rather easy to get to the summit!


A more difficult landscape . . .

Tough to get from here

to here

BundlePrice

Per-articlePrice


A more difficult landscape . . . Made a little easier . . .

BundlePrice

Per-articlePrice

Use economic knowledge to:

reduce the search space!



select better starting values!Bundle

Price

Per-articlePrice




FeePer-articlePrice




FeePer-articlePrice


supply gradient information!


Problem 1: Pricing Problem

Firm Sells Information Goods Consumer demand uncertain Many different pricing schedules possible General rule: Higher profits from schedules that are harder to learn. What schedule? No analytical solution!


The Pricing Problem: Results

adjusting Linear Two-part NonlinearBlock

Adaptive uses knowledge to move among schedules!


Problem 2: Battle of the Agents

Highly complex environment Large Search space Actions of competitor warp my landscape. Result: Computer science algorithms, without economic knowledge, perform quite poorly


Computer algorithm, no economic knowledge

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

x 104

0. 5

0

0.5

1

1.5

2

2.5

3

3.5

4x 10

4

Iteration

Average Profit per Iteration for Zero knowledge Producers

Equilibrium Profit

Equilibrium


Computer algorithm, with economic knowledge

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

x 104

0

0.5

1

1.5

2

2.5

3

3.5

4x 10

4

Iterations

36000

zero-knowledge

gradient

tremble

noisenoise+adjacency

number of categories

equilibrium profit

Gradient info

}

Reduce search space

}

Better starting values


That’s all folks!

Software Agents in Economic Environments Robert S. Gazzale Ph.D. Candidate, Department of Economics Jeffrey MacKie Mason Professor, Dept. of Economics.

Documents

nash equilibrium slide

gazzale mackie mason

gazzale mackie mason

gazzale mackie mason

equilibrium matters

nash equilibrium trust

carat nsf ibm

alleviating equilibrium