1 Market Based Control of Complex Computational Systems Nick Jennings [email protected].

1

Market Based Control of Complex Computational Systems

Nick Jennings

[email protected]

2

The Complex Systems Challenge

Building software that operates effectively in environments that:– Have no centralised control– Are highly interconnected– Are in constant state of flux– Are highly unpredictable– Involve multiple, individually-motivated actors

3

The Complex Systems Landscape

Grid Computing

Semantic WebWeb Services

Agent Based Computing

Service descriptionService discoveryService composition

Flexible interoperation &reasoning in heterogeneous

environments

Robust, large scale open systems

AutonomyRich interactions

“Brain meets Brawn”

Semantic integration

SemanticGrid

OGSA uses WSstandards

PervasiveComputing

Peer-to-Peer

eCommerce

AutonomicComputing

4

• Entities offer services in an institutional setting

• Entities connect to services– Service discovery

– Service composition

– Service procurement

• Entities enact services– Flexible & context sensitive

service delivery

The Computational Model

Agent

EnvironmentSphere of influence

Electronicinstitution

Interaction

Organisationalrelationship

(Jennings, 2000 & 2001)

5

“encapsulated computer system, situated in some environment, and capable

of flexible autonomous action in that environment in order to meet its objectives”

Agents as Service Providers & Consumers

6

“encapsulated computer system, situated in some environment, and capable

of flexible autonomous action in that environment in order to meet its objectives”

Agents as Service Providers & Consumers

• control over internal state and over own behaviour

7

“encapsulated computer system, situated in some environment, and capable

of flexible autonomous action in that environment in order to meet its objectives”

Agents as Service Providers & Consumers

• control over internal state and over own behaviour

• experiences environment through sensors and acts through effectors

8

“encapsulated computer system, situated in some environment, and capable

of flexible autonomous action in that environment in order to meet its objectives”

Agents as Service Providers & Consumers

• reactive: respond in timely fashion to environmental change• proactive: act in anticipation of future goals

• control over internal state and over own behaviour

• experiences environment through sensors and acts through effectors

9

Negotiation as de facto Form of Interaction

Agree appropriate service contracts– Service composition– Service procurement

• Fixed price offerings– Catalogues

• Dynamic pricing– Negotiations– Auctions

Historical precedentEconomic efficiency

10

permissible participants e.g. buyers, sellers & third parties

interaction states e.g. accepting bids, auction closed

events causing state transitions e.g. bid, time out, bid accepted

valid actionsbid, ask, propose, accept, reject,

counter-proposal, critique

reward structureswho pays & who gets paid for what

Computational Service Economies

Mechanism Design

“rules of the game”

(Dash et al., 2003)

11

Computational Service Economies

shaped by interaction protocol

decision making employed to achieve trading objectives

– from very simple to very complex

maximise benefit– to self (self interest) and/or– to group (social welfare)

Mechanism Design

“rules of the game” “how to succeed in the game”

Agent Strategies

(Dash et al., 2003)

permissible participants e.g. buyers, sellers & third parties

interaction states e.g. accepting bids, auction closed

events causing state transitions e.g. bid, time out, bid accepted

valid actionsbid, ask, propose, accept, reject,

counter-proposal, critique

reward structureswho pays & who gets paid for what

12

Computational Service Economies

shaped by interaction protocol

decision making employed to achieve trading objectives

– from very simple to very complex

maximise benefit– to self (self interest) and/or– to group (social welfare)

Mechanism Design

“rules of the game” “how to succeed in the game”

Agent Strategies

(Dash et al., 2003)

permissible participants e.g. buyers, sellers & third parties

interaction states e.g. accepting bids, auction closed

events causing state transitions e.g. bid, time out, bid accepted

valid actionsbid, ask, propose, accept, reject,

counter-proposal, critique

reward structureswho pays & who gets paid for what

Game theory analyses interactions to determine likely outcomes and

equilibria

13

The Market-Based Control Project• Market-Based Control (MBC):

– paradigm for controlling computer systems using economically-inspired techniques

• Market mechanisms used to: – generate and predict emerging system properties,

• although decisions are made independently by local agents that each have their own aims and objectives

– a market is a self-organising system, directed by mechanism

• The proposition:– MBC is good for effectively controlling and managing

complex, adaptive, distributed computational systems

14

Objectives• Develop and evaluate core MBC technologies

• Automated mechanism design– Automate design of market mechanisms to achieve

a desired set of global goals– Adapt to a changing environment and changing

(priority of) objectives– Predict and automate design of agent strategies

• Apply MBC solutions to design and manage complex, distributed computational systems

15

Project Applications

• Utility data centres– MBC to allocate computational resources & achieve

a robust, scalable service

• Distributed content delivery within p2p networks– MBC to regulate sharing of content

• Decentralised control and scheduling of multiple robots– MBC to provide incentives for cooperation and to

achieve global goals

16

Research Highlights

• Competing sellers in online auctions• Strategies for bidding in multiple auctions• Empirical game theory to select mechanisms

and strategies for complex markets• Adaptive auctions

17

Research Highlights

• Competing sellers in online auctions• Strategies for bidding in multiple auctions• Empirical game theory to select mechanisms

and strategies for complex markets• Adaptive auctions

18

• Often strong competition among sellers in online auctions

– How many eBay auctions yesterday?A) 10

B) 100

C) 1000

19

• Often strong competition among sellers in online auctions

• Seller’s choice of mechanism & auction parameters affect buyer’s choice of seller– How should bidder choose between auctions/sellers?– How should a seller set its parameters?

• Focus on seller’s reserve price & sealed-bid auctions

20

Mediator

Model of Competing Sellers

Auction Auction

Buyers

SellerSeller

• Set & announce Auction Fees

• Set & announce Reserve Price

• Select seller• Bid in auctions

Seller

Auction

21

Shill Bidding• Competing sellers reduces optimal reserve price and

expected revenue (compared to isolated auctions)• Avoid by shill bidding:

– Seller disguised as buyer to bid in own auction.

• Illegal and undesired, but hard to detect – But mediator can use auction fees to deter it

• Use Evolutionary Simulation to:– Evaluate effectiveness of different types of auction fees in

deterring shill bidding– Measure market efficiency

22

Results with Auction FeesFraction of auctions

won by shill bidsAllocative efficiency

CP= closing priceRD = difference between reserve and closing prices

23

Observations

• Competition among sellers affects choice of mechanism and auction parameters– Important to take competition into account when designing

mechanisms and bidder strategies

• Sellers can decide to shill bid in order to improve profits

• Mediator (such as eBay) can deter shill bidding and increase efficiency by setting appropriate auction fees

24

International Competition

• Made proposal to have new game in the Trading Agents Competition Foundation– TAC Market Design

• “Reverse” Trading Agents Competition

– Design mechanisms with varying:• Clearing policy• Information revelation policy• Auction fees

25

Research Highlights

• Competing sellers in online auctions• Strategies for bidding in multiple auctions• Empirical game theory to select mechanisms

and strategies for complex markets• Adaptive auctions

26

Bidding in Multiple Auctions

• Different start/finish times– Simultaneous, sequential, or hybrid

• Heterogeneous: – N single-unit auctions– 1st/2nd price sealed bid, English or Dutch– Each can have different number of bidders

• Multiple items

Find optimal best response

simultaneous

sequential

hybrid

27

Heuristic Strategies• Setting too complex to analyse theoretically

and find optimal strategies

• Heuristic strategies:– Choose the thresholds

• Single auction dominant strategy (DOM)• Equal threshold (EQT)

– Choose the auction• Exhaustive search (ES)• Knapsack utility approximation search (KS)

• Trade-off between speed and complexity

28

• Heuristics close to optimal for this restricted case

• EQT better than DOM

• KS much more computationally efficient than ES

29

Research Highlights

• Competing sellers in online auctions• Strategies for bidding in multiple auctions• Empirical game theory to select mechanisms

and strategies for complex markets• Adaptive auctions

30

Empirical Game Theory• Game Theory is a mathematical theory which underpins

auction- and mechanism-design– very powerful and, at least in theory, can tell us what are the optimal

mechanism and strategies.

• But some markets too complex to analyse in practice using game theory. – too many participants and too many possible moves.

• Evolutionary methods do not always converge on robust strategies

• Empirical Game Theory: – emerging field combines principled game-theoretic analysis together

with computer simulation methods.– amenable to automation, so it may be used by agents themselves to

decide on market mechanisms.

31

Empirical Game Theory

• Analysing strategies in Double Auctions

• Find payoffs for strategies by repeated simulations

• Find mixture of these “pure” strategies that constitute a evolutionary game-theoretic equilibrium

32

Research Highlights

• Competing sellers in online auctions• Strategies for bidding in multiple auctions• Empirical game theory to select mechanisms

and strategies for complex markets• Adaptive auctions

33

Discrete Bid English Auctions

Fixed bid increment

34

Research Questions

• What effect do these discrete bid levels have on the auction properties?

• How should the auctioneer determine the discrete bid levels to use in any situation in order to maximise his revenue?

35

E º =mX

i=0

eº [F (l i + 1)¡ 1] ¡ eº [F (l i )¡ 1]

F (li+1) ¡ F (li )

hli£1¡ F (li )

¤¡ li+1

£1¡ F (li+1)

¤i

• We calculate the auction revenue by considering the probability of these three cases:

• Gives the final result:

• We can optimise this expression (analytically or numerically) to find the optimal discrete bid levels .

Calculating Auction Revenue

E =mX

i=0

li [P (case1;li ) +P (case2;li ) +P (case3;li )]

Discrete bid levels

implemented

Mean number of

biddersBidders’ valuation

distribution

l0 : : : lm

(David et al., 2005)

36

Optimal Bid Levels

• Uniform bidders’ valuation distribution

Reserve priceincreases

Bid incrementdecreases

37

Optimal Bid Levels

Increases expected revenue.

Decreases expected auction duration.

Increases expected auction efficiency.

Optimal discrete bid

levels

Fixed bid increment

Optimal discrete bid

levels

Fixed bid increment

Optimal discrete bid

levels

Fixed bid increment

38

Learning Auction Parameters

• To calculate optimal discrete bid levels we must know:– The bidders’ valuation distribution.– The number of participating bidders.

• Typically we do not know these parameters.– However, we can use Bayesian Machine Learning to

estimate these parameters – online.

39

Parameter Estimates

Optimal Bid Levels

AuctionPrior

Knowledge

Auction Closing Price

Learning Auction Parameters(Rogers et al., 2005)

40

• Bayesian machine learning is attractive for this application:– Makes use of our knowledge of how the auction closes.– Allows us to incorporate prior knowledge or experience.– Makes efficient use of the sparse training data

(observations of auctions).– Computationally efficient (no need to maximise multi-

dimensional functions).

Bayesian Machine Learning

41

Learning the Number of Bidders

42

Learning the Number of Bidders

43

Conclusions

• MBC prima facie candidate for controlling complex, distributed computational systems with autonomous self-interested components:– Computational game theory / Mechanism design– Evolutionary algorithms / Machine learning– Decision theory

• Ongoing research and goals: – design of mechanisms and strategies for MBC– gain understanding of and predict dynamic properties of

market-based computational systems– develop formal representation and tools

• Ultimate goal: automated mechanism design

44

Partners

http://www.iam.ecs.soton.ac.uk/projects/mbc.html