Decentralized vs. Centralized Economic Coordination of Resource Allocation in Grids

Decentralized vs. Centralized Economic Coordination of Resource Allocation in Grids

T. Eymann, M. ReinickeAlbert-Ludwigs-University, Freiburg (DE)O. Ardaiz, P. Artigas, L. Díaz de Cerio, F. Freitag, R. Messeguer, L. Navarro, D. RoyoUniversitat Politècnica de Catalunya, Barcelona (ES)

CATNET project – Open Research, Evaluation(3/2002-3/2003)

Problem and objective

Problem: Provisioning services Requiring (huge amount of) resources From large number of computers CDN, Grid and P2P

Objective: evaluation of decentralized mechanism for resource allocation, based on economic paradigm: Catallaxy. (compare against a centralized mechanism using an

arbitrator object)

Methodology: simulation Network simulator (javasim)

+ application network (catnet)

Resource infrastructuresContent distribution networks, GridPeer-to-Peer

Application networks on top, run in multiple resource locationsExample: word-processor requiring service for creation of PDF files Client: Look for nearest/cheapest svc. Instance Network: always provide svc, optimize

provisioning costs and network communication

Service control, resource allocationService control, resource allocation

Service control+resource allocation

Decentralized economic coordinationPrice generation and negotiationTrading resources and servicesRegulation of supply and demand in large and complex systemsCatallaxy

Catallaxy Basics (1)Catallaxy is an alternative word for “market economy” (Mises and Von Hayek of the Neo-austrian economic school)

“Fundamentally, in a system in which the knowledge of the relevant facts is dispersed among many people, prices can act to co-ordinate the separate actions of different people in the same way as subjective values help the individual to co-ordinate the parts of his plan.”

(Friedrich A. von Hayek, The Use of Knowledge in Society, 1945)

“The Market” as a technically decentralized, distributed, dynamic coordination mechanism Adam Smith’s “invisible hand” Hayek’s “spontaneous order” Walras’ “non-tâtonnement process”

How does Catallaxy avoid chaos and achieve order?

Spontaneous order of the participants

„Unplanned result of individuals' planful actions“ (Hayek)

Constitutive Elements of the Catallaxy Access to a Market

Knowledge about scarceness of resources is transported through price information

Constitutional Ignorance Self-interest and autonomy

of participants Ability to choose between

alternative actions

Institutions and Evolution "Institutions are frictions

which, like frictions in mechanical systems, by restricting movement may make controlled movement possible.” (Loasby 2000, p. 299).

Implementation of Norms, Rules, Objects

Learning Dynamic Co-Evolution Income expectations and

price relations stabilize development

Catallactic Information Systems – Internal model

Self-Interest Individual goals of the agent can be formalized (e.g. profit maximization) Agent attempts to prognose future world state Actions effect environmental state in order to achieve goals

Choice Agent can choose between diverse alternatives Agent can rank alternatives according to prognosed goal approximation Environment is worth-oriented domain (cf. Rosenschein/Zlotkin)

Constitutional Ignorance No agent can exactly prognose a future market state („future is blind“) No agent can exactly prognose a „best strategy“ (always historically

bound) You never step twice in the same river (Heraklit)

Strategy is sophisticated trial and error procedure at best Requires adaptive and learning strategy Learning procedures are based on subjective past experiences

Consequences for Application Development

Application must be a Worth-Oriented Domain Application Domain needs common

value denominator (money) Requires “money vs. Goods“ exchange However: if the application domain

already uses money, it can be directly modelled

Consequences for Application Development

Agent-based solution is always inferior to analytical optimizationCatallaxy is inverse scalable Works better, the larger the network isInformation The more information is available, the more

accurate are the choices The more agents, the more information existsComputation Computation is fully parallel (no central bottleneck) Solution always exists in the system (no non-

allocated resource)

What we could expect?Catallaxis good for certain situations: Load balancing Large systems: inherent cost of global/up-to-date

state information for resource allocation where autonomous and decentralized algorithms work well

Adaptive to changes: in demand, topology, location and number of resources evolutionary learningself-organisation (specially for non-uniform systems with “hot spots”) Centralized/de-centralized systems may have

oscillatory behaviour “constitutional ignorance” Centralized: tragedy of state info overload with scale; Decentralized: tragedy of commons

Catallactic Information systems

The Catnet network simulator

Client: computer program at host, requests serviceService Copy: instance of service, hosted in a resource computerResource: host computer with limited storage and bandwidth

We are measuring...Social Welfare: the sum of all utilities over all participants in a given timespan. Utility = Benefit - Cost, basic utility

function per participant.

Resource Allocation Efficiency (RAE): [Marketing] "fill rate", the ratio of matched

transactions divided by the number of all proposals. (#"accepts“/#"proposals“)

Comm.Cost= #messages * #hopsResponse time

Our goal: compare baseline/Catallactic

Quasi-staticVery dynamicLow node densityHigh node density

Dynamics: change: %node disconnection time (SC?)Node density: many small nodes / few large nodes

SWF

Reso

urce

Allo

catio

nEf

ficie

ncy

BW

utiliz

atio

n

Com

mun

icatio

nco

st

Reac

tion

time

~

C

B B B B

C CC C

C C CB C

~ B B B B

Syst

em

/

Message Flows (Baseline)

Message Flows (Catallactic)

Scenarios

Appropiate scale: 10th or 100th or 1000th nodes … Change (dynamics): Movement / failure, creation (R) Change of demand (C)

Location of demand (which clients) Characteristics (many, including temporal

distribution)

Density: Fragmentation of resource capabilities

Same global amount of resources: P2Pmany small PC, CDNfew large servers /

Demand

From several clientsAt the same time, at different timesRequests with different price/priorityRate: #requests/second distribution in time, space.Deterministic, random

Dynamics

Node density (1/concentration)P2P

GridCDN

Very-dynamic (Many disconn.)

Quasi-static (few disconn.)

low high

Exp. 1

Exp. 2Fixed

netw

ork

sMob

ile, ad

-hoc,

overl

oad

ed

netw

ork

s

E1.2 E1.3

E2.3

E2.2

Dynamics

Node density (1/concentration)P2P

GridCDN

Very-dynamic (Many disconn.)

Quasi-static (few disconn.)

low high

Exp. 1

Exp. 2Fixed

netw

ork

sMob

ile, ad

-hoc,

overl

oad

ed

netw

ork

s

E1.2 E1.3

E2.3

E2.2

Catallactic better than Centralized

Ongoing work

0

10

20

30

40

50

60

70

80

90

100

10 s50 s75 s100 s125 s150 s

RAE (%)

C/B C/B C/B C/BC/B

C/B C/B C/B C/BDynamics

Density/#SC: 0/5 1/25 2/75

Network of 105 nodes, 75 Clients, 105 ResourcesResource Density/#SC: 0/5, 1/25, 2/75

500 Client requests for service, during 10, 50, 75, 100, 125, 150 seconds

Conclusions

Initial simulation results prove that a decentralized, economic model works better in certain situations. “Better” is a combination of factors (SWF)

Promising: Large scale Dynamic Saturation Resource allocation