Distributed Market Broker Architecture for Resource Aggregation in Grid Computing Environments Morihiko Tamai, Naoki Shibata*, Keiichi Yasumoto, and Minoru Ito Nara Institute of Science and Technol ogy, *Shiga University
May 25, 2015
Distributed Market Broker Architecture for Resource Aggregation in
Grid Computing Environments
Morihiko Tamai, Naoki Shibata*, Keiichi Yasumoto, and Minoru Ito
Nara Institute of Science and Technology,*Shiga University
Background
• PC Grid and peer-to-peer computing systems (SETI@home, GIMPS and cell computing) – Achieve more than super computer power at
extremely low cost.– Aggregated computing power is used for
public objectives.
• We want to use these computing power for personal objectives– e.g., video encoding
Background (cont.)
• Problem:– Resource providers wouldn’t allow strangers
to use their computers without appropriate incentive
• In Cell computing (by NTT DATA), incentives for PC users are provided by giving them electronic currency.
• But, users can only provide resources. They can’t use other people’s resources.
• Popcorn[4], Java Market[5], JaWS[6], Nimrod-G[7], etc.
• Each user is resource seller or buyer.• Resource seller receives e-money by offering
his/her computational resource.• Resource buyer pays e-money for using
computational resource provided by seller.• Appropriate price of each resource is automatically
decided based on supply and demand.
Market-based Resource Sharing
• Seller wants to find buyer with highest price.• Buyer wants to find seller with lowest price.
• The existing systems utilize a market broker placed in a network node, which mediates between sellers and buyers.
Market-based Resource Sharing (cont.)
Our Goal
• However, in the existing systems…– Market broker is implemented by centralized server,
and with no detailed consideration of its scalability– Can handle only one-to-one order matching
But, if a user wants to use relatively large computation power, it is desirable to match a multiple set of sell-orders to one buy-order.
• Our contributions:– Decentralized market broker architecture.– One-to-many order matching.
Outline
1. Introduction
2. Framework for market-based resource sharing
3. Distributed Market Broker Architecture
4. Experiments and Results
5. Conclusion
Sell/buy Orders• Resource type
– CPU power (MIPS), memory amount (MB), disk amount (GB), occupation time of resource (min)
• Resource vector– r = (r1, r2, …, rn)
– Hereafter, for simplicity, we suppose n = 2.• Sell/buy orders
– Tuple of resource vector and sell/buy price: (r, price)
– e.g., sell-order s = ((1000MIPS, 80MB), 200)
Order Matching
• Matching condition (same as NYSE)– Condition A:
– Condition B: If there are multiple sell/buy orders which satisfy condition A, one with the highest/lowest price is selected for sell/buy orders.
Sell-order: S Buy-order: B
S.r1
S.r2 S.price
B.r1
B.r2 B.price
Order Matching (cont.)
Market brokerSellers buyers
((3, 3), 4) ((2, 1), 2)
Buy price is less than the sell price
Order Matching (cont.)
Market brokerSellers buyers
((3, 3), 4)
((2, 2), 3)
((2, 1), 2)
((1, 1), 5)Match
From condition B, the buy-order matches the second sell-order, because the sell price is lower.
Outline
1. Introduction
2. Framework for market-based resource sharing
3. Distributed Market Broker Architecture
4. Experiments and Results
5. Conclusion
Logical Space for Orders
CPU power0
Mem
ory
am
ount
Sell-order s2 =((80,40), 100)
Sell-order s1 =((60,70), 90)
max
max
Map each order to the point of the two-dimensional logical space
s1 (90)
60
70
s2 (100)
80
40
Logical Space for Orders (cont.)
Sell-order s1 =((60,70), 90)
CPU power0
max
Mem
ory
am
ount
max
Sell-order s2 =((80,40), 100)
s2 (100)
s1 (90)
Buy-order b
Valid region of
sell-order s1
Dividing Order Space
0max
maxm2m1 m3
m5m4 m6
m8m7 m9
• Divide the whole space into sub-regions.• Then, one server node is assigned to each sub-region.
Dividing Order Space (cont.)
0max
maxm2m1 m3
m5m4 m6
m8m7 m9
• When a user registers a new order, he/she sends the order to the one of the server nodes.
s1 (70)
Dividing Order Space (cont.)
0max
maxm2m1 m3
m5m4 m6
m8m7 m9
• When a user registers a new order, he/she sends the order to the one of the server nodes.
• If new sell-order s1 = ((80,40), 70) is sent to m8, it is forwarded to m6 using CAN routing [11].
• Finally, s1 is retained by m6. s1 (70)
s1 (70)
Dividing Order Space (cont.)
0max
m2m1 m3
m5m4 m6
m8m7 m9
Problem: s1 and b can’t match even though they satisfy matching condition, because they are retained by different server nodes.
Buy-order b (75)
s1 (70)
Order Replication
0max
m2m1 m3
m5m4 m6
m8m7 m9
• Make appropriate server nodes hold replicas of the order.
s1 (70)
Problem: s1 and b can’t match even though they satisfy matching condition, because they are retained by different server nodes.
Order Replication (cont.)
0max
m2m1 m3
m5m4 m6
m8m7 m9
s1 (70)• Make appropriate server nodes hold replicas of the order.• Use flooding based delivery method to send replication messages.
Problem: s1 and b can’t match even though they satisfy matching condition, because they are retained by different server nodes.
Order Replication (cont.)
0max
s2 (80)
s3
s4
s5
maxm2m1 m3
m5m4 m6
m8m7 m9
s1 (70)
Problem: server nodes on the left below are overloaded to retain replicas.
Order Replication (cont.)
0max
max
s3
m2m1 m3
m5m4 m6
m8m7 m9
s2 (80) s1 (70)• pay attention to sell price of sell-orders.• we can realize that m7 and m8 do not need to retain replicas of s2, because s2.price > s1.price
Problem: server nodes on the left below are overloaded to retain replicas.
Buy-order b (75)
One-to-many Order Matching
• We suppose there are n sub-tasks.
• Let {b1, b2, …, bn} denotes the set of buy-orders for the sub-tasks. We call each bi sub-buy-order.
• We suppose the total budget is specified by user to buy resources for n sub-tasks.
• We want to match all sub-buy-orders simultaneously.
One-to-many Order Matching (simple method)
First, we think about a simple method.
0 max
maxm2m1 m3
m5m4
m8m7 m9
s2 (80) s1 (70)
s3 (100)
b1b2
• two sub-tasks {b1, b2}.• total budget is 150.• In simple method, we register b1, b2 as independent buy-orders.• Problem: how to specify buy prices of the buy-orders?
One-to-many Order Matching (simple method)
First, we think about a simple method.
0 max
maxm2m1 m3
m5m4
m8m7 m9
s2 (80) s1 (70)
s3 (100)
b1b2
• two sub-tasks {b1, b2}.• total budget is 150.• In simple method, we register b1, b2 as independent buy-orders.• Problem: how to specify buy prices of the buy-orders?
(80)(70)
One-to-many Order Matching (simple method)
First, we think about a simple method.
0 max
maxm2m1 m3
m5m4
m8m7 m9
s2 (80) s1 (70)
s3 (100)
b1b2
• two sub-tasks {b1, b2}.• total budget is 150.• In simple method, we register b1, b2 as independent buy-orders.• Problem: how to specify buy prices of the buy-orders?
(75)(75)
One-to-many Order Matching (our method)
0 max
m2m1 m3
m5m4
m8m7 m9
s2 (80) s1 (70)
s3 (100)
b1b2
• We allow buyers to specify wild card as buy price.• Each server node finds a sell-order with the lowest price which satisfies matching condition. • Then, each server node sends the buyer tentative match message.
( * )( * )
buyer
One-to-many Order Matching (our method)
0 max
m2m1 m3
m5m4
m8m7 m9
s2 (80) s1 (70)
s3 (100)
b1b2
• Then, buyer checks if the total budget is enough to buy all sell-orders.• If so, the buyer sends confirmation messages.• Finally, server nodes confirm the one-to-many order matching.
( * )( * )
buyer
Budget >= 150 ?
Outline
1. Introduction
2. Framework for market-based resource sharing
3. Distributed Market Broker Architecture
4. Experiments and Results
5. Conclusion
Experiment 1
• To evaluate what extent our distributed architecture can mitigate each server load.
• Settings:– 15000 of sell-orders and buy-orders are generate
d.– The number of resource types: 2.– The amount to sell or buy each resource: unifo
rm random values in [0:100].– Sell/buy price: Gaussian distribution whose mea
n and standard deviation are (v1+v2) and 30. (Here, vi denotes the amount of i-th resource)
Experiment 1 (cont.)
• We measured average number of messages received per node.
• The messages are categorized into three:(1) buy/sell orders sent from clients.
(2) replication messages for orders.
(3) deletion messages for replicas.
Overhead due to decentralization of server nodes
ScalabilityA
vera
ge
Nu
mb
er
of M
ess
ag
es
rece
ive
d p
er
serv
er n
ode
• The number of messages received by each server node can be reduced significantly by increasing the number of server nodes.• The load is balanced among server nodes uniformly.
Experiment 2• To evaluate the difference of matching results
between our method and centralized server node.• Users’ satisfaction degree of matching result:
– When a new order o1 matches an order o2 which is already retained by server node,
• D = o1.buyPrice – o2.sellPrice, if o1 is buy-order.• D = o2.buyPrice – o1.sellPrice, if o1 is sell-order.
– In the case of centralized server node, o2 which maximizes D is always selected for o1.
• DG: value of D when using centralized server node.– In the case of decentralized server node, value of D is
equal or less than DG because there are transmission delay among server nodes.
– Define users’ satisfaction degree of matching result as DL / DG.
• DL: value of D when using decentralized server nodes.
Difference of matching results compared with centralized server node
The case of centralized server node
Our architecture achieves almost the same satisfaction degrees (>= 0.95) as the case of centralized server node
Transmission delay among server nodes (ms)
Experiment 2
• To evaluate effectiveness of one-to-many order matching method.
• We measured probabilities of successful one-to-many matching using proposed method, and compared it to the case of simple method.
• Simple method:– We generated n buy-orders using the same settings
of the first experiment, and registered these orders as independent orders (n is determined at random according to exponential distribution whose mean is 10).
– Matching fails if there is a buy-order which is not matched to sell-order until the end of simulation time.
• Our one-to-many matching method:– We used the same buy orders as simple method.– The total budget is the sum of buy prices of n buy-
orders.
Experiment 2 (cont.)
Efficiency of one-to-many order matching
Our method achieves higher matching probability than the simple method.
Conclusion
• Our architecture…– can significantly reduce the number of
messages received by each server node.– greatly increases the availability of resources
using one-to-many matching method.
• Future work…– Dynamic division method of each sub-region
assigned to a server node will be treated.
Thanks
Introduction of Time Slot
0max
m2m1 m3
m5m4 m6
m8m7 m9
Problem: when a large number of orders are arrived at one server node in a short time period, the server node may be overloaded due to transmitting replication messages.
s1 (70)After T
After T• Aggregate replication messages during every time interval T.
Order Deletion
0max
maxm2m1 m3
m5m4 m6
m8m7 m9
• Problem: s1 matches to b1 and b2.• To keep consistency, when a server node matches an order to a replica, let the server node check if the original order exists or not.• If original exists, deletes the original.• If not, the matching is canceled.
s3 (90)
s1 (70)s2 (80)
Buy-order b1 (75)
Buy-order b2 (75)
One-to-many Order Matching (cont.)
0max
maxm2m1 m3
m5m4
m8m7 m9
s2 (80) s1 (70)
• When a server node receives sub-buy-order, it finds a sell-order with the lowest price which satisfies matching condition, and sends the buyer tentative match message.
s3 (100)
b1b2
buyer
One-to-many Order Matching (cont.)
0max
maxm2m1 m3
m5m4
m8m7 m9
s2 (80) s1 (70)
• Buyer checks whether the total budget is enough to buy all sell-orders.• If so, the buyer sends confirmation messages.
s3 (100)
b1b2
buyer
One-to-many Order Matching (cont.)
0max
m2m1 m3
m5m4
m8m7 m9
s2 (80) s1 (70)
• Each server node checks whether the sell-order still exists.• If so, server node changes the state of sell-order to intermediate state, and sends ack message. • Otherwise, it sends nack message.
s3 (100)
b1b2
buyerack
One-to-many Order Matching (cont.)
0max
maxm2m1 m3
m5m4
m8m7 m9
s2 (80) s1 (70)
• If the buyer receives ack messages from all server nodes, the sub-buy-orders finally matches to the sell-orders, and buyer sends commitment messages. s3 (100)
b1b2
buyer
b1 = ((40,50), undefined). b2 = ((70,40), undefined).
commit
One-to-many Order Matching (cont.)
0max
maxm2m1 m3
m5m4
m8m7 m9
s2 (80) s1 (70)
• Otherwise, if at lease one nack message received, buyer sends cancel messages to free intermediate state of sell-orders. s3 (100)
b1b2
buyer
b1 = ((40,50), undefined). b2 = ((70,40), undefined).
cancel
More Detailed Settings for the Experiment 3
• Orders generated:– 15000 sell-orders– 13500 buy-orders– 1500 buy-orders which include n buy-orders
(Here, n is determined at random according to exponential distribution whose average is 10)
(1000)
(100)