Grid Economics and Business Models: A Gridbus Perspective Rajkumar Buyya Grid and Distributed Systems (GRIDS) Laboratory Dept. of Computer Science and Software Engineering The University of Melbourne Melbourne, Australia www.gridbus.org WW Grid Grid User Meet, Bengaluru, India
42
Embed
Grid Economics and Business Models: A Gridbus Perspective Rajkumar Buyya Grid and Distributed Systems (GRIDS) Laboratory Dept. of Computer Science and.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Grid Economics and Business Models: A Gridbus Perspective
Rajkumar BuyyaGrid and Distributed Systems (GRIDS) LaboratoryDept. of Computer Science and Software EngineeringThe University of MelbourneMelbourne, Australia
www.gridbus.org
WW GridGrid User Meet, Bengaluru, India
2
Outline
Introduction Utility Networks and Grid Computing
Global Grids and Challenges Security. Resource management, pricing and service
models Service Oriented Grids and Grid Economy
SOGA, Grid Market Directory, Grid Bank, Broker Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary
3
4 Essential Utilities and Delivery Networks
(1) Water
(2) Electricity
(3) Gas
(4) Telephone
4
(5) IT services as the fifth utility (water, electricity, gas, telephone, IT)
eScienceeBusiness
eGovernmenteHealth
MultilingualeEducation
…
5
Leading to Grid (computing) Paradigm:Cyberinfrastructure for sharing resources
•Inspired by Power Grid!
•* A service-oriented/utility computing paradigm that enables seamless
sharing of geographically distributed, autonomous resources (for PROFIT).
•* This was the original aim of building Internet although it ended up in
giving birth to email!
6
A Bird Eye View of World-Wide Grid Environment
Grid Resource Broker
Resource Broker
Application
Grid Information Service
Grid Resource Broker
databaseR2R3
RN
R1
R4
R5
R6
Grid Information Service
7
Various Types of Grid Services
Computational Services – CPU cycles SETI@Home, NASA IPG, TeraGrid, I-Grid,…
Data Services Data replication, management, secure
access--LHC Grid/Napster Application Services
Access to remote software/libraries and license management—NetSolve
Interaction Services eLearning, Virtual Tables, Group
Communication (Access Grid), Gaming Knowledge Services
The way knowledge is acquired and managed—data mining.
Utility Computing Services Towards a market-based Grid computing:
Leasing and delivering Grid services as ICT utilities.
Computational Grid
Data Grid
ASP Grid
Interaction Grid
Knowledge Grid
Utility Grid
8
Worldwide Grid Spending
After the year 2006, business popularity of Grid computing is expected to be accelerate exponentially:
Especially, the financial services and ERP services is expected to take major parts in the expense
Source: Insight Research Corp.
9
Grid Challenges
Security
Resource Allocation & Scheduling
Data locality
Network Management
System Management
Resource Discovery
Uniform Access
Computational Economy
Application Construction
10
Some Grid Initiatives Worldwide
Australia Nimrod-G Gridbus DISCWorld GrangeNet. APACGrid ARC eResearch
Brazil OurGrid, EasyGrid LNCC-Grid + many others
China ChinaGrid – Education CNGrid - application
Europe UK eScience EU Grids.. and many more...
India Garuda
Japan NAGERI
Korea...N*Grid
SingaporeNGP
USA Globus NASA IPG AccessGrid TeraGrid Cyberinfrasture
Industry Initiatives IBM On Demand
Computing HP Adaptive Computing Sun N1 Microsoft - .NET Oracle 10g Satyam – Grid Practice Infosys, Wipro, TCS StorageTek –Grid..
Public Forums Global Grid Forum Australian Grid Forum
Conferences: CCGrid Grid HPDC E-Science
http://www.gridcomputing.com
1.3 billion – 3 yrs
1 billion – 5 yrs
450million – 5 yrs
486million – 5 yrs
1.3 billion (Rs)
27 million
2? billion
120million – 5 yrs
11
Grid (Market) Participant Types and Application Category
commercialscientific
free trading
regulation
Publiccomputing
(SETI@Home, Alchemi, UD)
National provider(K*Grid, TeraGrid, Garuda/IndiaGrid, UKGrid, AusGrid)
Private enterprises(Satyam, IBM, Sun)
Application Category
SharingModel
12
mix-and-match (service)
Object-oriented
Internet/partial-P2P
Network enabled Solvers
Economic-based Utility / Service-Oriented
ComputingNimrod-G
13
The Gridbus Project @ Melbourne:Enable Leasing of ICT Services on
Demand
WWG
World Wide Grid!On Demand Utility
Computing
Gridbus
Distributed Data
14
The Gridbus Project @ GRIDS Lab, The University of Melbourne: Toolkit for Creating and Deploying e-Research Applications on Utility Grids
The Gridbus Project @ GRIDS Lab, The University of Melbourne: The Gridbus Project @ GRIDS Lab, The University of Melbourne: Toolkit for Creating and Deploying eToolkit for Creating and Deploying e--Research Applications on Utility GridsResearch Applications on Utility Grids
Gridbus
Distributed Data
http://www.gridbus.org
• Gridbus is a “open source” Grid R&D project with focus on Grid Economy, Utility Grids and Service Oriented Computing.
– Grid Bank: Accounting and Transaction Management
– Visual Tools for Creation of Distributed Applications
– Grid Service Broker and Scheduling
– Workflow Management Engine
– GridSim Toolkit
– Libra: SLA-based Resource Allocation
15
Grid Economy: Methodology for Sustained Resourced Sharing and Managing Supply-and-Demand for Resources
16
New challenges of Grid Economy
Resource Owners How do I decide prices ? (economic models?) How do I specify them ? How do I translate price to resource allocation ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? …..
Resource Consumers How do I decide expenses ? How do I express QoS requirements ? How do I trade between timeframe & cost ? How do I map jobs to resources to meet my QoS needs? …..
They need mechanisms and technologies for value expression, value translation, and value enforcement.
17
Grid Entities and Architecture
GSP site scheduler
accounting
Grid consumer
MarketMaker
GSP global scheduler
broker
Resource Provider
End usersPrivate enterprises
National providers
GSP site scheduler
Resource Provider
18
Grid Node N
A Reference Service-Oriented Architecture for Utility Grids
Grid Consumer
Pro
gra
mm
ing
En
viro
nm
ents
Grid Resource Broker
Grid Service Providers
Grid Explorer
Schedule Advisor
Trade Manager
Job ControlAgent
Deployment Agent
Trade Server
Resource Allocation
ResourceReservation
R1
Misc. services
Information Service
R2 Rm…
Pricing Algorithms
Accounting
Grid Node1
…
Grid Middleware Services
…
…
HealthMonitor
Grid Market Services
JobExec
Info ?
Secure
Trading
QoS
Storage
Sign-on
Grid Bank
Ap
pli
cati
on
s
Data Catalogue
19
Gridbus and Complementary Technologies – realizing Utility
On Demand Assembly of Services: Putting Them All Together
ASP Catalogue
Grid Info Service
Grid Market Directory
GSP(Accounting Service)
GridbusGridBank
GSP(e.g., UofM)
PEGSP
(e.g., VPAC)
PE
GSP(e.g., IBM)
CPUorPE
Grid Service (GS)
(Globus)
Alchemi
GS
GTS
Cluster Scheduler
Job
8
GridResource Broker
2
Visual Application Composer
Application CodeExplore
data1
36
45
Resu
lts9 7
Results+
Cost Info
10
11
Bill
12Data Catalogue
21
Alchemi: .NET-based Enterprise Grid Platform & Web Services
InternetInternet
InternetInternet
Alchemi Worker Agents
Alchemi Manager
Alchemi Users
Web Services
Web Services
•SETI@Home like Model•General Purpose•Dedicated/Non-dedicate workers•Role-based Security•.NET and Web Services•C# Implementation•GridThread and Job Model Programming•Easy to setup and use• Widely in use!
22
Some Users of Alchemi
Tier Technologies, USALarge scale document processing using Alchemi framework
CSIRO, AustraliaNatural Resource Modeling
The Friedrich Miescher Institute (FMI) for Biomedical Research, SwitzerlandPatterns of transcription factors in mammalian genes
Satyam Computers Applied Research Laboratory, IndiaMicro-array data processing using Alchemi framework
The University of Sao Paulo, BrazilThe Alchemi Executor as a Windows Service
stochastix GmbH, GermanyAsynchronous Excel Tasks using ManagedXLL and Alchemi .Net Grid Computing framework.
Many users in Universities: See next for an example.
The Gridbus Grid Service Broker for Data Grid
Applications
Builds on the Nimrod-G Computational Grid Broker and
Application Development Environment: XML-based language for composition of task farming
(legacy) applications as parameter sweep applications. Task Farming APIs for new applications. Web APIs (e.g., Portlets) for Grid portal development. Threads-based Programming Interface Workflow interface and Gridbus-enabled workflow
engine. Resource Allocation and Scheduling
Dynamic discovery of optional computational and data nodes that meet user QoS requirements.
Case Study: High Energy Physics and Data Grid The Belle Experiment
KEK B-Factory, Japan Investigating fundamental
violation of symmetry in nature (Charge Parity) which may help explain the universal matter – antimatter imbalance.
Collaboration 400 people, 50 institutes
100’s TB data currently
30
Australian Belle Data Grid Testbed
Grid Service Broker
Replica Catalog
AARNET
NWS NameServer
VirtualOrganization
Analysis Request
Analysis Results
CertificateAuthority
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
GRIDS Lab, University of Melbourne
Dept. of Physics,University of Sydney
ANU, Canberra
Dept. of Computer Science, University of Adelaide
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Intel Pentium 2.0 Ghz, 512 MB RAM
Dept. of Physics,University of Melbourne
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
VPACMelbourne
31
Case Study: Event Simulation and Analysis
B0->D*+D*-Ks
• Simulation and Analysis Package - Belle Analysis Software Framework (BASF)• Experiment in 2 parts – Generation of Simulated Data and Analysis of the distributed data
Analyzed 100 data files (30MB each) were distributed among the five nodes
32
Belle Data Grid (GSP CPU Service Price: G$/sec)
Grid Service Broker
Replica Catalog
AARNET
NWS NameServer
VirtualOrganization
Analysis Request
Analysis Results
CertificateAuthority
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
GRIDS Lab, University of Melbourne
Dept. of Physics,University of Sydney
ANU, Canberra
Dept. of Computer Science, University of Adelaide
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Intel Pentium 2.0 Ghz, 512 MB RAM
Dept. of Physics,University of Melbourne
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NA
G$4
G$4
Datanode
G$6VPAC
MelbourneG$2
33
Belle Data Grid (Bandwidth Price: G$/MB)
Grid Service Broker
Replica Catalog
AARNET
NWS NameServer
VirtualOrganization
Analysis Request
Analysis Results
CertificateAuthority
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
GRIDS Lab, University of Melbourne
Dept. of Physics,University of Sydney
ANU, Canberra
Dept. of Computer Science, University of Adelaide
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Intel Pentium 2.0 Ghz, 512 MB RAM
Dept. of Physics,University of Melbourne
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NA
G$4
G$4
Datanode
G$6VPAC
MelbourneG$2
34
31
38
31
30
3336
32
35
Deploying Application Scenario
A data grid scenario with 100 jobs and each accessing remote data of ~30MB