Page 1
New Frontiers in Cloud Computing for Big Data/IoT Applications
Dr. Rajkumar Buyya Cloud Computing and Distributed Systems (CLOUDS) Lab School of Computing and Information Systems The University of Melbourne, Australia
www.cloudbus.org www.buyya.com www.manjrasoft.com
Major Sponsors/Supporters
Manjrasoft
Dr Rajkumar BuyyaChief Executive Officer
Manjrasoft Pty LtdOffice No. 7.22, Doug McDonell BuildingMelbourne University - Parkville Campus
Melbourne, VIC 3010, Australia P: +61-3-8344 1344 | M : +61-431799078
E: [email protected] | W: http://www.manjrasoft.com
ManjrasoftInnovative Solutions for Cloud Computing
Page 2
2
Manjrasoft
The Next Revolution in IT The Big Switch in IT
Classical Computing
Buy & Own
Hardware, System
Software, Applications
often to meet peak needs.
Install, Configure,
Test, Verify
Manage
..
Finally, use it
$$$$....$(High CapEx)
Cloud Computing
Subscribe
Use
$ - pay for what you
use, based on QoS
Every 1
8 m
on
ths?
Page 3
3
Manjrasoft
Outline
“Computer Utilities” Vision and Promising IT Paradigms/Platforms
Cloud Computing and Related Paradigms Trends, Definition, Cloud Benefits and Challenges
Market-Oriented Cloud Architecture SLA-oriented Resource Allocation
Global Cloud Exchange
Aneka: A Cloud Application Platform
Summary and Thoughts for Future
Page 4
4
Manjrasoft
Waves of Computing Architecture
1960 1970 1980 1990 2000
Mainframe
Client Server
Web
Cloud
Minicomputer
Disruptor: Virtualisation
2010
Source: Geng Lin
Page 5
5
Manjrasoft
“Computer Utilities” Vision: Implications of the Internet
1969 – Leonard Kleinrock, ARPANET project “As of now, computer networks are still in their
infancy, but as they grow up and become sophisticated, we will probably see the spread of “Computer Utilities”, which, like present electric and telephone utilities, will service individual homes and offices across the country”.
During the last 45 years, several advances have taken place in both “computing” and “communications” areas that are turning the vision of “Computer Utilities” in to a reality.
Page 6
6
Manjrasoft
Computing Paradigms and Attributes: Realizing the ‘Computer
Utilities’ Vision
Web
Data Centres
Utility Computing
Service Computing
Grid Computing
P2P Computing
Market-Oriented Computing
…
Cloud Computing
Internet of Things
Big Data
Fog Computing
..
-Ubiquitous access
-Reliability -Scalability -Autonomic -Dynamic discovery &
composiability -QoS -SLA - …
+
Paradigms
Attributes/Capabilities
-Trillion $ business per year?
Page 7
7
Manjrasoft
A Data-Driven World
Big Data for
Big Science
Big Data/Clouds for
Big Population
Page 8
8
Manjrasoft
Internet (network of computers) to Internet of Things (IoT)
IoT will consist of ~50 billion objects by 2020.
IoT economic impact of $11 trillion/year by 2025.
Page 9
9
Manjrasoft
Outline
“Computer Utilities” Vision and Promising IT Paradigms/Platforms
Cloud Computing and Related Paradigms Trends, Definition, Cloud Benefits and Challenges
Market-Oriented Cloud Architecture SLA-oriented Resource Allocation
Global Cloud Exchange
Aneka: A Cloud Application Platform
Summary and Thoughts for Future
Page 10
10
Manjrasoft
Compute
Storage
Applications
Development and Runtime Platform
Public Clouds
Subscription-Oriented Cloud Services: X{compute, apps, data, ..}
as a Service (..aaS)
Clients
Other Cloud Services
Govt. Cloud Services
Private Cloud
Cloud Manager
Page 11
11
Manjrasoft
Cloud Services
Infrastructure as a Service (IaaS)
CPU, Storage: Amazon.com,
Google Compute, ….
Platform as a Service (PaaS)
Google App Engine, Microsoft
Azure, Manjrasoft Aneka..
Software as a Service (SaaS)
SalesForce.Com
Infrastructure as a Service (IaaS)
Software as a Service (SaaS)
Platform as a Service (PaaS)
Manjrasoft
....
Page 12
12
Manjrasoft
Cloud Deployment Models
Private/Enterprise Clouds
Cloud model run within a company’s own Data Center / infrastructure for internal and/or partners use.
Public/Internet Clouds
3rd party, multi-tenant Cloud
infrastructure & services:
* available on
subscription basis
Hybrid/Inter Clouds
Mixed usage of private and public
Clouds: Leasing public cloud services
when private cloud capacity is insufficient
Page 13
13
Manjrasoft
Cloud Applications
•Scientific/Tech Applications •Business Applications •Consumer/Social Applications
Science and Technical Applications
Business Applications
Consumer/Social Applications
Page 14
14
Manjrasoft
Cloud Computing Challenges: Dealing with too many issues
Uhm, I am not quite
clear…Yet another
complex IT paradigm?
Billing
Utility & Risk
Management
Scalability
Reliability
Software Eng.
Complexity
Programming Env.
& Application Dev.
Page 15
15
Manjrasoft
Outline
“Computer Utilities” Vision and Promising IT Paradigms/Platforms
Cloud Computing and Related Paradigms Trends, Definition, Cloud Benefits and Challenges
Market-Oriented Cloud Architecture SLA-oriented Resource Allocation
Global Cloud Exchange
Aneka: A Cloud Application Platform
Summary and Thoughts for Future
Page 16
16
Manjrasoft
Market-Oriented Cloud Architecture: QoS negotiation and SLA-based Resource Allocation
Dispatcher
VM
Monitor
Service Request
Monitor
Pricing Accounting
Service Request Examiner and
Admission Control
- Customer-driven Service Management
- Computational Risk Management
- Autonomic Resource Management
Users/
Brokers
SLA
Resource
Allocator
Virtual
Machines
(VMs)
Physical
Machines
Page 17
17
Manjrasoft
Cloud to expand (lease more resources) with increase in demand
Dispatcher
VM
Monitor
Service Request
Monitor
Pricing Accounting
Service Request Examiner and
Admission Control
- Customer-driven Service Management
- Computational Risk Management
- Autonomic Resource Management
Users/
Brokers
SLA
Resource
Allocator
Virtual
Machines
(VMs)
Physical
Machines
Multi-Clouds
Page 18
18
Manjrasoft
Cloud to shrink (unlease resources) with decrease in demand
Dispatcher
VM
Monitor
Service Request
Monitor
Pricing Accounting
Service Request Examiner and
Admission Control
- Customer-driven Service Management
- Computational Risk Management
- Autonomic Resource Management
Users/
Brokers
SLA
Resource
Allocator
Virtual
Machines
(VMs)
Physical
Machines
Page 19
19
Manjrasoft
Outline
“Computer Utilities” Vision and Promising IT Paradigms/Platforms
Cloud Computing and Related Paradigms Trends, Definition, Cloud Benefits and Challenges
Market-Oriented Cloud Architecture SLA-oriented Resource Allocation
Global Cloud Exchange
Aneka: A Cloud Application Platform
Summary and Thoughts for Future Infrastructure as a Service (IaaS)
Software as a Service (SaaS)
Platform as a Service (PaaS)
Page 20
20
Manjrasoft
Aneka: Cloud Application Platform (CAP) for Resource-Intensive/Elastic Apps
Multiple Infrastructures
Multi-core Cluster Grid Cloud
Thread Task ... MapReduce
2100 2100 2100 2100
2100 2100 2100 2100
Aneka
Multiple Applications
1. SDK
2. Runtime
World-first platform supporting multiple Cloud programming models (Task, Thread, MapReduce)
SDK (Software Development Kit) containing APIs for multiple programming models and tools
Runtime Environment for managing application execution on Clouds
Suitable for Development of Enterprise
Cloud Applications
Cloud enabling legacy applications
Portability for Customer Apps: Enterprise ↔ Public Clouds
.NET/Win ↔ Mono/Linux
Page 21
22
Manjrasoft
Aneka as a Cloud Application Platform
Datacenters Clusters
Desktop PCs Public Clouds
Platform
Infrastructure
Applications & Services Application Development
Cloud Middleware
Aneka
Scheduling & Execution
Monitoring Billing
SLA & QoS User Management
Application Management Dynamic
Provisioning
Bag of Tasks Dist. Threads MapReduce …
Page 22
23
Manjrasoft
Aneka - Dynamic Management of Application Tasks and Cloud Resources
Application
Aneka Application
Aneka
Manager
Aneka
Workker/Executor
Manager / Executor
Aneka Threads
Page 23
24
Manjrasoft
Aneka Plug-ins for Multi-Clouds
XenServer Pool Aneka + Xen Server
private Cloud
VMWare Pool Aneka + VMWare
private Cloud
Amazon EC2 Pool Aneka + public EC2
Windows Azure Aneka + public Azure
OpenStack Aneka+ OpenStack
Executors/Schedulers
Executor
Client Libraries
Executors
Scheduler
Public Cloud
publicly available resources (physical and virtual)
Scheduler
internet
Private Cloud
private enterprise network
VPN (virtual resources)
Executor
Page 24
Deployment Case Studies
Page 25
26
Manjrasoft
Aneka used in deployment of many Applications on Clouds
China Southern Railways
Locomotive design CAD rendering
Indian Space Research Organization (ISRO)
Satellite Image Processing
Washington State University
Privacy-aware Medical Data Processing on Clouds
Thapar University
Agriculture-as-a-Service on Clouds
Melbourne University and MSRIT University
Health-care (ECE data analysis) on Clouds
Medical application data mining
Page 26
27
Manjrasoft
(GoFront: A unit of China Southern Railway Group)
Aneka utilizes idle desktops
(30) to decrease task time
from days to hours
Time
(in hrs)
Single
Server
Aneka
Cloud
Raw Locomotive Design Files
(Using AutoDesk Maya) Using Maya
Graphical Mode
Directly
Case 1: Single Server
4 cores
server
Aneka Maya Renderer
Use private
Aneka Cloud
GoFront Private Aneka
Cloud
LAN network
(Running Maya Batch Mode on demand)
Case 2: Aneka
Enterprise Cloud Manjrasoft
Application: Locomotive design CAD rendering
Page 27
28
Manjrasoft
Indian Space Research Organization (ISRO) - ADRIN
Receiving station processing
Archiving
Aneka Cloud
Distribution
Private Cloud Citrix Xen
Public Cloud Amazon AWS
Manjrasoft
Page 28
29
Manjrasoft
Mobile Cloud: Delegate Resource
Intensive Tasks to Clouds Max{User Experience/Energy}
Page 29
30
Manjrasoft
Health Care: SaaS Cloud for ECG Sensor Data Analysis
Embedded Bluetooth Enabled
Data Processor & Communication Module
ECG Sensor ModuleECG Data Analysis Software
Wireless ECG Sensor
Sa
aS
Pa
aS
Iaa
S
Blu
eto
oth
connectivity
Wireless/ Mobile 3G Network
1
2
3
4
6
5
A User
Millions of users
User
Requests
User Requests
User QoS based Scaling of
Compute Resources
Dynamic Scalable Runtime
Page 30
31
Manjrasoft
IEEE Scalable Computing Challenge
at CCGrid 2010 Conference
Page 31
32
Manjrasoft
`
manjr
asoft
.com
Page 32
34
Manjrasoft
Outline
“Computer Utilities” Vision and Promising IT Paradigms/Platforms
Cloud Computing and Related Paradigms Trends, Definition, Cloud Benefits and Challenges
Market-Oriented Cloud Architecture SLA-oriented Resource Allocation
Global Cloud Exchange
Aneka: A Cloud Application Platform
Summary and Thoughts for Future
Page 33
35
Manjrasoft
Summary
Several Computing Paradigms have promised to deliver “Computing Utilities” vision: Cloud Computing is the most recent kid in the block
promising to turn vision into reality
Many exciting business and consumer applications
Aneka is a rapidly emerging as a platform for: Building Clouds and their Apps rapidly – reducing
the cost/time of application Software Engineering.
Already used in engineering, life science, gaming, and business intelligence applications.
Market-oriented Clouds starting to become real. Amazon Cloud Marketplace, Virtual Appliance,…
Page 34
36
Manjrasoft
Fog Computing: Integration of Edge and Clouds for IoT Apps
Private Cloud
Provider A Provider B
Private Cloud
Public Cloud
Fog Computing
Computing is dynamically
distributed across cloud sites
and network elements based
on QoS requirements.
…IoT Sensors
Page 35
37
Manjrasoft
A Reference Book
Morgan Kauffman, USA McGraw Hill, India China Machine Press, China
Page 36
38
Manjrasoft
Thanks for your attention!
Are there any
Questions?
Comments/Suggestions
We welcome you to contact: Study/Research with Us | To setup a Cloud Lab
http:/www.cloudbus.org | www.Manjrasoft.com [email protected] | [email protected]
Manjrasoft