Cluster and Grid Computing Information Session / Introduction Dr. Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Lab. The University of.
Post on 25-Dec-2015
222 Views
Preview:
Transcript
1
Cluster and Grid ComputingInformation Session / Introduction
Dr. Rajkumar BuyyaGrid Computing and Distributed Systems (GRIDS) Lab. The University of MelbourneMelbourne, Australiawww.gridbus.org
WW Grid
IAUT
2
Computing and Communication Technologies Evolution: 1960-2010!
* Sputnik
1960 1970 1975 1980 1985 1990 1995 2000
* ARPANET
* Email* Ethernet
* TCP/IP* IETF
* Internet Era * WWW Era
* Mosaic
* XML
* PC Clusters* Crays * MPPs
* Mainframes
* HTML
* W3C
* P2P
* Grids
* XEROX PARC worm
COM
PUTI
NG
Com
mun
icati
on
* Web Services
* Minicomputers * PCs
* WS Clusters
* PDAs* Workstations
* HTC
2010
* e-Science
* Computing as Utility
* e-Business
* SocialNet
ControlCentralised Decentralised
IAUT
3
Three Laws of Computing:Defining the growth rate of network & computing speed, and their value.
• Moore’s Law. – Transistors on a single
chip doubles ~ every 18–24 months.
• Gilder’s Law. – Aggregate bandwidth triples
~ every year.
• Metcalfe’s Law. – The value of a network may
grow exponentially with the number of participants.
1,000,000,000,000
100,000,000,000
1970
Moore/Transistors
Gilder/Bandwidth
Metcalf/NetworkNodes
10,000,000,000
1,000,000,000
100,000,000
10,000,000
1,000,000
100,000
10,000
1,000
100
101
1975 1980 1985 1990 1995 2000 2005 2010
2,300 6,000 29,000 275,000 1.2 mil 5.5 mil 42 mil 252 mil 1.344 bil
50 50 56 1,544 45,000 145,000 10 mil 2.43 bil 200.49 bil
4 111 200 10,000 300,000 1 mil 140 mil 3.5 bil 300 bil
Source: Cambridge Energy Resource Associates
4
How to Run Applications Faster ?
• There are 3 ways to improve performance:–Work Harder–Work Smarter–Get Help
• Computer Analogy–Using faster hardware–Optimized algorithms and techniques used to solve
computational tasks–Multiple computers to solve a particular task
IAUT
5
Resource Hungry Applications
• Solving grand challenge applications using modeling, simulation and analysis
Life Sciences
CAD/CAM
Aerospace
Military ApplicationsDigital Biology Military ApplicationsMilitary Applications
Internet & Ecommerce
IAUT
6
Shrek:A movie Rendered using Clusters
IAUT
7
Lord of Rings
IAUT
8
2100
2100 2100 2100 2100
2100 2100 2100 2100
Personal Device SMPs or SuperComputers
LocalCluster
GlobalGrid
SERV ICES+
PERFORMANCE
Inter PlanetGrid
•Individual•Group•Department•Campus•State•National•Globe•Inter Planet•Universe
Administrative Barriers
EnterpriseCluster/Grid
Computing is Scaling: Towards Inter-Planetary Level
IAUT
9
Computer Food Chain: Causing the demise of specialize systems
•Demise of mainframes, supercomputers, & MPPsIAUT
10
History: Clustering of Computers for
Collective Computing
1960 1990 1995+1980s 2000+
PDAClusters
IAUT
11
What is Cluster ?
• A cluster is a type of parallel or distributed processing system, which consists of a collection of interconnected stand-alone computers cooperatively working together as a single, integrated computing resource.
• A node– a single or multiprocessor system with memory, I/O facilities, & OS
• A cluster:– generally 2 or more computers (nodes) connected together– in a single cabinet, or physically separated & connected via a LAN– appear as a single system to users and applications – provide a cost-effective way to gain features and benefits
IAUT
12
Cluster Architecture
Sequential Applications
Parallel Applications
Parallel Programming Environment
Cluster Middleware
(Single System Image and Availability Infrastructure)
Cluster Interconnection Network/Switch
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
Sequential ApplicationsSequential Applications
Parallel ApplicationsParallel Applications
IAUT
13
High Performance Cluster (dedicated mode)
IAUT
14
A typical Cluster Computing Environment
PVM / MPI/ RSH
Applications
Hardware/OS
???
IAUT
15
The missing link is provided by cluster middleware/underware
PVM / MPI/ RSH
Applications
Hardware/OS
Middleware
PVM / MPI/ RSH
IAUT
16
Middleware Design Goals
• Complete Transparency (Manageability):– Offer a single system view of a cluster system..
• Single entry point, ftp, telnet, software loading...
• Scalable Performance:– Easy growth of cluster
• no change of API & automatic load distribution.
• Enhanced Availability:– Automatic Recovery from failures
• Employ checkpointing & fault tolerant technologies
– Handle consistency of data when replicated..
IAUT
17
HPCC Books, 2 Volumes - Prentice Hall, 1999 Edited by R.Buyya with contributions from over 100 leading researchers
(www.buyya.com/cluster/)(www.buyya.com/cluster/)IAUT
18
Definition: What is a Cluster?
• A cluster is a type of parallel or distributed processing system, which consists of a collection of interconnected stand-alone computers cooperatively working together as a single, integrated computing resource.
• “stand-alone” (whole computer) computer that can be used on its own (full hardware and OS).
IAUT
Cluster Applications
• Numerous Scientific & engineering Apps.• Business Applications:
– E-commerce Applications (Amazon, eBay);– Database Applications (Oracle on clusters).
• Internet Applications:– ASPs (Application Service Providers);– Computing Portals;– E-commerce and E-business.
• Mission Critical Applications:– command control systems, banks, nuclear
reactor control, star-wars, and handling life threatening situations.
19
20
Cluster of Clusters - Hyperclusters
Scheduler
MasterDaemon
ExecutionDaemon
SubmitGraphicalControl
Clients
Cluster 2
Scheduler
MasterDaemon
ExecutionDaemon
SubmitGraphicalControl
Clients
Cluster 3
Scheduler
MasterDaemon
ExecutionDaemon
SubmitGraphicalControl
Clients
Cluster 1
LAN/WAN
IAUT
21
Adoption of the Approach
IAUT
22
Towards Grid Computing
IAUT
23
What does Grid mean?/ (It means different things to different people)
• IBM– On Demand Computing
• Microsoft– .NET
• Oracle– 10g
• Sun– N1 – Sun Grid Engine
• HP– Adaptive Enterprise
• Amazon– Elastic Compute Cloud Services
• United Devices and related companies:– Harvesting Unused Desktop resources
IAUT
24
What is Grid?[Buyya et. al.]
• A type of parallel and distributed system that enables the sharing, exchange, selection, & aggregation of geographically distributed “autonomous” resources:
– Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc;
– Software – e.g., ASPs renting expensive special purpose applications on demand;
– Catalogued data and databases – e.g. transparent access to human genome database;
– Special devices/instruments – e.g., radio telescope – SETI@Home searching for life in galaxy.
– People/collaborators.
depending on their availability, capability, cost, and user QoS requirements.
Widearea
IAUT
25
A Bird Eye View of World-Wide Grid Environment
Grid Resource Broker
Resource Broker
Application
Grid Information Service
Grid Resource Broker
databaseR2 R3
RN
R1
R4
R5
R6
Grid Information Service
IAUT
26
با • که است اینترنت بر مبتنی گرید توسط جهانی پردازشی پروژه . می شود انجام داوطلبان کامپیوتر از سرواژه\ SETI استفاده
Search for Extra-) Terrestrial Intelligence هوش برای جستجو. است( فرازمینی
کالیفرنیا • برکلی دانشگاه فضایی علوم البراتوار در پروژه این . جهان مردم پردازشگرهای به وابسته پروژه این می شود اداره
دادن قرار اختیار در با می تواند کس هر که طوری به است،. کند کمک پروژه این پیشرفت در اینترنت طریق از خود کامپیوتر
داده هایی • در شونده تکرار سیگنال های یافتن پروژه این هدفآوری جمع جهان رادیویی تلسکوپ های سوی از که است
که. داشت انتظار می توان چیزی، چنین یافتن صورت در می شود. باشد فرستاده را امواج این فرازمینی هوشمند موجود یک
IAUT
27
Grid Challenges
IAUT
28
Grid Challenges and Technologies
Security
Resource Allocation & Scheduling
Data locality
Network Management
System Management
Resource Discovery
Uniform Access
Computational Economy
Application ConstructionIAUT
29
Classes of Grid Services / Types of Grids
• Computational Services – CPU cycles– Pooling computing power: SETI@Home, TeraGrid,
AusGrid, ChinaGrid, IndiaGrid, UK Grid,… • Data Services
– Collaborative data sharing generated by instruments, sensors, persons: LHC Grid, Napster
• Application Services– Access to remote software/libraries and license
management—NetSolve • Interaction Services
– eLearning, Virtual Tables, Group Communication (Access Grid), Gaming
• Knowledge Services– The way knowledge is acquired, processed and
managed—data mining.• Utility Computing Services
– Towards a market-based Grid computing: Leasing and delivering Grid services as ICT utilities.
Computational Grid
Data Grid
ASP Grid
Interaction Grid
Knowledge Grid
Utility Grid
infrastructure
Use
rs
IAUT
30
Grid & Related Paradigms
Utility Computing• Computing “services”• No knowledge of provider• Enabled by grid technology
Utility Computing• Computing “services”• No knowledge of provider• Enabled by grid technology
Distributed Computing• Loosely coupled• Heterogeneous• Single Administration
Distributed Computing• Loosely coupled• Heterogeneous• Single Administration
Cluster• Tightly coupled• Homogeneous• Cooperative working
Cluster• Tightly coupled• Homogeneous• Cooperative working
Grid Computing• Large scale• Cross-organizational• Geographical distribution• Distributed Management
Grid Computing• Large scale• Cross-organizational• Geographical distribution• Distributed Management
Hiro Kishimoto IAUT
31
Some Characteristics of GridsNumerousresources
Different securityrequirements
& policies
Resources areheterogeneous
Geographicallydistributed
Different resourcemanagementpolicies
Connected byheterogeneous, multi-level networks
Owned by multiple organizations &
individuals
Unreliable resources and environments
IAUT
32
Grid Realization Steps/Requirements
• Step 1:– The integration of individual components into a combined networked
resource (single system image cluster).• Step 2:
– Low-level middleware to provide a secure and uniform access to services provided by different resources.
• Step 3:– User-level middleware to support application development and
aggregation of distributed resources.• Step 4:
– The construction of Grid applications and Web portals.
IAUT
33
Networked Resources across Organizations
Computers Networks Data Sources Scientific InstrumentsStorage Systems
Local Resource Managers
Operating Systems Queuing Systems Internet ProtocolsLibraries & App Kernels
Distributed Resources Coupling Services
Information QoSProcess
Application Development and Deployment Environment
Languages/Compilers Libraries Debuggers Web tools
Resource Management and Scheduling:
Applications and Portals
Prob. Solving Env.Scientific…CollaborationEngineering Web enabled Apps
Trading
…
…
…
…
FABRIC
APPLICATIONS
SECURITY LAYER
Security Data
CORE MIDDLEWARE
USER LEVEL MIDDLEWARE
Monitors
Layered Grid ArchitectureA
daptiv
e M
anagem
ent
Au
tonomic/ G
rid Econ
omy
IAUT
34
Figure 3 : Logging into the portal.
Drug DesignMade Easy!
Click Here for Demo
IAUT
35
Conclude with a comparison to the Electrical Grid………..
Where we are ????
Courtesy: Domenico LaforenzaIAUT
36
Alessandro Volta in Paris in 1801 inside French National Institute shows the battery while in the presence of
Napoleon I
Fresco by N. Cianfanelli (1841) (Zoological Section "La Specula" of National History Museum of Florence University)
IAUT
37
….and in the future, I imagine a WorldwidePower (Electrical) Grid …...What ?!?!
This is a mad man…
Oh, monDieu !
IAUT
38
2008 - 1801 = 207 Years(Recent R&D: Delivering Internet services via Electric cables).
IAUT
39
4 Essential Utilities and Delivery Networks
(1) WaterWater Distribution Network
(4) TelephoneTelecom Networks
(3) Gas
(2) ElectricityPower Grid
40
(5) Computing: Grid Delivery IT services as the 5th utility (Power Grid inspiration)
eScienceeBusinesseGovernmenteHealthMultilingualeEducation…
41
Summary
• Grid Computing has a great potential as it enabling sharing of distributed resources
• Enables creation of Cyberinfrastructure for eScience and eBusiness applications.
• Has potential to create a Service-Oriented Computing industry.
• Grids Hold a Great Promise!– Learn it, Enjoy it, Use it, Build it, Work on it!
42
Cloud Computing
43
ابر Cloud computing تکنولوژی
( استانداردها و فناوری ملی موسسه :NISTتعریف ابری( رایانش از
” اساس بر آسان دسترسی كردن فراهم برای است مدلی ابری رایانشو تغيير قابل رایانشی منابع از مجموعه ای به شبكه طريق از كاربر تقاضای : و ) کاربردی برنامه های ذخیره سازی، فضای سرورها، شبکه ها، مثل پیکربندیبه( نیاز یا و منابع مديريت به نياز کمترین با بتواند دسترسی اين که سرویس ها
“. گردد تامین و شده فراهم سرعت به سرویس فراهم کننده مستقيم دخالت
اصلی سرويس ومكانيسم ابر انواع :های
سرویس • عنوان به (IaaS)زیرساخت
سرویس • عنوان به (PaaS)پلتفرم
سرویس نرم• عنوان به (SaaS)افزار
44
اصلی سرويس ومكانيسم ابر انواع های
Tradition
al
• بسته نرم افزار
ی• نرم افزار
سفارشی
• اجاره نرم افزار
SaaS
• Software as a
Service
نرم افزار
به عنوان
سرویس
) SaaS)
• اشتراك
خدمات
نرم افزاری
• با قيمت ها و
مدل های تجاری
متفاوت
PaaS
• Platform as a
Service
پلتفرم به عنوان
سرویس
) PaaS)
• اشتراك
سكوی
توسعه
نرم افزار به
عنوان
مجموعه ای
از خدما
ت
IaaS
• Infrastructure as a
Service
زیرساخت
به عنوان
سرویس
) IaaS )• اشترا
ك در زيرساخت های فن آوری
اطالعات به
عنوان
مجموعه ای
از خدما
ت
45
سازی مدل پیاده های
ابر •خصوصی
ابر •عمومی
ابر •ترکیبی
ابر میان•
46
ابر تکنولوژی مزایای
نرم یکامپيوتر یها هزينه وکم تر افزاری
کارآي یافزايشو یافزار نرم یارتقا سريع
آخرين داشتن اختيار در و دائماسناد نسخه
ذخيره نامحدود یساز ظرفيت داده به بيشتر اطمينان قابليت
ها سخت یافزار استقاللکمتر ینگهدار یها هزينه داده بيشتر ها امنيتسيستم یسازگار بين بيشتر
ها عامل
تجهيزات و ابزار به نياز عدمويژه
یپذير دسترسیپذير انعطاف ظرافت و چابکی مکان و دستگاه به نابستگی مستاجری چند پذیری مقیاس ایمنی و امنیتپذیری سنجش
47
فناوری چالش اين پذيرش در موجود های
دائم اينترنت یاتصال :پرسرعت بهدائم اتصال به اتصال ینياز با و دارد اينترنت نم کم یاينترنت یها به کار ی سرعت
کند.
باشند ی ويژگ محدود است ممکن :هاغن ی ويژگ عدم امکانات و همتا یمبتن یکاربرد یها برنامه یها اندازه به وب یبر
خود رایانه شخصی های
کاف یها داده امنيت از است ممکن شده برخوردار یذخيرهنباشند:
رو یها داده - بر شده شده یذخيره توزيع ماشين چندين بين در هستند ابر ایمن . و - مفقود صورت فيزيک شخصی یها داده یدر پشتيبان نسخه محل یهيچ در یيا
نیست. اختيار رو) بر شده ذخيره اسناد تمام که اين دانلود یمگر خود شخصی رایانه روي بر را ابر
کار چنين کمي کاربران � معموال که کنند(. ی م یکنيد
48
گرید رایانش با ابری رایانش تفاوت
مشبک • برد: رایانش کار به ن به شبکه یک در موجود کامپیوتر چندین منابعمساله یک روی بر کار برای همزمان صورت
• : ابری طور به رایانش به که می دهد اجازه کوچک تر کاربردی برنامه چندین . شوند اجرا همزمان
گرید • ابر یک روی بر . ساختمی توان نیست شدنی آن برعکس ولیگرید • به نسبت ابر راحتتر بسیار گسترش و استفادهابر • در پیچیده کاربر واسط به نیاز عدمهای • حوضچه وجود علت به گرید به نسبت ابر بیشتر پایداری و ثبات
لزوم صورت در پشتیبانی و ذخیره برای اختصاصی
VS.
top related