Transcript
8/8/2019 Grid Intro
1/46
Introduction to
Grid Computing withHigh Performance ComputingMike Griffiths
White Rose Gride-Science Centre of Excellence
8/8/2019 Grid Intro
2/46
IntroductionHigh Performance Grid Computinge-ScienceThe Evolving Grid
The Local Compute Node IcebergRegistration
Ou tline
8/8/2019 Grid Intro
3/46
Obj ectives
What is grid computing?How the grid assists with problem solving lifecycleIdentify and Explain BuzzwordsRemove Hype
8/8/2019 Grid Intro
4/46
Pro bl em so lving lifecyc le
Problem definition and requirements captureModel development Languages (FORTRAN, C, C++, Java etc.) Model Building SDKs Matlab and clones
Packages (ANSYS, FLUENT, CFX)
8/8/2019 Grid Intro
5/46
Pro bl em so lving lifecyc le
Problem solving environment specialized software for solving one class of problems Application user interface, portal
Model testing Validation, verification
Results production Scheduling tasks over the grid
Analysis and Visualisation
8/8/2019 Grid Intro
6/46
Grid Techno logies
8/8/2019 Grid Intro
7/46
Simulation of large complex systemsLarge scale multi site data mining,distributed data setsShared virtual reality
Interactive collaborationReal-time access to remote resources.
Grid Techno logies
8/8/2019 Grid Intro
8/46
W hat Is Grid Computing
Virtualisation of resourceIncrease processing power Secure and flexible collaborationThe Grid Problem
8/8/2019 Grid Intro
9/46
E lectric Power Generation Ana logy
InformationGeneratorsInformation Distri b uted
Over the Grid
Customer Access toInformationGrid
8/8/2019 Grid Intro
10/46
Pcwe b opedia.com
A form of networking . Unlike conventional networksthat focus on communication among devices, gridcomputing harnesses unused processing cycles of all computers in a network for solving problems toointensive for any stand-alone machine.
8/8/2019 Grid Intro
11/46
IBM Definition
Grid computing enables the virtualization of distributed computing and data resources such asprocessing, network bandwidth and storage capacityto create a single system image, granting users andapplications seamless access to vast IT capabilities.
Just as an Internet user views a unified instance of content via the Web, a grid user essentially sees asingle, large virtual computer.
8/8/2019 Grid Intro
12/46
S un Microsystems
Grid Computing is a computing infrastructure thatprovides dependable, consistent, pervasive andinexpensive access to computational capabilities.
8/8/2019 Grid Intro
13/46
The Grid Pro bl em
Grid problem, flexible, secure, coordinatedresource sharing among dynamic collections of individuals, institutions, and resourceswhat werefer to as virtual organization s. From The Anatomy of the Grid by Foster, Kesselman and
Tuecke.
8/8/2019 Grid Intro
14/46
VirtualOrganisations
8/8/2019 Grid Intro
15/46
Grid Characteristics
Ne tworks High Bandwidth
Computing - Tflops
Data storag ePe ta byt e
TheG rid
8/8/2019 Grid Intro
16/46
Cluster GridBeowulf clustersEnterprise Grid, Campus
Grid, Intra-GridDepartmental clusters,
servers and PC networkUtility Grid
Access resources over internet on demand
Global Grid, Inter-gridWhite Rose Grid, National Grid Service,
Particle physics data grid
Types of Grids
8/8/2019 Grid Intro
17/46
Three Uses of Grid Computing
Compute grids
Data gridsCollaborative grids
8/8/2019 Grid Intro
18/46
Distri b uted S upercomputing
Compute C lusters
S chedu lers sun grid engine, p b sGrid aggregates computationa l resources to compute largecomp lex pro bl emsFast networks ena bl ing true para lle l computation and sharedmemory processingS e lect compute resources according to Time and Financia l constraints
8/8/2019 Grid Intro
19/46
Architectures for High PerformanceComputing
Supercluster
e.g. Blue Gene (65536 dual processors in 64 cabinets)Clusters e.g. iceberg Parallel applications using MPISymmetric multiprocessors e.g. 4 processor shared memory V40 node on iceberg
Shared memory programming Open MPVector Processor E.g Amdhal VP at MCC (80s and 90s)
8/8/2019 Grid Intro
20/46
High Throughput App licationsProblems divided into many tasks
Grid schedules tasksSeti@home
The mother of @home projects Spin off for companies such as
Entropia and United DevicesOther @home projects
Folding@home, fightAIDS@home,Xpulsar@home
Condor
Cycle scavenging from spare PCs
8/8/2019 Grid Intro
21/46
S tatistics for SE TI at Home(13/09/2004)
Tota l Last 24 Hours
Users 5115495 2715
Resu lts received 1532818080 3248739
Tota l CPU time 2045520.287 years 2510.9 years
Floating PointOperations
5.562175e +21 1.267008e+19(146.64 TeraFLOPs/sec)
Average CPU timeper work unit
11 hr 41 min 24.2 sec 6 hr 46 min 10.6 sec
8/8/2019 Grid Intro
22/46
SE TI@homes Most Promising Candidates
8/8/2019 Grid Intro
23/46
Grid TypesData Grid
ComputingNetwork storeslarge volume of data acrossnetworkHeterogeneous
data sources
E ngine f light data
Air line
Maintenance Centre
European data center
London Airport New York Airport
American data center
GridDiagnostics centre
8/8/2019 Grid Intro
24/46
Grid Types - Co lla b orativeInternet videoconferencingCollaborative Visualisation
8/8/2019 Grid Intro
25/46
e- S cience
More science relies on c omputational experimentsMore large, geographically disparate, collaborativeprojectsMore need to share/lease resources Compute power, datasets, instruments, visualization
8/8/2019 Grid Intro
26/46
e-Science Centres
Centres of Excellence
Regional Centres
8/8/2019 Grid Intro
27/46
8/8/2019 Grid Intro
28/46
e- S cience Requirements
Simple and secure access to remote resourcesacross administrative domainsMinimally disruptive to local administration policiesand usersLarge set of resources used by a single computation
Adapt to non-static configuration of resources
8/8/2019 Grid Intro
29/46
8/8/2019 Grid Intro
30/46
The E vo lving Grid
8/8/2019 Grid Intro
31/46
Comprising of two data clusters and two computeclusters.Offer a significant resource for the UK e-Sciencecommunity.Clusters are located at Manchester (data cluster), Oxford (compute cluster), CCLRC (data cluster) and White Rose Grid (compute cluster).
More sites Lancaster Wesc Bristol
8/8/2019 Grid Intro
32/46
E GEE
The EGEE project brings together experts from over 27countries Build on recent advances in Grid technology.
Developing a service Grid infrastructure in Europe,available to scientists 24 hours-a-day.
8/8/2019 Grid Intro
33/46
Avai la bl e Grid S ervicesAccess GridWhite Rose Grid Grid research HPC ServiceNational Grid Service Compute Grid Data Grid (SRB)
National HPC Services HPCx and CSAR (part of NGS)Portal Services
8/8/2019 Grid Intro
34/46
8/8/2019 Grid Intro
35/46
S heffie ld Grid Node: Hardware
AMD based supplied by Sun Microsystems
Processors: 320Performance: 300GFLOPsMain Memory: 800GBFilestore: 9TBTemporary disk space: 10TBPhysical size: 8 racksPower usage: 50KW
8/8/2019 Grid Intro
36/46
S heffie ld Grid Node: Hardware,part 2
160 Processors Grid pp community
160 Processors General Use 20 x V40 each with 4x64 bit AMD Opteron (2.4GHz) and 16GB
shared main memory. 40 x V20 each with 2x64 bit AMD Opteron (2.4 GHz) and 4GB
shared main memory
Comparing L2 Cash AMD Opteron 1MB Ultrac sparc III Cu (Titania) 8MB
8/8/2019 Grid Intro
37/46
S heffie ld Grid Node: Hardware, part 3
Inside a V20unit.
8/8/2019 Grid Intro
38/46
S heffie ld Grid Node: Hardware 4
Two main Interconnect types gigabit (commodity),
Myrinet (more specialist) Gigabit Supported as standard good for job farms, and
small to mid size systems Myrinet High End solution for large parallel applications
has become defacto standard for clusters (4Gb/s)
8/8/2019 Grid Intro
39/46
S heffie ld Grid Node: Hardware
64bit v 32 bit Mainly useful for programs requiring large memory
available on bigmem nodes Greater Floating Point accuracy Future-proof: 32-bit systems are becoming obselete in HPC
8/8/2019 Grid Intro
40/46
S heffie ld Grid Node: S oftware 1
O pteron
Redhat 64bitScientific
Linux
Portland,GNU
DDT
MPICH
Sun GridEngine v6
Ganglia
8/8/2019 Grid Intro
41/46
S heffie ld Grid Node: S oftware 2Maths and Statistical Matlab7.0, scilab 3.1
R+ 2.0.1Engineering and Finite Element Fluent 6.2.16, 6.1.25 and 6.1.22 als gambit, fidap and tgrid Ansys v90 Abaqus CFX 5.7.1 DYNA 91aVisualisation IDL 6.1 OpenDX
8/8/2019 Grid Intro
42/46
S heffie ld Grid Node: S oftware 3
Development
MPI, MPICH-gm OpenMP Nag, 20 ACML
Grid Globus 2.4.3 (via gpt 3.0) SRB s-client tools to follow
8/8/2019 Grid Intro
43/46
RegistrationLocal User Account
Obtain an e-Science CertificateRegister with the White Rose GridApply for NGS Resource
Go to the linkhttp://www.shef.ac.uk/wrgrid/access/index.html
8/8/2019 Grid Intro
44/46
W hy o b tain an e- S cience Certificate
Enables secure single sign on to the White Rose
GridUse portals e.g. the WRG Application portalAccess WRG, NGS, egee
8/8/2019 Grid Intro
45/46
For More InformationThe White Rose Grid www.wrgrid.org.uk
The National e-Science Centre www.nesc.ac.uk
The Globus Project www.globus.org
Global Grid Forum www.gridforum.org
8/8/2019 Grid Intro
46/46
top related