grid-iitr

8/9/2019 grid-iitr

1/36

Grid Computing&

Applications

INDIAN INSTITUTE OF TECHNOLOGY

ROORKEE

8/9/2019 grid-iitr

2/36

Networking and Computing

Technologies Advancements

* Web Services

1960 1970 1975 1980 1985 1990 1995 2000

T

echno

logiesIntroduc

ed

* ARPANET

* Email* Ethernet

* TCP/IP

* Internet Era * WWW Era

* Mosaic

* XML

* PC Clusters

* Crays * MPPs

* Mainframes

* HTML

* W3C

* P2P

* Grids

COMPUTING

NETWORKIN

G

* Minicomputers * PCs

* WS Clusters

* PDAs* Workstations

* HTC

8/9/2019 grid-iitr

3/36

Internet and WWW Growth

1

10,000

100,000

1,000,000

10,000,000

1969 1970 1975 1980 1985 1990 1995 2000

10

100

1,000

4

Internet Hosts

WWW ServersNumberin

millions

8/9/2019 grid-iitr

4/36

Why Grids ? Large Scale Explorationneeds themKiller Applications.

Solving grand challenge applications using

computermodeling, simulation and analysis

Life Sciences

CAD/CAM

Aerospace

Military ApplicationsDigital Biology Military ApplicationsMilitary Applications

Internet &

Ecommerce

8/9/2019 grid-iitr

5/36

Cluster of Clusters - Hyperclusters

Scheduler

MasterDaemon

ExecutionDaemon

SubmitGraphicalControl

Clients

Cluster 2

Scheduler

MasterDaemon

ExecutionDaemon

SubmitGraphicalControl

Clients

Cluster 3

Scheduler

MasterDaemon

ExecutionDaemon

SubmitGraphicalControl

Clients

Cluster 1

LAN/WAN

8/9/2019 grid-iitr

6/36

Grid: Towards Internet Computing

for (Coordinated) Resource Sharing

- Unification of geographically distributed resources

Grid enables:

bResource Sharing

b

SelectionbAggregation

8/9/2019 grid-iitr

7/36

What is Grid ?

A paradigm/ infrastructure that enables the sharing,selection, & aggregation of geographically distributedresources like:

Computers PCs, workstations, clusters, supercomputers, laptops,notebooks, mobile devices, PDA, etc;

Software e.g., ASPs renting expensive special purpose applications ondemand;

Catalogued data and databases e.g. transparent access to humangenome database;

Special devices/instruments e.g., radio telescope SETI@Homesearching for life in galaxy.

People/collaborators.

[depending on their availability, capability, cost, anduser QoS requirements] for solving large-scaleproblems / applications.

8/9/2019 grid-iitr

8/36

A Typical Grid Computing

Environment

Grid Resource Broker

Resource Broker

Application

Grid Information Service

Grid Resource Broker

databaseR2R3

RN

R1

R4

R5

R6

Grid Information Service

R7

IIT Roorkee (GRB)

8/9/2019 grid-iitr

9/36

Building and Using Grids requires...

Services that make our systems Grid Ready! Security mechanisms that permit resources to be

accessed only by authorized users. New Programmingtools that make our applications

Grid Ready!. Tools that can translate the requirements of an

application into requirements for computers, networks,and storage.

Tools that perform resource discovery, trading,composition, scheduling and distribution of jobs andcollects results.

8/9/2019 grid-iitr

10/36

Grid@IITR

8/9/2019 grid-iitr

11/36

Grid@IITR using Alchemi

What is Alchemi? Enterprise grid framework and runtime machinery to

create a high-throughput computing environment byharnessing distributed resources .NET-based (Windows)

Voluntary execution (cycle stealing) or Dedicated execution LAN or Internet

Programming environment Independent grid threads (.NET API)

File-based jobs (input, executable, output) Web service for interoperability with other grid

middleware File-based jobs

Monitoring, administration tools

8/9/2019 grid-iitr

12/36

Why .NET based ?

Why .NET/Windows? More than 90% of machines worldwide run variants of

Microsoft Windows operating system. Hence designingfor Windows is seen as key factor in industry adoption

of grid computing technology Many features of the new .NET platform can be

leveraged

Support multiple languages - write API/libraries once inany .NET language and make use from any other .NETsupported language.

8/9/2019 grid-iitr

13/36

Alchemi Architecture

Alchemi Manager

e-Science Application e-Business Application e-EngineeringApplication

Windows-based machines with .NET Framework

Precompiled executables

e-Commerce Application

Alchemi Executor Alchemi Executor Alchemi Executor

Alchemi Jobs(XML representation)

Grid Threads (.NET objects)

Alchemi .NET API(Object-Oriented Grid Programming

Environment)

AlchemiConsoleInterface

Alchemi Cross-PlatformManager

8/9/2019 grid-iitr

14/36

Role-basedSecurity

Authentication:Simple username /password

Authorization:Role-basedpermissions

Auditing:All jobs/threadsexecuted arerecorded in adatabase and linkedto user account usedfor Authentication

8/9/2019 grid-iitr

15/36

8/9/2019 grid-iitr

16/36

As a Node for Global Grids

Cross-PlatformManager WebServices Interface

Alchemi grids asnodes (classical gridmodel)

mgrid broker

grid node(Globus-based)

e e e e

8/9/2019 grid-iitr

17/36

Grid Components:Alchemi

Grid application Consists of

independent gridthreads

Manager central

controller Discovery, scheduling,

dispatching,monitoring

Cross PlatformManager Web service interface

Executor workeragent

User Runs grid applications Monitoring

E

M

E EE

X

E

U

Custom GridMiddleware

M

X

E

U User Node

Manager Node

Executor Node

Cross PlatformManager Node

Legend

8/9/2019 grid-iitr

18/36

Execution Model

Dedicated Executor 2-way communication between Executor

and Manager

Voluntary Executor 1-way communication between Executor

and manager (Executor works from behindfirewalls)

Dual benefit Flexible resource management Flexible deployment

8/9/2019 grid-iitr

19/36

Performance Evaluation: StandaloneNode (High Precision Pi Calculation)

0

50

100

150

200

250

300

350

400

450

1000 1200 1400 1600 1800 2000 2200

Thread Size (no. of digits of Pi)

ExecutionTime(s

econds)

1 Executor

2 Executors3 Executors

4 Executors

5 Executors

6 Executors

8/9/2019 grid-iitr

20/36

Performance Monitor:High Precision PiCalculation

Grid - Single Executor

8/9/2019 grid-iitr

21/36

Performance Monitor:High Precision PiCalculation

Grid - Eight Executor

8/9/2019 grid-iitr

22/36

Performance Result:Pi Calculation SingleNode

Digits to Calculate: 1000 Total Time Taken: 00:03:39.8125000

8/9/2019 grid-iitr

23/36

Performance Result:Pi Calculation ThreeNodes

Digits to Calculate: 1000 Total Time Taken: 00:00:11.3437500

Grid Database

8/9/2019 grid-iitr

24/36

Grid Database@IITR

8/9/2019 grid-iitr

25/36

Grid@IITR

Applications

8/9/2019 grid-iitr

26/36

Types of Grid Applications

Sequential dusty deck codes.

Data Parallel: Synchronous tightly coupled;

Loosely synchronous.

Asynchronous: Irregular in time and space;

Difficult to parallelise to exploit the massiveparallelism.

Embarrassingly Parallel.

8/9/2019 grid-iitr

27/36

Distributed HPC (Supercomputing): Computational Science.

High-Capacity/Throughput Computing: Large scale simulation/chip design & parameter studies.

Content Sharing (free or paid): Sharing digital contents among peers (e.g., Napster)

Remote software access/renting services: Application service provides (ASPs) & Web services.

Data-intensive computing: Drug Design, Particle Physics, Stock Prediction...

On-demand, real-time computing: Medical instrumentation & Mission Critical.

Collaborative Computing: Collaborative design, Data exploration, education.

Service Oriented Computing (SOC): Computing as Competitive Utility: New paradigm, new

industries, and new business.

P2P/ Grid Applications

8/9/2019 grid-iitr

28/36

Ad Hoc Mobile Network Simulation

Ad Hoc Mobile Network Simulation: Network performance underdifferent microwave frequencies and different weather conditions

8/9/2019 grid-iitr

29/36

Drug Design: Data Intensive

Computing on Grid

It involves screening millions

of chemical compounds

(molecules) in the Chemical

DataBase (CDB) to identifythose having potential to

serve as drug candidates.

Protein

Molecules

Chemical Databases

(legacy, in .MOL2 format)

8/9/2019 grid-iitr

30/36

MEG (MagnetoEncephaloGraphy)Data Analysis on Grid: Brain Activity Analysis

ife-electronics laboratory,IST

Data Analysis

Provision of expertise inthe analysis of brain function

Provision of MEG analysis

Data Generation

Nimrod-G

64 sensors MEG

Results

Analysis All pairs (64x64) of MEG data byshifting the temporal region of MEG dataover time: 0 to 29750: 64x64x29750 jobs

World-Wide Grid[deadline, budget, optimizationpreference]

1

5

4

3

2

[Collaboration with Osaka University, Japan]

8/9/2019 grid-iitr

31/36

SETI@home: Search for

Extraterrestrial Intelligence at Home

8/9/2019 grid-iitr

32/36

Content Sharing P2P

C

8/9/2019 grid-iitr

33/36

Collaborative Engineering

Components of an AG NodeComponents of an AG Node

Digital Video

Digital Video

Digital Audio

NETWORK

MixerControl

Computer

NTSC Video

RGB Video

Analog Audio

Video

Capture

Computer

DisplayComputer

Audio

CaptureComputer

Echo

Canceller

Group to group interactions.Human collaboration across

distributed locationsRemote visualizations, virtual meeting,

seminars,etc.Uses grid technologies for secure

communication etc.May have interaction with scientific apps.

Access GRID: http://www-fp.mcs.anl.gov/fl/accessgrid/

8/9/2019 grid-iitr

34/36

Image-Renderinghttp://www.swin.edu.au/astronomy/pbourke/povray/p

arallel

8/9/2019 grid-iitr

35/36

Future of Grid

Access to any resources, for anyone, anywhere,anytime, from any platform portal (super)computing .

Application access to resources from the wallsocket!

Many applications provide solutions in real-time. Choice of working: office vs home vs . . .

Collaboratories for distributed teams. Monitoring and steering applications through

wireless devices (PDAs etc.).

8/9/2019 grid-iitr

36/36

Conclusions

The HPC will be dominated by Peer-to-PeerGrid of clusters.

Adaptive, scalable, and easy to use Systemsand End-User applications will be prominent.

Access electricity, internet, entertainment(music, movie,), etc. from the wall socket!

An Economics based Service Oriented Grid

Computing computing needed for eventual successof Grids! The impact of Grid on 21st century economy will be

the same as electricity on 20th century economy.