Recent advances in GRID use developed at CNES: The Virtual Private Facilities (WGISS-20, Kiev )

JP Antikidis, WGISS20-Kiev, 12 sept 2005

Recent advances in GRID use developed at CNES: The Virtual Private Facilities

(WGISS-20, Kiev )

Earth Observation as a tool for Services Virtual Private facilities & GRIDs The Emulation way (EPs versus IPs) Practical consequences

Jean Pierre AntikidisCNES Programme & Strategy Directorate-ARP"Space Information Systems”[email protected]


To day EO large scale objectives

World-wide follow-on ozone

World-wide follow-on of sea state (MERCATOR)

World-wide follow-on of vegetation index

World-wide follow-on of crop and food resources

GMES programme

GEO endeavour

....

Can we afford on the long run to pile-up investments as we do today ?


What is a Software based service ?

It is a set of Hardware , Software and user interfaces connected in such a way a given end-user service is provided.

To make it available we need practically:

# To decide on the needed HW arrangement

# To decide on a suitable SW running on this HW

# To decide on machines interconnections and interfaces with final users

This can be made on a real fashion (Private Facilities) or rely on an exact virtual equivalent (Virtual Private facilities).

=> One possible way to implement a VPF is to “Emulate” the needed HW/SW/Files and Connections topology


Some upcoming conceptsService is the Goal and is achieved by creating Hw/Sw facilities:

The Network story

PN: Private Network

VPN: Virtual PrivateNetwork=VP+distr.Networks

X. Jiang, D. Xu, “VIOLIN: Virtual Internetworking on OverLay INfrastructure”, Perdue Un.

Combined Services (Network+computers+data)

PF: Private Facilities

VPF: Virtual Private facilities=PF+GridsEmulation is one way to create a VPF =>EVPF


Exemple of to-day computer systems definition for a given Service


A Software service created as a VPF

VPF=MACHINES+PROCESSES+FILES+CONNECTIONS

Machine 1

Machine 7

Machine 3

Machine 2

Machine 4

Machine 6

Machine 5

Sw3

Sw7

Sw6Sw5

Sw4

Sw3

Sw2Sw1

Real File 1

Real File 2 Real access to Web service Real File 3

Virtual File


Experiment Setup(ref:violin project)

Physical Cluster(ITaP)

Two mutuallyisolated virtual clusters

VM

Physical Switch

VS VS




The HEAVEN paradigm

"The result produced by running an "emulator" is equivalent to the result produced by the real system."HEAVEN can "mimic" any required system

Instead of creating complex processing system, perhaps better describe them in a powerfull and fully flexible computing topology and make the resulting emulation running as a replacement for the real system.


Simple emulation

Game machine Emulation on a PC

PC emulation in a PC (looks stupid but …)

SW1

SW2SW2

SW1


Somewhat more interesting…

Emulation of a complex PC set of machines with data storage & multi-Hw/Sw layout ("Still stupid but not that much in a GRID based system") Key issue: Should power compute resources becomes cheap and flexible enough, perhaps better exchange flexibility and full Sw compatibility with raw CPUs power=>> GRID is bringing the needed characteristics for such an evolution

SW1

SW2

SW3SW1

SW2

SW3


A Software service created as a EVPF

EVPF= Emulation of (MACHINES+PROCESSES+FILES+CONNECTIONS)

Machine 1

Machine 7

Machine 3

Machine 2

Machine 4

Machine 6

Machine 5

Sw3

Sw7

Sw6Sw5

Sw4

Sw3

Sw2Sw1

Real File 1

Real File 2 Real access to Web service Real File 3

Virtual File

Exchange Virtual File


Building elements for a given machine

Emulated machine Standard SW Exchange Files File services

Machine “n”

Software “n”

Virtual Real


Multitasking environnement

Reproduces the full topology needed for a given VPF

Machine “n”

Software “n”

Machine “n”

Software “n”

Machine “n”

Software “n”

Files & data





Multitasking environement implementation

1) Create as many machine Emulators as needed machines by either:

- Creating a machine HW emulator and plug a genuine O/S on it

- Emulate a full compound of machine+O/S emulator

2) Plug without change the applications softwares associated to any of the needed machines

3) Describe inter-machine data exchanges with respect to their content (can correspond to real or virtual files)

The result behave in an asynchronous fashion as the full equivalent to the real implementation


“With VMWare, you have the option to run your choice of 'primary' OS at all times. Then, when you need access to the other OS all you need to do is start up a vmware session. VMware includes file system shares to facilitate getting files betweeen one environment and the other, and it automatically supports a seamless NAT translation that allows your virtual machine to have full access to the Internet”

The Emulation Packet concept (VMWare based example)

Emulation Packet (1 EP per Machine to reproduce)

EP


HW EMULATOR

O/S IMPLEMENTATION

FILES DEFINITIONS (METADATA)

“Emulation Packet”

APPLICATION SW

Autonomous processing packet containing:The HW emulator (To be developed once for a given machine brand)The genuine O/S (in theory the one already available)The native user application SW (no change)The Files description associated to the given machine (as created by the GUI)

STANDALONE “Emulation Packet” (1 EP per Machine)EP


EPs versus IPs (Grid versus Internet)

Same stand with “EP” concept (these are much bigger packets of course !) in which Emulation packets once generated becomes fully autonomous with respect to their location or timing. This allows the full compatibility of millions of EPs at a time corresponding to perhaps several thousand of “Private facilities” that look from the user viewpoint as a pseudo HW/SW service implementation under his full control.

The efficiency of Internet principle rely on the IP concept by which a data packet once submitted to the system becomes autonomous (no time nor routes relations in between the packets)

This allow Internet to handle on an unvisible fashion Billions of packets at a time without need for strong centralised means.


EP

EPEPEP

EP

EPEPEP

Power Computing Resources

Graphical Interface for Service developer

Emulation Packet prep.XML descr.

End userReal interfaces

Information provider

Virtual Private Facilities created by Emulation

UpperWare

UpperWare


GRID environement implementation

1) Create as many Emulation Packets (“EP”s) machine Emulators as needed machines by either:

- Creating a machine HW emulator and plug a genuine O/S on it

- Emulate a full compound of machine+O/S emulator

2) Plug without change the applications softwares associated to any of the needed machines

3) Describe inter-machine data exchanges with respect to their file content (can correspond to real or virtual files)

The result behave in an asynchronous fashion as the full equivalent to the real implementation


HEAVEN Systems and services deployments onto a GRID based on autonomous “Emulations Packets”

EP EP

EP

EP

EPEP

EPEP

EPEP

EPEP

UPPERWARE

EPEPEPEP

GRID

GUI

Files Web Service

Interfaceto GRID

Ftp?

Urls ?


• A grid can generate significant value from idle resources but:– Users must be given absolute control of their environment.– Owners must retain absolute control of their machines.

• Virtualisation/Emulation enables this to happen:– Administration can be simple enough to become a fixed cost.– Security concerns can be minimised almost to the “anonymous” level.

• We still need new technology to make this happen, but we may be very close……

Quote CNES=>>> VFC, EVFC,WAG are such a Tool

CONCLUSION, Quote: "Overview of New technologies for using Grids by :Rhys NewmanManager of Interdisciplinary Grid Development, Oxford University


Virtualisation is the Key

• To provide these properties, reliably and uniformly on heterogeneous resources, the user’s environment must be “Virtual”.

• Simplifies the administration:– The “service end” of eScience can focus on the provision of

“virtual hardware”– The “user end” of eScience can focus on developing applications

in a standardised environment.

• Imagine M users and N owners, each wanting their concerns addressed:– Without Virtualisation: MxN negotiations.– With Virtualisation: M+N negotiations.

Recent advances in GRID use developed at CNES: The Virtual Private Facilities (WGISS-20, Kiev )

Documents