The EDGeS project receives Community research funding XtremWeb-HEP & EGEE CSST-HUST-Wuhan Octobre 11-15th, 2010 Oleg Lodygensky - LAL - [email protected].

The EDGeS project receives Community research funding

XtremWeb-HEP & EGEEXtremWeb-HEP & EGEE

CSST-HUST-WuhanCSST-HUST-Wuhan

Octobre 11-15th, 2010Octobre 11-15th, 2010

Oleg Lodygensky - LAL - Oleg Lodygensky - LAL - [email protected]@lal.in2p3.fr

2Authors : O. Lodygensky HUST : XtremWeb-HEP 2

EDGeSEDGeSBOINC-based DGs

XtremWeb/XWHEP-based DGs

localXtremWeb DG

IN2P3200 PCs

300 (150%)

new public XtremWeb DGEDGeS@home

1.000 PCs

EGEE VO

publicXtremWeb DG

IN2P3

300 PCs6 (2%)

publicXtremWeb DGAlmereGrid

3.000 PCs?

new public BOINC DG

EDGeS@home

10.000 PCs

publicBOINC DG

SZDG

30.000 PCs77.000 (256%)

publicBOINC DG

Extremadura

70.000 PCs22.500 (32%)

localBOINC DG

Westminster1.500 PCs

1881 (125%)

localBOINC DGCorrelationSystems

publicBOINC DGAlmereGrid

0 PCs1.700

publicBOINC DGIbervicis

0 PCs24.000

3HUST : XtremWeb-HEPAuthors : O. Lodygensky 3

EDGeSEDGeS

desktopgrid.vo

voms

lb wmsbdii

an EGEE VO

BOINC-EGEEbridge

EGEE>DGbridge CE

UoW queueSDZGr queueAlmere queue

Fundecyt queue

BOINC-based DGs

UoW, Correlation

Systems local DGs

SZDGr,

Almere,

Fundecytpublic DGs

AR

UICoreService Resource

lfc

bridge

bridge

bridge

bridge

myproxy

ce

wms

ce

4

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

Bridging EGEE and Desktop Grids

Grid User

X509 proxy

VOMS Proxy

Submits Job with VOMS proxy

Publishes available Resources

Pushes Job

Log

Log

VOMS Admin

Manages VO

Site Admin

Manages Site

Accesses Data with VOMS proxy

Accesses Data with VOMS proxy

Gives Job Status

Gives Accounting and Auditing

VOMS Server

AccountingLogging & Bookkeeping

Meta-scheduler(WMS)

Site Computing Resource

Site Storage

Resource

Grid Admin

Sends back Output Sandbox Sends back

Output Sandbox

VOMS Proxy =X509 proxy with

VOMS extensions

EGEE (Service Grid)Well defined infrastructure:- authentication- authorization- logging and bookkeeping- quality of service (QoS)

A strong effort of standardization

Volunteer PCs

Volunteer PCs

Desktop Grids : Boinc, XWHEPUndefined infrastructure:- anonymity- volatility

How to bridge ? - security - monitoring - logging

DG User

?

Server(scheduler)

5

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

XWHEP->EGEEXWHEP->EGEEresource sharingresource sharing

9

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

XWHEP :XWHEP :XtremWeb by High Energy PhysicsXtremWeb by High Energy Physics

LAL :•LCG Tiers-2•EGEE seed resource• XtremWeb by High Energy Physics (XWHEP) developer & maintainer

XWHEP is based on XtremWeb 1.8.0. by INRIA.

http://www.xtremweb-hep.org/spip.php?rubrique16

XWHEP ensures security at different levels:

• authentication :✓ SSL and X509 certificates

• authorization and confidentiality :✓ user rights management✓ user groups✓ access rights

• volunteer nodes integrity :✓ application repository✓ sandboxing

• monitoring and bookeeping :✓all messages are logged

XWHEP environment➡ multi users➡ multi applications

10

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

XWUser Job

Execution Sandbox

Userdata andapplications

Dynamically downloadedXW user dataandapplication

Externaldata

servers

Server certificate

Server public key

Job Mgt

DeployedXW-Client UI

DeployedXW-Computing

Service

PC

Volunteer PC

Data Sandbox

All communicationsare encrypted

Data Mgt

XW Services

Local I/O

XWHEP : Architecture, authenticationXWHEP : Architecture, authenticationand resource integrityand resource integrity

Authentication: - login/password - X509 cert. Node integrity:

- sandboxing

Authentication: - login/password - X509 cert.

XW Coordinator(scheduler)

XW Data Repository

24

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

VOMS Server

Meta-scheduler(WMS)

DG User

jLite : proxy init

XW Coordinator(scheduler)

jLiteby Oleg Sukhoroslovhttp://code.google.com/p/jlite/

Public worker

Group worker

Private worker

User X.509 Cert

User X.509 proxy

Server certificate

Server public key

Security, monitoring and logging are the main issues in Pilot Jobs. (http://edms.cern.ch/document/855383)

XWHEP->EGEEXWHEP->EGEEbridgingbridging

XW Bridge

Submit Pilot jobw/ user X509 proxy

Retreived signedDG user job

DG user job submissionwith X509 proxy

DownloadDG userX509 proxy

Pilot Job

Private worker

WN

Site Computing Resource

Pilot Job submitted as EGEE Job

DG user job deployment, status, results

27

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

EDGeS MonitoringEDGeS Monitoring

28

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

How to port an application fromHow to port an application fromXWHEP to EGEEXWHEP to EGEE

•EGEE is made of Linux computing resourcesEGEE is made of Linux computing resources•XWHEP applications must have a Linux XWHEP applications must have a Linux

versionversion

•There is no specific need to recompile nor There is no specific need to recompile nor relink binariesrelink binaries

29

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

EGEE->XWHEPEGEE->XWHEPresource sharingresource sharing

30

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

Bridging EGEE and Desktop Grids

Grid User

X509 proxy

VOMS Proxy

Submits Job with VOMS proxy

Publishes available Resources

Pushes Job

Log

Log

VOMS Admin

Manages VO

Site Admin

Manages Site

Accesses Data with VOMS proxy

Accesses Data with VOMS proxy

Gives Job Status

Gives Accounting and Auditing

VOMS Server

AccountingLogging & Bookkeeping

Meta-scheduler(WMS)

Site Computing Resource

Site Storage

Resource

Grid Admin

Sends back Output Sandbox Sends back

Output Sandbox

VOMS Proxy =X509 proxy with

VOMS extensions

EGEE (Service Grid)Well defined infrastructure:- authentication- authorization- logging and bookkeeping- quality of service (QoS)

A strong effort of standardization

Volunteer PCs

Volunteer PCs

Desktop Grids : Boinc, XWHEPUndefined infrastructure:- anonymity- volatility

How to bridge ? - security - monitoring - logging

DG User

?

Server(scheduler)

31

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

gLite ensures security:

• authentication :✓ X509 cert. and VOMS ext.

• authorization and confidentiality :✓ VOMS server

• monitoring and bookeeping

EGEE->XWHEP bridge :EGEE->XWHEP bridge :authentication, authorization, logging, authentication, authorization, logging,

monitoringmonitoring

XWHEP ensures security at different levels:

• authentication :✓ SSL and X509 certificates

• authorization and confidentiality :✓ user rights management✓ user groups✓ access rights

• volunteer nodes integrity :✓ application repository✓ sandboxing

• monitoring and bookeeping :✓all messages are logged

35

Volunteer PCs

Volunteer PCsXtremWeb

User

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

QM WS client

QM

XW

plu

gin

WS

Int

erfa

ce

P-GRADE Portal

EDGeSEDGeSARAR

EGEE -> XtremWebEGEE -> XtremWebresource sharingresource sharing

VOMS Server

LCG CE

Meta-scheduler(WMS)

EDGeS CE XW Coordinator(scheduler)

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky 8

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky 10

Standard gLite tools usage

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky 17