EGEE-II INFSO-RI- 031688 Enabling Grids for E-sciencE www.eu-egee.org EGEE and gLite are registered trademarks The EGEE project and the gLite middleware Gergely Sipos ([email protected]) EGEE “User training and Induction” Activity deputy manager MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences
32
Embed
EGEE-II INFSO-RI-031688 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks The EGEE project and the gLite middleware.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
CoreGRID Summer School 2007 – Budapest, 3-7 September 3
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Introduction
• The EGEE project:– 4 year project, funded by the EU (EGEE, EGEE-II)– Seamless Grid infrastructure for e-Science, available for
scientists 24 hours-a-day
• EGEE:1 April 2004 – 31 March 2006
71 partners in 27 countries, federated in regional Grids
• We are now in 2nd phase of the project
EGEE-II:1 April 2006 – 31 March 2008
91 partners in 32 countries grouped into 13 federations
• EGEE-III proposal is under preparation
Transition to self-sustainable European Grid service infrastructure
CoreGRID Summer School 2007 – Budapest, 3-7 September 4
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
EGEE goals – international e-infrastructure
Network infrastructure& Resource
centres
Op
era
tio
ns
, S
up
po
rt a
nd
tr
ain
ing
Collaboration
International Grid
• Build, deploy and operate a consistent, robust a large scale production grid service that– Links with and build on
national, regional and international initiatives
• Improve and maintain the middleware in order to deliver a reliable service to users
• Attract new users from research and industry and ensure training and support for them
CoreGRID Summer School 2007 – Budapest, 3-7 September 5
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Production Service
Real Time Monitor – Java tool– Displays jobs
running (submitted through brokers)
– Shows jobs moving around world map in real time, along with changes in status
http://gridportal.hep.ph.ic.ac.uk/rtm/ (snapshot 16 January 2007)
CoreGRID Summer School 2007 – Budapest, 3-7 September 6
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
• From April 2006, natural continuation of EGEE– Expanded consortium – Emphasis on providing an infrastructure
increased support for applications interoperate with other infrastructures more involvement from Industry
SA: service activities- establishing operations
NA: network activities- supporting users and applications
JRA: “joint research activities”- e.g. hardening middleware
EGEE-II
CoreGRID Summer School 2007 – Budapest, 3-7 September 7
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
EGEE Infrastructure
Country participating
in EGEE
> 200 sites in 40 countries~ 36 000 CPUs~ 5 PB storage98k jobs/day> 200 Virtual Organizations⇨The world’s largest multi-disciplinary Grid
CoreGRID Summer School 2007 – Budapest, 3-7 September 8
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Who is using EGEE?
• >200 VOs from several scientific domains– Astronomy & Astrophysics– Civil Protection– Computational Chemistry– Comp. Fluid Dynamics– Computer Science/Tools– Condensed Matter Physics– Earth Sciences– Fusion– High Energy Physics– Life Sciences
• Further applications under evaluation
98k jobs/day
Applications have moved from testing to routine and daily usage
~80-90% efficiency
CoreGRID Summer School 2007 – Budapest, 3-7 September 9
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Virtual organisations and grids
• What is a Virtual Organisation?– People in different organisations seeking to cooperate and share
resources across their organisational boundaries – E.g. A research collaboration
• Each grid is an infrastructure enabling one or more “virtual organisations” to share and access resources
• Each resource is exposed to the grid through an abstraction that masks heterogeneity, e.g.– Multiple diverse computational platforms– Multiple data resources
• Resources are usually owned by VO members. Negotiations lead to VOs sharing resources
CoreGRID Summer School 2007 – Budapest, 3-7 September 10
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
EGEE Infrastructure
• Production service (production VOs)– Based on gLite 3.0 middleware– With new resource centres and new applications encouraged to
participate– Stable, well-supported infrastructure, running only well-tested and
reliable middleware
• Pre-production service– Run in parallel with the production service (restricted no. of sites)– First deployment of new versions of the middleware – Applications test-bed
• GILDA training testbed To be used in the afternoon– https://gilda.ct.infn.it/testbed.html– Complete suite of Grid elements and applications
Testbed, CA certification, VOs, monitoring– Everyone can register and use
GILDA for training and testing
CoreGRID Summer School 2007 – Budapest, 3-7 September 11
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Basic services of the EGEE grid
Computing Element
Storage Element
Site X
Information System
Submit job
Submit job
queryRetrieveoutput
Retrieveoutput
Resource Broker
User Interface
publishstate
File and Replica Catalog
Authorization Service (VO Management Service)
query
createcredential
CoreGRID Summer School 2007 – Budapest, 3-7 September 12
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Who provides the resources?!
ServiceService Provider Note
User interfaceUser interface User / institute / VO Computer with client software
Resource BrokerResource Broker VOs - EGEE does not fund RBs
Information SystemInformation System Grid operations - EGEE funded effort
File and replica catalogFile and replica catalog VOs - EGEE does not fund catalogs
Computing Element Computing Element (cluster)(cluster)
VOs - EGEE does not fund CEs
Scalability requires that VOs provide resources to match average need
Storage Element Storage Element VOs - EGEE does not fund SEs
VO or application specific services
User / institute / VO To extend the capabilities of the core infrastructure
CoreGRID Summer School 2007 – Budapest, 3-7 September 13
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
gLite security – the user’s viewpont
• User’s checklist– User obtains grid certificate from
a recognized Certificate Authoritywww.gridpma.org
– User finds and joins a VOhttp://cic.gridops.org/
– User gets account on the UI machine (and in a Grid portal)
– User connects to UI by ssh and generates proxy
(OR User logs it to a portal service and generates proxy)
• VO administrator’s checklist– Receive user’s join request– Add user’s identity and role into
VOMS DB– Create account on UI
UI
CA
VO mgr
Annually
VO database
Grid Security
Infrastructure
VOMS
Once
VO services: broker, CE, SE, …
CoreGRID Summer School 2007 – Budapest, 3-7 September 14
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Security – viewpoint of the grid
User Interface
Storage ElementBroker
Computing Element
User’s public key
User’s private key
Proxy
Generate VO database
VOMS server
Proxy+
VO role
Proxy+
VO role
Proxy+
VO role
Proxy+
VO role
CoreGRID Summer School 2007 – Budapest, 3-7 September 15
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Generating proxy – user command
[giorgio@glite-tutor giorgio]$ ls -l .globus
total 8
-rw-r--r-- 1 giorgio users 1613 Jun 7 15:16 usercert.pem
-r-------- 1 giorgio users 1914 Jun 7 15:16 userkey.pem
CoreGRID Summer School 2007 – Budapest, 3-7 September 29
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Summary - EGEE and gLite
• EU founded Grid project: 2004-2006 EGEE; 2006-2008 EGEE-II– Build, deploy and operate a consistent, robust a large scale production
grid service that– Improve and maintain the middleware in order to deliver a reliable
service to users – Attract new users from research and industry and ensure training and
support for them • gLite middleware: Bag of services that address key distributed
system issues:– Security, Data management, Job management, Monitoring
• Basic components– User interface access service for end users– Workload Management System resource broker– Computing Element computing service, a job queue– Storage Element File storage– File catalog File registry, virtual file system– Information System database of SEs and CEsAll built onto Grid Security Infrastructure (GSI)
• High level community or application specific tools
CoreGRID Summer School 2007 – Budapest, 3-7 September 30
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Furhter information, references
• EGEE
– http://www.eu-egee.org/
• gLite middleware
– http://www.glite.org
• gLite manuals, documentation
– http://glite.web.cern.ch/glite/documentation/ (gLite user guide)
• Recommended External Software Packages for Egee CommuniTies
– http://egeena4.lal.in2p3.fr/
CoreGRID Summer School 2007 – Budapest, 3-7 September 31