Enabling Grids for E-sciencE www.eu-egee.org EGEE is a project co-funded by the European Commission under contract INFSO-RI- 508833 Web Services, EGEE and Richard Hopkins Training Team, National e-Science Centre, Edinburgh, UK [email protected]National Grid Services Induction March 11 2005
43
Embed
Enabling Grids for E-sciencE EGEE is a project co-funded by the European Commission under contract INFSO-RI-508833 Web Services, EGEE and.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Enabling Grids for E-sciencE
www.eu-egee.org
EGEE is a project co-funded by the European Commission under contract INFSO-RI-508833
Web Services, EGEE and
Richard HopkinsTraining Team, National e-Science Centre, Edinburgh, [email protected]
National Grid Services InductionMarch 11 2005
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 2Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEOutline and Acknowledgements
• Grids and Web Services• EGEE• gLite
Acknowlegements
• This talk has drawn on material produced by many people, including - • Tony Hey• Mike Mineter• EGEE colleagues
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 3Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencESpecialised DistributedComputing
Grids come within the general field of
• Parallel / Distributed Computing
A computational task involves coordination of components which occur
• Simultaneously
And/or
• At physically separated locations
The Distribution dimension -
Multi-processor
ProcessorPool
Embedded Systems
Flexible Manufacturing . . .
GRIDS
Parallel Distributed
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 4Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEGrids & Web Services
A very inclusive formulation of “Grid Computing” is
• Coordination of computational components
• Processing / storage resources
• Of up to international level of geographic separation
• Crossing organisation boundaries
A (Web) service is a
• S/W system designed to support interoperable machine-to-machine interaction over a network.
Similar concepts
• For grids, the focus is on a computational model – Running a job Marshalling the data needed
• For web services – the focus is on service inter-operability A service can be any functionality you want to provide via the web
Need to bring these technologies together
• First we look closer at Web Services
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 5Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEWeb Services Approach
Web Services is the next step in the automation of inter-enterprise interaction
Web Browsing
• Human travel agent provides “organise holiday” service by surfing the web to look for and invoking services – book a hotel; book a plane; book a car hire; ….; confirm bookings of best options to meet client needs.
Web Services
• The aspiration of Web services is to provide a framework that allows that same model to be used in writing an application –
• which is itself becomes an “organise a holiday” service, finding and using useful services
Mode human intervention at –
service provider service consumer
E-mail
Web browsing
Web Services
Yes Yes
No Yes
No No
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 6Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEService Interaction
I book planes I book car Rentals
I book hotels
I locate services
I organise holidays Get a car rental quote
locate service ask for quote
Is quote good enough?YesReserve it
… get other resources reserved
Confirm booking
Service Identifier
I convert currency
I know the weather
quote
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 7Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEEssentials of WS Approach
Need to achieve effective cooperation even though
• the different services are produced by different organisations, without any design collaboration
• the services are autonomously evolving
• Loose coupling –
minimum prior shared information between the designer of the two components of an interaction
This requires
• Self-description – Meta data
• Tolerance of partial understanding
Collaboration is on defining generic standards,
rather than on specific design
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 8Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencECore WS
Basic Requirements
• Service Description – WSDL • Web services Definition language, defines -
logical service –
Operations. Message structures for each Physical – protocol, location
• Allows service user to bind to the service Statically – use WSDL to generate code Dynamically – using -
• Service Registry and Discovery Services – UDDI• Universal Description, Discovery and integration• Allows one service to register with a registry
known by potential users• User can discover it and obtain its description
• Usability Negotiation – Policies
I book hotels
PolicyQuote- held - 10 minsProv Reserv. Held - 2 hours….
I locate services
register
I organise holidays
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 9Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencECore WS
Give me a quote for …
Basic Requirements
• Communication Protocol - SOAP
• Request - Response
• Other patterns – e.g notification
Ask car rental service to notify me of any special deals
To notify me if my reservation hold time is about to expire
• Common Language – XML Documents
• Tag-value pair
• Defined by a Schema
• Used for messages, policies, service definition
• Specific extensibility mechanism
• Self Description – tagged values, WSDL
• Necessary for loose coupling
• Autonomous evolution
• Autonomous evolution – discovery service – UDDI
or out of band
Type = QuoteValidity = 1 hourPrice = £200….
Quote
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 10Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEEvolving Standards
• Collaboration is on defining generic standards, not specific design
• Two main standards bodies –
• W3C – actually produces “recommendations” – web community
• OASIS – industry – IBM, Microsoft, Sun, ….
• These standards are factored to allow partial adoption and combination
• The core standards
• WS-I – clarifications to aid interoperability
• Higher level standards built on them
XML*
SCHEMAS*WSDL*
DTD
Core WS
SOAP*
*WS-Interoperability
WSRF
WS-addressing
WS-notificationWS-security
WS-TransactionFramework
UDDI*
…
WS-MetaDataExchange
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 11Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencECore WS
• XML – the standard format for all information
• SCHEMA –the standard language for defining the structure (syntax/type) of a unit of information
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 12Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencESome Further Standards
• WS-Security – Framework for authentication and confidentiality
• WS-Transaction Framework – for robustness of correlated interactions, e.g two phase – provisionally book everything, then confirm everything
• UDDI – standard repository interface (included in WS-I)
• WS-MetaDataExchange – how to communicate meta-data
• ….
XML*
SCHEMAS*WSDL*
DTD
Core WS
SOAP*
*WS-Interoperability
WSRF
WS-addressing
WS-notificationWS-security
WS-TransactionFramework
UDDI*
…
WS-MetaDataExchange
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 13Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEWSRF-Related Standards
• WS-Addressing - For communication of identities between services
• WS-Notification - Framework of notification interaction – subscribe, publish
• WSRF – Web Services Resource Framework
– E.g. a quote is a persistent entity which will need to be identified in subsequent interactions to finalise a provisional booking – a resource
– Consistent standard framework for creating, identifying, destroying resources
XML*
SCHEMAS*WSDL*
DTD
Core WS
SOAP*
*WS-Interoperability
WSRF
WS-addressing
WS-notificationWS-security
WS-TransactionFramework
UDDI*
…
WS-MetaDataExchange
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 14Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencE Grid Services
Web Service Grids
• Grid middleware can be seen as a special application area of web services dealing mainly with –
VOs data replication job execution
• Many advantages to basing grids on web services Cooperation between autonomously evolving components Mix-and-match of components Combining Aspects of grids with other Web Service application
domains Rapid Development of higher-level services Leverage industrially-produced support packages
Now led by HEP, Bio-info• Administrative and policy decision framework in order to share
resources at pan-European scale (and beyond) Areas such as AAA (Authentication, Authorisation, Accounting) End-to-end issues (Network related) Funding Policies (Grid economics) Resource Sharing Policies Usage Policies
• EGEE project is tackling most of the above issues
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 23Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencE
In the first 2 years EGEE will
• Establish production quality sustained Grid services • 3000 users from at least 5 disciplines• integrate 50 sites into a common infrastructure• offer 5 Petabytes (1015) storage
• Demonstrate a viable general process to bring other scientific communities on board
• Propose a second phase in mid 2005 to take over EGEE in early 2006
Pilot New
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 24Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEEGEE Organisation
• 70 leading institutions in 27 countries, federated in regional Grids
• ~32 M Euros EU funding for first 2 years starting April 2004 (matching funds from partners)
• Leveraging national and regional gridactivities
• Promoting scientific partnershipoutside EU
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 25Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEActivities Definition
• Network Activities• NA1: Project Management• NA2: Dissemination and Outreach• NA3: User Training and Induction• NA4: Application Identification and Support• NA5: Policy and International Cooperation
• Service Activities• SA1: Grid Support, Operation and Management• SA2: Network Resource Provision
• Joint Research Activities• JRA1: Middleware Reengineering + Integration• JRA2: Quality Assurance• JRA3: Security• JRA4: Network Services Development
Emphasis in EGEE is on operating a productiongrid and supporting the end-users
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 26Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEOperations - Introduction
• Strategy has been to • have a robust certification and testing activity,
• simplify as far as possible what is deployed, and to make that robust and useable.
• In parallel construct the essential infrastructure needed to operate and maintain a grid infrastructure in a sustainable way.
• Current service based on work done in LCG – culminating in the current service (“LCG-2”)• Now at the point where in parallel we need to deploy and understand gLite –
whilst maintaining a reliable production service.
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 27Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEEGEE Service Activities
• Create, operate, support and manage a production quality infrastructure
• Offered services:• Middleware deployment and
installation• Software and documentation
repository• Grid monitoring and problem tracking• Bug reporting and knowledge
database• VO services• Grid management services
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 28Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencESA1 – Operations Structure
• Operations Management Centre (OMC):• At CERN – coordination etc
• ROC is responsible for ensuring problem is resolved
• ROC oversees regional RCs
• ROCs responsible for organising the operations in a region
• Coordinate deployment of middleware, etc
• CERN coordinates sites not associated with a ROC
CIC
CICCIC
CICCIC
CICCIC
CICCIC
CICCIC
RCRC
RCRC RCRC
RCRC
RCRC
ROCROC
RCRC
RCRC
RCRCRCRC
RCRCRCRC
ROCROC
RCRC
RCRC RCRC
RCRC
RCRC
ROCROC
RCRC
RCRC
RCRC
RCRC
ROCROC
OMCOMC
RC = Resource Centre
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 30Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencE(Human) Networking Activities
• Dissemination and Outreach: 5% of EGEE budget• Dissemination – to actively promote and raise awareness of the EGEE project• Outreach – to identify and contact potential new user communities
• Training and Induction: 4% of EGEE budget• Induction – to introduce and orient - users and members• Training – to create, collate, make available and deliver material and courses
• Application Identification and Support• Process for selecting new application areas• Supports selected VO’s in porting applications
• International cooperation
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 31Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencE Dissemination
• 1st project conference, Cork, April • 2nd conference in The Hague
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 40Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencEApplication communities and EGEE
• LCG and Bio-informatics from day 1
• New application communities are selected by the EGEE Generic Applications Advisory Panel:• For new applications• See: EGEE web site (NA4 activity) and also
• Also working with DILIGENT:• Virtual digital data libraries
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 41Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencE
• Prototyping short development cycles for fast user feedback –
Requirements
Planning & Design
Implementation
Testing
Development Cycle
Globus 2 based Web services based
gLite-2gLite-1LCG-2LCG-1
Deployment : Prototype Infrastructure → Pre-production service → Production Service
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 42Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencE
Open Source Software License
• The existing EGEE grid middleware (LCG-2) is distributed under an Open Source License developed by EU DataGrid project• Derived from modified BSD - no restriction on usage
(academic or commercial) beyond acknowledgement
• Approved by Open Source Initiative (OSI)
• Same approach for new middleware (gLite)• New license agreed by partners is derived from the
EDG license and takes into account feedback from the World Intellectual Property Office (WIPO)
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 43Richard Hopkins, Training Team, NeSC
Enabling Grids for E-sciencE
EGEE www.eu-egee.orgLCG lcg.web.cern.ch/LCG/“Concertation event” and Second EGEE conference:http://public.eu-egee.org/conferences/2nd/programme/outline.html
The Grid Cafe www.gridcafe.org•More EU sites:
•http://www.cordis.lu/ist/grids/fp6_grid_projects.htm•http://www.gridstart.org/concertation_mtg.shtml•“e-Infrastructures Reflection Group http://www.e-irg.org
•NeSC www.nesc.ac.uk
Further Information
NGS Induction, NeSc, March 11 2005 – Web Service Grids - 44Richard Hopkins, Training Team, NeSC