Software from Science for Science Steven Newhouse, Director.

Software fromScience for Science

Steven Newhouse, Director

©2

Community Engagement July 2004: ‘The Roadtrip’

Functional Issues: Security, Running Jobs, Moving Data, …

Autumn 2006: SUPER Study of User Priorities for e-Infrastructure for e-Research Non-functional issues:

Meta-data, file management, VOs, … Support: Tools, services, training and consultancy Infrastructure: Authentication, software licensing and reliable

consistent environments Good News: Problems have changed over two years! Bad News: Users are happier but not happy!

©3

Providing Solutions

Higher-level e-Infrastructure Services

Scientists doing Science

Lower-level e-Infrastructure Services

Data Resources

Bio

info

rmat

ics

Eng

inee

ring

Art

s &

Hum

aniti

es

Indu

stria

l Par

tner

s

Compute Resources

Communities

Hig

h E

nerg

y P

hysi

cs

gLite (EGEE)

GLOBUS TOOLKITOMII-UK

©4

Our Mission…

OMII-UK aims to provide software and support to enable a sustained future for

the UK e-Science community and its international collaborators

•Promote the use of good-quality open-source software•Reduce the risk of moving to new e-infrastructure world•Recognise distinct user communities: by domain and function

©5

Manchester

Southampton

Edinburgh

University of Manchester

Electronics and Computer Science

University of EdinburghEuropean Bioinformatics Institute

The OMII-UKPartnership

Cambridge

•Southampton: 14 FTEs•Manchester: 9 FTEs•Edinburgh: 8 FTEs•Community: ~8 FTEs

©7

User Communities

AppliedResearchDomain

Casual User(Novice

or Infrequent)

Intensive User(Expert

or Focused)

Technologists

Assemblersof domain

Components/Services/Tools

Buildersof domain

Components/Services/Tools

Assemblers of generic

Components/Services/Tools

Builders of generic

Components/Services/Tools

Providers

VO Managers

ResourceOwners

Helpdesk &Training

SystemAdministrators

Applied e-Researchers Technology Specialists e-Infrastructure Providers

Users

Applied Technology Specialists e-Infrastructuree-Researchers (domain & generic) Providers

©8

Software for Science from Science Fund short-term open-source community

development work Responsive mode

(http://www.omii.ac.uk/projects/funding.jsp) Calls

Accessing Grid from the Desktop Portets Support Grid API

Support consolidation from one project to enable reuse in others (NB: Not research)

Increase confidence & accelerateadoption of open-source software

©9

e-Science Community

Informationabout

softwarethat you

have founduseful

Open SourceDevelopment Community

Information aboutsoftware that you

are developing thatothers might use

Contributing softwareinto the

Repository

Open SourceDevelopers funded by OMII-UK

in the community

Website & Wiki SoftwareRepository

Software Catalogue

Register a softwareproject or activity

Website Interactions

©10

OMII-UK User Community

Download individual software components

directly from repositoryOMII-UK Software Release

Software componentsintegrated and testedto form the OMII-UK

software release

-

AppliedDomain

Researchers

Technologists Providers

• Information about the community• Advice & consultancy• Community forums & feedback• Support & Training• Partnerships to provide software

Website & Wiki SoftwareRepository

Software Catalogue

©11

Scientists using software Obtaining OMII-UK Software Components

Direct from the development team NeSCForge, SourceForge, project website

From the OMII-UK repository Component release contributed by the developers

From the OMII-UK release Integrated and quality assured collection of many software

components Installation wizards to ‘expert’ or ‘typical’ installs

Support and training around these components

©12

Mature Software Components Secure web services hosting environment (Tomcat, Axis, …) Composing workflows across secured resources (Taverna) Web service to integrating heterogeneous data resources

(OGSA-DAI) Web service for jobs submission and job monitoring (GridSAM) Registry for service publishing, discovery & annotation

(Grimoires) Additional functionality to support OGSA-DAI & Taverna EGEE performance testing

Supporting scientific workflows through web services (BPEL) Integration of open-source components, training & support

©13

Running Projects Collecting & publishing resource details in a registry

(Open Grid Manager) Monitoring and reporting of resources Lightweight probes recording data in Grimoires

Authorisation Integration SAML 1 Assertion port-type Service PERMIS decision making engine

Matchmaking and brokering across web services (Knoogle) Consume GridSAM, Grimoires, BPEL & Taverna

Client & server for distributed visualisation (RAVE)

©14

Completed projects Reliable Messaging (FIRMS)

Implementation of WS-ReliableMessaging and WS-Reliability specifications

Distributed Events (FINS) Implementatino of WS-Eventing

Simplified lightweight interface to running applications (AHE) Uses WSRF::Lite to provide secure Perl WS

Using BPEL to run simulation & visualisation applications through GridSAM (MANGO)

©15

New Projects Programmatic Access to the Grid using SAGA

(Simple API Grid Applications) C++ & Java implementation Use of OMII-UK and Globus middleware

Command line and graphical desktop interface (OGRSH) Using OGF specifications: BES, ByteIO, Naming

Integration of GridSAM with the PBS/Torque, LSF, gLite compute element and Microsoft’s CCS.

Integration of the Basic Execution Service into the Condor client to form Condor-BES.

©16

New Projects - Portlets Managing Shibboleth Authorisation

(SPAM-GP) Portlet to specify attribute release policies Portlet to specify resource access policies

E.g. portlets, web service, artefact, resource, … Artefact Sharing Framework (WHIP)

Infrastructure to share artefacts Upload/download from desktop client Attach policy (Shibboleth) to govern access Instantiate by sharing different workflows

©17

New Projects - Portlets Application Portlet Designer

Application Designer to provide customised portlet interface to JSDL job submission

Use of Artefact Sharing Framework to share designed portlets Authorised deployment and undeployment portlets

National Grid Service (NGS) Jobs & Data Portlets Refactor the ‘new’ NGS portlets to use SAGA Integrate into OMII-UK portlet environment (GridSphere)

Access Grid Client Portlet (applet & client) combination Manage (transparent) switching between uni/multi cast Effectively replace existing AG3 client app.

©18

Summary What can you do to get involved?

Let others know about your project Contribute a release of your software Join the beta-testing programme Download the complete software release or a component

Release Timetable Development releases (3.3.x) Production release in April 2007 (3.4.0)

More Information: Web: www.omii.ac.uk Contact: [email protected] Mail: [email protected]