EMI INFSO-RI-261611 NA3 Sustainability Morris Riedel (JUELICH) Peter Stefan (NIIF) et al. EMI Vision Meeting, Geneva, 14 th -15 th December 2011.

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

NA3 & Sustainability

Morris Riedel (JUELICH)Peter Stefan (NIIF)

et al.

EMI Vision Meeting, Geneva, 14th-15th December 2011

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Objectives & Review Recommendation

20/09/2011 EMI Sustainability 2

• DoW Sustainability – Will be improved by establishing collaboration programs with

commercial companies, adopting off-the-shelf components to reduce maintenance costs and to facilitate easier adoption by wider user communities.

– The definition together with the resource providers of measureable Service Level Agreements will provide the base for establishing more standard service provision business models.

• Recommendation 6– Improve the sustainability plans and commit to a specified period of

support for EMI software and services after the end of the project (e.g. the Monte Bianco release), which is more convincing than the currently anticipated 3 months.

• How long can we achieve a commitment to EMI from partners?

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Three areas of work towards sustainability

20/09/2011 EMI Sustainability 3

• Inspire and confirm commitment from key stakeholders/users– ScienceSoft (Alberto later) for the community

• Adopt lesson learned from industry and successful open source models– Company SysFera seems to make real money– Learn from RedHat (‘support just costs’)

• Adopt standards to enable re-use of EMI components ‘outside the DCI ecosystem’ – Expand usage via standard-based components– Enable switch products in unsustainable times

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Key Questions to Answer

20/09/2011 EMI Sustainability 4

• Roundtable at EMI AHM Padua Questions• Discussed Lighthouse Challenges in NA3• NA3 Comments to Alberto’s Draft proposal• Timelines and Planning with stakeholders

– EGI runs one year longer: how we deliver EMI?• Funding of developers

– Obey to the release process (or easier process)– Engage further in ‘Inter-component’ work – Sustain common developments (STS, EMI Registry,

common authN lib, EMI-ES?, etc.)– Developments to follow trends (e.g. Oauth)

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

EMI Mission Statement

20/09/2011 EMI Sustainability 6

The European Middleware Initiative (EMI) project represents a close collaboration of the major European middleware providers - ARC, gLite, UNICORE and dCache - to explore and implement sustainable models to support, harmonise and evolve the distributed computing and data management middleware for deployment in EGI, PRACE and other distributed e-Infrastructures

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Applications Integrators, System Administrators

EMI Middleware Evolution

20/09/2011 EMI Sustainability 7

Standards,New technologies (clouds)Users and Infrastructure

Requirements

EMI Reference Services

3 years

Before EMI After EMI

Specialized services, professional support

and customization

Standard interfaces

Standard interfaces

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Complementary to Technology Strategies

20/09/2011 EMI Sustainability 8

• Understand and define ‘EMI value proposition’ far beyond the DCI space

• Sustainability: Inspire and confirm commitment from key stakeholders/users

• Adopt lesson learned from industry and successful open source models

• Contribute to open standards to enable re-use of EMI components ‘outside the DCI ecosystem’

• Expand usage ‘beyond traditional users’

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Sustainability Drivers

11/04/2011 EMI Roadmap 9

Expansion of the user base

Decrease of costs

Commercial activities

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Examples: Open Source Models

20/09/2011 EMI Sustainability 10

Community model

Macro R&D Infrastructures

model

Support contracts or Subscription

model

Dual-licensing or Commercial model

Apache, Eclipse

Red Hat, Canonical

MySQL, Zarafa

EMI and other publicly funded projects (“unstable” future)

From Open Source to long-term sustainability: Review of Business Models and Case studies, Chang et al.

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Open Source Models

20/09/2011 EMI Sustainability 11

Community model

Macro R&D Infrastructures

model

Support contracts or Subscription

model

Dual-licensing or Commercial model

Incubation in the right condition becomes

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Some lessons learned from RedHat

20/09/2011 EMI Sustainability 12

• Specific concrete advice for an open source community– Clearly have an idea/plan what you offer; who

would pay for this? What is the “EMI value”?– Identify active and strong team leaders – they

are the key in the open source community– Stop “free” support model of today, it is an

important income on support models (week, premium) to fund the core set of people

With thanks to Francois Lucatelli (~10 years in RedHat)

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

EMI Value for Technical Users

• A streamlined middleware distribution available from a well-defined place– EMI Repository http://eticssoft.web.cern.ch now,

EPEL/Debian as component mature– Regularly published service releases

• Better integration with Fedora/EPEL (and compatible OSs) and Debian

• Open to external contribution– Source packages fully available– A single patch submission channel (GGUS)

20/09/2011 EMI Sustainability 13

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

EMI Value for End Users

• Stable middleware services delivered with standard Operating Systems– EMI products from different technical areas (compute,

data, security, infrastructure) work seamlessly together, well tested

– 10 years experience of ‘cutting edge HPC and HTC’• Open Source model allows

– More rapid and transparent improvement of quality; contributions from different sources

– Value-added services from experts– Open competition brings better quality

20/09/2011 EMI Sustainability 14

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

EMI Value for EGI[-InSPIRE|.eu]

• Open, transparent software releases– EMI Inter-product ecosystem, well-tested

• Possibility to implement revenue streams from value-added professional services (support, customization, outsourcing) via commercial SLAs

• Possible involvement of commercial companies in the provision of services, thanks to the standard open source approach20/09/2011 EMI Sustainability 15

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Points to stimulate the discussions

• Scientific institutes miss the organizational structure and capabilities to ‘go business’– ‘Business-oriented legal departments’: patents, trust

insurances, IPR issues– ‘Marketing departments’: influence 1000^x at the same

time instead of 10-100 per community– ‘Maintenance vs. research’: software stability vs. scientific

innovation conflicts; different careers• Two possibilities now

– Institutes establish necessary departments/skills– Let commercial companies do support and focus on

research

20/09/2011 EMI Sustainability 16

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Next Steps

• Strategic planning (new NA3 work package)– Work plans in current deliverable and include new

activities; also ‘products and scientific results‘– ‘Change’ the way we work towards described models;

implement initial ideas in project lifetime– Concrete EMI product factsheets of services including key

usage models, maturity, etc. – Talk, but more important – listen – to the community;

what they want; who would pay?– Align with the broader (scientific) community– EMI products in EPEL; case studies; ‘Works with EMI’,

address ‘market’ (i.e. users) requirements…

20/09/2011 EMI Sustainability 17

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Thank you

11/04/2011 18EMI Roadmap

EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Add-on: interview Technicolor

• TECHNICOLOR Interview– Movies on demand, home entertainment, etc.

• Discussions reveal interesting common interests and ‘the same problems’– ‘Sharing movies in the neighborhood’ Vos– Delegating rights to see a movie to another

delegation of end-user rights/roles– Need for security standards SAML/XACML

• Next steps– Discussions whether EMI products might help– In NA3 Tracker…

20/09/2011 EMI Sustainability 19

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Add-on: interview UK SSI

• UK Software Sustainability Institute– Scientific software in the UK, OMII-UK follow-on

• Sustainability discussions– Some UK institutions provide partially people– Wide range of products: GridSAM, MPI codes, etc.– Hard to get funding from UK academic councils for

software maintenance only• Next steps

– Work together with the UK SSI towards a EU-wide open software foundation/approach

20/09/2011 EMI Sustainability 20

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Add-on: e-IRG Panel – Software role?

• e-IRG Meeting Poznan – last week– Panel with ESFRI co-chair, EGI, PRACE, user

community representatives• Question to the panel: role of software?

– Software can be only maintained via advancement– Each user community is responsible for its

software stack – open whether 24/7 is realistic• Next steps

– Follow e-IRG and make the case that software do matter if you want to create e-Infrastructures

20/09/2011 EMI Sustainability 21