A Hep Software Collaboration - A proposal Ian Bird CERN, 17 th July 2013 July 17, 2013 [email protected] 1
Ian Bird CERN, 17 th July 2013 July 17, 2013
Jan 18, 2018
Background The Concurrency Forum has successfully brought the community together ( Pere) - To focus on problems of more optimally using modern computer architectures - Sharing knowledge across HEP - Forum for dissemination The scope of the problem we need to address for the next 10+ years is significant - Needs to draw in available expertise - Needs to provide training and education for next generation of physicists We should not re-invent the wheel – HEP may have lost some skills in this area - Other sciences have not – there is expertise and experience available We need a more formal framework within which to plan our (HEP) activities and which allows us to construct collaborative activities where necessary July 17, 2013
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
A Hep Software Collaboration- A proposal
Ian BirdCERN, 17th July 2013
July 17, 2013
Background• The Concurrency Forum has successfully brought the community
together ( Pere)- To focus on problems of more optimally using modern computer architectures- Sharing knowledge across HEP- Forum for dissemination
• The scope of the problem we need to address for the next 10+ years is significant- Needs to draw in available expertise- Needs to provide training and education for next generation of physicists
• We should not re-invent the wheel – HEP may have lost some skills in this area- Other sciences have not – there is expertise and experience available
We need a more formal framework within which to plan our (HEP) activities and which allows us to construct collaborative activities where necessary
July 17, 2013
Some of the risks• There are a lot of activities on HEP-HPC• The danger of
- Diverging- Reinventing the wheel (inside and outside HEP)- Spending time in wrong directions
• ...is very real• Relations with other labs & sciences & industry is essential for
items such as- HPC & concurrency programming- Data preservation- Algorithms- Big Data- Future of Grid / Cloud / …- and so on…
July 17, 2013
Resource constraints• Optimistically(!) funding for computing in the next few
years will be ~flat- Worse – it competes with detector upgrades – there is only 1
pot of money, and this sharing is not being managed by the experiments
- we must optimise our use of [CPU, memory/storage, bandwidth, power] & human resources
• HEP cannot go it alone- External funding for computing people will be limited, - HEP must build synergies with other sciences and
organisations - Must be sure to focus our scarce efforts on the real problems
that we have
July 17, 2013
HEP is not (just) LHC• LHC – long term programme with
significantly increasing data rates • ILC/CLIC• TLEP• FAIR• Etc.• Will all be facing the same problem –
significant need for resources in a time of limited funding
July 17, 2013
A HEP software collaboration• Formalise the Concurrency Forum as a collaboration• Provide a framework to
- Increase level of collaboration and involvement- Coordinate and review existing efforts- Recognise and credit the contributions of FAs, institutes and
individuals (shown to be important in the past)- Roadmaps, priorities for investigation and development- Create new projects, make them visible, and provide review- Launch new projects where R&D need is clear- Clarify that the scope is not limited to LHC – but is HEP-wide- Also a mechanism to collaborate and bring in existing
expertise from other sciences and industry- Would very much help in bidding for funds (especially if this
was joined up EU/US etc)July 17, 2013
A Software Collaboration• We should also recognise that HEP is a very
successful community – we are well integrated compared to almost every other science
• BUT: we are fragmented in computing- Notable exceptions – GEANT4, ROOT- All HEP experiments do essentially the same
workflows, but we always (without fail) start over again for the software for every experiment
• We should not miss the huge opportunity to build commonality
July 17, 2013
More commonality?• Common distributed computing models• Common software frameworks for processing and analysis• Common libraries – now that incorporate all the best
practices of high performance computing (vectorised, multi-threaded, parallel, …)
• This would allow us to have a common optimisation team/expertise/… to help experiments optimise their code- Commonality will help bring the best practice to all of HEP codes
(in the same way GEANT4 incorporates physics understanding)• HEP is also missing a huge opportunity to provide our
expertise to the wider scientific community- ROOT to supplement R, Matlab, ???- etc
July 17, 2013
Commonality – why?• Cannot expect to train the entire community to
become experts in HPC/Parallel software; must encode the developments/optimisations in common frameworks and libraries
• Developing concurrent frameworks and parallel algorithms will be a significant effort and we cannot afford to duplicate it
July 17, 2013
R&D Techlab• PH/SFT and IT are establishing a lab to support the
development and benchmarking of parallel software on a variety of architectures- Would be activity as part of the software collaboration- Cluster of machines of interesting architectures- Environment for benchmarking and comparison of
software developments- Provide access to range of code development and
analysis tools to aid in parallelising and vectorising code- Means to collaborate with interested industrial partners,
eg via openlab
July 17, 2013
But what about broader HEP computing?
• Software optimisation is only 1 aspect• How will HEP manage computing in 10-15
years?- How should the models change, with Terabit
networks, computing advances across the board in science, consumer, social, …
- How can HEP make the best/optimal use of developments from other areas?
- For how much longer does HEP run all/part of its own computing?
July 17, 2013
How does HEP computing adapt?
• HEP-wide project on future computing?- Coordinate all the projects in HEP that address future
evolution of computing and software- Governance by the HEP community itself- Launch new projects in areas where there are identified holes- Act as the focal point for collaboration with other sciences and
industry• We have experience in this level of collaboration• We have a bigger problem now than we have had in the
past• HEP computing needs to take a change of direction now
July 17, 2013
Timelines• The software collaboration should be formed
by the end of this year• The broader scope of a project for HEP
future computing needs some community discussion- Aim for a workshop in Q4 2013?
July 17, 2013
Related Documents
