The Hartree Centre & The Daresbury Research Collaboratory in association with IBM

The Hartree Centre&

The Daresbury Research Collaboratory in association with IBM

Jonathan Follows

HPC activities at

STFC Daresbury Laboratory

NEW

e-Infrastructure and HPC Capital Investment

• £145M funded by BIS covering– High capacity networks– Massive data storage– Advanced software development for massively parallel

computers– Space infrastructure– HPC

• National Tier 0/1 facilities• Distributed Tier 2/3 facilities

From BIS Presentation To Cabinet Office

“Capital Projects in Support of Software Development Activities”

From BIS Presentation To Cabinet Office

• ICE-CSE Phases 1 and 2 (Hartree, STFC)• Rationale for investment

– Focus on leveraging new collaborations and investment into Daresbury to promote UK economic growth and rebalance the economy;

– Bring together communities from academia, Government and industry to develop a programme of multi-disciplinary projects that will focus on developing multi-scale, efficient and effective modelling and simulation software;

From BIS Presentation To Cabinet Office

• Provide a step-change in analytical and predictive capabilities across many strategic themes, including energy, life sciences, environment, materials, security;

• In addition, the centre will develop and demonstrate services to a diverse range of end users to demonstrate the value of software development to them and the positive effects on their products;

• The investment in ICE-CSE will provide:– Capability to host truly world class, next generation

HPC systems;– Equip it with world leading computer systems as

software development and demonstration platforms;• Applications areas that will benefit directly from the

investment include– Earth Observation data-mining, Square Kilometre Array

data receiving/mining, genome sequencing, financial transactions and predictions, seismic exploration data interpretation

From BIS Presentation To Cabinet Office

• Lack of investment will require Daresbury to continue to work with better equipped partners and procure run-time elsewhere in order to develop its new algorithms and applications software.

• This will result in a smaller, slower applications development programme with reduced competitiveness of UK industry

From BIS Presentation To Cabinet Office

IDC said ….• Software leadership will become the New

Battleground– Only 1% of HPC codes can exploit 10,000 or more cores– There May Be More Emphasis on Software … Finally

• Big Data Methods Will Start to Transform the HPC Market, Including Storage– HPC and commercial Big Data are starting to collide

• Petascale Performance on Big Systems Will Create New Business Opportunities

• The HPC Staffing Shortage Will Grow More Acute

Return on investment• UK tax revenues exceed 35% of GDP• Forecast tax revenue for 2016 is >£700,000,000,000• Forecast GDP for 2016 is >£2,000,000,000,000• A 0.1% increase in GDP leads to additional

annual£700,000,000 in tax revenues (net taxes and national insurance contributions) (and IDC and others forecast a much greater increase in GDP than this)

• So a £145,000,000 investment shows a pretty decent return on investment; even a 0.01% increase in GDP pays for itself after 2 years

• And, by the way, we paid £6,300,000 in VAT for our HPC systems!

Aims of the Centre (1)• Drive the adoption of “HPC” in UK industry

– to improve their competitiveness and generate wealth Next-generation applications

• Delivering step-change improvements in scientific capabilities

• Develop world-class “HPC” technology & skills in Daresbury Laboratory

Aims of the Centre (2)• Develop & enhance research collaboration

– nationally & internationally– on key focus areas for the UK

• Foster innovation in academia & commerce

• Educational focus on computational science & engineering

• Establish a better career structure for staff – who support computational science and engineering in academia

An “ecosystem” for HPC• The Hartree Centre needs to be, and is designed to

be complementary to other HPC facilities in the UK– HECToR, ARCHER, new Tier 2 systems (such as HPC

Midlands), University HPC systems

• One model for the Hartree Centre is one in which the entire system can be reserved for one purpose for a period of time – akin to a telescope facility

• Software generation and a wide range of collaborators are the focus areas for the Hartree Centre

Douglas Rayner Hartree• Hartree – Fock • Appleton – Hartree Equation

• Differential Analyser• Numerical Analysis

• Father of Computational Science

Douglas Hartree with Phyllis Nicolson at the Hartree Digital Analyser at Manchester University

Hartree Digital Analyser

• He said in 1946

It may well be that the high-speed digital computer will have as great an influence on civilization as the advent of nuclear power.

• He wrote Numerical Analysis in 1952 which became a classic of the subject.

Anyone intending to undertake a serious piece of calculation should realise

that adequate checking against mistakes is an essential part of any satisfactory numerical process.

No one, and no machine, is infallible, and it may fairly be said that the ideal to aim at is not to avoid mistakes entirely, but to find all mistakes that are made, and so free the work from any unidentified mistakes.

The Project• To establish a world leading centre with a world

leading service ethos– Phase A (Initiation):

• Establish the staff, operating procedures, and systems • Establish the IBM Collaboratory

– Phase B (On-going): • On-going operation of the centre• Detailed governance structure

How (1)• Purchase of internationally competitive DONE

– massively parallel systems – a high-throughput system– a data-intensive computation system– advanced visualization systems– 145 days from launch of tender to contract signature– Majority of capital spend in 2011-12 financial year

• Establish a collaboration with IBM DONE– for R&D and business development– as part of a single contract for £31½m

• STFC funding for recurrent spend DONE

How (2)• Establishing the centre with a service ethos

• Tackle with industry major computational problems.

• Promote uptake of “HPC” to commerce

• Foster collaboration & inward investment to DL

• Outreach programme – raise awareness with the public and policy-makers

How(3)

It is important to place a higher value upon the position of “scientific programmer” and also “data curator” in the academic environment and to offer more career opportunities to these staff.

Scientific programmers combine the knowledge of the underlying scientific discipline with implementation, optimisation and parallelisation for high-end systems: they are important in obtaining highly-efficient application implementations.

– Dominic Tildsley, e-Infrastructure report

Our hardware• Data centre upgrade – water cooling• 7 racks of BG/Q 1.46Pflop/s• IBM iDataPlex with 8192 cores 170Tflop/s

– Including 8x4TB ScaleMP large memory systems

• ~6PB of DDN disk storage with GPFS• Tape library• Visualisation systems in Daresbury and Rutherford

Appleton Laboratory• Computers undergoing final installation and

commissioning• Visualisation waiting on some building work to

complete

BusinessInnovation

Department

Scientific Computing

(CSED &e-Science) Hartree

Centre

Daresbury Research Collaboratory in association with IBM

Other Collaborators

Initial Governance

Advisory Group

User Group

Hartree CentreManagement Team

IBM

STFC

Operational Teams

Visualization Advisory Group

External Input

Main Elements of PlanArea One Word Description

Systems Support Hardware

Applications Support Software

User Support People

Science & Engineering R&D

Business Development Pipeline

Governance Monitoring

Marketing & Communications

Information

The Service• Includes all aspects of our work

– Users, R&D projects, collaborators (commerce & academia), public, policymakers

– The service is the centre, not just the delivery of computer systems

• World Class– Process for everything– Metrics

• But only those which matter in the context of our centre– Those processes under continuous improvement

Structure

Core IT InfrastructureNetworks, Security, Server Admin, User Desktop, …

User Management & User Support

Commercial ApplicationsSupport

In House ApplicationSupport

Computational Science & Engineering Support

Domain Specific Support

User

The HPC Service

HPC System Administration

Service Manager

Serv

ice R

&D

Progress

Interim Management Team In Place

Project Managers In Place

Project Plan Nearly Ready

Systems Nearly Accepted

Centre Structure & Governance Initial version agreed

Staffing levels Initial version agreed

Early Access Users Panel approved 15 projects

Thank you for listening

Any Questions?