Two Types of Supercomputer developments Yutaka Ishikawa RIKEN AICS University of Tokyo 1 2014/09/03 http://computing.ornl.gov/ workshops/exascale14/ Session 2: Deployed Ecosystems and Roadmaps for the Future Smoky Mountains Computational Sciences and Engineering Conference
14
Embed
Two Types of Supercomputer developments Yutaka Ishikawa RIKEN AICS University of Tokyo 1 2014/09/03 Session.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Two Types of Supercomputer developments
Yutaka IshikawaRIKEN AICS
University of Tokyo
1
2014/09/03
http://computing.ornl.gov/workshops/exascale14/
Session 2: Deployed Ecosystems and Roadmaps for the Future
Smoky Mountains Computational Sciences and Engineering Conference
Supercomputers in Japan
2014/09/03
FLAGSHIP Machine
K Computer
1
10PF
Riken
9 Universitiesand National Laboratories
HPCI (High Performance Computing Infrastructure) is formed from those machines, called leading machines
Features: Single sign-onShared storage (Distributed file system)
As of Jun 2012
Each supercomputer center has one, two or more supercomputers.
Each supercomputer center replace their machines every 4.5 to 6 years.
2
Procurement Policies in Supercomputer Centers
2014/09/03
• Flagship-Aligned Commercial Machine (FAC)– Acquiring a machine whose architecture is the same of the
flagship machine.• Complimentary Function Leading Machine (CFL-M, CFL-D)
– Acquiring a machine whose architecture is different than the flagship machine, e.g. vector machine.
– CFL-M: a commercial machine provided by a vendor– CFL-D: a new machine developed by both a vendor and
• Operating System (Linux and McKernel)• Programming Languages (Fortran, C/C++,
Xcalable MP)• Communication Library (MPI-3)• Math Libraries• File System• Batch Job System
2014/09/03 6
DevelopmentProcurement• McKernel
– Light Weight Microkernel• Xcalable MP
– Parallel Programming Language
• MPICH with Low-level Communication Facility
Linux + McKernel
• Concerns– Reducing memory contention– Reducing data movement among cores– Providing new memory management– Providing fast communication– Parallelizing OS functions achieving less
data movement• New OS mechanisms and APIs are
revolutionarily/evolutionally created and examined, and selected
• Linux with Light Weight Micro Kernel– IHK (Interface for Heterogeneous Kernel)
• Loading a kernel into cores• Communication between Linux and the kernel
– McKernel• Customizable OS environment
– E.g. environment without CPU scheduler (without timer interrupt)
2014/09/03 7
Core
McKernel
Linux Kernel
Dae
mon
Core Core
Use
r pr
oces
s
Use
r pr
oces
s
Dae
mon
Dae
mon
Core
Interface for Hetero. Kernels
System call to LMKSystem call to Linux
Running on both Xeon and Xeon-phi environments
IHK and McKernel have been developed at the University of Tokyo and Riken with Hitachi, NEC, and Fujitsu
PostT2K OS Environment being developped
• Linux Kerne l+ McKernel– Several variations of McKernel are provided for applications– Linux Kernel resides, but an McKernel is selectively loaded for each
application
2014/09/03 8
Linux kernel is residentApp A on McKernel
without CPU scheduler Is invoked
Finish
App C on McKernel with Segmentation is invoked
Finish
App B on McKernel with CPU scheduler Is invoked
Finish
App D on Linux Is invoked
Finish
XcalableMP(XMP) http://www.xcalablemp.org
What’s XcalableMP (XMP for short)? A PGAS programming model and language
for distributed memory , proposed by XMP Spec WG
XMP Spec WG is a special interest group to design and draft the specification of XcalableMP language. It is now organized under PC Cluster Consortium, Japan. Mainly active in Japan, but open for everybody. Project status (as of Nov. 2013)
XMP Spec Version 1.2 is available at XMP site. new features: mixed OpenMP and OpenACC , libraries for collective communications.
Reference implementation by U. Tsukuba and Riken AICS: Version 0.7 (C and Fortran90) is available for PC clusters, Cray XT and K computer. Source-to- Source compiler to code with the runtime on top of MPI and GasNet.
9
Poss
iblit
yof
Per
form
ance
tun
ing
Programming cost
MPI
Automaticparallelization
PGAS
HPF
chapel
XcalableMPXcalableMP
Poss
iblit
yof
Per
form
ance
tun
ing
Programming cost
MPI
Automaticparallelization
PGAS
HPF
chapel
XcalableMPXcalableMP
int array[YMAX][XMAX];
#pragma xmp nodes p(4)#pragma xmp template t(YMAX)#pragma xmp distribute t(block) on p#pragma xmp align array[i][*] to t(i)
main(){int i, j, res;res = 0;
#pragma xmp loop on t(i) reduction(+:res)for(i = 0; i < 10; i++)for(j = 0; j < 10; j++){
array[i][j] = func(i, j);res += array[i][j];
}}
add to the serial code : incremental parallelization
data distribution
work sharing and data synchronization
Language Features Directive-based language extensions for
Fortran and C for PGAS model Global view programming with global-view
distributed data structures for data parallelism SPMD execution model as MPI pragmas for data distribution of global
array. Work mapping constructs to map works
and iteration with affinity to data explicitly. Rich communication and sync directives
such as “gmove” and “shadow”. Many concepts are inherited from HPF
Co-array feature of CAF is adopted as a part of the language spec for local view programming (also defined in C).
XMP provides a global view for data parallel
program in PGAS model
Code example
10
Roles of PC Cluster Consortium
Development, Maintenance and Promotion
2014/09/03
Members: Univ. of Tsukuba, Univ. of Tokyo, Titech, AMD, Intel, Fujitsu, Hitachi, NEC, Cray, …
IHK, McKernel, LLC, XMP
Integration of other open sources, e.g., MPICH
Distribution as open source Promotion
PostT2KPostK
Vendor
Contribution Contribution
Vendor
ContributionContribution
SupportSupport
PC cluster consortium was established in 2001. The original mission was to contribute to the PC cluster market through the development, maintenance, and promotion of cluster system software based on the SCore cluster system software and Omni OpenMP compiler, developed by the Real World Computing Partnership funded by the Japanese government from 1992 for 10 years.
International Collaboration between DOE and MEXT
2014/09/03 11
PROJECT ARRANGEMENTUNDER THE IMPLEMENTING ARRANGEMENT
BETWEENTHE MINISTRY OF EDUCATION, CULTURE, SPORTS, SCIENCE AND TECHNOLOGY
OF JAPANAND
THE DEPARTMENT OF ENERGY OF THE UNITED STATES OF AMERICACONCERNING COOPERATION IN RESEARCH AND DEVELOPMENT IN ENERGY AND
RELATED FIELDS
CONCERNING COMPUTER SCIENCE AND SOFTWARE RELATED TO CURRENT AND FUTURE HIGH PERFORMANCE COMPUTING FOR OPEN SCIENTIFIC
RESEARCH Yoshio Kawaguchi (MEXT, Japan)and William Harrod(DOE, USA)
Purpose: Work together where it is mutually beneficial to expand the HPC ecosystem and improve system capability
– Each country will develop their own path for next generation platforms
– Countries will collaborate where it is mutually beneficial• Joint Activities
– Pre-standardization interface coordination– Collection and publication of open data– Collaborative development of open source software– Evaluation and analysis of benchmarks and
architectures– Standardization of mature technologies
• Kernel System Programming Interface• Low-level Communication Layer• Task and Thread Management to Support Massive
Concurrency• Power Management and Optimization• Data Staging and Input/Output (I/O) Bottlenecks • File System and I/O Management• Improving System and Application Resilience to Chip
Failures and other Faults• Mini-Applications for Exascale Component-Based
Performance Modelling
Technical Areas of Cooperation
12
Concluding Remarks
• Ecosystem– Co-development of system software stack for a leading
machine (PostT2K) and the flagship machine (PostK)– Beneficial to users
• Continuity of System Software and Programming Language from leading machines to the flagship machine
– Contribution to open source community• Shared and Enhanced by the community
• Schedule
2014/09/03
PostT2K
PostK
ProcurementSoftware Development
Operation
Basic Design Design and ImplementationManufacturing, Installation,
and TuningOperation
2014/09/03 13
1. The overall theme of SMC2014 is "Integration of Computing and Data into Instruments of Science and Engineering".
2. Our session is focused on "Deployed Ecosystems and Roadmaps for the Future ". We will be focusing on current experiences and challenges in deploying large scale computing capabilities and our plans and expectations on how future systems will be made available to our scientists and engineers.
3. Consistent with this topic, we are inviting you share your vision for how the computational ecosystem may continue to develop to serve the scientific and engineering challenges of the future.
4. The three other panels in our conference will focus on "Strategic Science: Drivers of Future Innovation", "Future Architectures to Co-Design for Science", and "Math and Computer Science Challenges for Big Data, Analytics, and Scalable Applications".