OpenPOWER Roadmap towards CORAL - HPC …hpcadvisorycouncil.com/events/2016/swiss-workshop/pdf/Wednesday24...OpenPOWER Roadmap towards CORAL Klaus Gottschalk ... xCAT • IBM Platform

OpenPOWER Roadmap towards CORALKlausGottschalkHPCArchitectPOWERPlatform

HPCAdvisoryCouncilMeetingLugano,March23rd,2016

FundamentalforcesareacceleratingchangeinourindustryProcessortechnologyalonecannolongerprovidethepriceperformancegainsnecessary tosustain

Moore’sLaw

Scale-out & Hyperscale Data Centers

Hybrid Cloud

Open Solutions

ITconsumptionmodelsareexpandingto:

Abetterprocessorandfullsystemstackopeninnovationrequired

Price

/Perform

ance

Moore’sLaw

ProcessorTechnology

2000 2020

Firmware/OSAcceleratorsSoftwareStorageNetwork

2

• Data&infrastructuresilos

• Datainducedbottlenecks

• Blindlyaddingcapacity

• Dataduplicationandmoresilos

• Adhoccloudusage

Complexity

Cost

Availability

Traditionalcompute&storageapproacheswon’tsuffice…

3

Price

/Perform

ance Semiconductor

TechnologyandProcessors

2000 2020

IBMHPCstrategy

POWER8era

ShiftbacktowardstheMoore’sLawpredictionthrough:1. IBMHPCInnovation(processorarchitectureenhancement,scalablefilesystems,workflowmanagement)2. Acceleration throughpartnerecosystem(e.g.NVIDIAGPUsdeliver2Xperf/wattand~3Xperformance)

POWER9 era

IBMHPCInnovation

Acceleration

Moore’sLaw,ClientExpectations

4

IBMVision:Data-CentricComputing

• Hardwareandsoftwaretosupport&enablecomputeindata• AllowworkloadstorunwheretheyrunbestMinimizeDataMotion

• Introduce“active”systemelementsincludingnetwork,memory,storage

• Globalaccesstoshareddata

EnableComputeEverywhere

• Balanced,composable architecture• Modularandupgradeabledesignscalableupto100sofracksModularity

• Userealworkloads/workflowstodrivedesignpoints• Co-designforcustomervalue

Application-drivendesign

5

IBMHPCInnovationsandInvestments:OurCommitmenttoHPC

ProcessorDesign• POWER8,POWER9,14nmProcessorDesign

ProcessorInterfaces• CAPI,POWERwithNVLinkTechnology

Filesystems&StorageSolutions• IBMSpectrumScale• ElasticStorageServer• HPSS• IBMFlashSystem

Compilers,Development,Libraries• IBMXL,Parallel Environment,ESSL

MPIandClusterProgramming• IBMPlatformMPI,OpenMP 3.1and4.0Development

Workload&ClusterManagement• IBMPlatformClusterManager,xCAT

• IBM PlatformLSFPowerandData-AwareScheduling

6

Highlights:IBMHPCaroundtheWorld2014-2015

IBM,Mellanox,andNVIDIAawarded$325MU.S.DepartmentofEnergy’sCORALSupercomputers

GENCIannouncescollaboration withIBMon

speedingthepathtoexascale computing.

DESYcollaborateswithIBMResearchselectsSoftwareDefinedStorageSDI,IBM

Powerforparticleacceleratordata

UKGovernmentInvests£115Million inBigData&CognitiveComputing

ResearchwithSTFCandIBM

LeadingoilandgasfirmselectsIBMforworkload

management

LSUselectsacomprehensiveIBMHPCsolutionforgenomicsresearch

IntegratedPortfolio:IBMPowerSystemsSoftwareDefinedInfrastructureIBMStorageIBMResearch

Withmany,manymorearoundtheworld… 7

OpenPOWER:OpenArchitectureforHPC &BigData

ProcessorIPLicensing

OpenInterfaces

Systems&Software

LicensingIPtoenablesemiconductorpartnerslikeSuzhouPowercore tobuildPOWERchips

TightintegrationusingCAPI&NVLink withAccelerators(NVIDIA,Xilinx,Altera),Networking(Mellanox),Storage(CAPIFlash)

EnablingInnovativePOWER-basedserversfromPartners&OpenCompute andSharingOpenSourceSoftwareincludingFirmware&Hypervisor

8

MajorWinsEndorsetheStrategyandEstablishthePathForward

IBM,Mellanox,andNVIDIAawarded$325M

U.S.DepartmentofEnergy’sCORALSupercomputers

IBM&UK’sSTFC PartnerforBigData&CognitiveComputing Researchin£313MPartnership

CORAL: Leadership Class Supercomputers 5X – 10X HIGHER APP PERF THAN CURRENT SYSTEMS

US,UK,GermanScientificResearchCommunitiesSelectOpenPOWERorCommittoOpenPOWERCollaboration

PADCOpeningOctober12th,2015

Jülich Supercomputing CentrePOWERAccelerationandDesignCenter(PADC) RoadmapTowardCORAL

190+ Open Power Foundation Members

10

1900+ApplicationsRunonPOWERBigData&

MachineLearningCloud MobileEnterprise

Major Linux Distros

HPC

miniDFTCTHBLASTBowtieBWAFASTAHMMERGATKSOAP3STAC-A2SHOC

Graph500Ilog

CHARMMGROMACSNAMDAMBERRTM

GAMESSWRF

HYCOMHOMME

LESMiniGhostAMG2013OpenFOAM

11

POWER8:ProcessorPerformanceLeadership

MemoryBuffer

DRAMChips

POWER812Cores,96Threads4LevelLargeCachesUpto1TBpersocket

Upto230GB/ssustained

FasterCores8ThreadsPerCore

LargerCachesDirectAccelerator

Interconnect

3xHigherMemoryBandwidth,1TB

MemoryperSocket

12

HPCApplicationPerformance

0,0

0,5

1,0

1,5

2,0

2,5

3,0Re

lativ

ePe

rforman

ce

HaswellBased POWER8– S822LC

POWER8:Upto2.5xFasteronApplicationsoverIntelHaswellx86CPUs

13

SharedSystemMemoryviaCoherentInterface 2.5xFasterCPU-GPUConnectionviaNVLink

OpenPOWER:TightintegrationusingCAPI&NVLink

POWERCPUs

Memory

FPGA

CAPICoherentAcceleratorProcessorInterface

14

IBMOpenPOWER-basedHPC Roadmap

2015 2016 2017

POWER8POWER8+

POWER9

OpenPowerCAPI Interface

NVLink

EnhancedCAPI &NVLink

Connect-IBFDRInfinibandPCIe Gen3

ConnectX-4EDR Infiniband

CAPI overPCIe Gen3

ConnectX-5Next-GenInfiniband

EnhancedCAPI overPCIe Gen4

MellanoxInterconnectTechnology

IBMCPUs

NVIDIAGPUs KeplerPCIe Gen3

VoltaEnhancedNVLink

PascalNVLink

IBMNodes15

NewIBM“LC”PowerSystemsforLinux

812LCCPU

Memory

1xPOWER8CPU10cores,2.9-3.3GHz

Upto1TeraByteMemory115GB/sMemoryBandwith14Drives(84TB,HDD,SSD)

OptimizedforHadoop,Spark,In-MemoryAnalytics

822LCCPU

Memory

OptimizedforDatabasesandCloudWorkloads

CPU2xPOWER8CPU

10coreseach,2.9-3.3GHzUpto1TeraByteMemory

230GB/sMemoryBandwith2Drives(2TB,HDD,SSD)

822LCHPC

CPU

MemoryBuiltforHPC andDeep

Learning

CPU2xPOWER8CPU

10coreseach,2.9-3.3GHzUpto1TeraByteMemory

230GB/sMemoryBandwith2xNVIDIATeslaK80GPUs

GPU GPU

16

IBMSoftwareDefinedInfrastructurefordiverseworkloads

IBMPlatformComputing

BigData/Hadoop

HighPerformanceAnalytics

HighPerfComputing

Makelotsofcomputers&storagelooklikeoneresourcepool

ScaleUpandOut

Prioritizedmatchingofsupplywithdemand

Benefitsü Efficientutilizationü HighAvailabilityü HighThroughputü HighPerformanceü SLAPrioritizationü Reducedcost

IBMSpectrumStorage

x86 Linuxonz

IBMFlashSystem

IBMHybridStorage IBMTapeStorage

On-premise,On-cloud,HybridInfrastructure(heterogeneous distributedcomputing andstorageenvironment)

OtherCommercial

Other

NewGenApp

Framework

Other

17

IBMPlatformLSF10.1(Q22016)

• Productivity-centric, enterprise-class workload management for HPC environments

• Intelligent policy-based data management• Optimized for both high throughput and traditional

HPC applications

12xlesswork

150xlesswork

6xlesswork

LSF9.1.2(2013) LSF9.1.3.3(2015)5.8million jobs/hr

Highthroughputsyntheticbenchmarkexternallyauditedandverified

Superiorrepeatableperformance=betterbusinessoutcomes

IBMConfidentialuntilannounced

EaseofUse•Understandwhyworkloadisnotrunning

•Understandwhenworkwillrun

HPCAccessibility•Webenabledworkflows•Simplifymigrationtothecloud

PerformanceatScale

• Delivering nextevolution ofperformanceandscalability

18

Wellcome TrustSangerInstituteUsinganalyticstooptimizehighperformancecomputingcapacityacross15,000cores

ProblemThe Wellcome Trust Sanger Institute, a non-profit research organization based in Hinxton, England, wanted to accelerate results to meet scientific journal publication deadlines and secure funding for future projects.

Solution• Platform LSF® policy-based scheduler• IBM® Platform™ Analytics with Custom dashboards showing cluster

utilization according to criteria such as job, job submitter & memory use• IBM® Platform™ Process Manager for workflow design and executionOutcome• Each job runs on the optimal compute node increasing throughput across

the HPC environment• Researchers understand the factors that reduce processing efficiency,

and take action to improve performance• A clear record of efficient compute utilization enables a

strong business case for future growth investments

Industry: ResearchProducts: IBMPlatform™LSF®

IBMPlatformAnalyticsIBMPlatformProcessManager

Overview

Profile

“AnalyticsisoneofthemostsignificantITinvestmentsthatwehavemade.WeneededtheIBMsolution todelivertheHPCutilizationrequiredtohelpourresearchteamsgenerateresultsrapidly,meettheirpublicationdeadlinesand,ultimately,securenewfunding.”

Dr.PeterClaphamPrincipalSystemsAdministrator,InformaticsSystemsGroup,Wellcome TrustSangerInstitute

19

IBMSpectrumScale4.2andIBMElasticStorageServer

• Reducecapitalandoperatingexpenseforstoragesupportingverylargescaledata• Enableacommonpoolofstorageformodernworkloadswithunifiedfileandobjectstorage• Policy-drivencompressionandQualitiesofService

• Accelerateanalyticsapplications• EliminateneedtomovedatabackandforthfromHadoop/HDFSstoragesiloswithintegratedHDFS

• AutomateBigDatadeploymentwithApacheAmbari support

• Speedandsimplifydeployment• NewSpectrumScaleGUIconsistentwithSpectrumStoragefamily• ExpandmanagementoptionswithSpectrumControlintegration• Fullyfunctionalvirtualmachinedemonstrationlowersbarrierstoadoption

AvailablesinceNovember20(IBMSpectrumScale)November6(IBMElasticStorageServer)

20

DESY- ResearchinstituteinGermanyAcceleratingexperimentaloutcomeswithIBMSoftwareDefinedInfrastructure

Solution components

The Challenge Deutsches Elektronen-Synchrotron needed a solution that could keep up with one of the most powerful x-rays in the world, rapidly delivering experimental results to researchers

The SolutionA high-performance IBM storage and compute solution to support increasingly sophisticated experiments and offers analysis-as-a-service, allowing scientists to tap into DESY’s particle accelerator facilities and access the data from anywhere in the world.

The Results§ Accelerates research with data available for analysis in

4 minutes, down from 2-3 days§ Accommodates more demanding projects, handling 20GB/second§ Scales up for continuing innovation

• IBM® Elastic Storage™ Server• IBM Spectrum Scale• IBM Platform HPC• IBM Power Systems™ running Linux• IBM Advanced Business Partner:

MCS Moorbek Computer Systeme GmbH

"The ability to provide scientists with data on very short time scales has changed the way experiments are done at the beam line.”--Steve Aplin, senior scientist 21

ActionableIntelligencewithOptimizedPerformanceIBMFlashSystemandSpectrumScale

IBMFlashSystem poweredbyFlashCore™Technologyisoptimizedtodeliverextremeperformance,reliability,andlowlatencyinthemostdemandingenvironments

IBM FlashSystem

All active files

Eliminate costly performance bottlenecks

IBMFlashSystem

Hot Files

HDD Storage All other files

Data Metadata

IBM FlashSystemSpectrum Scale

Primary storage

1MillionIOPS

ResponseTimesunder0.5millisecond

Over6GB/ssequential

readthroughput

Asacachedevice Asastoragetier Asametadatastoragedevice

22

LeadingObjectStorageInnovation:SpectrumScaleandCleversafe

IBMofferscomplementaryObjectstorageofferings- eachdeliveringindustryleadinginnovationsfordifferentworkloads/usecases

• Cleversafe isdesignedtoenablesecure,largescaleactivearchivesandcloud-basedcontentrepositorieswherefocusisonlowestcostatscale• Erasurecodingdeliverscost-efficientreliability,requiringlessspacethantriplecopyapproaches• UniquekeylessencryptiontechnologyprovidesstrongsecurityforObjectarchives

• SpectrumScaleofferslargescaleFile& Objectstoragethatsupportshighperformanceprocessingandbigdataanalytics• Supportsmultiplearchitectures:storagerichservers;NASclusters;globallydistributedgrids• POSIXsupportforscale-outOLTPdatabasesanddatawarehouses• IntegratedlifecyclemanagementacrossFlash,DiskandTapetiers

23

HighPerformanceStorageSystem(HPSS)

ImprovedstorageeconomicsanddataavailabilityOpenStack Swift

• OakRidgeNationalLaboratorycutredundanttapecostsby75%with4+PHPSSRAITandupto872MB/sfile-transfers

• Enterprisetaperecommendedaccessorder(RAO)forfastertaperecalls

• HPSSTrashCan

• Filesystemandtraditionaldiskstoragetiers• ProductionreadyonPower8

• OpenStack “SwiftOnHPSS”providesobjectinterfaceforfilesharing

• Stopcorruptedfilesfrommakingittotapewithend-to-enddataintegrity

• 219MB/spertapedriveusing2.6MBfiles

24

IBMishelpingourclients…

ü Expandfromtraditionalcompute-centrictoadata-centricHPCmodelü Efficientlymanagethelifecycleofalldataincludingfilesandobjectsü Achievenewinsightsfromalldatawhileimprovingdataeconomicsü Store,analyzeandprotectdatawithahyperscale convergedinfrastructure

MaximizePerformance

EnableNewWorkloads

OptimizeEfficiency

25

MoreInformationFormoreinformation:

• Highperformancecomputing(HPC)andtechnicalcomputingsolutions• http://www-03.ibm.com/systems/power/solutions/high-performance-computing/

• http://www-03.ibm.com/systems/technicalcomputing/

• FindIT:yourgatewaytofindandusethetopassetsyouneedforanygivensalessituation• https://findit.cloud.dst.ibm.com

• Redbook:ImplementinganIBMHigh-PerformanceComputingSolutionon

IBMPOWER8• http://www.redbooks.ibm.com/abstracts/sg248263.html?Open=

26

Questions?

27