OPEN CCQ Federated High Performance Computing with
OPENCCQFederated High Performance Computing with
Where to start with HPC in the Cloud?
Here – Combine all the services yourself?
Clus
ter i
s Rea
dy
In Minutes your Cluster is ready
Collaboration
• Have the ability to create collaborations
• Invite other collaborators to CloudyCluster
• Initially can share Google Drive Folders
• Oauth and Shib / InCommonsupport for Web Authentication
Self-Service Elastic HPC
Scheduler
CCQ
EFS S3
Auto-ScalingCompute
OrangeFSHPCParallelStorage
DDB
HPCJob
Login
WebDAViRODSGlobus
CreateafullyoperationalHPCClusterinminutes,completewith:• Storage: OrangeFSonEBS,S3,EFS
• Compute: JobDrivenElasticComputethroughCCQ
• Scheduler: withv.1.2Torque/Maui&SLURMwithCCQMetaScheduler
• HPCLibraries:Boost,Cuda Toolkit,Docker,FFTW,FLTK,GCC,Gengetopt,GRIB2,GSL,Hadoop,HDF5,ImageMagick,JasPer,NetCDF,NumPy,Octave,OpenCV,OpenMPI,PROJ,R,Rmpi,SciPy,SWIG,WGRIB,UDUNITS
• HPCSoftware:Ambertools,ANN,ATLAS,BLAS,Blast,Blender,Burrows-WheelerAligner,CESM,GROMACS,LAMMPS,NCAR,NCL,NCO,nwchem,OpenFoam,papi,paraview,QuantumEspresso,SAMtools,WRF
• YoucanalsoInstallyourowninacustomAMI• AllfromaneasytouseWebUIfrommobile,tabletordesktop• iRODS andXDMoD aretargetedforFallrelease.
Torque/MauiSlurm
CCQ - Elastic HPC Dispatching
Scheduler
DynamoDB
Login Instance
Public Subnet
Submit the Job
Through CCQ
CCQ holds job determines and
launches instances needed
CCQ Sends the job to the
scheduler when ready
Scheduler launches the Job normally
If no jobs are in the queue for that
instance type near the billing hour, instances
are terminated
Compute Groups
SchedulerTorqueSlurm
OpenCCQ Project (Federated HPC)On Prem to Cloud bursting using heuristics and hints to determine where the job goes; supporting multiple schedulers and environments
Job Routing• On Premise• Cloud• Nat. ResourcesData MovementJob to Data
For $100 AWS Credit you can fill out the form at:http://www.cloudycluster.com/ccvisualhpc/
Discussion and Questions?
Come see a demo at our table