High Performance Deepak Singh Principal Product Manager Cloud Computing
High Performance Cloud Computing
Jan 25, 2015
Talk given at a customer site recently
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
High Performance
Deepak SinghP r i n c i p a l P r o d u c t M a n a g e r
Cloud Computing
Via butteryflysha under a CC-BY license
Image: Simon Cockell under CC-BY
High Scale Computing
using a large number of computers at the same time to solve a problem
2
High ThroughputComputing
1
scale out
“embarassingly parallel”
constraints
constrained by capacity
constrained by capacity
More molecules
More simulations
More dimensions
Bigger systems
constrained by time
constrained by time
Upcoming conference
Grant submissions
Impatience!
Exploratory “spike” run
EC2
Elastic Compute Cloud
EC2
elastic
programmatic
ec2-run-instances
AWS CloudFormation
EC2 instance types
standard “m1”high cpu “c1”
high memory “m2”
http://aws.amazon.com/ec2/instance-types/
EC2
inst
ance
type
s
standard “m1”high cpu “c1”
high memory “m2”
http://aws.amazon.com/ec2/instance-types/
EC2
inst
ance
type
s
ec2-terminate-instances
rapid provisioning
10K in 45 minutes
design patterns
optimize for throughput
Tasks
Instances
Tasks
Instances
Queue
Tasks
Instances
Queue
vertical scaling
Increase instance size
Tasks
Instances
Queue
Increase instance size
Tasks
Instances
Queue
horizontal scaling
Increase instance count
Tasks
Instances
Queue
Tasks
Instances
Queue
Results
Store
Tasks
Instances
Queue
Results
Store
On-premise
Tasks
Instances
Queue
Results
Store
On-premise
Tasks
Instances
Queue
Results
Store
On-premise
optimize for cost
optimize for costmaximize bang for buck
on-demand instances
reserved instances
spot instances
ideal for batch
persistent requests
all or nothing
use cases galore
Credit: Angel Pizzaro, U. Penn
ClusterComputing
2
tightly coupled
MPI
Cluster Compute
Dual Intel X5570
“Nehalem”23GB RAM
HVM1.7TB scratch
GPGPU
Cluster Compute
10 gig E
PlacementGroup
Cluster Compute
Placementgroup
231
450
Cores 7040
Rmax 41.82
Rpeak 82.51
GPGPU
2 x Tesla
M2050
Getting Started
http://aws.amazon.com/hpc
4 steps
15 minutes
performance
WIEN2K Parallel Performance
H size 56,000 (25GB)Runtime (16x8 processors)
Local (Infiniband) 3h:48Cloud (10Gbps) 1h:30 ($40)
1200 atom unit cell; SCALAPACK+MPI diagonalization, matrix size 50k-100k
Credit: K. Jorissen, F. D. Villa, and J. J. Rehr (U. Washington)
customer examples
Computational Fluid Dynamics
Example Use Case #1
Dynamic Clusters
40-180 CC1 instances
Molecular Dynamics
Example Use Case #2
Steady Usage
32-40 CG1 instances
Machine Learning
Example Use Case #3
Spiky, Experimental
8-20 CG1 instances
Customer Case Study: Bioproximity
http://aws.amazon.com/solutions/case-studies/bioproximity/
Customer Case Study: cyclopic energy
http://aws.amazon.com/solutions/case-studies/cyclopic-energy/
OpenFOAM®
Customer Case Study: PSR
http://aws.amazon.com/solutions/case-studies/psr/
44,000 CPU hrs in Oct 2010
Stochastic Dual Dynamic Programming
familiar tools
Oracle Grid Engine
MIT StarCluster
LSF
Moab/Torque
Condor
StackIQ Rocks+
Slurm
[email protected] Twitter:@mndoci
http://slideshare.net/mndocihttp://mndoci.com
Inspiration and ideas from Matt Wood, James Hamilton
& Larry Lessig
Credit” Oberazzi under a CC-BY-NC-SA license
Related Documents
