What is MCSR?
Who is MCSR?What Does MCSR Do?Who Does MCSR Serve?What Kinds of Accounts?Why Does Mississippi Need Supercomputers?What Kinds of Research?What Kinds of Instruction?What Kinds of Workshops?How Much Does it Cost?What Kinds of Software?What Supercomputers and Clusters?
What is MCSR?
Mississippi Center for Supercomputer Research
Established in 1987 by the Mississippi Legislature
Mission: Enhance Computational Research Climate at Mississippi’s 8 Public Universities
also: Support High Performance Computing (HPC) Education in Mississippi
What Does MCSR Do?
We make Mississippi scientists:- more competitive for federal grants- more productive in research
We provide extraordinary learning opportunities for Mississippi college students- instructional accounts- computing workshops- helpdesk support
Who Does MCSR Serve?
MCSR serves faculty, researchers, and students at all of Mississippi’s 8 public universities
• Alcorn State University• Jackson State University• Mississippi Sate University• Mississippi Valley State University• The University of Southern Mississippi• Delta State University• The University of Mississippi• Mississippi University for Women
Who Uses More MCSR Computing Time?
MSU; 38771; 4%
USM; 25279; 3%
UM; 172946; 19%
JSU; 663177; 73%
Other; 2145; 0%
CPU Hours Used by IHLAll MCSR SystemsFY 2009 (1st QTR)
2% 1% 10%
86%
SWEETGUM
1%28%
71%
REDWOOD
7%5%
16%
71%
MIMOSA
What Types of Computing Access is Available?
Research Accounts- provided for faculty and student “researchers”- good for the duration of employment or enrollment
Instructional Accounts- provided at an instructor’s request- for all students enrolled in a semester course- valid for the duration of the semester- can be “converted” to research account
Research vs. Class Accounts?
New Class Accounts; 266; 73%
New Research Accounts; 100;
27%
MCSR Accounts Created FY 08
New Class Accounts
New Research Accounts
Who Uses Research Accounts?
usm6%
msu5%
UM41%
jsu46%
other2%
MCSR Research Accounts by IHL
September 2008
usm
msu
um
jsu
other
Who Uses Instructional Accounts?
UM; 2
DSU; 8
USM; 1
Courses Supported with MCSR Class Accounts, FY 2008
UM
DSUUSM
UM; 5
DSU; 8
USM; 1
Courses Supported with MCSR Class Accounts, FY 2007
UMDSU
USM
UM; 7
DSU; 5
USM; 1
Courses Supported with MCSR Class Accounts, FY 2006
UM
DSUUSM
MVSU; 2
UM; 13
DSU; 2 USM; 1
Courses Supported with MCSR Class Accounts, FY 2005
MVSU
UM
DSU
USM
What Types of Courses Use MCSR?
Over 82 University Courses Supported since 2000
C/C++, Fortran, MPI, OpenMP, MySQL, HTML,Javascript, Matlab, PHP, Perl, ….
http://www.mcsr.olemiss.edu/education.php
Why Do Mississippi Researchers Need Supercomputers?
Economics: researchers in a poor state like Mississippi can still make a big splash.
Computational simulations are faster, cheaper, and safer than laboratory experiments alone.
What Kinds of Research @ MCSR?90% of MCSR calculations are computational chemistry
Cleanup of high explosive materials.Design of high energy density rocket fuelsThe chemical underpinnings of high powered lasersMutation studies of enzyme activityDesigning weather-proofing coatings for machinery
Other AreasHurricane forecastingBlast resistant CoatingsBetter 3-D imaging for diagnosing brain tumors
What Types of Workshops by MCSR?• MCSR consultants taught over 140 free seminars in
FY08.• Over 60 training topics available, and growing. • Fixed schedule or on-demand.• Unix/programming, Math Software, Stats Software,
Computational Chemistry Software
Do Researchers and Students Pay to Use MCSR?
No. MCSR services are provided at no cost to the individual, department, or institution.
Funded researchers may ask for priority access.
Mississippi researchers may claim the value of MCSR computing services received as an in-kind contribution from their institution when seeking federal grants.
How Much Does MCSR Cost Mississippi?
FY88
FY89
FY90
FY91
FY92
FY93
FY94
FY95
FY96
FY97
FY98
FY99
FY00
FY01
FY02
FY03
FY04
FY05
FY06
FY07
FY08
FY09
$-
$500,000
$1,000,000
$1,500,000
$2,000,000
$2,500,000
$3,000,000
MCSR Budget by Fiscal Year
Original
Revised
Special Funds
What Is the Value of MCSR to Mississippi?
RETURN ON INVESTMENT =
GRANTS SUPPORTED / MCSR BUDGET
For FY 2008:
$32,832,097 / $845,535
= $38.30 per dollar spent
What Software Environments @ MCSR
• Programming C/C++, FORTRAN, Java, Perl, PHP, MPI…
• Science/EngineeringPV-Wave, IMSL, GSL, Math Libraries, Abaqus
• Math/StatisticsSAS, SPSS, Matlab, Mathematica
• Chemistry Gaussian, Amber, NWChem, GAMESS, CPMD, MPQC, MolPro, GROMACS
What Supercomputers @ MCSR?
Supercomputers at MCSR: sweetgum
- SGI Origin 2800 128-CPU Supercomputer- 64 GB of shared memory
Supercomputers at MCSR: redwood
- 224 CPU SGI Altix 3700 Supercomputer- 224 GB of shared memory
Supercomputers at MCSR: mimosa
- 253 CPU Intel Linux Cluster – Pentium 4- Distributed memory – 500MB – 1GB per node- Gigabit Ethernet
Supercomputers at MCSR: mimosa
Supercomputers at MCSR: sequoia
- 22 nodes- 176 cores- 352 GB Memory- 20 TB Storage- InfiniBand Interconnect
Supercomputers at MCSR: sequoia
Introduction to Parallel Programming at MCSR
• Message Passing Computing– Processes communicate via calls to message passing library routines– Programmers “parallelize” algorithm and add message calls– At MCSR, this is via MPI programming with C or Fortran
• Sweetgum – Origin 2800 Supercomputer (128 CPUs)• Mimosa – Beowulf Cluster with 253 Nodes• Redwood – Altix 3700 Supercomputer (224 CPUs)• Sequoia – Altix XE 310 InfiniBand Cluster (176 cores)
• Shared Memory Computing– Threads coordinate/communicate results via shared memory variables– Care must be taken not to modify the wrong memory areas– At MCSR, this is via OpenMP programming with C or Fortran on
sweetgum, redwood, or sequoia
Speed-Up
http://www.mcsr.olemiss.edu/Engr692_TimingWorshkeet.xls
What MCSR Systems for USM Class Accounts
SweetgumMPI or OpenMP1 to 16 CPUsUp to 900mb per CPUPBS scripts preferred
#PBS –l ncpus=4Interactive computations will be killed after 30 minutesQueues: SM-4P, SM-8P, MM-8P, MM-16P Processors: Mix of 195 MHz and 300 MHzO/S: Irix (like Unix)Compilers: SGI’s Fortran, C/C++, GNU C/C++, w/ SGI MPT
What MCSR Systems for USM Class Accounts
MimosaMPI1 to 18 nodes400 GB Memory per nodePBS Scripts Only (no interactive jobs allowed)
#PBS –l nodes=4Queues: MCSR-CAProcessors: single 1.4 GHz P4/nodeO/S: SUSE Linux 10.3Compilers: Portland Group (PGI) Fortran, C/C++ w/mpichqstat –f (to find out what nodes your job is running on)
What MCSR Systems for USM Class AccountsSequoia
OpenMP (multiple processors on the same node)MPI (multiple processors on the same or different nodes)Hybrid (OpenMP within node, MPI across nodes)1 to 4 nodes, 1 to 8 CPUs per nodePBS Queues: SM-4P (for up to 4 CPUs on 1 node)
MCSR-Test (up to 8 CPUs on each of 4 nodes)PBS Scripts Only (no interactive jobs allowed)#PBS –l nodes=4:ppn=8 (to run on all 8 CPUs of all 4 nodes)#PBS –l ncpus=8:select= 16 GB Memory per node (2 GB per CPU)
qstat –f (to find out what nodes your job is running on)
Sequoia for USM Class AccountsTo run on 4 nodes, and 8 processors per node (32 processes)
#PBS –l nodes=4:ppn=8To run on 2 nodes, and 4 processors per node (8 processes)
#PBS –l nodes=2:ppn=4To run on 1 node, and up to 8 processors (OpenMP)
#PBS –l nodes=1:ppn=8To run on 8 processors, regardless of number of nodes
#PBS –l ncpus=8To run 8 processors, with preferences about node placement
#PBS –l ncpus=8#PBS –l place=scatter (distribute across as many nodes as can)#PBS –l place=pack (pack processes onto as few nodes as can)#PBS –l place=free (place processes on first available processors)
Parallel Efficiency