Stanford: Solving the HPC Data Deluge

HPC Storage at Stanford University!

William Law Stanford Research Computing Center wlaw@stanford.edu!

We are a new center!

Working across disciplines, across the university.!

The grand tradition is for each research cluster to be a stand alone island.!

Backups? Who needs ‘em.! Oh, we do.!

Storage!

• AFS

• box.com

• GPFS

• Lustre

• NFS

• Full HW solutions (panasas, isilon, netapp)!

Data Transfer!

• Aspera (Enterprise, Faspex)

• Globus Online

• box.com!

Concerns!

•  IOPS

•  Storage Growth

•  Sharing/Transfer/Workflows

•  Data Integrity

•  Backups

•  Long Term Data Retention/Sustainability (who pays?)

•  Security and Compliance (and dealing with PHI)

•  Cost!

Path Forward!

Standardize platforms Use open protocols for access Provide access to common storage pools across resources!