Data Management User Guide

ORNL is managed by UT-Battelle for the US Department of Energy

Data Management User Guide

Suzanne Parete-KoonOak Ridge Leadership Computing Facility

2 Presentation_name

Data Management Users Guide

• We have organized a New Data Management User Guide Look for the magenta icon on the systems guide page:

3 Presentation_name

Data Management Policy

• Details the official data management policy of the OLCF.

• Must be agreed to by PI and users before they get access to the project.

• Includes the Policy on: • Data storage and placement• Data Retention, Purge, & Quotas• Data Prohibitions & Safeguards• Software

4 Presentation_name

Directory Structures and filesystem

• Where to store your data and for how long• For example, did you know that Projects have a 50

GB storage area that is not purged on the Network File Service.– /ccs/proj/[projid]

5 Presentation_name

Data Transfer

• These pages list our Data Transfer Hardware and Software and give detailed examples of how to user it.

• For example, did you know that:• OLCF has two interactive data transfer nodes, 10 batch

schedulable data transfer nodes, and 3 batch scheduled data transfer nodes dedicated for transfers with the High performance storage system.

• OLCF has a supported globus endpoint olcf#dtn’

• A page that gives you a complete tutorial about how to obtain the Open Science Grid Certificate needed to use globus at OLCF

6 Presentation_name

Workflow • These pages explain how to cross-submit jobs between

OLCF resources and give example workflows.