FNAL - LQCD Documentation

New Users / Account Renewal

User Authentication

Kerberos and SSH Troubleshooting

Project Allocations

SLURM Batch System

Software Details

Hardware Details

Data Management

Filesystem Details

Mass/Tape Storage Details

Globus Online

Getting Help


LIVE Cluster Status

Contact Us

Fermilab LQCD Data Management Guidelines and Policies

The Scientific Program Committee (SPC) allocates resources during each program year. In addition to CPU or GPU hours, each project may be allocated some amount of disk or tape storage space. Each project should have a plan for manageing their data over its lifetime. These plans need to include knowing how much space you will need, considering where you will store copies of precious data and knowing what you will do with the data at the end of the program year. The project's PI may want to name an individual to be the responsible data mangaer who will oversee this plan.

Responsibility for the data

The project/user is the owner of the data and is responsible for managing and preserving this data. The Fermilab page regarding Filesystems details which of our storage is backed up by LQCD-Admin staff and which is not. Regardless of these backups, users should save copies of precious files in some other location.

For example: our Lustre filesystem is not backed up and is meant to be used as a volatile storage space. Users are responsible for keeping copies of precious data at some other location or on some other media.

To that end, if your project is going to need storage space at some other site/institution, please make arrangements for that space before you need it. Storage requests can take time to get approvals and should be submitted before you need the space.

Quota Management

Your alocated storage in the Lustre filesystem will have a quota set, based on the disk storage allocation from the Scientific Program Committee. We will set up a Unix group (GID) named similarly to your project name. The quota will be set within Lustre at the group level.

Since we manage the quotas based on group ownership, we require that all of the files and directories under the project's top level directory (ie. /lqcdproj/projectname) be owned by the allocated group. We use Unix level "Sticky bits" so that any files created under your top level directory will inherit group ownership from the parent. LQCD-Admin staff will review group ownerships as needed. Any files or directories that are associated with an incorrect group will be updated to show ownership by the proper allocation GID.

The home areas for all users are based on a user (UID) level quota. These are entirely seperate from the project's Lustre quota. Which user owns a file in the Lustre storage has no significance as far as Lustre quota tracking is concerned.

Tape vs. Disc storage

Magnetic tape storage provides bulk data storage that is currently less expensive per GigaByte than either magnetic or solid state disk storage. Magnetic tape costs for data in total are a significant recurring cost for the project. Projects may request short-term tape storage for the active life of the project and its extensions. Projects are expected to remove short-term data once a project becomes inactive.

Projects may request long-term magnetic tape storage for critical results or community data such as QCD gauge configurations. Typically, storage space is granted for the lifetime of the tape media, and a single migration to new tape technologies or media. Additional migrations may incur significant costs to the researchers and the USQCD collaboration. Projects must request long-term storage as part of their proposal and data management plan. The amount of storage and hosting institution will be negotiated by the SPC, EC, and the USQCD facilities.

End of the program year data housekeeping

At the end of the allocation year, each project must arrange to clear their old data from the Lustre storage so that space can be allocated to the new projects in the new program year. The project's PI or a designated data manager will be responsible for this data mangement task.

Several scenarios are possible:

Volitile or Working data should be deleted ASAP

Any data that has already been copied elsewhere or stored in an archival storage system should be deleted as soon as possible. This frees up space right away for the new allocations.

We ask that the migration and clearing of any data that remains at this point be completed within two months. If additional time will be needed, we ask that the PI or data manager send details of their plan and schedule for clearing the remaining space to lqcd-admin@fnal.gov. We can then discuss the plan and set a reasonable deadline for completing this data housekeeping.

Transfer to a new allocation

If your project is continuing under a new allocation, any data that will still be needed and useful can be brought into that new allocation.

Data already copied elsewhere

There should already be copies of precious data elsewhere. Once the data manager verifies that the copies exist, this data can be deleted from the Lustre storage area.

Remaining data should be copied elsewhere

Any precious data that remains should be copied to another site. See our Globus Online page for information on using Globus to transfer files out of Lustre.

The final deadline

Any data that remains three months after the end of the program year is subject to deletion by LQCD-Admin staff, unless an extension has been negotiated.

This page last updated 16 April 2019. If you have questions or feedback regarding this policy, please send email to hpc-admin@fnal.gov.