⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

Disk Quotas and Usage Rate Display

Principle

Quotas ensure fair access to disk resources. They prevent a group of users from consuming all the space and preventing other groups from working. At IDRIS, quotas limit both the amount of disk space and the number of files (inodes). These limits apply per user for your (single) HOME space and per project for your WORK and STORE spaces (for each of your projects).

You can check the usage of your disk spaces using one of the two commands presented on this page:

• idr_quota_user for a view of your personal usage as a user;
• idr_quota_project for an overview of your project and the consumption of each of its members.

note

The command idrquota is no longer available. It was the first command for viewing quotas that was deployed on Jean Zay. The commands idr_quota_user and idr_quota_project are an evolution of it.

Recommendation

Remember to regularly check the usage of your disk spaces using one of the two commands above: if you reach the quota, it can have consequences on your SLURM jobs.

Exceeding Quotas

When a user or project reaches its quota or is exceeding the quota, no warning email is sent. However, you are informed by error messages such as Disk quota exceeded when you manipulate files in the affected disk space or your binary writes to this space.

Attention

When one of the quotas is reached (volume or number of files/directories),

• You can no longer create files in the affected disk space. This can therefore disrupt running jobs if they were launched from this space or write to this space.
• Editing a file when at the disk quota limit can reduce its size to zero, thus erasing its content.

Remarks

• Remember to check the common disk spaces $ALL_CCFRWORK and $ALL_CCFRSTORE which are also taken into account for disk usage calculations.
• A recurring cause of quota exceedance (especially on the HOME) is the use of personal Conda environments. Please refer to the dedicated Python page for best practices on Jean Zay.

When you are blocked or about to be:

• Try to clean up the affected disk space by deleting obsolete files.
• Archive directories you no longer access or rarely access to reduce the number of files.
• Move your files/archives to another space depending on their usage (see the page on disk spaces).
• The project leader or their deputy can request an increase in the STORE space quotas and the WORK space volume via the Extranet interface.

The idr_quota_user Command

By default, the idr_quota_user command returns your personal usage as a user for your HOME and all disk spaces of your active project. For example, if your active project is abc, you will see an output similar to this:

idr_quota_user HOMEINODE:   |██-------------------------------| U: 9419/150000  6.28%STORAGE: |████████████████████████████████-| U: 2.98 GiB/3.00 GiB  99.31%
ABC STOREINODE:   |---------------------------------| U: 1/100000  0.00%           G: 12/100000  0.01%STORAGE: |---------------------------------| U: 4.00 KiB/50.00 TiB  0.00% G: 48.00 KiB/50.00 TiB  0.00%
ABC WORKINODE:   |███▒▒▒---------------------------| U: 50000/500000 10.00%       G: 100000/500000  20.00%STORAGE: |██████████▒▒▒▒▒▒▒▒▒▒-------------| U: 1.25 TiB/5.00 TiB  25.00% G: 2.5 TiB/5.00 TiB 50.00%
The quotas are refreshed daily. All the information is not in real time and may not reflect your real storage occupation.

In this example output, your personal usage is represented by the "full" bar, and quantified on the right after the letter U (for "User"). This personal usage is also compared to the project's overall usage, which is represented by the "shaded" bar, and quantified after the letter G (for "Group").

Remarks

• This command provides information about your HOME, which is personal to you and limited by a per-user quota. You are therefore blocked when you reach 100% of your HOME.
• For the WORK and STORE spaces, the quotas are per project, so you can be blocked by another project member (100% at the group level).
• The return of the command idr_quota_user is immediate but is not real-time information: the data is only updated once a day overnight.

You can refine the information returned by the idr_quota_user command by adding one or more of the following arguments:

• --project def to display the usage of a project different from your active project (here def);
• --all-projects to display the usage of all projects you are attached to;
• --space home work to display the usage of one (or more) specific disk space(s) (here HOME and WORK).

The full help for the idr_quota_user command is accessible using the -h option:

idr_quota_user -h 

The idr_quota_project Command

By default, the idr_quota_project command returns the disk usage of each member of your active project for all disk spaces associated with the project. For example, if your active project is abc, you will see an output similar to this:

 idr_quota_project  PROJECT: abc SPACE: WORKPROJECT USED INODE: 34373/500000 6.87%PROJECT USED STORAGE: 1.42 GiB/5.00 TiB 0.03%┌─────────────────┬─────────────────┬─────────────────┬─────────────────┬──────────────────────┐│      LOGIN      │     INODE ▽    │     INODE %     │     STORAGE     │      STORAGE %       │├─────────────────┼─────────────────┼─────────────────┼─────────────────┼──────────────────────┤│      abc001     │            29852│            5.97%│       698.45 MiB│                 0.01%││      abc002     │             4508│            0.90%│       747.03 MiB│                 0.01%││      abc003     │                8│            0.00%│         6.19 MiB│                 0.00%││      abc004     │                1│            0.00%│           0.00 B│                 0.00%││      abc005     │                1│            0.00%│           0.00 B│                 0.00%│└─────────────────┴─────────────────┴─────────────────┴─────────────────┴──────────────────────┘PROJECT: abc SPACE: STOREPROJECT USED INODE: 13/100000 0.01%PROJECT USED STORAGE: 52.00 KiB/50.00 TiB 0.00%┌─────────────────┬─────────────────┬─────────────────┬─────────────────┬──────────────────────┐│      LOGIN      │     INODE ▽    │     INODE %     │     STORAGE     │      STORAGE %       │├─────────────────┼─────────────────┼─────────────────┼─────────────────┼──────────────────────┤│      abc001     │                2│            0.00%│         8.00 KiB│                 0.00%││      abc002     │                2│            0.00%│         8.00 KiB│                 0.00%││      abc003     │                2│            0.00%│         8.00 KiB│                 0.00%││      abc004     │                2│            0.00%│         8.00 KiB│                 0.00%││      abc005     │                1│            0.00%│         4.00 KiB│                 0.00%│└─────────────────┴─────────────────┴─────────────────┴─────────────────┴──────────────────────┘The quotas are refreshed daily. All the information is not in real time and may not reflect your real storage occupation.

For each disk space attached to the project, a summary of the overall usage is displayed, followed by a detailed table of the usage of each project member.

Remarks

• This command does not provide information about your HOME because it is personal to you and not linked to a project.
• For the WORK and STORE spaces, the quotas are per project, so you can be blocked by another project member (100% at the level of one of the PROJECT USED ... lines), but the detail per user allows you to contact the person who is using the most space.
• The return of the command idr_quota_project is immediate but is not real-time information: the data is only updated once a day overnight.

You can refine the information returned by the idr_quota_project command by adding one or more of the following arguments:

• --project def to display the usage of a project different from your active project (here def);
• --space work to display the usage of one (or more) specific disk space(s) (here WORK);
• --order storage to sort the information in descending order of the values of a given column (here the STORAGE column).

The full help for the idr_quota_project command is accessible using the -h option:

idr_quota_project -h 

General Remarks

• The projects you are attached to correspond to the Unix groups listed by the idrproj command.
• Quotas are not monitored in real time and may not represent the current state of your disk space usage. The commands idr_quota_user and idr_quota_project are updated every morning.
• To know the real-time volume in bytes and inodes of a given directory my_directory, you can execute the commands du -hd0 my_directory and du -hd0 --inodes my_directory, respectively. Unlike the commands idr_quota_..., the execution time of the commands du can be significant and proportional to the size of the directory and the number of files it contains.
• For the WORK and STORE spaces, the displayed usage rates include both the usage of your personal space ($WORK or $STORE) and your usage of the common space ($ALL_CCFRWORK or $ALL_CCFRSTORE).