Ouessant : Disk spaces

Three distinct disk spaces (HOME, WORKDIR and TMPDIR) are accessible to users on Ouessant. Each space has specific characteristics adapted to its usage which are described below. The paths to access these spaces are stored in 3 variables of the shell environment: $HOME, $WORKDIR, et $TMPDIR.

The HOME

$HOME : This is the home directory during an interactive connection. This space is intended for frequently-used small-sized files such as shell environment files, tools, and potentially sources and libraries of limited size (in space and in number of files). The characteristics of the HOME are as follows:

  • The HOME is a permanent space.
  • It is accessible in interactive or in batch jobs.
  • It is the home directory during an interactive connection. It can also be accessed via the $HOME variable :

    $ cd $HOME 
  • It is submitted to quotas per user which are intentionally rather low: 1 GiB (1 GB), by default. The IDRIS command quota_u allows you to see the real situation of your disk space occupation.
  • Attention : At the present time, the HOME is not backed up on the Ouessant prototype (contrary to the IDRIS production machines).

The WORKDIR

$WORKDIR : This is a permanent work and storage space which is usable in batch. In this space, we generally store large-sized files which are used to run batch jobs: data files, executable files, result or restart files, submission scripts and very large source files. The characteristics of WORKDIR are:

  • It is a permanent space.
  • It is not backed up.
  • It is accessible in interactive or in batch jobs.
  • It is composed of 2 sections:
    • A section in which each user has an individual part, accessed with the command:

      $ cd $WORKDIR
    • A section common to the UNIX group to which the user belongs. The files to be shared by all the group members can be placed here, accessed by the command:

      $ cd $COMMONDIR
  • It is submitted to group quotas : 1 TiB (1.1 TB), by default. The IDRIS command quota_u -w allows you to see the real situation of your disk occupation and that of each of your group members.

Usage recommendations:

  • The WORKDIR is not backed up. We are never protected from the risk of accidental manual destruction (rm) or a disk failure.

Attention :

  • Since batch jobs can run in the WORKDIR, the files are directly accessible in read/write (permanent space) and do not need to be explicitly copied. However, because several of your jobs can be run at the same time, you must create a unique execution directory for each of your jobs. In addition, the disk space is submitted to group quotas and your job execution can suddenly stop if the quotas are reached. Therefore, you must be aware of both your own activity in this disk space and that of your colleagues. For these reasons, you may prefer running your batch jobs in the TMPDIR.

The TMPDIR

$TMPDIR : An execution directory for batch jobs. The following are the characteristics of the TMPDIR:

  • TMPDIR is a temporary directory.
  • It is only accessible in batch jobs by using the $TMPDIR variable.
  • It is automatically created when a batch job begins: It is, therefore, unique to each batch job.
  • It is automatically destroyed at the end of this job: You must, therefore, copy the important files on a permanent disk space (WORKDIR, for example) before the end of the job.
  • TMPDIR is not submitted to the quotas, as is HOME or WORKDIR. However, some security quotas are put in place to avoid the situation where a user could unintentionally fill up all of the disk space because of an accidental usage error.

Usage recommendations :

General advice for using the TMPDIR :

  • For each execution, we assume that the necessary input files for the execution (restart or executable) have previously been stored on a permanent file system (HOME ou WORKDIR).
  • For each batch job execution, we advise you to be in the TMPDIR.
  • Using the cp command, copy the necessary files from the WORKDIR into the TMPDIR.
  • Launch the execution in the TMPDIR.
  • Before the batch job finishes, you must backup the significant files (those that are used regularly or might be post-processed) in a permanent file system (HOME or WORKDIR), by using the cp command.

Comments:

  • As the performance in read/write is identical for WORKDIR and TMPDIR, you can avoid making copies between these two directories if the code is able to read or write the files directly into the HOME or the WORKDIR;
  • TMPDIR, like WORKDIR, is a GPFS disk space whose bandwidth (about 50 GiB/s in read/write) is shared by all the users. In consequence, the input/output performance can vary as it can be slowed down in the case of exceptionally high-volume usage.

HOME and WORKDIR: Data security

To improve the protection of your data stored in your HOME and WORKDIR spaces, we recommend that you respect the security policy put in place at IDRIS.