⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

Multi-project User

On the Jean Zay machine, each user has a single login but this can be linked to multiple scientific projects.

My Projects

On the Jean Zay machine, each scientific project is identified by a Unix group. IDRIS has developed the idrproj command to list your projects and select a project (see idrproj -h). You also have access to the idrenv command, which lists the environment variables defined for each project (see idrenv -h for more details).

The idrproj Command

By default, the idrproj command displays the various Unix groups referencing your various projects. Example with a login linked to two projects grp1 and grp2:

idrproj
Available projects:-------------------  grp1 (123456)   grp2 (167890) [default][active]

Note that for each project, the command indicates:

• the reference Unix group (here grp1 and grp2),
• the IDRIS project number (here 123456 and 167890), different from the DARI project number,
• which project is active (indicated by [active]): the project under which you are currently working,
• and which project is selected by default (indicated by [default]): the project automatically selected when connecting to the machine and therefore active by default.

Changing the Default Project

To change the default project ([default]), use the -d option of the idrproj command. In the previous example, the command required to change the default group from grp2 to grp1 would be:

idrproj -d grp1Password:Default project changed to grp1

note

Changing the default project requires the user password.

Then verify the default project:

idrproj
Available projects:-------------------  grp1 (123456) [default]  grp2 (167890) [active]

Attention

This does not change the active project (here gpr2) of the current session, so you are still working under the same project! However, the change of default project remains effective for subsequent connections to the machine. Therefore, this new default project (here gpr1) will be the active project for all new connections. Thus, after reconnecting, the display of the idrproj command for this example will look like this:

idrproj
Available projects:-------------------  grp1 (123456)  [default] [active]  grp2 (167890)

Changing the default project is particularly useful for switching to another project if your current project has no more computing hours (hour allocation exhausted or project not renewed) or if you wish to calculate/work on another project for some time.

Attention

It may happen that you no longer have a default project (and therefore no active project) if the one you had was deleted because it was not renewed. The variables $IDRPROJ, $WORK, $STORE and $SCRATCH will then no longer be defined. You just need to apply the default project change procedure above and reconnect to the machine to restore a correct computing environment.

Note that you can also activate a project temporarily (i.e. for the current session only) as indicated in the "Changing the Active Project" paragraph below.

The idrenv Command

By default, the IDRIS idrenv command simply displays all the environment variables ($WORK, $SCRATCH, $STORE, "common" spaces, $IDRPROJ) related to the default project which is automatically selected and activated upon connection (in this example grp1) as well as all the environment variables referencing the ad-hoc directories of your other projects (in this example grp2):

idrenv
export ALL_CCFRSCRATCH=/...scratchpath.../grp1/commun;export ALL_CCFRSTORE=/...storepath.../grp1/commun;export ALL_CCFRWORK=/...workpath.../grp1/commun;export CCFRHOME=/...homepath.../gengrp/login;export CCFRSCRATCH=/...scratchpath.../grp1/login;export CCFRSTORE=/...storepath.../grp1/login;export CCFRWORK=/...workpath.../grp1/login;export HOME=/...homepath.../gengrp/login;export IDRPROJ=grp1;export grp1_ALL_CCFRSCRATCH=/...scratchpath.../grp1/commun;export grp1_ALL_CCFRSTORE=/...storepath.../grp1/commun;export grp1_ALL_CCFRWORK=/...workpath.../grp1/commun;export grp1_CCFRSCRATCH=/...scratchpath.../grp1/login;export grp1_CCFRSTORE=/...storepath.../grp1/login;export grp1_CCFRWORK=/...workpath.../grp1/login;export grp2_ALL_CCFRSCRATCH=/...scratchpath.../grp2/commun;export grp2_ALL_CCFRSTORE=/...storepath.../grp2/commun;export grp2_ALL_CCFRWORK=/...workpath.../grp2/commun;export grp2_CCFRSCRATCH=/...scratchpath.../grp2/login;export grp2_CCFRSTORE=/...storepath.../grp2/login;export grp2_CCFRWORK=/...workpath.../grp2/login;export SCRATCH=/...scratchpath.../grp1/login;export STORE=/...storepath.../grp1/login;export WORK=/...workpath.../grp1/login;

Remarks

• The environment variable $IDRPROJ references the active project to which you are currently attached (echo $IDRPROJ).
• Variables whose names contain the character string CCFR are defined to provide harmonisation between the various national computing centres.
• Variables whose names begin with grp1, grp2 allow access to the data (directories) of the corresponding projects regardless of the one on which you are currently working.

Changing the Active Project

To change the active project (indicated by [active]) for the current working session, you must use the -d option of the IDRIS idrenv command and pass everything as an argument to the Unix eval command. Thus, with the initial example, to activate the grp1 project:

eval $(idrenv -d grp1)

then verify the active project:

idrproj
Available projects:-------------------  grp1 (123456) [active]  grp2 (167890) [default]

Attention

In this case, the change of active project (here gpr1) is effective only for the working session in which the command is executed.

Remark

This does not change the default project (here gpr2) which remains the active project for subsequent connections to the machine. Thus, after reconnecting, the display of the idrproj command for this example will look like this:

idrproj
Available projects:-------------------  grp1 (123456)  grp2 (167890)  [default] [active]

warning

Since the default project is not changed when changing the active project, the display of the idrenv command (without option) will not correspond to that of the active project. To obtain the correct display, you must then use the idrenv -d $IDRPROJ command.

Best Practices in Multi-project

• The first thing to do is always to check which project you are currently attached to:

  • either via the content of the environment variable $IDRPROJ:

    echo $IDRPROJgrp2

    Remark: The environment variable $IDRPROJ references the active project to which you are currently attached;
  • or via the IDRIS idrproj command:

    idrproj
    Available projects:-------------------  grp1 (123456)  grp2 (167890) [default] [active]

    Remark: one of your projects is automatically selected by default (indicated by [default]).

• Then, if necessary, switch to another of your projects on which you wish to work:

  • For a change of active project valid only for the current working session:

    eval $(idrenv -d grp1)

  • For a change of default project which will only be active for subsequent connections, thus implying a disconnection/reconnection:

    idrproj -d grp1

• Then verify that the change has been made:

  • either via the content of the environment variable $IDRPROJ:

    echo $IDRPROJgrp1

  • or via the IDRIS idrproj command:

    idrproj
    Available projects:-------------------  grp1 (123456) [active]  grp2 (167890) [default]

Thus, you ensure that you create your files in the directories $WORK, $SCRATCH, $STORE, "common" spaces which are indeed those attached to the active project: you can then work (create/edit files, compile, ...) with a perfectly defined environment.

However, to submit Slurm jobs or for interactive executions, you will imperatively need to specify the hour allocation on which the hours consumed by your calculations should be deducted. Without this precision, you risk deducting the hours consumed on a project other than the one on which you are currently working.

You will therefore obligatorily need to add:

• either the Slurm directive #SBATCH --account=... in the header of the submission script,
• or the --account=... option to the sbatch, salloc or srun commands,
• and taking care to indicate the correct project (my_project) and the correct type of hours (cpu, v100, a100 or h100):
  • Thus for CPU hours, either as an argument to the sbatch, salloc and srun commands:

    sbatch ... --account=my_project@cpu job_cpu.slurm

    or via the Slurm directive in a batch job:

    #SBATCH --account=my_project@cpu

  • And for GPU hours (here v100), either as an argument to the sbatch, salloc and srun commands:

    sbatch ... --account=my_project@v100 job_gpu.slurm

    or via the Slurm directive in a batch job:

    #SBATCH --account=my_project@v100

Remark: the value of my_project corresponds to the value of the environment variable $IDRPROJ which references the active project to which you are currently attached: echo $IDRPROJ.