⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

Running a hybrid MPI/OpenMP job in batch

Jobs are managed across all nodes by the Slurm software.

To submit a hybrid MPI + OpenMP job in batch on Jean Zay, you need to:

Create a submission script:

intel_mpi_omp.slurm
#!/bin/bash
#SBATCH --job-name=Hybride         # nom du job
#SBATCH --nodes=2                  # Nombre de noeuds
#SBATCH --ntasks-per-node=4        # Nombre de processus MPI par noeud
#SBATCH --cpus-per-task=10         # nombre de threads OpenMP par tache MPI
# /!\ Attention, la ligne suivante est trompeuse mais dans le vocabulaire
# de Slurm "multithread" fait bien référence à l'hyperthreading.
#SBATCH --hint=nomultithread       # 1 thread par coeur physique (pas d'hyperthreading)
#SBATCH --time=00:10:00            # Temps d’exécution maximum demandé (HH:MM:SS)
#SBATCH --output=Hybride%j.out     # Nom du fichier de sortie (%j est remplacé par le numéro du travail)
#SBATCH --error=Hybride%j.out      # Nom du fichier d'erreur (ici commun avec la sortie)

# on se place dans le répertoire de soumission
cd ${SLURM_SUBMIT_DIR}

# nettoyage des modules charges en interactif et herites par defaut
module purge

# chargement des modules
module load ...

# echo des commandes lancées
set -x

# nombre de threads OpenMP
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

# binding OpenMP
export OMP_PLACES=cores

# exécution du code
srun ./executable_mpi_omp

Submit this script using the sbatch command:

sbatch intel_mpi_omp.slurm

Notes:

We recommend compiling and running your code in the same software environment by loading the same modules.
The --hint=nomultithread option ensures the reservation of physical cores (no hyperthreading).
The memory allocated for the job is proportional to the number of CPU cores requested. For example, if you request 1/4 of the physical CPU cores of a node, you will have access to 1/4 of its RAM memory. It is important to be consistent with the configuration of the nodes used to avoid overcharging of hours while benefiting from the memory you are entitled to. You can consult our documentation on this subject: Memory allocation with Slurm.
In these examples, it is assumed that the executable executable_mpi_omp is located in the submission directory, i.e., the directory in which you are located when using the sbatch command: the $SLURM_SUBMIT_DIR variable is automatically set by Slurm.
The output file of the calculation will also be located in the submission directory. It is created at the beginning of the job execution; editing or modifying it during the job execution may disrupt it.
The module purge is made indispensable by Slurm's default behaviour: the modules you have loaded in your environment at the time you run sbatch are taken into account in the submitted job.
The use of the srun command is indispensable when you request multi-task execution. We discourage the use of mpirun on Jean Zay, only srun guarantees a distribution in accordance with the resource specifications requested in your submission file.
All jobs have resources defined by a partition and a "Quality of Service" QoS (Quality of Service) set by default in Slurm. You can modify these limits by specifying a partition and/or a QoS as indicated in our documentation detailing CPU partitions and QoS.
For multi-project accounts as well as those with CPU and GPU hours, it is essential to specify the hour allocation on which to deduct the computing hours of the job as indicated in our documentation detailing the management of computing hours to ensure that the hours consumed by your jobs are deducted from the correct allocation.

Notes:​

Notes: