Skip to main content
⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

Using CUDA MPS

Presentation​

The multi-process service MPS (Multi-Process Service) is a compatible implementation variant of the CUDA programming interface. The MPS execution architecture is designed to allow cooperative multi-process CUDA applications, typically for MPI work, to use the Hyper-Q features on the latest NVIDIA GPUs. Hyper-Q allows simultaneous processing of CUDA kernels on the same GPU, which can improve performance when the GPU's computing capacity is underutilised by a single application process.

Usage​

The CUDA MPS service is available by default in the various CUDA modules provided to users.

For a batch multi-GPU MPI distributed job, CUDA MPS can be enabled via the -C mps option.

Attention

The node must be reserved exclusively via the --exclusive option.

  1. You can refer to the examples below:
multi_gpu_mpi_mps.slurm
#!/bin/bash
#SBATCH --job-name=multi_gpu_mpi_mps # nom du job
#SBATCH --exclusive                  # on réserve le noeud de manière exclusive 
#SBATCH -C mps                       # activation du service MPS
# Il est possible d'utiliser uniquement des GPU V100 16 Go ou 32 Go
##SBATCH -C mps&v100-16g             # decommenter pour reserver uniquement des GPU V100 16 Go
##SBATCH -C mps&v100-32g             # decommenter pour reserver uniquement des GPU V100 32 Go
# Ici, reservation de 1 noeud avec 4 GPU par noeud et 40 taches MPI :
#SBATCH --nodes=1                    # on demande un noeud
#SBATCH --ntasks-per-node=40         # nombre de taches par noeud
#SBATCH --gres=gpu:4                 # nombre de GPU par noeud (max 4)
# Le nombre de CPU par tache doit etre adapte en fonction de la partition utilisee
# de sorte à ce que ntasks-per-node x cpus-per-tasks = nombre total de coeurs CPU
#SBATCH --cpus-per-task=1            # nombre de CPU par tache (ntasks-per-node x cpus-per-tasks = 40)
# /!\ Attention, "multithread" fait reference à l'hyperthreading dans la terminologie Slurm
#SBATCH --hint=nomultithread         # hyperthreading desactive
#SBATCH --time=00:10:00              # temps maximum d'execution demande (HH:MM:SS)
#SBATCH --output=multi_gpu%j.out     # nom du fichier de sortie (%j est remplacé par le numéro du travail)
#SBATCH --error=multi_gpu%j.out      # nom du fichier d'erreur (ici commun avec la sortie)

# Nettoyage des modules charges en interactif et herites par defaut
module purge

# Chargement des modules
module load ...

# Echo des commandes lancees
set -x

# Le code doit etre compile avec les modules compatibles avec la partition choisie
# Execution du code
srun ./executable_multi_gpu_mpi
  1. Then submit the script via the sbatch command:
sbatch multi_gpu_mpi_mps.slurm
Attention

Even if you are working on only part of the node, it must be reserved exclusively to enable MPS. The entire node will therefore be billed.

Remarks:​

  • We recommend compiling and running your code in the same software environment by loading the same modules.
  • The --hint=nomultithread option ensures the reservation of physical cores (no hyperthreading).
  • The memory allocated for the job is proportional to the number of CPU cores requested. For example, if you request 1/4 of the physical CPU cores of a node, you will have access to 1/4 of its RAM memory. It is important to be consistent with the configuration of the nodes used to avoid overcharging of hours, while benefiting from the memory to which you are entitled. You can consult our documentation on this subject: Memory allocation with Slurm.
  • In these examples, it is assumed that the executable executable_multi_gpu_mpi is located in the submission directory, i.e. the directory in which you are located when using the sbatch command: the $SLURM_SUBMIT_DIR variable is automatically set by Slurm.
  • The output file of the computation will also be located in the submission directory. It is created at the beginning of the job execution; editing or modifying it during the job execution may disrupt it.
  • The module purge is made indispensable by Slurm's default behaviour: the modules you have loaded in your environment at the time you run sbatch are taken into account in the submitted job.
  • The use of the srun command is indispensable when you request a multi-task execution. We advise against using mpirun on Jean Zay, only srun guarantees a distribution in accordance with the resource specifications requested in your submission file.
  • Jobs all have resources defined by a partition and a "Quality of Service" QoS positioned by default in Slurm. You can modify the limits by specifying a partition and/or a QoS as indicated in our documentation detailingGPU partitions and QoS.
  • For multi-project accounts and those with CPU and GPU hours, it is essential to specify the hour allocation on which to deduct the computing hours of the job as indicated in our documentation detailing themanagement of computing hours to ensure that the hours consumed by your jobs are deducted from the correct allocation.

Documentation​

Theofficial documentation at NVIDIA.

Your opinion matters!

To give your feedback, report an error, or suggest an improvement, click here:

quick anonymous questionnaire

This questionnaire is temporary and will take less than a minute, so take the opportunity!