Jean Zay: APS

APS (Application Performance Snapshot) is a performance analysis tool provided with Intel VTune Amplifier. The sampling can be done very simply in command line and produces result files in various formats (text, HTML, VTune).

• Different versions of APS are accessible via the diverse Intel Parallel Studio XE versions installed on the machine.
• APS is perfectly adapted to MPI and OpenMP hybrid applications.

Which APS version is used depends on the Intel environment that you are using. You can access the different Intel environments with the module command.

The module avail intel-all command indicates the available versions:

$ module avail intel-all
intel-all/2016.4(16.0.4)  intel-all/2019.2(19.0.2)  intel-all/2020.0  
intel-all/2018.1(18.0.1)  intel-all/2019.4(19.0.4)  intel-all/2020.1  
intel-all/2018.5(18.0.5)  intel-all/2019.5(19.0.5)  intel-all/2020.2

Before working with this tool then, it is necessary to execute a command such as:

$ module load intel-all/2019.4

Note that the intel-all/XXXX.Y.Z modules contain complete Intel environments including the compilers, the MKL mathematical library and the MPI library, as well as the code analysis tools.

Once the module is loaded, usage of APS is realised in two steps:

1. Execution of the application via APS (in command line)
2. Analysis/visualisation of results in a chosen writing format:
   • With a text editor (batch job output file)
   • On line with the aps-report command (see the aps-report -h command)
   • Or, using a Web browser

Important: We strongly recommend that you use the same Intel environment version during compilation, execution and your code analysis.

The simplest way is to do the sampling in command line in your Slurm scripts: You just add the aps command into the srun call. For this command to be recognized, you must not forget to load the appropriate intel-all/20XX.Y module in your job.

To get help regarding the aps command options, you simply need to type aps --help.

At the end of the sampling, APS writes its files in a directory for which the name can be specified via the -r=directory_name or the --result-dir=directory_name option. Specify a sub-directory for writing these files in a permanent disk space (such as the WORK or SCRATCH).

Important: If the directory already exists, the current execution will write over the preceding execution results.

• Here is a submission example for a pure MPI code (without OpenMP threads) :

  job_aps_mpi.slurm

  #!/bin/bash
  #SBATCH --job-name=aps_mpi       # name of job
  #SBATCH --ntasks=4               # total number of MPI processes
  #SBATCH --ntasks-per-node=4      # number of MPI processes per node
  # /!\ Caution, "multithread" in Slurm vocabulary refers to hyperthreading.
  #SBATCH --hint=nomultithread     # reserves physical and not logical cores
  #SBATCH --time=00:30:00          # maximum execution time requested (HH:MM:SS)
  #SBATCH --output=%x_%j.out       # name of output file
  #SBATCH --error=%x_%j.out        # name of error file (same as output file)
   
  ## Choice of an Intel environment with APS
  module load intel/2019.4
   
  ##echo of commands
  set -x
   
  ## Execution in the SCRATCH
  cd $SCRATCH
   
  ## Sampling of the binary ./my_bin_exe
  ## The report files are generated automatically after the sampling.
  srun aps --collection-mode=mpi ./my_bin_exe

  
  Important: The diverse APS versions cannot use the Intel sampling drivers because they are not installed on the compute nodes. Consequently, certain analyses, activated via the --collection-mode option, are not available on Jean Zay. Notably, the --collection-mode=all option, which is the default value, is not usable. Therefore, you must specify the --collection-mode=mpi option for a pure MPI code.

• Here is a submission example for a hybrid MPI/OpenMP code:

  job_aps_mpi_omp.slurm

  #!/bin/bash
  #SBATCH --job-name=aps_mpi_omp   # name of job
  #SBATCH --ntasks=4               # total number of MPI processes
  #SBATCH --ntasks-per-node=4      # number of MPI processes per node
  #SBATCH --cpus-per-task=10       # number of OpenMP threads
  # /!\ Caution, "multithread" in Slurm vocabulary refers to hyperthreading.
  #SBATCH --hint=nomultithread     # reserves physical and not logical cores
  #SBATCH --time=00:30:00          # maximum execution time requested (HH:MM:SS)
  #SBATCH --output=%x_%j.out       # name of output file
  #SBATCH --error=%x_%j.out        # name of error file (same as output file)
   
  ## Choice of an Intel environment with APS
  module load intel/2019.4
   
  ##echo of commands
  set -x
   
  ## Execution in the SCRATCH
  cd $SCRATCH
   
  ## Sampling of the binary ./my_bin_exe
  ## The report files are generated automatically after the sampling.
  srun aps --collection-mode=mpi,omp ./my_bin_exe

  
  Important: The diverse APS versions cannot use the Intel sampling drivers because they are not installed on the compute nodes. Consequently, certain analyses, activated via the --collection-mode option, are not available on Jean Zay. Notably, the --collection-mode=all option, which is the default value, is not usable. Therefore, you must specify one of the following options:

  • --collection-mode=mpi to effectuate an MPI sampling
  • --collection-mode=omp to effectuate an OpenMP sampling
  • --collection-mode=mpi,omp to effectuate an MPI and OpenMP sampling

You will find global information about your code at the end of your job's output file: the MPI sub-programmes called, the functions consuming the most elapsed time, etc.

The aps-report tool does not require a remote graphical display. It can be directly called from an interactive session on Jean Zay by providing it with the name of the directory containing the results to display (aps_result_yyyymmdd, by default).
Here is a call example in order to see the data pertaining to each MPI process in your console:

$ aps-report -t -D aps_result_yyyymmdd

aps will also generate a file in HTML format (aps_report_yyyymmdd_hhmmss.html, by default) but there is not a web browser on Jean Zay which would allow you to consult it.

• Beginner's guide of Intel's APS Getting Started Guide
• APS can be freely downloaded from https://software.intel.com/performance-snapshot.