Ada: VASP

Introduction

VASP : Vienna Ab initio Simulation Package. Ab initio quantum chemistry software.

Available versions

  • 4.6.35
  • 5.2.2
  • 5.2.12
  • 5.3.5 (default version)
  • 5.4.1, hybrid

This software is available on Ada in parallel (MPI or hybrid).

Three executable files are supplied, to be chosen during the execution in function of your needs:

  • vasp : standard version
  • vasp_gamma : version optimised for the calculations at gamma point, faster (from 30 to 50 %) but only allows calculations to gamma point.
  • vasp_nc : version for calculations of non-collinear magnetic structures and spin-orbit coupling (activated by some keywords in INCAR) but uses more memory.

You can consult the documentation on the official site of VASP for more information.

Attention : Access to this application is reserved to users/laboratories having a licence from the developers of VASP. We will verify your status before allowing you to execute files. Contact IDRIS support for more information.

If you have a licence for the 4.6 versions but you would like to access the 5.2 versions, you must purchase a new licence. The 5.2 licence also opens access to the 5.3.x versions.

Additional versions

The installation of VASP includes additional tools, including the ones developed by The University of Texas.

To access them, the names of the executable files are suffixed by:

  • _vts : transition state tools (NEB)
  • _dosbader : projection of the density of state on the Bader volumes
  • _tbdyn : dynamics et metadynamics using different thermostats
  • _beef : BEEF-vdW fonctional

These tools are installed firstly on the VASP default version. If you would like other tools or their installation on other versions, contact IDRIS support.

Launching script

Following is an example of a LoadLeveler submission script for Ada, calculating in the WORKDIR directory:

job.ll
# @ job_name         = job
# @ output           = $(job_name).$(jobid)
# @ error            = $(job_name).$(jobid)
# @ job_type         = parallel
# @ total_tasks      = 64
# @ wall_clock_limit = 1:00:00
# @ queue
 
### Module initialisation ###
module load vasp
 
### Command echoes ###
set -x
 
### Run the calculation ###
poe vasp

Following is an example of a LoadLeveler submission script for Ada, calculating in the TMPDIR directory :

job.ll
# @ job_name         = VASP
# @ job_type         = parallel
# @ output           = $(job_name).$(jobid)
# @ error            = $(job_name).$(jobid)
# @ wall_clock_limit = 01:00:00
# @ total_tasks      = 64
# @ queue
 
### Module initialisation ###
module load vasp
 
### Command echoes ###
set -x
 
### Copy to the TMPDIR ###
cp ./* $TMPDIR
 
### Run the calculation ###
cd $TMPDIR
poe vasp 
 
### Copy to the submission folder ###
cd -
cp $TMPDIR/* .

The module load vasp command loads the VASP default version. If you wish to use a different version, refer to the documentation on the module command.

Specific parameters to the software

Pseudopotentials

Some PAW pseudopotential data sets (version 5.2 for the calculations in LDA and PBE, including the pseudos for the GW calculations) are available in a directory accessible via the environment variable $PSEUDO_VASP. After the module command is executed, you may access this by simply typing:

cd $PSEUDO_VASP

In interactive mode, you can access this from the Ada front-end; alternatively, you may access it through integration into your script: for example,

module load vasp
cd $PSEUDO_VASP/PBE.52
cat H_h/POTCAR O_h/POTCAR > $WORKDIR/myfolder/POTCAR
cd -
poe vasp

Van der Waals kernel

Some specific calculations require the file vdw_kernel.bindat. It is available in the folder $PSEUDO_VASP.

Parameters of the INCAR file

The parallelisation parameters of the INCAR file are *very* sensitive on Ada. It is necessary to add the following:

LPLANE = .TRUE.
NPAR = optimal value
LSCALU = .FALSE.
NSIM = 4

The NPAR value

The optimal value of NPAR depends on the system studied and the number of cores. If you enter an incorrect value, this may result in a doubled execution time, or in the worst case, a segmentation fault, a RSPHERE error (NPAR too small), or even a calculation which diverges. Nevertheless, to help you, here are the optimal values obained for a test case :

MOR zeolite cell, doubled and exchanged 1 time (289 atoms), PBE, 400 eV, Γ-point, 2 geometry steps:

Version Number of
cores
NPAR Version Number of
cores
NPAR
4.6.35
standard
16 8 4.6.35
gamma
16 1
32 2 32 2
64 4 64 4
128 16 128 16
Version Number of
cores
NPAR Version Number of
cores
NPAR
5.2.2
standard
16 4 5.2.2
gamma
16 4
32 1 32 4
64 4 64 2
128 32 128 8
Version Number of
cores
NPAR Version Number of
cores
NPAR
5.2.12
standard
16 1 5.2.12
gamma
16 1
32 32 32 2
64 64 64 4
128 16 128 32
256 32 256 32
Version Number of
cores
NPAR Version Number of
cores
NPAR
5.3.3
standard
8 8 5.3.3
gamma
16 4
16 16 32 16
32 32 64 4
64 64 128 16
128 16 256 32
256 32

Note 1 : For the nc and vts versions, you should use the same parameters as for their corresponding standard versions.

Note 2 : Depending on the versions and parameters chosen, a WARNING message may appear at the beginning of the OUTCAR file indicating that your NPAR is not optimal. Since this did not occur during the above test, you don't need to take heed of this warning in the majority of cases (as you can see on the test, and contrary to what is announced in the WARNING, there isn't a mathematical rule for this parameter).

Note 3 : Beginning with version 5.3, VASP introduces the parameter NCORE as a replacement of the parameter NPAR. In the test case, using this parameter instead of NPAR slightly degrades performance. In addition, it would be necessary to test each possible value (as was done for NPAR) and this is not planned for the time being.

Note 4 : You must choose NPAR = number of cores for calculations using a functional hybrid or for GW or RPA calculations.

Parallelisation on the K points (KPAR)

Beginning with version 5.3, VASP permits parallelisation on the k points. This method is very advantageous for parallel acceleration and permits, depending on the system being studied, using more computing cores and, thereby, obtaining a faster result in an efficient way. Of course, if you have only one k point (gamma), use the vasp_gamma version because it is very efficient in this case: The execution time can be reduced by half (maximally) for the same number of compute cores and it introduces a deviation on the total energy of only 10-4 eV.

A new keyword, the KPAR parameter, manages this parallelisation in the INCAR file. It designates the number of attributed compute core groups for each k point (or groups of k points, if there are more). This parameter must be equal, therefore, to a lower multiple of the total number of cores. Before proceeding with any study (as noted in a message in the OUTCAR file), do some tests on your own study systems to check your results as this new parallelisation method is still being developed. Inappropriate parameters under certain conditions can cause the computation to stop abruptly without necessarily producing an error message.

No strict overall rule has been observed concerning this parameter. However, some global observations have been made:

  • NPAR must be adjusted in function of the KPAR: In the preceding NPAR table, choose NPAR in function of the division result, number of cores/ KPAR. If you enter an inappropriate value, VASP will use it anyway, or the largest value possible up to the inappropriate value; this is very detrimental to performance in the majority of cases.
  • The more irreducible k points there are, the more it is possible to use a large number of cores efficiently.
  • KPAR values in function of the system:
Small system

(H2O, 1000Å3, 800eV)
Tested with  meshes:
2×2×2
2×2×4
5×5×5
 
no. k points ≤ no. of cores * The best performance seems to be obtained if no. of cores / KPAR = 8
* Example: For 63 irreducible k points, with 128 compute cores, choose KPAR = 16.
no. k points > no. of cores * The best performance seems to be obtained if no. of cores / KPAR = 4
* Example : For 63 irreducible k points with 32 compute cores, choose KPAR = 8.
Medium and larger-sized systems

(zeolite, 289 atoms)
Tested with  meshes:
1×1×2
2×2×2
no. k points
= 1
* Launch with vasp_gamma !
no. k points
> 1
* The best performance seems to be obtained if no. of cores/ KPAR = 16, except …
* If KPAR is larger or equal to the number of irreducible k points of the system, then it is necessary to choose KPAR = number of irreducible k points, except…
* If the number of k points is not a lower multiple of the total number of compute cores, then it is necessary to choose KPAR so that no. of cores / KPAR = 16, but the performance may not be optimal.

Summary example 1 : There are 50 irreducible k points for my modest-sized system and I would like to use 128 compute cores. I choose KPAR=16 (as per the instructions, 128 / 8 = 16). The number of cores per KPAR group is 128 / 16 = 8. According to the NPAR table for VASP standard version 5.3.3, when there are 8 cores it is necessary to take NPAR=8. Conclusion: KPAR=16 and NPAR=8.

Summary example 2 : There are 8 irreducible k points for my large-sized system and I would like to use 256 compute cores. According to the above table, I should choose KPAR=16 (256 / 16 = 16) but the KPAR > no. of k points so I must choose KPAR=8. The number of cores per KPAR group is 256 / 8 = 32; therefore, according to the NPAR table for VASP standard version 5.3.3, when there are 32 cores it is necessary to choose NPAR=32. Conclusion: KPAR=8 and NPAR=32.

Note : Remember that each group of compute cores is going to process one or more different k points. Consequently, each group of cores must have sufficient resources to be able to “contain” the complete unit cell. If this is not the case, the calculation can be blocked with an error message such as insufficient virtual memory.

Hybrid parallelisation

Version 5.4.1 allows using the threaded Intel Math Kernel Library (MKL). When parallel efficiency becomes weak in function of the number of MPI processes, it is possible to use threads to compensate. This version is also interesting if your system does not have enough memory when staying in pure MPI parallelisation. When the chosen number of MPI processes is efficient, nothing needs to be modified or added to the launching script. If you wish to make use of the threads, here is the corresponding launching script:

job.ll
# @ job_name         = job
# @ output           = $(job_name).$(jobid)
# @ error            = $(job_name).$(jobid)
# @ job_type         = parallel
# @ total_tasks      = 128
# @ parallel_threads = 2
# @ wall_clock_limit = 1:00:00
# @ queue
 
### Module initialisation ###
module load vasp/5.4.1
 
### Command echoes ###
set -x
 
### Run the calculation ###
export MKL_NUM_THREADS=2
poe vasp_std

The optimal number of threads is generally 2 but you may use more if, for example, your system does not have enough memory or if using more threads is still efficient for your system. As shown in the above script example, the number of parallel_threads and MKL_NUM_THREADS should be in agreement; if not, you will not obtain a good performance.

Documentation

All the documentation may be found on the VASP site. You may also find information in their discussion forum.