Ada: Quantum ESPRESSO

Introduction

Quantum ESPRESSO is an integrated suite of codes which permits calculating electronic structures as well as nanometer-scale materials. This software is based on the DFT (Density Functional Theory) method using plane waves and pseudo-potentials (norm-conserving and ultrasoft).

Availability

The hybrid OpenMP/MPI parallel versions are installed. The available versions are:

  • 5.0.1
  • 5.0.2 (until 26 may 2016)
  • 5.1.0
  • 5.3.0 (since 26 may 2016)
  • 5.4.0
  • 6.0.0

To know which versions are installed, type the following on Ada :

module avail espresso

Note that as of version 6.0.0, the default environment cannot be used anymore to execute Quantum Espresso. Instead you should use the Intel MPI environment and modify your submission script accordingly (see the example bellow). 

Launching script

The following is an example of a launching script for a Quantum Espresso versions from 6.0.0 hybrid calculation (with 32 MPI processes and 4 OpenMP threads per process) executed in the WORKDIR data space :

job.ll
# @ job_name         = job
# @ output           = $(job_name).$(jobid)
# @ error            = $(job_name).$(jobid)
# @ job_type         = mpich
# @ total_tasks      = 32
# @ nb_threads  = 4
# @ resources = ConsumableCpus($(nb_threads))
# @ environment = NTASKS=$(total_tasks); OMP_NUM_THREADS=$(nb_threads)
# @ wall_clock_limit = 20:00:00
# @ queue
 
### Module Initialisation ###
module load espresso
 
### Command echoes ###
set -x
 
### Run the calculation ###
mpirun -np $NTASKS pw.x < file.input > file.out

The following is an example of a launching script for a Quantum Espresso versions prior to 6.0.0 hybrid calculation (with 32 MPI processes and 4 OpenMP threads per process) executed in the WORKDIR data space :

job.ll
# @ job_name         = job
# @ output           = $(job_name).$(jobid)
# @ error            = $(job_name).$(jobid)
# @ job_type         = parallel
# @ total_tasks      = 32
# @ parallel_threads = 4
# @ wall_clock_limit = 20:00:00
# @ queue
 
### Module Initialisation ###
module load espresso
 
### Command echoes ###
set -x
 
### Run the calculation ###
poe pw.x < file.input > file.out

The following is an example of a launching script for a Quantum Espresso hybrid calculation (with 128 MPI processes and 1 OpenMP thread per MPI process) executed in the TMPDIR data space :

job.ll
# @ job_name         = job
# @ output           = $(job_name).$(jobid)
# @ error            = $(job_name).$(jobid)
# @ job_type         = parallel
# @ total_tasks      = 128
# @ parallel_threads = 1
# @ wall_clock_limit = 20:00:00
# @ queue
 
### Module initialisation ###
module load espresso
 
### Command echoes ###
set -x
 
### Copy to TMPDIR ###
cp ./* $TMPDIR
 
### Run the calculation ###
cd $TMPDIR
poe pw.x < file.input > file.out
 
### Copy to the submission folder ###
cd -
cp $TMPDIR/* .

To use a particular version (for example, version 5.3.0), type the following :

module load espresso/5.3.0

Documentation

Results

  • For performance reasons, it is recommended to use version 5.3.0. On the Ada machine, a scalability gain of about 1.5 to 2.5 (depending on the test case) was observed when compared to version 5.0.2.
ADA (64 cores) - TEST CASE LP 1) ADA (64 cores) - TEST CASE FL 2)
MPIxOMP 5.0.2 (s) 5.3.0 (s) GAIN MPIxOMP 5.0.2 (s) 5.3.0 (s) GAIN
64×1 2403.48 946.17 2.54 64×1 3755.62 2532.51 1.48
32×2 2450.88 1297.67 1.89 32×2 4236.90 2725.13 1.55
16×4 2636.93 2225.57 1.18 16×4 - 4896.02 -
  • Quantum Espresso is a hybrid code (possibiity of using several OpenMP threads per MPI process). On the ADA machine, using only one OpenMP thread per MPI process seems to give the best performance.

SCALABILITY - ADA - Q.E. 5.3.0 - TEST CASE LP 3)
MPIxOMP Elapsed time (s) Memory (Go)
64×1 946.17 3.6
128×1 571.02 2.2
192×1 388.02 1.7
256×1 281.50 1.4
320×1 264.30 1.3
512×1 239.52 1.1
  • Version 5.4.0 : a very slight improvement in elapsed time
ADA (64 cores) - TEST CASE LP 4) ADA (64 cores) - CAS TEST FL 5)
MPIxOMP 5.0.2 (s) 5.3.0 (s) 5.4.0 (s) MPIxOMP 5.0.2 (s) 5.3.0 (s) 5.4.0 (s)
64×1 2403.48 946.17 943.83 64×1 3755.62 2532.51 2365.57
32×2 2450.88 1297.67 1309.86 32×2 4236.90 2725.13 2645.08

The test cases are energy optimisations.

Test case LP - input file: pw.in

1) , 3) , 4) “LP”: 400 bands, 300k plane waves, 8 k points
2) “FL”: 300 bands, 94k plane waves, 4 k points
5) “FL”: 300 bands, 94k plane waves, 4 k points