Ergon: Usage of the Ergon archive server

The deployment of this new archive server coincides with the new IDRIS policy regarding the storage of results. We recommend using this server for archiving purposes only and not for intermediate storage. We advise our users to use the WORKDIR space (common to the Turing and Ada computers) for all of their active data; this space constitutes the work directory of the batch jobs. During a simulations campaign, files used to start a new execution, even large-sized files, should be stored in this work directory: data files, executable files, result files or calculation restarts. In this way, it will not be necessary to transfer the result files each time to the archive machine followed by re-transferring them in the other direction before beginning the next step in the simulation.

This WORKDIR space has large group quotas which can be increased by a simple justified request from the project manager (IDRIS Extranet). Nevertheless, as this work space is not backed up, it is necessary to regularly copy the important or sensitive files into the Ergon archive server by using the specialised commands: mfput and mfget.

Information for the user:

  • Each IDRIS user has an account on the Ergon archive server.
  • The password and the Ergon connection shell (either bash or tcsh) are shared by the Ada, Adapp and Turing machines. You just need to change it on one of the machines and it will also be effective on the others.
  • The interactive connection on Ergon is still possible but it is recommended, in general, to work directly from the Adapp machine where all the pre/post-processing tools, commands and libraries are available.
  • No pre/post-processing software or library will be installed on Ergon.
  • The Ergon storage space, called HOME, is subjected to quota both in the number and volume of files.
  • All the commands related to file management in HOME are available on both Ergon and Adapp. The Ergon HOME space is directly accessible from Adapp with read/write permissions by using the environment variable $ARCHIVE (through the intermediary of a GPFS mount).
  • There is no direct access to the Ergon HOME from the Ada and Turing compute machines. You must explicitly transfer the files from one machine to another by using the secure mfget/mfput commands. On the compute machines, it is recommended to uncouple the file transfer phases (from/towards the archive server) from the compute steps by using multi-step jobs and the archive class (Turing: Multi-step jobs ; Turing: Multi-step jobs with file transfers using Ergon ; Ada: Execution of multi-step and cascade jobs).
  • For exchanges with the exterior, it is preferable to connect on Ergon for compressing or reorganising the data (tar, zip) and to initiate these transfers.