Skip to main content

46 posts tagged with "flashinfo"

Le Flash Info est la lettre électronique envoyée à tous les utilisateurs de l'IDRIS. Son objectif est d'informer ses lecteurs des dernières nouveautés concernant l'IDRIS. Son rythme de parution dépend de l'information à diffuser.

View all tags

Flash Info No 2025-07

IDRIS
IDRIS
Computing center
⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

Summary:

  • Power outage: Tuesday 8 April 2025 (reminder)
  • 3rd Deep Learning for Science Day on Thursday 5 June 2025 at the CNRS headquarters
  • IDRIS Training

  • Power outage: Tuesday 8 April 2025 (reminder)

As previously announced, maintenance of the IDRIS technical infrastructure requires the centre to be shut down tomorrow, Tuesday 8 April. The Jean Zay machine will be unavailable all day.

The User Support service will be closed. The website will remain accessible and you can follow the availability of the machines on the usual page: http://www.idris.fr/statut.html.

  • 3rd Deep Learning for Science Day on Thursday 5 June 2025 at the CNRS headquarters

IDRIS and the CNRS Fidle National Training Action are organising the third Deep Learning for Science Day (JDLS) on Thursday 5 June from 9 am to 5:30 pm at the CNRS headquarters, Paris Michel Ange. Speakers from various scientific fields will present their use of artificial intelligence methods throughout the day. It is also possible to participate in the day by taking part in the "My project in 3 minutes" session and/or the poster session that will follow. Information and registration at https://jdls-2025.sciencesconf.org.

  • IDRIS Training

Remember to register now for the IDRIS training sessions scheduled for the coming months:

  • Hybrid MPI/OpenMP Programming, 5-6 June
  • Jean Zay Workshop, 12-13 June
  • OpenMP/MPI, 16-20 June
  • Optimised Deep Learning on Jean Zay, 24-27 June

For more information on the IDRIS training catalogue and registration procedures: http://www.idris.fr/formations/catalogue.html.


You are receiving this newsletter because you are a user of the IDRIS machines.

You can view this issue of the IDRIS Flash Info and previous issues on the IDRIS website: http://www.idris.fr/flash-info.html

Migration of the operating system to Red Hat 9.4

IDRIS
IDRIS
Computing center
⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

Hello,

During tomorrow's maintenance (April 1st, 2025), the operating system of the Jean Zay supercomputer will be updated to Red Hat 9.4 (from 9.2 currently). This update aims to provide a recent software environment on Jean Zay.

This operating system version change should have a limited impact on your use of the machine. The majority of your executables should continue to work without any action on your part; in case of issues, we recommend that you try to recompile your codes.

The GNU compilers bundled with the operating system will be updated to version 11.4.1. The "gcc/11.3.1" module will become an alias for a new "gcc/11.4.1" module to ensure an easy transition.

In case of problems, contact the IDRIS Support.

Best regards, The IDRIS user support team

Flash Info No 2025-05

IDRIS
IDRIS
Computing center
⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.
  • GPU HPC and AI Hackathon: only a few days left to apply
  • Panoram'IA: meet on Friday 28/02 at 10 am
  • CEEMS: an energy measurement tool on Jean Zay
  • IDRIS Training

  • GPU HPC and AI Hackathon: only a few days left to apply

We remind you that the deadline for registration for our GPU HPC and AI hackathon is set for 4 March. As a reminder, this event, organised jointly with NVIDIA, will take place on 13 May (remotely) and on 20, 21 and 22 May (at the IDRIS premises).

Registration is via the event page (in English): https://www.openhackathons.org/s/siteevent/a0CUP00000sO9LV2A0/se000364.

  • Panoram'IA: meet on Friday 28/02 at 10 am

IDRIS support invites you on Friday morning 28/02 at 10 am for a live broadcast of its "Panoram'IA" show. This show will feature our guest Jérôme Louradour, who will share his experience with the pretraining of Lucie on our Jean Zay supercomputer. The live broadcast and replays are available on our YouTube channel "Un oeil sur l'IDRIS": https://www.youtube.com/@idriscnrs.

  • CEEMS: an energy measurement tool on Jean Zay

As announced at the last User Committee meeting, the CEEMS energy measurement tool is now available on Jean Zay: see the page http://www.idris.fr/jean-zay/jean-zay-doc-energie.html for more information.

  • IDRIS Training

Remember to register now for the IDRIS training sessions scheduled for the coming months:

  • MPI, from 18 to 21 March
  • OpenMP, from 26 to 28 March
  • Modern Scientific C++, from 9 to 11 April
  • Introduction to OpenACC and OpenMP GPU, from 16 to 18 April
  • Hybrid MPI/OpenMP Programming, from 5 to 6 June
  • Jean Zay Workshop, from 12 to 13 June
  • OpenMP/MPI, from 16 to 20 June

For more information on the IDRIS training catalogue and registration procedures: http://www.idris.fr/formations/catalogue.html.

Flash Info No 2025-04

IDRIS
IDRIS
Computing center
⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

[Version française ci-dessous]

IDRIS is hosting its fifth IDRIS Open Hackathon (https://www.openhackathons.org/s/siteevent/a0CUP00000sO9LV2A0/se000364) from May 13 to May 22, 2025 and we are actively seeking applications (deadline: March 4th).

If you are not familiar, this hands-on coding event helps researchers and engineers to optimise, accelerate, and scale their HPC and AI applications on Jean Zay by pairing them with expert programming mentors whose guidance can take your HPC and AI projects to the next level.

This year NVIDIA invites you to an informational webinar “Getting Hackathon Ready” focused on AI applications, with the participation of IDRIS:

Join us on February 18th, 11AM CET to learn more about how to participate in this unique opportunity.

By attending the webinar, you will:

  • Get an overview of the upcoming IDRIS Open Hackathon (https://www.openhackathons.org/s/siteevent/a0CUP00000sO9LV2A0/se000364) and how it can benefit your AI research;
  • Examine AI use cases advanced at previous hackathons;
  • Understand how Jean Zay can support your AI projects; and
  • Get the chance to meet AI specialists from NVIDIA for a private consultation on your project.

Space is limited, sign up now: https://bit.ly/IdrisGHR.

For a glimpse of the projects that benefited from attending last year’s event, read the 2024 IDRIS Open Hackathon recap: https://www.openhackathons.org/s/article/IDRIS-Contributing-to-the-Excellence-of-Scientific-Research.

We hope you will join us to learn more and receive personalised expert support.

Should you have any questions, please contact events@openhackathons.org.


IDRIS is organising its 5th Hackathon (https://www.openhackathons.org/s/siteevent/a0CUP00000sO9LV2A0/se000364) from 13 to 22 May 2025 and we are looking for interested teams to participate (registration deadline: 4 March).

What is a hackathon? It is an event dedicated to scientific code developers aimed at helping researchers and engineers to optimise, accelerate, and scale their HPC or AI applications on Jean Zay. To do this, you will be supported by programming expert mentors whose advice will boost your HPC or AI projects.

This year NVIDIA invites you to an informational webinar "Getting Hackathon Ready" focused on AI applications, with the participation of IDRIS:

Join us on 18 February at 11am to discover how to make the most of this opportunity.

By participating in the seminar, you will be able to:

  • Get an overview of the upcoming IDRIS Hackathon (https://www.openhackathons.org/s/siteevent/a0CUP00000sO9LV2A0/se000364) and how it can benefit your AI research;
  • Discover AI use cases that have benefited from previous hackathons;
  • Understand how to take advantage of Jean Zay for your AI projects;
  • Meet AI specialists from NVIDIA for a private exchange about your project.

The number of places is limited, register now: https://bit.ly/IdrisGHR.

For an overview of the projects that benefited from last year's hackathon, you can read the IDRIS Hackathon 2024 recap: https://www.openhackathons.org/s/article/IDRIS-Contributing-to-the-Excellence-of-Scientific-Research.

We hope you will participate to learn more and receive personalised support.

If you have any questions, please do not hesitate to contact events@openhackathons.org (in English).

Flash Info No 2025-03

IDRIS
IDRIS
Computing center
⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

Migration of the DSDIR space to Lustre

[English version below]

Hello,

During the maintenance on January 21, 2025, the DSDIR environment variable was modified and now points to a new Lustre storage space: "/lustre/fsmisc/dataset".

All databases and models downloaded to DSDIR since the beginning of Jean Zay have been migrated to this new space. This new space is accessible from all Jean Zay nodes, including the nodes of the new H100 extension.

This new space will now be used for downloading public datasets or models. The old disk spaces "/gpfsdsdir/dataset" and "/gpfsdswork/dataset" are now deprecated and will no longer be accessible as of the end of January.

If you are already using the DSDIR environment variable in your scripts, the data migration will have no impact on your runs. However, if you are using hardcoded paths, you will need to update them. We invite you to take this opportunity to switch to systematic use of the DSDIR variable, as in the following script for example:

''' import os from datasets import load_dataset dataset = load_dataset(os.environ['DSDIR']+"/HuggingFace/wikimedia/structured-wikipedia") '''

As a reminder, DSDIR is a dedicated disk space for storing public databases or models needed for the use of Artificial Intelligence tools. It is visible to all Jean Zay users. For more information: http://www.idris.fr/eng/jean-zay/cpu/jean-zay-cpu-calculateurs-disques-eng.html#the_dsdir

Please do not hesitate to contact the support team at assist@idris.fr if you encounter any issue.

Best regards, The IDRIS support team

GPU HPC and AI Hackathon at IDRIS in May 2025

IDRIS
IDRIS
Computing center
⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

For the fifth consecutive year, IDRIS is organising, together with NVIDIA, a GPU HPC and AI Hackathon.

It will take place on 13 May (remotely) and from 20 to 22 May 2025 (on-site at IDRIS). The registration deadline is 4 March.

GPU Hackathons allow teams of three or four developers or researchers to improve the performance of their HPC or AI codes under the guidance of one or more mentors, who are experts in GPU programming and come from universities, national laboratories, computing centres, government institutions and manufacturers.

You do not need to be an IDRIS user to participate in this event, so feel free to share this message with your colleagues. You will have access to IDRIS's computing resources, allowing you to test your code on recent GPUs, including NVIDIA H100s.

For HPC projects, it could be an opportunity to start porting a code to GPU using OpenACC, OpenMP, Kokkos, GPU libraries, CUDA, etc., or to optimise an existing GPU implementation. If your codebase is large, it is recommended to prepare a representative mini-application to be more efficient during the event.

For AI projects, this hackathon offers a unique opportunity to explore fine-tuning, developing or evaluating state-of-the-art models such as LLMs and GenAI applications. You will be able to optimise your models using advanced techniques such as mixed precision, multi-GPU or multi-node parallelisation (DDP, PP, TP, CP), using standard frameworks such as PyTorch or TensorFlow, or specialised tools such as DeepSpeed, Megatron-LM, Nanotron or NVIDIA NeMo. Although full pre-training is out of reach within the hackathon's time frame, preparing a pre-training code for large-scale models is still possible.

Registration is via the event page (in English): https://www.openhackathons.org/s/siteevent/a0CUP00000sO9LV2A0/se000364.

Flash Info No 2025-01

IDRIS
IDRIS
Computing center
⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.
  • Table of Contents:

  • Happy New Year!

  • Next IDRIS UC Meeting: Wednesday 22 January 2025

  • Panoram'IA: Join us on Friday 24/01 at 10 am

  • IDRIS Training


  • Happy New Year!

IDRIS wishes all its users a very happy 2025!

  • Next IDRIS UC Meeting: Wednesday 22 January 2025

The next IDRIS User Committee (UC) meeting will take place on Wednesday 22 January 2025 at the IDRIS premises. Do not hesitate to send your requests to: cu-elus at idris.fr. More information: http://www.idris.fr/cu.html

  • Panoram'IA: Join us on Friday 24/01 at 10 am

IDRIS support invites you to "Panoram'IA" on Friday morning 24/01 at 10 am: the monthly live video magazine covering scientific and technical AI news. The live stream and replays are available on our YouTube channel "Un oeil sur l'IDRIS": https://www.youtube.com/@idriscnrs.

  • IDRIS Training

Register now for the upcoming IDRIS training sessions scheduled over the next few months.

Coming up very soon:

  • Practical Introduction to Deep Learning (PIDL), 3 and 4 February
  • Deep Learning Architectures (ArchDL), 5 and 6 February

Other scheduled training sessions:

  • MPI, 18 to 21 March
  • OpenMP, 26 to 28 March
  • Modern Scientific C++, 9 to 11 April
  • Introduction to OpenACC and OpenMP GPU, 16 to 18 April

For more information on the IDRIS training catalogue and registration procedures: http://www.idris.fr/formations/catalogue.html.


Flash Info No 2024-29

IDRIS
IDRIS
Computing center
⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

Adaptation to the new terms of the Anaconda license

[English version below]

Hello,

Since summer 2024, the use of Anaconda repositories has been subject to pricing (https://legal.anaconda.com/policies/). As IDRIS does not plan to purchase a license, a few measures have been put in place on Jean Zay to limit requests to paid repositories as much as possible.

1/ The new conda environments installed for you by IDRIS support will be provided using Miniforge and no longer using Anaconda. Older Anaconda-based environments will remain available and can still be completed on request. Support will take care to target open repositories such as conda-forge.

2/ The behavior of any existing conda environment on Jean Zay has been modified by deploying a configuration file /etc/conda/.condarc on all nodes with internet access (login nodes, prepost, compil, archive, visu). As a result, all installations performed by conda now target the open repository conda-forge as a priority.

3/ Despite the deployment of the new configuration file, some requests to Anaconda repositories may persist if you are using Anaconda or Miniconda. We ask you to be attentive during your installations. Finally, we strongly advise you to work with Miniforge from now on to avoid this problem. A “miniforge/24.9.0” module is already available, and new versions will be installed regularly. On the contrary, we no longer plan to maintain Anaconda modules in the future.

Please do not hesitate to contact assist@idris.fr if you have any question on this subject.

Best regards, The IDRIS Support Team


Hello,

Since summer 2024, the use of Anaconda repositories has been subject to pricing (https://legal.anaconda.com/policies/). As IDRIS does not plan to purchase a license, a few measures have been put in place on Jean Zay to limit requests to paid repositories as much as possible.

1/ The new conda environments installed for you by IDRIS support will be provided using Miniforge and no longer using Anaconda. Older Anaconda-based environments will remain available and can still be completed on request. Support will take care to target open repositories such as conda-forge.

2/ The behavior of any existing conda environment on Jean Zay has been modified by deploying a configuration file /etc/conda/.condarc on all nodes with internet access (login nodes, prepost, compil, archive, visu). As a result, all installations performed by conda now target the open repository conda-forge as a priority.

3/ Despite the deployment of the new configuration file, some requests to Anaconda repositories may persist if you are using Anaconda or Miniconda. We ask you to be attentive during your installations. Finally, we strongly advise you to work with Miniforge from now on to avoid this problem. A “miniforge/24.9.0” module is already available, and new versions will be installed regularly. On the contrary, we no longer plan to maintain Anaconda modules in the future.

Please do not hesitate to contact assist@idris.fr if you have any question on this subject.

Best regards, The IDRIS Support Team


Flash Info No 2024-28

IDRIS
IDRIS
Computing center
⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.
  • Panoram'IA: Appointment Friday 15/11 at 10 am
  • DSDIR not accessible from the H100 partition
  • Shutdown of the old STORE space at the end of November 2024
  • IDRIS Training

  • Panoram'IA: Appointment Friday 15/11 at 10 am

IDRIS support invites you this Friday morning 15/11 at 10 am to "Panoram'IA": the monthly live video magazine that covers the scientific and technical news of AI. This session's programme includes: AI news, an expert intervention: "QROA: The art and science of jailbreaking LLMs" and our selection of papers with Papers Storm. The live stream and replays are available on our YouTube channel "Un oeil sur l'IDRIS": https://www.youtube.com/@idriscnrs.

  • DSDIR not accessible from the H100 partition

We would like to remind you that the DSDIR space, where we store public datasets and models useful to the community (http://www.idris.fr/jean-zay/gpu/jean-zay-gpu-dataset.html), is currently not accessible from the new H100 partition of Jean Zay. Equipment is currently being installed to enable its migration to a new Lustre-based storage system that will be accessible from all partitions. This new space should be operational by the end of 2024. In the meantime, if you need to use a dataset or model available in the DSDIR from an H100 node, we recommend copying it to your SCRATCH.

  • Shutdown of the old STORE space at the end of November 2024

Following the migration of the STORE space to a new Lustre-based storage system during the summer, the old STORE space remained accessible in read-only mode using the environment variable "$OLDSTORE" (http://www.idris.fr/jean-zay/modifications-extension-jean-zay-h100.html#cas_particulier_du_store). The definitive shutdown of this space will take place at the end of November 2024.

  • IDRIS Training

Remember to register now for the IDRIS training sessions scheduled for the rest of the year and early 2025:

  • HPC Debugging, 22 November
  • SIMD Vectorisation, 26 November
  • Optimised Deep Learning on Jean Zay, 14, 15, 16 and 17 January
  • MPI, 18, 19, 20 and 21 March
  • OpenMP, 26, 27 and 28 March

For more information on the IDRIS training catalogue and registration procedures: http://www.idris.fr/formations/catalogue.html.


Flash Info No 2024-27

IDRIS
IDRIS
Computing center
⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

[English version below]

Hello,

Following today's maintenance (Tuesday 1st October), several changes may impact you.

  • QoS name changes for the A100 partition

To better manage resource sharing on the machine, specific QoS have been defined for the A100 partition. If you explicitly used the QoS "qos_gpu-t3" or "qos_gpu-dev" in your job submissions targeting this partition, you will need to use "qos_gpu_a100-t3" or "qos_gpu_a100-dev" instead. The QoS "qos_gpu_a100-t3" is used by default and can be omitted.

The CPU and V100 partitions are not affected by this change.

The documentation has been updated accordingly: http://www.idris.fr/jean-zay/gpu/jean-zay-gpu-exec_partition_slurm.html#les_qos_disponibles.

  • Use of QoS via JupyterHub

If you wish to specify a QoS when using the Slurm launcher on JupyterHub, you will now need to specify it manually in the "Extra #SBATCH directives" field.

  • JupyterHub IP address change

The IP address of our JupyterHub instance has been modified. It is now 130.84.132.56. This change may impact you if your organisation applies IP address filtering for outgoing connections. If you encounter difficulties connecting to JupyterHub, we suggest contacting your IT service to inform them of this change.

As a reminder, the range of IP addresses used for IDRIS machines and services is as follows: 130.84.132.0/23. We recommend authorising the entire range rather than specific IP addresses to avoid being affected by future internal changes to our infrastructure.

  • Opening of the H100 partition

Users who have already obtained H100 hours can now use them. You can refer to the example below:

#!/bin/bash
#SBATCH --job-name=mon_travail # nom du job
#SBATCH -A xyz@h100 # comptabilite a utiliser, avec xyz le trigramme de votre projet
#SBATCH -C h100 # pour cibler les noeuds H100
# Ici, reservation de 3x24=72 CPU (pour 3 taches) et de 3 GPU (1 GPU par tache) sur un seul noeud :
#SBATCH --nodes=1 # nombre de noeud
#SBATCH --ntasks-per-node=3 # nombre de tache MPI par noeud (= ici nombre de GPU par noeud)
#SBATCH --gres=gpu:3 # nombre de GPU par noeud (max 4 pour les noeuds H100)
# Sachant qu'ici on ne reserve qu'un seul GPU par tache (soit 1/4 des GPUs),
# l'ideal est de reserver 1/4 des CPU du noeud pour chaque tache:
#SBATCH --cpus-per-task=24 # nombre de CPU par tache (1/4 des CPUs ici)
# /!\ Attention, "multithread" fait reference a l'hyperthreading dans la terminologie Slurm
#SBATCH --hint=nomultithread # hyperthreading desactive

Note that the default modules are not compatible with the H100 partition. To use the software environment specific to this partition, you must load the "arch/h100" module: http://www.idris.fr/jean-zay/cpu/jean-zay-cpu-doc_module.html#modules_compatibles_avec_la_partition_gpu_p6. This must be done in your submission scripts as well as in your terminal if you need to compile codes.

If you do not yet have H100 hours, the project manager can make a request on the eDARI portal if necessary.

Do not hesitate to contact assist@idris.fr if needed.

Best regards, The IDRIS support team


Dear Jean Zay user,

Several changes might affect you after today's maintenance operations (Tuesday October 1st):

  • QoS name changes for the A100 partition

In order to more precisely manage the resource sharing of the machine, specific QoS have been defined for the A100 partition. If you used to explicitly specify "qos_gpu-t3" or "qos_gpu-dev" in your Slurm jobs targeting the A100 partition, you now have to use "qos_gpu_a100-t3" or "qos_gpu_a100-dev" instead. Note that the "qos_gpu_a100-t3" QoS is used by default and may be omitted.

The CPU and V100 partitions are not affected by these changes.

The online documentation has been updated: http://www.idris.fr/eng/jean-zay/gpu/jean-zay-gpu-exec_partition_slurm-eng.html#available_qos

  • Use of QoS through JupyterHub

If you wish to specify a QoS when using Slurm on JupyterHub, you now have to do it manually in the "Extra #SBATCH directives" field.

  • JupyterHub IP address change

The IP address of our JupyterHub instance has been modified. It is now 130.84.132.56. This change might impact you if your institution applies an IP address filtering of outgoing connections. If you run into difficulties when connecting to JupyterHub, we invite you to contact your local administrator to mention this change.

As a reminder, the set of IP addresses used for the IDRIS machines and services is the following: 130.84.132.0/23. We recommend authorising the complete set rather than specific IP addresses so as not to be affected by potential future internal changes of our infrastructure.

  • Opening of the H100 partition

Users who were already granted H100 computing hours may now use them. An example submission script is as follows:

#!/bin/bash
#SBATCH --job-name=my_job # job name
#SBATCH -A xyz@h100 # account to use, with xyz the 3 letter code of your project
#SBATCH -C h100 # to target H100 nodes
# Example reservation of 3x24=72 CPU (for 3 tasks) and 3 GPU (1 GPU per task) on one node:
#SBATCH --nodes=1 # number of nodes
#SBATCH --ntasks-per-node=3 # number of MPI tasks per node (= number of GPU requested per node here)
#SBATCH --gres=gpu:3 # number of GPU requested per node (max. 4 for H100 nodes)
# Since here only one GPU per task is requested (i.e., 1/4 of the available GPUs)
# the best way to proceed is to book 1/4 of the node's CPU for each task:
#SBATCH --cpus-per-task=24 # number of CPU per task (1/4 of the CPUs here)
# /!\ Caution, "multithread" in Slurm vocabulary refers to hyperthreading.
#SBATCH --hint=nomultithread # hyperthreading deactived

Note that the default modules are not compatible with the H100 partition. In order to use the software environment dedicated to this partition, you need to load the "arch/h100" module: http://www.idris.fr/eng/jean-zay/cpu/jean-zay-cpu-doc_module-eng.html#modules_compatible_with_gpu_p6_partition. This is needed for your submission scripts but also in your shell when compiling codes.

If you do not have H100 computing hours yet, your project manager may ask for supplementary hours ("au fil de l'eau") on the eDARI portal if necessary.

Do not hesitate to contact assist@idris.fr if needed.

Best regards, The IDRIS support team

Your opinion matters!

To give your feedback, report an error, or suggest an improvement, click here:

quick anonymous questionnaire

This questionnaire is temporary and will take less than a minute, so take the opportunity!