Jean Zay: Access to JupyterHub

The IDRIS teams have installed JupyterHub, a tool enabling the usage of Jupyter Notebooks and other applications like MLflow and Dask via a web interface, without having to initiate an SSH connection to the supercomputer beforehand. A procedure was installed for connecting to this tool and thereby obtain access to the front ends as well as the compute nodes via Slurm.

The presentation of JupyterHub which was given to the user committee on 16/09/2022 is available here (Important: This presentation was developed before the introduction of the SSH-spawner which enables the creation of a JupyterLab instance on a front-end node.)

JupyterHub is accessed from the following site: https://jupyterhub.idris.fr. The connection procedures are similar to those for accessing the Jean Zay front ends:

• Use the same user name and the same password as for accessing the Jean Zay machine.

• IP addresses will still be filtered by the firewall. You can add a new IP address via the FGC form.

If you usually access Jean Zay via a proxy machine, you can use SSH dynamic port forwarding to access our JupyterHub instance.

Use the following command to start the port forwarding:

ssh -D localhost:9080 -N -f proxy_machine_login@proxy_machine

The parameters proxy_machine_login and proxy_machine must be replaced by your login on the proxy machine and its address. The -N -f options allow starting the port forwarding without starting an interactive shell on the proxy machine.

Warning: Static port forwarding will not work to use JupyterHub. For more information regarding the differences between static and dynamic port forwarding, you can read the following document: How to set up SSH dynamic port forwarding on Linux.

You then need to configure your web browser so that it uses the dynamic port forwarding that you just set up. The configuration process can be slightly different depending on the browser you use but it should be easy to adapt from the example below.

If you are using Firefox, you need to click on the “Settings” item of the main menu then locate the “Network settings” section and click on the “Settings…” button.

You can then reproduce the configuration visible on the screenshot below which enables using a local “SOCKS” proxy (127.0.0.1 address) on port 9080 as defined when starting the port forwarding:

Once your browser is configured, you can access our JupyterHub instance directly at https://jupyterhub.idris.fr.

Note: When using those settings, all connections made from your browser go through your proxy machine. If you would like to avoid this behavior, you may define different profiles in your browser or just use another browser to access JupyterHub.

If you use a VPN to access Jean Zay but cannot connect to our JupyterHub instance, please contact your local IT team to confirm whether the VPN is configured such that the connection to https://jupyterhub.idris.fr is routed through the VPN.

To launch a notebook, it is necessary to create a JupyterLab instance on a node.

You can choose to create the instance in two ways: either on a front-end node (to have Internet access, for example), or on a compute node through reserving resources via Slurm. In each of these two cases, you must select the various options (work space, allocated resources, accounting, etc.).

IDRIS provides kernels which are based on modules installed on Jean Zay. You will find here all the Python,Tensorflow and PyTorch versions which are installed. You can create a new notebook with the chosen kernel via the launcher:

It is also possible to change the kernel on a previously created notebook by clicking on the name of the kernel on the top right of the screen.

This will open a dialogue box in which you can select the kernel you wish to have.

Other more advanced functionalities are available on JupyterHub such as the utilization of Tensorboard, MLFlow, creation of personalized kernels, etc. These functionalites are described in the JupyterHub technical documentation: https://jupyterhub.idris.fr/services/documentation/