Jean Zay: Database Management

There are various disk spaces on Jean Zay for storing your data. These disk spaces have different characteristics and are more or less relevant depending on how you want to use your data.

On this page we present some good practices for managing your database on Jean Zay.

Reminder about the disk space characteristics

The characteristics of the different disk spaces are summarized in the figure below and detailed on the disk spaces page of the documentation. A summary table of the main disk spaces is also found on that page.

Good practices

1/ The DSDIR reflex for public databases

If you work with public databases, we can download them for you in the DSDIR shared space. The data will be accessible to all users. This solution allows you to avoid saturating your disk spaces and to share the resources.

This space provides the same performance as the WORK: a 100GB/s bandwidth for reading/writing.

If your database is personal or under a license which is too restrictive, you must take charge of its management yourself in the disk spaces of your project, as described in the next sections.

2/ Keep a copy as an archive in the STORE

Downloading a database could require a substantial amount of resources (computation and time). If you take charge of your database management, we advise you to keep a copy as an archive in the STORE (or in the WORK, if this doesn’t saturate the space).

3/ WORK or SCRATCH?

During development or production phases, you can choose to store your database either in the WORK or in the SCRATCH by considering the following pro and cons:

WORK SCRATCH
+ No automatic file deletion procedure - Unused files are deleted after 30 days
+ Backed up space (snapshots) - Space not backed up
- 100GB/s bandwidth for reading/writing + 500GB/s bandwidth for reading/writing
- Quotas of 5TB and 500 kinodes per project + Very large safety quotas (2.5PB shared by all users)

Comment: If you store your database in the SCRATCH, it is important to keep an archived copy in the STORE. In this way, you will be able to retrieve the data that would be deleted by the automatic cleaning procedure of the SCRATCH.

4/ Contact the support team in case of problems

If you encounter some difficulties to import or manage your database on Jean Zay, the IDRIS support team is here to help ( ).