⚠ INFORMATION
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.

Course: LLM specialization

---

Responsible : Theodora Pazakou
Instructors : Members of the IDRIS Support Team

This training aims to equip participants with the essential skills to adapt and effectively use Large Language Models (LLMs). It covers the main model adaptation techniques, including fine-tuning, Prompt Engineering, Retrieval-Augmented Generation (RAG), Parameter-Efficient Fine-Tuning (PEFT), and model alignment. The course also introduces multimodality and agentic systems, two key concepts for designing advanced LLM-based applications.

Sign up

Target audience

---

Target audience

This training is intended for individuals with a background in deep learning who wish to gain practical expertise in the main techniques for adapting and evaluating Large Language Models (LLMs), as well as in the concepts of multimodality and agentic systems. It is aimed at a wide range of professionals, including engineers, researchers, developers, PhD students, and data scientists who want to learn how to effectively adapt LLMs to downstream tasks.

Prerequisites

---

Prerequisites

• Master the fundamentals of deep learning.
• Be proficient in Python.
• Have a strong foundation in PyTorch.

Format and duration

---

Format and duration

This training lasts 4 days. Registration starts at 09:00, with classes beginning at 09:30. The average end time is 17:30 (which may vary depending on the group due to the hands-on sessions).

It is held in person only at the IDRIS premises in Orsay (91).

Attendance

Minimum: 8 people;
Maximum: 18 people.

Course content

---

Learning objectives:

• Acquire the theoretical basics of Transformers (attention mechanism, language modelling, etc.)
• Learn about different Fine-tuning methods (classical, LoRA, etc.) and Prompt Engineering techniques (RAG, Chain of Thought, etc.)
• Set up an environment for training and optimising Large Language Models (LLMs): training loop, evaluation, result tracking, data cleaning.
• Understand the principles of multimodality and agentic systems, and know how to integrate them into LLM-based applications.
• Apply theory to a specific use case.

Syllabus:

Day 1

• Theory of Transformers (attention mechanism, language modelling, etc.)
• Classical Fine-tuning
• Evaluation and metrics for Large Language Models

Day 2

• Setting up a framework for LLM development (MLflow, multi-GPU, etc.)
• Textual data cleaning
• Prompt Engineering
• Retrieval-Augmented Generation (RAG)

Day 3

• Parameter-Efficient Fine-Tuning (PEFT)
• Synthetic data generation and RAG evaluation
• LLM alignment (DPO, RLHF, etc.)

Day 4

• Deployment (inference)
• Multimodality
• Agents and tool calling

INFO

For effective execution of the hands-on sessions, these sessions will take place on the Jean Zay supercomputer. A workstation with access to the IDRIS supercomputer is provided to learners. Prior experience in using a supercomputer, as well as prior access to it, are not required.

Course materials

---

Course materials

INFO

All course materials, including slides, will be made available on a GitHub repository.

Upcoming sessions

---

To view the dates of upcoming sessions for this training, visit the following page:

🗓️ Upcoming sessions

Registration

For the next Minerva session (23-26 of March 2027), registration is free but mandatory on our training web server.

Sign up