AI Profiling Tools
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.
This page was translated by an AI (LLM) with a cursory human check and is awaiting full review.
Profiling is an essential step in code optimisation. Its purpose is to target the most time- or memory-consuming execution steps and to visualise the distribution of the workload between the GPU and CPU.
Profiling an execution is a very time-consuming operation. This study is therefore generally carried out over a few iterations of your training. The creation of logs is done a posteriori, at the end of the job. It is not possible to visualise the profiling during its execution. The visualisation of logs can therefore be done either on Jean Zay or on your own local machine (sometimes it will be easier to do it on your own machine).
NVIDIA also provides a specific profiler for Deep Learning called DLProf. Coupled with Nsight, its GPU Kernel debug tool, it allows for the collection of an extremely comprehensive set of information. However, the implementation can be somewhat complex and significantly slow down the execution of the code.
Profiling solutions with PyTorch and TensorFlow
Both libraries offer profiling solutions. For now, we document:
With PyTorch:
- PyTorch Profiler
- DLProf (NVIDIA)
With TensorFlow:
Each of these profiling tools is capable of tracing the activity of the GPU.
The following comparative table lists the capabilities and possibilities of each of these tools.
| Profiler | Ease-of-use | Slow down code | Overview | Recommendations | Trace view | Multi GPU Dist. View | Memory profile | Kernel View | Operator View | Input View | Tensor Core efficiency view |
|---|---|---|---|---|---|---|---|---|---|---|---|
| PyTorch TensorBoard | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | . | ✓ |
| PyTorch Native | ✗ | ✓ | ✗ | . | ✓ | . | ✓ | . | . | . | . |
| DLProf + PyTorch | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | . | ✓ | ✓ | . | ✓ |
| TensorFlow TensorBoard | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | . | ✓ | . |
Legend:
- ✓ : feature present and positive sentiment
- ✗ : feature present and negative sentiment (difficulty or current version limited)
- . : feature absent