Skip to content

CUDA

๐Ÿ“ Short definition

NVIDIAโ€™s software platform that lets programs use NVIDIA GPUs for computation, and therefore run AI models on these cards.

๐Ÿ“– Detailed definition

NVIDIA defines it as โ€œa parallel computing platform and programming modelโ€ that speeds up computation by harnessing the GPU1. It includes the driver, the compiler and optimized libraries on which frameworks (PyTorch) and inference engines (vLLM, SGLang, TensorRT-LLM) rely.

CUDA only runs on NVIDIA hardware. Other vendors have their own stack:

VendorSoftware stackRole
NVIDIACUDAreference of the AI ecosystem1
AMDROCmโ€œthe AMD open software stack for GPU-accelerated computingโ€2
AppleMetal, MLXMLX is โ€œan array framework for machine learning on Apple siliconโ€3

๐Ÿ’ก Why it matters for on-premise AI

Something new (model, format, optimization) often lands on CUDA first. Choosing an NVIDIA machine simplifies installing the engines; choosing AMD or Apple means checking that the target engine and model are supported on ROCm or MLX.

โš ๏ธ Common pitfalls

  • Assuming a โ€œGPUโ€ tool works on any GPU: many only support CUDA, or support ROCm and Metal with a delay.
  • Ignoring the version: an engine often requires a minimum CUDA and driver version.

๐Ÿ“š To go deeper

๐Ÿ”— See also

Footnotes

  1. NVIDIA, CUDA Programming Guide (โ€œCUDA is a parallel computing platform and programming model developed by NVIDIAโ€ฆ by harnessing the power of the GPUโ€), accessed 2026-10-11. https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html โ†ฉ โ†ฉ2

  2. AMD, What is ROCm? (โ€œROCm is the AMD open software stack for GPU-accelerated computingโ€), accessed 2026-10-11. https://rocm.docs.amd.com/en/latest/what-is-rocm.html โ†ฉ

  3. Apple ml-explore, MLX โ€” GitHub repository (โ€œMLX is an array framework for machine learning on Apple siliconโ€, unified memory model), accessed 2026-10-11. https://github.com/ml-explore/mlx โ†ฉ

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue โ†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.