Skip to content

NCCL

πŸ“ Short definition

NVIDIA software library handling collective ops (AllReduce, AllGather, Broadcast) across GPUs, using NVLink, InfiniBand, or PCIe per topology.

πŸ“– Detailed definition

NCCL (pronounced β€œnickel”) underpins PyTorch, vLLM, TensorRT-LLM, and Ray for synchronizing gradients or activations across GPUs.

Main operations:

  • AllReduce: sum/average a tensor on all GPUs (distributed training and Tensor Parallelism).
  • AllGather: each GPU collects fragments from others.
  • Broadcast: sends one tensor from a master GPU to all others.

NCCL auto-detects topology (NVLink > InfiniBand > PCIe > Ethernet) and optimizes communication paths.

πŸ’‘ Why it matters for on-prem AI

Invisible but critical layer: if NCCL is not configured for NVLink or InfiniBand, all inter-GPU traffic goes over PCIe and sharply degrades Tensor Parallelism throughput.

⚠️ Common pitfalls

  • NCCL_P2P_DISABLE=1 disables direct peer-to-peer transfers β€” a common misconfiguration that can cut performance fivefold.
  • NCCL logs show detected topology: always verify NVLink is selected on HGX nodes.

πŸ”— See also

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.