Skip to content

NVLink

πŸ“ Short definition

Dedicated hardware link accelerating exchanges between compatible NVIDIA GPUs.

πŸ“– Detailed definition

NVLink is the protocol and cabling technology connecting NVIDIA GPUs at very high speed, bypassing PCIe. Throughput varies by generation:

VersionTotal bidirectional bandwidthCompatible GPUs
NVLink 3 (A100)600 GB/sA100 SXM
NVLink 4 (H100)900 GB/sH100 SXM
NVLink 5 (B200)1,800 GB/sB200 (with NVSwitch)

NVLink vs NVSwitch: NVLink is point-to-point between two GPUs; NVSwitch is the chip forming a non-blocking fabric among all GPUs in an HGX node.

πŸ’‘ Why it matters for on-prem AI

NVLink throughput is a prerequisite for efficient Tensor Parallelism. Without NVLink, inter-GPU traffic over PCIe (~64 GB/s) becomes the bottleneck.

⚠️ Common pitfalls

  • NVLink is gone on workstations since Ada Lovelace. RTX workstation (RTX 6000 Ada, RTX PRO 6000 Blackwell) and consumer (RTX 40xx, RTX 50xx) cards no longer have a physical NVLink connector. NVIDIA removed external NVLink bridges from those lines. You cannot link two RTX PRO 6000 via NVLink β€” the connector does not exist. NVLink today is strictly for SXM GPUs (A100, H100, H200, B200) in HGX/DGX systems.
  • Two GPUs in point-to-point NVLink do not reach full NVSwitch fabric bandwidth.
  • Do not confuse β€œsupports CUDA” with β€œsupports NVLink” β€” a card can run CUDA/vLLM with zero NVLink capability.

πŸ”— See also

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.