NVLink
π Short definition
Dedicated hardware link accelerating exchanges between compatible NVIDIA GPUs.
π Detailed definition
NVLink is the protocol and cabling technology connecting NVIDIA GPUs at very high speed, bypassing PCIe. Throughput varies by generation:
| Version | Total bidirectional bandwidth | Compatible GPUs |
|---|---|---|
| NVLink 3 (A100) | 600 GB/s | A100 SXM |
| NVLink 4 (H100) | 900 GB/s | H100 SXM |
| NVLink 5 (B200) | 1,800 GB/s | B200 (with NVSwitch) |
NVLink vs NVSwitch: NVLink is point-to-point between two GPUs; NVSwitch is the chip forming a non-blocking fabric among all GPUs in an HGX node.
π‘ Why it matters for on-prem AI
NVLink throughput is a prerequisite for efficient Tensor Parallelism. Without NVLink, inter-GPU traffic over PCIe (~64 GB/s) becomes the bottleneck.
β οΈ Common pitfalls
- NVLink is gone on workstations since Ada Lovelace. RTX workstation (RTX 6000 Ada, RTX PRO 6000 Blackwell) and consumer (RTX 40xx, RTX 50xx) cards no longer have a physical NVLink connector. NVIDIA removed external NVLink bridges from those lines. You cannot link two RTX PRO 6000 via NVLink β the connector does not exist. NVLink today is strictly for SXM GPUs (A100, H100, H200, B200) in HGX/DGX systems.
- Two GPUs in point-to-point NVLink do not reach full NVSwitch fabric bandwidth.
- Do not confuse βsupports CUDAβ with βsupports NVLinkβ β a card can run CUDA/vLLM with zero NVLink capability.
π See also
NVIDIA switch that connects multiple GPUs into a fully non-blocking NVLink fabric inside a node.Strategy for distributing an LLM by splitting weight matrices across multiple GPUs on the same node.Architecture using several GPUs in one machine or a cluster to increase memory and throughput; the gains depend on the interconnect (PCIe, NVLink).Standard bus connecting CPU, GPU, SSD and other peripherals; it limits CPU-to-GPU offloading and Tensor Parallelism on multi-GPU setups without NVLink.Stacked memory with very high bandwidth, used on professional AI accelerators.