PCIe
π Short definition
Standard bus linking CPU, GPU, SSD, and other peripherals.
π Detailed definition
In local AI, PCIe carries data between system RAM and discrete GPU (offloading) and between GPUs in multi-GPU setups without NVLink.
Maximum throughput by generation:
| Generation | Unidirectional x16 | Bidirectional x16 |
|---|---|---|
| PCIe 4.0 | ~32 GB/s | ~64 GB/s |
| PCIe 5.0 | ~64 GB/s | ~128 GB/s |
| PCIe 6.0 | ~128 GB/s | ~256 GB/s |
For comparison, NVLink 4 (H100) offers 900 GB/s β about 14Γ more than a PCIe 5.0 x16 link.
π‘ Why it matters for on-prem AI
PCIe weighs on offloading when weights are copied to the GPU (the computation of layers left in RAM is itself bounded by DDR5), and it is the limiting factor for Tensor Parallelism on desktop setups without NVLink.
β οΈ Common pitfalls
- Theoretical PCIe bandwidth is divided by the number of GPUs sharing the same CPU controller.
- PCIe 5.0 x8 (common on mid-range motherboards) offers only as much as PCIe 4.0 x16.
π See also
Dedicated hardware link that connects NVIDIA GPUs at very high speed, bypassing the PCIe bus; it is gone from recent workstation and consumer cards.Technique that places part of the model in RAM or on SSD when VRAM is insufficient, at the cost of reduced throughput.Architecture using several GPUs in one machine or a cluster to increase memory and throughput; the gains depend on the interconnect (PCIe, NVLink).Architecture where CPU, GPU and sometimes NPU share a single memory pool, avoiding some copies over PCIe; found on Apple Silicon and some recent AMD APUs.