Skip to content

PCIe

πŸ“ Short definition

Standard bus linking CPU, GPU, SSD, and other peripherals.

πŸ“– Detailed definition

In local AI, PCIe carries data between system RAM and discrete GPU (offloading) and between GPUs in multi-GPU setups without NVLink.

Maximum throughput by generation:

GenerationUnidirectional x16Bidirectional x16
PCIe 4.0~32 GB/s~64 GB/s
PCIe 5.0~64 GB/s~128 GB/s
PCIe 6.0~128 GB/s~256 GB/s

For comparison, NVLink 4 (H100) offers 900 GB/s β€” about 14Γ— more than a PCIe 5.0 x16 link.

πŸ’‘ Why it matters for on-prem AI

PCIe weighs on offloading when weights are copied to the GPU (the computation of layers left in RAM is itself bounded by DDR5), and it is the limiting factor for Tensor Parallelism on desktop setups without NVLink.

⚠️ Common pitfalls

  • Theoretical PCIe bandwidth is divided by the number of GPUs sharing the same CPU controller.
  • PCIe 5.0 x8 (common on mid-range motherboards) offers only as much as PCIe 4.0 x16.

πŸ”— See also

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.