Skip to content

RDMA

πŸ“ Short definition

Network technique to read/write remote memory with low CPU overhead.

πŸ“– Detailed definition

In a classic network transfer (TCP/IP), data passes through the CPU (copy user space β†’ kernel β†’ NIC). RDMA bypasses that path: the NIC accesses application memory directly, with ~1–2 Β΅s latency and near-zero CPU load.

Two main implementations:

  • InfiniBand: dedicated fabric, natively lossless, HPC/high-performance AI standard.
  • RoCE: RDMA over Ethernet, lower cost but requires lossless configuration (PFC/ECN).

With GPUDirect RDMA, DMA goes further: the NIC accesses GPU VRAM directly, without passing through CPU RAM.

πŸ’‘ Why it matters for on-prem AI

As soon as AI workloads are distributed across multiple nodes, RDMA is what separates an efficient cluster from one limited by network latency.

⚠️ Common pitfalls

  • Comparing RDMA and TCP/IP without considering CPU load and microsecond latency.
  • Overestimating RDMA impact on small workloads that are mostly compute-bound.

πŸ”— See also

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.