Skip to content

InfiniBand

πŸ“ Short definition

Dedicated low-latency, high-bandwidth network (up to 400–800 Gb/s per link) built for HPC and GPU AI clusters.

πŸ“– Detailed definition

InfiniBand is both a protocol and physical fabric (cables, switches, HCA cards). It natively supports RDMA β€” GPUs can send data directly without the CPU.

Generations: HDR (200 Gb/s), NDR (400 Gb/s), XDR (800 Gb/s per port).

InfiniBand vs RoCE:

  • InfiniBand: dedicated fabric, natively lossless, lowest latency (~1 Β΅s), high cost.
  • RoCE: RDMA on standard Ethernet, cheaper but needs strict network config (PFC/ECN) to avoid packet loss.

πŸ’‘ Why it matters for on-prem AI

De facto standard for production GPU clusters (AI datacenters, HPC). Required for full-throughput Tensor Parallelism and Pipeline Parallelism across nodes.

⚠️ Common pitfalls

  • Needs dedicated hardware and HPC networking skills.
  • Mostly proprietary stack (NVIDIA/Mellanox): hard migration to Ethernet.
  • Total cost (switches, cables, HCAs) can exceed GPU server cost.

πŸ”— See also

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.