Skip to content

Thunderbolt

πŸ“ Short definition

Wired interface providing up to 80 Gb/s bidirectional between machinesβ€”used to link nodes in a desktop AI cluster (Exo).

πŸ“– Detailed definition

  • Thunderbolt 4: up to 40 Gb/s bidirectional.
  • Thunderbolt 5: up to 80 Gb/s bidirectional.

Between Macs, Thunderbolt 4 provides a local IP network connection (IP-over-Thunderbolt); since macOS 26.2, Thunderbolt 5 adds RDMA (MLX’s JACCL backend, also used by Exo); activation conditions are covered in AI Networking: RoCE, InfiniBand, and Thunderbolt. That bandwidth is enough for Exo Pipeline Parallelism on models up to 600B+.

In perspective: a Thunderbolt 5 link at 80 Gb/s β‰ˆ 10 GB/sβ€”about a hundred times less than an NVLink bus (1,800 GB/s). That is its fundamental limit.

πŸ’‘ Why it matters for on-prem AI

Desktop solution to link several Mac minis into a quiet, economical cluster without investing in datacenter RoCE/InfiniBand networking.

⚠️ Common pitfalls

  • RDMA only on Thunderbolt 5 under macOS 26.2+ (Thunderbolt 4: IP only); even with RDMA, throughput and latency remain far from NVLink or InfiniBand.
  • TTFT degrades sharply on long prompts (prefill moves a lot of data).
  • Not a substitute for datacenter networking at high concurrency.

πŸ“š Go deeper

  1. πŸ–₯️ Scenario C: Desktop Cluster (Exo + Thunderbolt in practice)
  2. 🌐 AI Networking: RoCE, InfiniBand, and Thunderbolt (full interconnect comparison)

πŸ”— See also

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.