Thunderbolt
π Short definition
Wired interface providing up to 80 Gb/s bidirectional between machinesβused to link nodes in a desktop AI cluster (Exo).
π Detailed definition
- Thunderbolt 4: up to 40 Gb/s bidirectional.
- Thunderbolt 5: up to 80 Gb/s bidirectional.
Between Macs, Thunderbolt 4 provides a local IP network connection (IP-over-Thunderbolt); since macOS 26.2, Thunderbolt 5 adds RDMA (MLXβs JACCL backend, also used by Exo); activation conditions are covered in AI Networking: RoCE, InfiniBand, and Thunderbolt. That bandwidth is enough for Exo Pipeline Parallelism on models up to 600B+.
In perspective: a Thunderbolt 5 link at 80 Gb/s β 10 GB/sβabout a hundred times less than an NVLink bus (1,800 GB/s). That is its fundamental limit.
π‘ Why it matters for on-prem AI
Desktop solution to link several Mac minis into a quiet, economical cluster without investing in datacenter RoCE/InfiniBand networking.
β οΈ Common pitfalls
- RDMA only on Thunderbolt 5 under macOS 26.2+ (Thunderbolt 4: IP only); even with RDMA, throughput and latency remain far from NVLink or InfiniBand.
- TTFT degrades sharply on long prompts (prefill moves a lot of data).
- Not a substitute for datacenter networking at high concurrency.
π Go deeper
- π₯οΈ Scenario C: Desktop Cluster (Exo + Thunderbolt in practice)
- π AI Networking: RoCE, InfiniBand, and Thunderbolt (full interconnect comparison)
π See also
Open-source P2P orchestrator to merge memory across machines into a local AI cluster.Strategy for distributing an LLM in layer slices across multiple machines.Dedicated hardware link that connects NVIDIA GPUs at very high speed, bypassing the PCIe bus; it is gone from recent workstation and consumer cards.Network technique that reads and writes remote memory without going through the CPU, with low latency; it comes in InfiniBand and RoCE implementations.Amount of data transferred per second between memory and the processor or GPU, in GB/s; the key indicator for estimating how smooth text generation will be.