Skip to content

Exo

πŸ“ Short definition

Open-source software that links several Apple Silicon Macs into a P2P AI cluster able to run very large models; as of Q4 2026, Exo relies solely on MLX (Linux on CPU only, Windows not supported).

πŸ“– Detailed definition

Exo (by Exo Labs) uses automatic discovery on the local network: each machine runs uv run exo, they detect each other and merge available memory. The model is distributed either as layer slices (Pipeline Parallelism: machine A runs early layers, passes activations to machine B, and so on), or via Tensor Parallelism over Thunderbolt 5 RDMA; Exo chooses automatically based on the bandwidth measured between nodes.

Macs are preferably linked by Thunderbolt 5 cables (RDMA on macOS 26.2+), the only desktop link fast enough to offset inter-machine latency.

πŸ’‘ Why it matters for on-prem AI

Lets you exceed a single machine’s limits without datacenter investment. A cluster of 2 to 4 Macs linked via Thunderbolt 5 RDMA (full mesh), or more Mac Minis in pipeline over IP, like the 8 Mac Minis in the 2024 Exo test, can host 200B–600B+ models inaccessible on one box.

⚠️ Common pitfalls

  • Wi-Fi or 1 Gbps Ethernet destroys performance: the network becomes the bottleneck.
  • TTFT explodes on long prompts (prefill moves a lot of data between nodes).
  • Poor support for high concurrency: not suited to 50 simultaneous users.
  • Slowing project: no release since v1.0.71 (April 2026); for a new Mac cluster, compare with distributed MLX (mlx.launch), maintained by Apple.

πŸ“š Go deeper

  1. 🌐 AI clustering: Exo and Ray (full chapter with benchmarks and use cases)
  2. πŸ–₯️ Scenario C: desktop cluster (Exo + Thunderbolt blueprint)
  3. Pipeline Parallelism (splitting strategy used)

πŸ”— See also

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.