Exo
π Short definition
Open-source software that links several Apple Silicon Macs into a P2P AI cluster able to run very large models; as of Q4 2026, Exo relies solely on MLX (Linux on CPU only, Windows not supported).
π Detailed definition
Exo (by Exo Labs) uses automatic discovery on the local network: each machine runs uv run exo, they detect each other and merge available memory. The model is distributed either as layer slices (Pipeline Parallelism: machine A runs early layers, passes activations to machine B, and so on), or via Tensor Parallelism over Thunderbolt 5 RDMA; Exo chooses automatically based on the bandwidth measured between nodes.
Macs are preferably linked by Thunderbolt 5 cables (RDMA on macOS 26.2+), the only desktop link fast enough to offset inter-machine latency.
π‘ Why it matters for on-prem AI
Lets you exceed a single machineβs limits without datacenter investment. A cluster of 2 to 4 Macs linked via Thunderbolt 5 RDMA (full mesh), or more Mac Minis in pipeline over IP, like the 8 Mac Minis in the 2024 Exo test, can host 200Bβ600B+ models inaccessible on one box.
β οΈ Common pitfalls
- Wi-Fi or 1 Gbps Ethernet destroys performance: the network becomes the bottleneck.
- TTFT explodes on long prompts (prefill moves a lot of data between nodes).
- Poor support for high concurrency: not suited to 50 simultaneous users.
- Slowing project: no release since v1.0.71 (April 2026); for a new Mac cluster, compare with distributed MLX (
mlx.launch), maintained by Apple.
π Go deeper
- π AI clustering: Exo and Ray (full chapter with benchmarks and use cases)
- π₯οΈ Scenario C: desktop cluster (Exo + Thunderbolt blueprint)
- Pipeline Parallelism (splitting strategy used)