🏢 Scenario B: SME Appliance (Unified Memory)
Your client (a law firm, medical practice, SME) needs a local assistant capable of processing confidential documents. The chosen model is a heavy LLM: either a quantized dense 70B (~40 GB of weights, the capacity benchmark of this blueprint), or — faster on this hardware — a light MoE such as gpt-oss-120b (~70 GB, 34–38 t/s measured on a 128 GB APU) or a 27–30B dense model such as Qwen3.8-27B (~18 GB in Q4)1.
As seen in Scenario A, a standard PC collapses due to CPU Offloading. Buying a multi-GPU server is very expensive, sounds like a jet taking off, and consumes a lot of electricity. The most elegant solution in 2026 is the Unified Memory Appliance.
🏗️ Hardware architecture
The goal is to have a single chip (SoC) where the CPU and GPU draw from the same large memory reserve. Three choices are available:
- Apple option (The silent standard): A Mac Studio M5 Max (up to 128 GB, 614 GB/s, from €2,999 incl. VAT) or M5 Ultra (up to 512 GB, 1.2 TB/s, from €6,599 incl. VAT)23.
- x86 PC option (Docker sovereignty): A workstation based on the AMD Ryzen AI Max PRO 400 APU (“Gorgon Halo”) with 192 GB RAM — on pre-order at Framework for $6,799 with delivery in November 2026; the previous 128 GB generation (Ryzen AI Max+ 395) is listed at €3,889 incl. VAT but out of stock as of 2026-10-094.
- NVIDIA option (CUDA without friction): A DGX Spark 128 GB (≈ $6,950 as of 2026-10-02) or its 64 GB variant — same 273 GB/s bus as Gorgon Halo, native CUDA and FP4; pricing and availability in APU & Mémoire Unifiée.
Estimated budget (Q4 2026): between €3,900 and more than €8,000 incl. VAT depending on chip and soldered memory — and rising: DRAM rises every quarter (see 💰 Comparaison TCO) and Framework announces memory price increases across all capacities for six months4. The prices quoted here are those of October 2026: re-check at purchase time. Physical advantages: Very low power consumption (often under 150W at full load), compact form factor, no excessive fan noise.
⚙️ Software stack
Here, the software stack differs depending on hardware chosen:
- On Mac Studio: MLX (
mlx_lm.server, or Ollama ≥ 0.40, which uses it by default on Apple Silicon) or llama.cpp via Metal (llama-serverdefault port changed to 9931 in October 2026)5. Both exploit the maximum bandwidth of unified memory. - On AMD Gorgon Halo (Linux): llama.cpp (Vulkan or ROCm) or SGLang (ROCm image for
gfx1151since 0.5.20), which brings server optimizations like Continuous Batching; vLLM on ROCm remains to be validated on this chip6.
Expected performance
Since the 40 GB model fits entirely in unified memory (which here acts as a huge VRAM), generation speeds are excellent and stable:
- Mac Studio (M5 Max, ~614 GB/s): theoretical bound of about 15 tokens/s during Decoding on a 70B Q4 (~40 GB), calculated with the formula from the memory bandwidth chapter based on the bandwidth announced by Apple in August 20262 — i.e. 10 to 15 t/s expected in practice, to be confirmed by a published benchmark.
- AMD Ryzen AI Max PRO 400 (~273 GB/s): on the order of 5 tokens/s measured (4.7–4.9 t/s on Llama 3.1 70B Q4_K_M, Strix Halo 128 GB, same bus)71 — consistent with the ~6.8 t/s theoretical bound given by the formula.
The concurrent KV Cache trap
📋 The architect’s verdict
✅ When to use this blueprint?
- This is the core target of on-premise AI for SMEs.
- Perfect for an “under the desk” deployment or in a small non-air-conditioned network rack.
- Excellent for running a sovereign local assistant or agent serving a dozen moderate concurrent requests.
❌ When to avoid this blueprint?
- If your client has unpredictable growth needs. Unified memory is soldered to the motherboard. It is impossible to add RAM to a Mac Studio or Gorgon Halo APU after purchase. If the company’s business model moves from 70B to 200B the following year, you will have to discard the machine and buy a new one.
To overcome this fixed capacity constraint while staying on affordable desktop hardware, the next blueprint proposes a scalable approach: The Desktop Cluster — connecting several machines via Thunderbolt.
🛡️ Backup and recovery (DRP)
What to back up on Blueprint B
| Data | Typical location | Frequency |
|---|---|---|
| Vector database (Qdrant / Chroma) | /qdrant/storage/ or Docker volume | Daily — API snapshot |
| Conversation histories (SQLite) | ~/.open-webui/data/ or Docker volume | Daily or hourly |
| Ollama / vLLM configuration | ~/.ollama/ or config.yaml | On every change (Git) |
| Fine-tuned LoRA adapters | Dedicated directory | After each training session |
| Base models (GGUF) | ~/.ollama/models/ | Low priority — re-downloadable |
Minimal recovery procedure
- Start a temporary instance (another Mac, sovereign cloud VM) with Ollama
- Restore the vector database from the latest snapshot
- Restore the SQLite history
- Point clients (Open WebUI, LiteLLM) to the new IP — on up-to-date versions: LiteLLM ≥ 1.106.9 / 1.108.6 / 1.109.1 depending on its line (see the fiche LiteLLM), and Open WebUI tracked release by release (see its fiche); never pin a version
Indicative RTO for Blueprint B: < 45 minutes with an up-to-date daily backup.
📚 Sources and references
Footnotes
-
Qwen, Qwen3.8-27B (dense, Apache 2.0, 262k context; Q4 via Ollama ≈ 18 GB), August 2026. https://huggingface.co/Qwen/Qwen3.8-27B · Ollama, qwen3.8. https://ollama.com/library/qwen3.8 · ignasivt, Strix Halo Guide (gpt-oss-120b ≈ 70 GB, 34–38 tok/s; Llama 3.1 70B Q4_K_M 4.7–4.9 tok/s on Ryzen AI Max+ 395 128 GB), re-read on 2026-10-10. https://github.com/ignasivt/strix-halo-guide ↩ ↩2
-
Apple Newsroom, Apple introduces new Mac Studio with M5 Max and M5 Ultra (M5 Max memory bandwidth 614 GB/s and M5 Ultra 1.2 TB/s, capacities up to 128 GB and 512 GB, Thunderbolt 5 clustering), 2026-08-25. https://www.apple.com/newsroom/2026/08/apple-introduces-new-mac-studio-with-m5-max-and-m5-ultra/ ↩ ↩2 ↩3
-
Apple, Mac Studio — Apple Store France (M5 Max 36 GB from €2,999 incl. VAT, M5 Max 64 GB €3,659 incl. VAT, M5 Ultra 96 GB from €6,599 incl. VAT, 512 GB option “late October”), captured 2026-10-09. https://www.apple.com/fr/shop/buy-mac/mac-studio ↩ ↩2
-
Framework, The 192GB Framework Desktop is open for pre-order (Ryzen AI Max+ PRO 495, 192 GB at $6,799, pre-orders from 2026-09-30, deliveries November 2026, warning about memory price increases), 30 September 2026. https://frame.work/blog/192gb-framework-desktop-open-for-pre-order · Framework, Framework Desktop — AMD Ryzen AI Max+ 395 (128 GB €3,889 incl. VAT, out of stock), captured 2026-10-09. https://frame.work/fr/fr/products/desktop-diy-amd-aimax300 ↩ ↩2
-
Ollama, Release v0.40.0 (MLX used by default on Apple Silicon), October 2026. https://github.com/ollama/ollama/releases/tag/v0.40.0 · ggml-org, llama.cpp — Release b11521 (
llama-serverdefault port changed from 8080 to 9931), 9 October 2026. https://github.com/ggml-org/llama.cpp/releases/tag/b11521 ↩ -
SGLang Project, Release v0.5.20 (ROCm image for
gfx1151Strix / Gorgon Halo), 18 September 2026. https://github.com/sgl-project/sglang/releases/tag/v0.5.20 · ignasivt, Strix Halo Guide (llama.cpp Vulkan / ROCm paths verified on gfx1151), re-read on 2026-10-09. https://github.com/ignasivt/strix-halo-guide ↩ -
ServeTheHome & ignasivt (GitHub), Strix Halo / Gorgon Halo 192GB Unified Memory Benchmarks (Expected decoding throughput on dense 70B model), May 2026. https://www.servethehome.com/amd-reveals-ryzen-ai-max-pro-400-series-192gb-ram-for-ai-systems/ · https://github.com/ignasivt/strix-halo-guide ↩
Incoming links
- ⚙️ Inference Engines: vLLM, Ollama, and TensorRT-LLM
- 🏗️ Possible Architectures
- 🏭 GPU rack servers
- 🏭 Scenario D: Datacenter (RoCE & Multi-GPU)
- 💰 TCO Comparison: On-Premise vs Cloud API
- 🖥️ Index — Hardware
- 🖥️ Scenario C: Desktop Cluster (Exo & Thunderbolt)
- 🖼️ Multimodality: Hardware Impact (VRAM & KV Cache)
- 🗺️ Choosing your local model
- 🤖 On-Premise Agents & Assistants
- 🚀 Zero to Hero Index
- AnythingLLM
- APU
- Open WebUI
- Zero to Hero: On-Premise AI