APU
π Short definition
Chip that integrates CPU, GPU (and often NPU) on one die or package, sharing one memory pool without PCIe copies.
π Detailed definition
An APU differs from a classic PC (CPU + discrete GPU) because processor and graphics memory are not physically separate. CPU and GPU address the same LPDDR5X pool.
Dominant APU platforms for LLM inference in 2026:
| Platform | Example | Allocatable VRAM | Bandwidth |
|---|---|---|---|
| Apple Silicon | Mac Studio M5 Max / M5 Ultra (since August 2026) | ~120 GB (of 128 GB) / ~240 GB (of 256 GB), via sysctl | 614 GB/s / 1.2 TB/s |
| AMD Gorgon Halo | Ryzen AI Max+ PRO 495 | 160 GB (BIOS, of 192 GB) | ~273 GB/s |
These figures allow running a quantized 70B model (~40 GB) with a large context window on a quiet desktop station β impossible on consumer high-end GPUs (32 GB VRAM max in consumer tiers, on the RTX 5090).
π‘ Why it matters for on-prem AI
The APU is the reference platform for Scenario B (SME appliance): self-contained, quiet, under ~60 W at light load, hosting a local assistant for teams of 10β50 people (2 to 20 concurrent users).
β οΈ Common pitfalls
- LPDDR5X is soldered: no post-purchase upgrade. Size the machine upfront.
- Shared CPU + GPU bandwidth: the more the OS and apps load RAM, the less the GPU gets during inference.
- Integrated NPUs (Apple Neural Engine, AMD XDNA) are not used by current LLM engines (Ollama, llama.cpp, vLLM) β inference stays on the iGPU.
π Go deeper
π See also
Architecture where CPU, GPU and sometimes NPU share a single memory pool, avoiding some copies over PCIe; found on Apple Silicon and some recent AMD APUs.Neural Processing Unit β AI accelerator in modern SoCs, useful for some tasks but limited for large LLMs.Very high-bandwidth memory attached to the GPU, storing weights, KV Cache and buffers; its capacity and bandwidth determine which models run without offloading.