Skip to content

APU

πŸ“ Short definition

Chip that integrates CPU, GPU (and often NPU) on one die or package, sharing one memory pool without PCIe copies.

πŸ“– Detailed definition

An APU differs from a classic PC (CPU + discrete GPU) because processor and graphics memory are not physically separate. CPU and GPU address the same LPDDR5X pool.

Dominant APU platforms for LLM inference in 2026:

PlatformExampleAllocatable VRAMBandwidth
Apple SiliconMac Studio M5 Max / M5 Ultra (since August 2026)~120 GB (of 128 GB) / ~240 GB (of 256 GB), via sysctl614 GB/s / 1.2 TB/s
AMD Gorgon HaloRyzen AI Max+ PRO 495160 GB (BIOS, of 192 GB)~273 GB/s

These figures allow running a quantized 70B model (~40 GB) with a large context window on a quiet desktop station β€” impossible on consumer high-end GPUs (32 GB VRAM max in consumer tiers, on the RTX 5090).

πŸ’‘ Why it matters for on-prem AI

The APU is the reference platform for Scenario B (SME appliance): self-contained, quiet, under ~60 W at light load, hosting a local assistant for teams of 10–50 people (2 to 20 concurrent users).

⚠️ Common pitfalls

  • LPDDR5X is soldered: no post-purchase upgrade. Size the machine upfront.
  • Shared CPU + GPU bandwidth: the more the OS and apps load RAM, the less the GPU gets during inference.
  • Integrated NPUs (Apple Neural Engine, AMD XDNA) are not used by current LLM engines (Ollama, llama.cpp, vLLM) β€” inference stays on the iGPU.

πŸ“š Go deeper

πŸ”— See also

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.