MLA (Multi-head Latent Attention)
π Short definition
A way of computing attention that keeps a compressed summary of each token in memory (a βlatent vectorβ) instead of its full keys (K) and values (V). The KV cache becomes much smaller.
π Detailed definition
In classic attention, each layer of the model stores, for each token, one K vector and one V vector per attention head. MLA compresses this information into a single small vector per token and per layer. At compute time, the model βunfoldsβ this vector to recover the K and V it needs.
An analogy: instead of keeping full notes on every speaker in a meeting, you keep a summary of a few lines from which you can recover the essentials.
MLA was introduced by DeepSeek-V2 (May 2024)1; for the quantified gains and the models that use it (DeepSeek-V3, Kimi K2), see πΎ KV Cache & Context.
π‘ Why it matters for on-premise AI
On an on-premise server, the VRAM available for the cache limits conversation length and the number of concurrent users. A model with MLA uses far less cache per token than a classic model of comparable size. The usual GQA formula does not apply: it greatly overestimates the memory needed.
β οΈ Common pitfalls
- Sizing an MLA model with the GQA formula (β¦): the result is heavily overestimated. Start from the model card or from the cache size reported by the engine.
- Confusing MLA with sparse attention: MLA reduces the space taken by each token; sparse attention reduces the number of tokens reread.
π To go deeper
- πΎ KV Cache & Context (section βWhen the model itself shrinks its memoryβ: MLA, sparse attention, linear attention)
π See also
Footnotes
-
DeepSeek-AI, DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model (arXiv:2405.04434; KV cache compressed into a latent vector, β93.3% vs DeepSeek 67B, equivalent to GQA with 2.25 groups), May 2024. https://arxiv.org/abs/2405.04434 β©