PagedAttention
π Short definition
Algorithm that manages KV Cache memory in non-contiguous blocks (like OS virtual memory), greatly reducing fragmentation.
π Detailed definition
In classic engines, the KV Cache is pre-allocated in contiguous VRAM blocks: unused regions stay wasted. PagedAttention splits the cache into fixed-size pages that can be allocated, freed, and shared dynamically.
Result: memory waste drops from 60β80% to under 4% according to the original paper (Woosuk Kwon et al., SOSP 2023)1. This enables Continuous Batching: requests are processed continuously without draining the server between each one, maximizing GPU throughput.
π‘ Why it matters for on-prem AI
This is the innovation that made vLLMβs continuous batching possible β together with block-based prefix caching (enabled by default), it explains why vLLM outperforms Ollama in multi-user production2. Without PagedAttention, the server wastes VRAM and cannot batch concurrent requests efficiently.
β οΈ Common pitfalls
- PagedAttention now designates a principle more than a component: vLLM removed the historical implementation in v0.25 (July 2026) in favor of its V1 engineβs attention backends, which keep block-based KV Cache management (
--block-sizeoption,vllm:kv_cache_usage_percmetric)3. Workstation-oriented engines rely on a different memory model:llama-server(llama.cpp) does do continuous batching (enabled by default, parallel slots via-np), but without a paged KV Cache or block sharing between requests, and Ollama by default processes only one request at a time per model (OLLAMA_NUM_PARALLEL=1). - Does not remove total capacity limits: if model + caches exceed total VRAM, OOM still occurs.
π Go deeper
- βοΈ Inference Engines (why vLLM > Ollama in production)
- πΎ KV Cache & Context (the cache mechanism PagedAttention optimizes)
π See also
Footnotes
-
W. Kwon et al., vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention (βexisting systems waste 60% β 80% of memoryβ, βa mere waste of under 4%β), 20 June 2023; Efficient Memory Management for Large Language Model Serving with PagedAttention (SOSP 2023). https://vllm.ai/blog/2023-06-20-vllm Β· https://arxiv.org/abs/2309.06180 β©
-
vLLM Project, Automatic Prefix Caching (
enable_prefix_cachingenabled by default, block hashing), accessed 2026-10-10. https://docs.vllm.ai/en/stable/features/automatic_prefix_caching/ β© -
vLLM Project, Release v0.25.0 (βPagedAttention has been removed β the legacy attention implementation is deleted now that V1/MRv2 backends are the standard pathβ), 11 July 2026. https://github.com/vllm-project/vllm/releases/tag/v0.25.0 Β· vLLM Project, Engine Arguments (
--block-size, block-based KV cache), accessed 2026-10-10. https://docs.vllm.ai/en/stable/configuration/engine_args/ β©