Skip to content

Memory Tree

πŸ“ Short definition

Long-memory pattern where documents are chunked, summarized, and arranged in a hierarchy so the agent loads only useful nodes into context.

πŸ“– Detailed definition

A Memory Tree replaces or complements a classic vector database. Instead of only retrieving chunks similar to a question, the agent walks a hierarchy: titles, short summaries, sub-summaries, then exact content if needed.

The goal is to reduce injected tokens and thus pressure on the context window and KV Cache.

πŸ’‘ Why it matters for on-prem AI

On a local machine, VRAM and memory bandwidth are limited. A Memory Tree helps keep context short, lowering TTFT and avoiding saturating the server with full documents.

⚠️ Common pitfalls

  • Assuming a Memory Tree replaces every vector-database use case.
  • Trusting stale summaries if the tree is not regenerated.
  • Forgetting to cite the exact source document after tree navigation.

πŸ“š Go deeper

  1. RAG & Agents: Memory Tree approach
  2. OpenHuman (popularized the pattern in its early versions, no longer implements it)

πŸ”— See also

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.