Skip to content

RAG

πŸ“ Short definition

Method that retrieves relevant passages from a document base, then generates a response with an LLM enriched by that context1.

πŸ“– Detailed definition

RAG (Retrieval-Augmented Generation) separates two phases:

  1. Retrieval β€” query β†’ embeddings β†’ similarity search in a vector database (or hybrid BM25 + vectors).
  2. Generation β€” concatenation of system prompt + retrieved chunks + user question β†’ model response.

Variants covered in the vault: Agentic RAG (retrieve/grade/rewrite loops), GraphRAG, Memory Tree for long contexts.

πŸ’‘ Why it matters for on-prem AI

  • Exploits internal knowledge without retraining the model β€” suited to GDPR and sovereignty constraints.
  • Reduces hallucinations if retrieval is good; does not eliminate them (see pitfalls).
  • Coupled with multi-tenant, requires strict index isolation (OWASP LLM08).

⚠️ Common pitfalls

  • Naive chunking: pieces too large or cut mid-sentence β†’ useless retrieval.
  • Confusing RAG with security: a poisoned document can trigger indirect prompt injection.
  • Ignoring evaluation: measure retrieval + faithfulness (RAGAS, golden dataset) β€” see evaluating a local model.
  • Oversizing the model to compensate for a poor index: optimize embeddings and re-ranking first.

πŸ“š To go deeper

  1. 🧩 RAG & Agents β€” full 2026 architecture
  2. πŸ§ͺ Evaluating a local model
  3. LangGraph β€” agentic orchestration

πŸ”— See also

Footnotes

  1. Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (NeurIPS 2020). https://arxiv.org/abs/2005.11401 ↩

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.