RAGAS
π Short definition
Framework that evaluates a RAG system by separating retrieval quality and fidelity of the generated answer.
π Detailed definition
RAGAS measures in particular:
- faithfulness: is the answer supported by the retrieved contexts?
- answer relevancy: does the answer actually address the question?
- context precision: are the retrieved passages relevant?
- context recall: does retrieval find the necessary information?
The value is diagnosing where the system fails: poor document retrieval, good source poorly used, off-topic answer, or hallucination.
π‘ Why it matters for on-prem AI
Many local assistants rely on RAG to use internal documents. RAGAS helps measure that layerβs quality without relying only on subjective impression.
β οΈ Common pitfalls
- Believing a good model compensates for poor retrieval.
- Measuring only the final answer without checking retrieved passages.
- Replacing all human review with an automatic score.
π See also
Architecture that combines document retrieval and LLM generation to ground answers in internal sources.Evaluation technique where a language model judges or scores responses.