answer relevancy
technique judged 70% likely a named technique or architecture
See how it has spread across media tiers
Mentions over time
Table view
| Week | Articles | Sources |
|---|---|---|
| 2024-W47 | 1 | 1 |
| 2026-W11 | 4 | 1 |
| 2026-W38 | 1 | 1 |
In their words
Every run fires the full set against the live backend and scores four metrics with an LLM judge correctness (does the answer match the expert answer?), answer relevancy , faithfulness (is every claim supported by the retrieved context?), and tool correctness (did the agent pick the right retrieval strategy?).
LLM evaluation uses three complementary approaches: RAGAS framework metrics (context precision, context recall, faithfulness, answer relevancy) for RAG systems, LLM-as-judge for scalable automated evaluation where a stronger model scores outputs against criteria, and A/B testing for comparing prompt or model changes in production.
- End-to-end answer quality — RAGAS metrics (faithfulness, answer relevancy, context precision) on multi-hop test questions, compared against the same questions answered by standard RAG This comparison — GraphRAG vs standard RAG on the same multi-hop test set — is the most convincing evidence for or against the investment.
Answer Relevancy (the off-topic detector) Section titled “Answer Relevancy (the off-topic detector)” What it measures: Whether the generated answer actually addresses the user’s question — not a related question, not a tangential topic, the actual question asked.
| | Answer Relevancy | Is the answer relevant to the question asked?
Figure 1: Output of the Ragas framework, showcasing metrics like faithfulness, answer relevancy, context recall, precision, relevancy, entity recall, and answer similarity.