EdTech Tracker

← Coding lens

answer relevancy

technique judged 70% likely a named technique or architecture

First seen
2024-11-24
Independent sources
3
Articles
6
Last 4 weeks vs prior
0 vs 0

See how it has spread across media tiers

Mentions over time

Articles per week (by publish date) using the term, in any spelling.
0242024-W472026-W112026-W38
Table view
WeekArticlesSources
2024-W4711
2026-W1141
2026-W3811

In their words

Every run fires the full set against the live backend and scores four metrics with an LLM judge correctness (does the answer match the expert answer?), answer relevancy , faithfulness (is every claim supported by the retrieved context?), and tool correctness (did the agent pick the right retrieval strategy?).

LLM evaluation uses three complementary approaches: RAGAS framework metrics (context precision, context recall, faithfulness, answer relevancy) for RAG systems, LLM-as-judge for scalable automated evaluation where a stronger model scores outputs against criteria, and A/B testing for comparing prompt or model changes in production.

- End-to-end answer quality — RAGAS metrics (faithfulness, answer relevancy, context precision) on multi-hop test questions, compared against the same questions answered by standard RAG This comparison — GraphRAG vs standard RAG on the same multi-hop test set — is the most convincing evidence for or against the investment.

Answer Relevancy (the off-topic detector) Section titled “Answer Relevancy (the off-topic detector)” What it measures: Whether the generated answer actually addresses the user’s question — not a related question, not a tangential topic, the actual question asked.

| | Answer Relevancy | Is the answer relevant to the question asked?

Figure 1: Output of the Ragas framework, showcasing metrics like faithfulness, answer relevancy, context recall, precision, relevancy, entity recall, and answer similarity.