human evaluation
technique judged 90% likely a named technique or architecture
See how it has spread across media tiers
Mentions over time
Table view
| Week | Articles | Sources |
|---|---|---|
| 2024-W26 | 1 | 1 |
| 2025-W40 | 1 | 1 |
| 2026-W09 | 1 | 1 |
| 2026-W11 | 6 | 1 |
| 2026-W41 | 1 | 1 |
In their words
How to Evaluate LLMs: A Complete Guide to LLM Evaluation From automated metrics to human evaluation — build a robust evaluation pipeline for your LLM applications.
This approach scales better than human evaluation while correlating well with human judgments when the criteria are well-defined.
Monitoring Checklist Section titled “Monitoring Checklist” - Routing accuracy: track how often the supervisor picks the correct agent (use human evaluation samples) - Sub-agent success rate: percentage of sub-agent calls that produce usable outputs - End-to-end latency: p50 and p95 across the full graph execution - Token consumption: total tokens per request — compare against single-agent baseline
(2020), foundational work on few-shot prompting Evaluation: - LMSYS Chatbot Arena — Human evaluation benchmarks and model comparisons - promptfoo — Open-source CLI for prompt testing and comparison Related Section titled “Related” - GenAI Engineer Roadmap — Complete career progression guide for GenAI engineers - LLM Evaluation Guide — Metrics and frameworks for measuring LLM system quality - RAG A
Beyond RAGAS: Custom Metrics, Human Evaluation, and A/B Testing Section titled “Beyond RAGAS: Custom Metrics, Human Evaluation, and A/B Testing” RAGAS covers the core failure modes, but production systems often need additional evaluation layers.
This is not a perfect substitute for human evaluation, but it enables continuous monitoring and regression detection.
Know which constraint is least flexible for your use case: - Customer-facing chat: Latency is king (users abandon after 3 seconds) - Batch document processing: Cost matters most (processing millions of documents) - Medical/legal advice: Quality dominates (errors have serious consequences) Common Failure Modes Section titled “Common Failure Modes” Retrieval Failures Section titled “Retrieval Failur
- Human evaluation — sample 100 responses per week, have domain experts label hallucinated claims.