EdTech Tracker

← Coding lens

golden dataset

technique judged 97% likely a named technique or architecture

First seen
2026-03-02
Independent sources
2
Articles
10
Last 4 weeks vs prior
0 vs 1

See how it has spread across media tiers

Mentions over time

Articles per week (by publish date) using the term, in any spelling.
03.572026-W092026-W112026-W40
Table view
WeekArticlesSources
2026-W0921
2026-W1171
2026-W4011

In their words

Latency (p50, p95), cost per query, refusal rate Build a golden dataset Start with 50 to 200 realistic questions, each paired with the source passages that answer it and, ideally, a reference answer.

Online Evaluation - Uses golden datasets with known answers - Fast iteration — run in CI on every commit - Catches prompt regressions immediately - Can measure RAGAS metrics automatically - Dataset may not reflect real user queries - Does not capture user satisfaction signals - Requires maintaining and updating datasets - Reflects actual user intent and behavior - Captures satisfaction through fee

First, offline evaluation: run a golden dataset through the classifier and measure misclassification rate — if more than 10% of complex queries get routed to Tier 1, the classifier needs tuning.

Eval tests score actual LLM outputs against golden datasets.

A/B Testing Prompts” A/B testing provides the only rigorous way to measure how a prompt change affects real users — offline evals tell you the golden dataset score, A/B tests tell you the business impact.

The evaluation process for chunking strategy selection: Step 1: Build a golden dataset.

For most RAG systems, this means running the A/B test on your full golden dataset at least once.

bi-encoders, when reranking is worth the latency cost - Agent patterns: ReAct, Plan-and-Execute, multi-agent orchestration - Caching strategies: Semantic caching, exact match caching, cache invalidation - Observability: Structured logging, distributed tracing, LLM-specific metrics You should be experimenting with: - Fine-tuning for specific use cases - Quantization and model compression - Self-hos