llm judge
technique judged 98% likely a named technique or architecture
See how it has spread across media tiers
Mentions over time
Table view
| Week | Articles | Sources |
|---|---|---|
| 2024-W12 | 1 | 1 |
| 2024-W26 | 1 | 1 |
| 2024-W27 | 1 | 1 |
| 2024-W44 | 1 | 1 |
| 2025-W31 | 1 | 1 |
| 2026-W09 | 2 | 1 |
| 2026-W11 | 11 | 1 |
| 2026-W14 | 1 | 1 |
| 2026-W17 | 1 | 1 |
| 2026-W30 | 1 | 1 |
| 2026-W31 | 1 | 1 |
| 2026-W32 | 1 | 1 |
| 2026-W33 | 1 | 1 |
| 2026-W37 | 1 | 1 |
| 2026-W38 | 2 | 2 |
| 2026-W39 | 1 | 1 |
| 2026-W40 | 12 | 3 |
| 2026-W41 | 4 | 1 |
In their words
Your response:""" ## Part 4: Prompt Optimization ### Automated Prompt Optimization python prompting/optimization.py from dataclasses import dataclass from typing import List, Callable import asyncio @dataclass class PromptCandidate: prompt: str score: float = 0.0 metadata: dict = None class PromptOptimizer: def init ( self, llm, evaluator: Callable[[str, str], float], # (prompt, expected) -> score
items ()} def evaluate_generation ( self , test_cases : List [ RAGTestCase ]) -> dict : """ Measure answer quality using LLM judge.
With evaluation, you: Catch regressions before users do Compare models objectively (GPT-4o vs Claude vs Llama) Optimize prompts with data, not vibes Meet compliance/quality requirements Evaluation Levels Level What It Tests Speed Cost Reliability Unit Tests Deterministic functions Fast $0 High Automated Metrics BLEU, ROUGE, BERTScore Fast $0 Medium LLM-as-Judge Quality, correctness, style Medium L
A cross-encoder returns a calibrated score per query-chunk pair, which is what the LLM judge was being prompted to imitate.
No LLM judge, no vibes.
The setup I ended up with is simple to state: the LLM judges, and my code does everything that can be computed.
Score generation with care LLM-as-judge is the practical way to grade faithfulness and relevance at scale.
It needs to orchestrate a multi-turn conversation between your agent and another model (or the same one with different prompts), and then pass the transcript to an LLM-as-judge to evaluate.