LLM Evaluation

What it is

Evaluating an LLM-based system means measuring output quality when there's often no single "correct" answer to check against — unlike a classifier with a clear accuracy metric, judging whether a generated response is good, accurate, and safe requires its own deliberate methodology.

Common approaches

• Reference-based metrics: comparing generated text against a known-good reference using metrics like BLEU or ROUGE (originally from machine translation/summarization) — cheap to compute, but correlate only loosely with actual human judgments of quality for open-ended generation.
• LLM-as-judge: using a (typically stronger, or differently-trained) LLM to score or compare responses against a rubric — scales far better than human review, but inherits its own biases (e.g. favoring longer or more confident-sounding answers) and needs validation against human judgments before being trusted.
• Human evaluation: the most reliable signal for subjective quality, but slow and expensive — usually reserved for validating that automated metrics/LLM-judges actually track what matters, and for high-stakes launches.
• Task-specific / grounded metrics: for RAG systems specifically, metrics like faithfulness (does the answer only claim what the retrieved context supports) and retrieval precision/recall (were the right chunks actually retrieved) isolate whether failures come from retrieval or generation.

Why it matters

• Evaluation is what turns "this demo looks impressive" into a defensible claim that a system actually works — without it, prompt or model changes are shipped on vibes, with no way to detect regressions.
• A held-out, representative test set (ideally with real user queries, not just hand-picked easy cases) matters more than which specific metric is used — a great metric on an unrepresentative test set still gives a false sense of confidence.
• Using the same model to both generate answers and judge them (rather than a different, typically stronger model) risks the judge systematically favoring its own generation style — a common evaluation-design mistake worth watching for.