Golden set: the eval baseline most pilots skip
A golden set is a fixed list of prompts, contexts, and expected outcomes you re-run after every model, prompt, or retrieval change. Without it, demos look fine and production drifts without a signal.
What belongs in the first set
- 30–100 cases covering top intents and known edge cases
- Expected answer shape, refusal, or retrieval source where relevant
- Frozen model and tool versions for each run
- Pass marks agreed before scoring, not after
Absence of a golden set is itself a finding. An LLM Quality Audit can bootstrap a candidate set from traces while scoring the live surface.
Related Arviser work
AI Readiness Check (from €2,900) maps whether you have gates at all. LLM Quality Audit (from €8,000) scores six areas including the golden set and ships a suite you can keep.
Next: How we score · Two-week timeline · Request scope