# Production sampling for LLM quality

Offline golden sets catch regressions before release. Production sampling catches drift after release: new intents, stale retrieval, model swaps the vendor pushed without telling you.

## A workable pattern

- Serve the user first. Score async. 
- Sample a slice (often 5–10%), stratified by topic or risk when you can. 
- Oversample high-risk surfaces (payments, policy, health-adjacent copy). 
- Turn recurring live failures into frozen regression cases. 

Track groundedness, refusal, cost, and latency by model, prompt, and retrieval version. An average that looks fine can hide one broken route.

## Where Arviser fits

LLM Quality Audit includes production sampling as one of six scored areas and leaves a suite you can keep running. Readiness maps whether any sampling path exists before you spend on scale.

**Next:** [CI gates](/guides/ci-gates-llm) · [Shared metrics](/guides/shared-metrics) · [How we score](/guides/how-we-score) · [Request scope](/#scope)
