How we score an LLM Quality Audit
Scores are agreed with you before the final readout. Each area uses a 1–5 scale.
| Score | Meaning |
|---|---|
| 5 | Meets the agreed threshold with headroom. Regressions gated. |
| 4 | Meets the threshold. Minor gaps documented. |
| 3 | Borderline. Go-live only with explicit risk acceptance. |
| 2 | Fails on material cases. Remediate before scale. |
| 1 | Unsafe or unmeasured. No-go until evidence is rebuilt. |
Six areas
- Golden set — coverage, labels, freshness vs the live surface.
- Groundedness and faithfulness — answers tied to allowed sources.
- Refusal and safety — policy breaches, jailbreaks, unsafe tool use.
- CI and regression — evals on change, or only on a laptop.
- Production sampling — live traffic feeding the suite, or a frozen demo set.
- Cost and latency — unit cost and delay under realistic load.
What the money buys
Engineering report with severity and engineer-day sizing. One-page exec summary. Repeatable suite. CI harness when repo access allows. 60-minute review with the person who ran the work.
From €8,000 for two weeks on an agreed surface. LLM Quality Audit · 24h PoC Spike · Open proof · Request scope