Arviser

How we score an LLM Quality Audit

Scores are agreed with you before the final readout. Each area uses a 1–5 scale.

Score Meaning
5 Meets the agreed threshold with headroom. Regressions gated.
4 Meets the threshold. Minor gaps documented.
3 Borderline. Go-live only with explicit risk acceptance.
2 Fails on material cases. Remediate before scale.
1 Unsafe or unmeasured. No-go until evidence is rebuilt.

Six areas

  1. Golden set — coverage, labels, freshness vs the live surface.
  2. Groundedness and faithfulness — answers tied to allowed sources.
  3. Refusal and safety — policy breaches, jailbreaks, unsafe tool use.
  4. CI and regression — evals on change, or only on a laptop.
  5. Production sampling — live traffic feeding the suite, or a frozen demo set.
  6. Cost and latency — unit cost and delay under realistic load.

What the money buys

Engineering report with severity and engineer-day sizing. One-page exec summary. Repeatable suite. CI harness when repo access allows. 60-minute review with the person who ran the work.

From €8,000 for two weeks on an agreed surface. LLM Quality Audit · 24h PoC Spike · Open proof · Request scope