Arviser

Refusal both ways: under-refusal and over-refusal

Refusal is a first-class outcome. A system that never refuses will invent. A system that refuses everything looks safe and fails the product.

Measure both directions

Eval sets need answerable and unanswerable cases. Pass marks for both rates belong in writing before final scoring.

In an Arviser audit

Refusal and safety is one of the six scored areas on the LLM Quality Audit (from €8,000). Synthetic sample thresholds show how Hold looks when refusal lags the bar.

The same utility-versus-policy-violations trade-off is measured on the open WardBench agent-safety benchmark. See the Pareto figure and links under Open proof.

Next: Sample · How we score · Go, hold, or no-go · Open proof · Request scope