# Golden-set and rubric workshop

100-Case RAG Evaluation Golden Set anatomy: 30 normal queries, 20 multi-hop reasoning, 20 hard retrieval edge cases, 15 unanswerable refusal cases, and 15 conflicting source cases with passage-level citations. Golden-set workshops produce rubric weights, annotator guidelines, and a versioned JSONL your team owns, often bundled with the €3,100 Eval CI harness. €2,400 PoC when the task definition is still fuzzy; €3,900 prototype assistant; €5,600 hardening; €8,500 MVP; audit from €8,000. Written scope arrives within 48 hours with numeric pass marks on accuracy, groundedness, p95 latency, and cost per thousand runs.

**Sprints from €2,400 PoC · €3,100 eval CI · €3,900 prototype · €5,600 hardening · €8,500 MVP · audit from €8,000** · Topic: `services-golden-set-workshop` · [Open proof](/#open-proof) · [Sample scorecard](/sample) · [golden-set-schema](https://github.com/nareyko/golden-set-schema)

HTML: [/services/golden-set-workshop](/services/golden-set-workshop) · Markdown: [/services/golden-set-workshop.md](/services/golden-set-workshop.md)


_Rubric and golden-set workshop_

## Why teams trust this stack with Arviser

- Stack-specific gates for Golden-set and rubric workshop, not generic AI slideware.
- Senior engineers (10+ yrs) with daily eval tooling; founder architecture review (30+ yrs IT).
- Proof links include Open proof and sample scorecards, commercial sprints stay fixed-scope.

## Dual path

### For CTO / Staff Eng

CTO path: lock the 100-case anatomy first (30 normal, 20 multi-hop, 20 hard retrieval, 15 unanswerable refusals, 15 conflicting sources with passage-level citations), then wire Promptfoo or DeepEval gates so production traces become a regression suite rather than a slide deck.

### For VP Product / CFO

Workshop delivers rubric, weights, and 50–100 golden rows your team owns, typically €3,100 as harness add-on or standalone week.

Typical route: 32h Eval CI (€3,100) or 24h PoC (€2,400) on the riskiest slice, then 60h Hardening (€5,600) or LLM Quality Audit (€8,000) before executive sign-off.

## Stack-specific failure modes we gate in eval CI

- Rubric weights copied from demo set mismatch production severity.
- Annotators disagree on groundedness for table-heavy PDF answers.
- Golden set never updated after pricing page redesign.

### Sample eval fragment (svc-golden)

```
rubric:
  groundedness: 0.35
  safety: 0.25
```

## Which sprint fits?

| Situation | Recommendation | Sprint |
| --- | --- | --- |
| Need a 100-case RAG golden set with citation traces | Golden-set workshop + 32h Eval CI | €3,100 · 1 week |
| Task definition still fuzzy | 24h Technical PoC on the narrow slice | €2,400 · 3–5 days |
| Pilot traffic soon after the set exists | 60h Hardening or 80h MVP | €5,600–€8,500 |

## FAQ

### Do you build Golden-set and rubric workshop under fixed scope?

Yes. We deliver written scope in 48h and fixed-hour packages; overruns inside agreed spec are on us.

### Can we start with eval CI only?

Yes, 32h Eval CI from €3,100 when a prototype exists but gates do not.

Reviewed by Vadim Nareyko, 30+ yrs IT · Last updated 2026-11-06

## Related

[Services hub](/services) · [Engineering hub](/engineering)

