Arviser

Arviser

Golden-set and rubric workshop

100-Case RAG Evaluation Golden Set anatomy: 30 normal queries, 20 multi-hop reasoning, 20 hard retrieval edge cases, 15 unanswerable refusal cases, and 15 conflicting source cases with passage-level citations. Golden-set workshops produce rubric weights, annotator guidelines, and a versioned JSONL your team owns, often bundled with the €3,100 Eval CI harness. €2,400 PoC when the task definition is still fuzzy; €3,900 prototype assistant; €5,600 hardening; €8,500 MVP; audit from €8,000. Written scope arrives within 48 hours with numeric pass marks on accuracy, groundedness, p95 latency, and cost per thousand runs.

Sprints from €2,400 PoC · €3,100 eval CI · €3,900 prototype · €5,600 hardening · €8,500 MVP · audit from €8,000 · Mention topic: services-golden-set-workshop in the form · Open proof · Sample scorecard · golden-set-schema

Why teams trust this stack with Arviser

  • Stack-specific gates for Golden-set and rubric workshop, not generic AI slideware.
  • Senior engineers (10+ yrs) with daily eval tooling; founder architecture review (30+ yrs IT).
  • Proof links include Open proof and sample scorecards, commercial sprints stay fixed-scope.

Dual path: For CTO / Staff Eng vs For VP Product / CFO

For CTO / Staff Eng

Technical delivery path

CTO path: lock the 100-case anatomy first (30 normal, 20 multi-hop, 20 hard retrieval, 15 unanswerable refusals, 15 conflicting sources with passage-level citations), then wire Promptfoo or DeepEval gates so production traces become a regression suite rather than a slide deck.

For VP Product / CFO

Budget and risk path

Workshop delivers rubric, weights, and 50–100 golden rows your team owns, typically €3,100 as harness add-on or standalone week.

Typical route: 32h Eval CI (€3,100) or 24h PoC (€2,400) on the riskiest slice, then 60h Hardening (€5,600) or LLM Quality Audit (€8,000) before executive sign-off.

Stack-specific failure modes we gate in eval CI

  • Rubric weights copied from demo set mismatch production severity.
  • Annotators disagree on groundedness for table-heavy PDF answers.
  • Golden set never updated after pricing page redesign.

Sample eval fragment (svc-golden)

rubric:
  groundedness: 0.35
  safety: 0.25

Which sprint fits?

SituationRecommendationSprint
Need a 100-case RAG golden set with citation tracesGolden-set workshop + 32h Eval CI€3,100 · 1 week
Task definition still fuzzy24h Technical PoC on the narrow slice€2,400 · 3–5 days
Pilot traffic soon after the set exists60h Hardening or 80h MVP€5,600–€8,500

FAQ

Do you build Golden-set and rubric workshop under fixed scope?

Yes. We deliver written scope in 48h and fixed-hour packages; overruns inside agreed spec are on us.

Can we start with eval CI only?

Yes, 32h Eval CI from €3,100 when a prototype exists but gates do not.

Reviewed by Vadim Nareyko, 30+ yrs IT · Last updated 2026-11-06

Request written scope or ask a question

Free written scoping and technical Q&A reply within 48 hours on workdays (Warsaw), or select Direct Founder Consultation (€250/h, 100% credited in 30 days) below. Controller: Arviser Sp. z o.o. Privacy. No patient data.

1-click prompt templates:
Optional details (role, industry, timezone)

Discuss your project