# RAG pipelines and tool agents

100-Case RAG Evaluation Golden Set anatomy: 30 normal queries, 20 multi-hop reasoning, 20 hard retrieval edge cases, 15 unanswerable refusal cases, and 15 conflicting source cases with passage-level citations. Retrieval and tool agents fail in production when rerankers trust stale titles, ReAct loops ignore 429 budgets, or citations cannot be verified. Arviser ships hybrid RAG and allowlisted agents with Promptfoo gates on grounding and tool misuse. €2,400 PoC isolates retrieval on your corpus; €3,900 prototype adds UI or API integration; €3,100 Eval CI encodes the 100-case set. Hardening €5,600; MVP €8,500; audit from €8,000.

**Sprints from €2,400 PoC · €3,100 eval CI · €3,900 prototype · €5,600 hardening · €8,500 MVP · audit from €8,000** · Topic: `engineering-rag-and-agents` · [Open proof](/#open-proof) · [Sample scorecard](/sample) · [golden-set-schema](https://github.com/nareyko/golden-set-schema) · [go-llm-gateway-audit](https://github.com/nareyko/go-llm-gateway-audit)

HTML: [/engineering/rag-and-agents](/engineering/rag-and-agents) · Markdown: [/engineering/rag-and-agents.md](/engineering/rag-and-agents.md)


_Retrieval, tools, MCP-lite_

## Why teams trust this stack with Arviser

- Stack-specific gates for RAG pipelines and tool agents, not generic AI slideware.
- Senior engineers (10+ yrs) with daily eval tooling; founder architecture review (30+ yrs IT).
- Proof links include Open proof and sample scorecards, commercial sprints stay fixed-scope.

## Dual path

### For CTO / Staff Eng

CTO path: build the retrieval or agent slice against the 100-case anatomy (30 normal, 20 multi-hop, 20 hard retrieval, 15 unanswerable, 15 conflicting with passage-level citations), keep tool allowlists typed, and leave CI fixtures your team can extend.

### For VP Product / CFO

Fixed-hour packages (€2,400–€8,500) de-risk AI spend: kill fees on Risk-Shared PoC when narrow hypotheses fail, 100% PoC credit toward MVP on Go, and no surprise T&M overruns inside written scope.

Route: 24h PoC (€2,400) if unknown feasibility, else 40h Prototype (€3,900); add 32h Eval CI (€3,100) before scale; harden at €5,600 or ship MVP at €8,500.

## Stack-specific failure modes we gate in eval CI

- Hybrid BM25+dense rerankers over-trust stale Confluence chunks after title-only edits.
- ReAct agents loop on HTTP 429 from internal APIs because retry budgets ignore tool cost.
- Citation snippets show HTML entities from CMS exports, causing users to trust wrong numeric tables.

### Sample eval fragment (rag-and-agents)

```
tests:
  - description: agentic RAG must cite primary doc
    vars: { q: "refund SLA for enterprise tier?" }
    assert:
      - type: llm-rubric
        value: answer uses only retrieved chunks
      - type: not-contains
        value: "As an AI language model"
```

## Which sprint fits?

| Situation | Recommendation | Sprint |
| --- | --- | --- |
| Need a 100-case RAG golden set with citation traces | Golden-set workshop + 32h Eval CI | €3,100 · 1 week |
| Hard unknown on retrieval or tools | 24h Technical PoC | €2,400 · 3–5 days |
| Working demo, no CI evals | 32h Eval CI harness | €3,100 · 1 week |

## FAQ

### Do you build RAG pipelines and tool agents under fixed scope?

Yes. We deliver written scope in 48h and fixed-hour packages; overruns inside agreed spec are on us.

### Can we start with eval CI only?

Yes, 32h Eval CI from €3,100 when a prototype exists but gates do not.

Reviewed by Vadim Nareyko, 30+ yrs IT · Last updated 2026-11-06

## Related

[Engineering hub](/engineering) · [Open proof](/#open-proof)

