Enterprise RAG Support Platform

A single pipeline runs end to end, each stage a pure typed function emitting the same trace row: session load → intent + clarify-check → query rewrite → hybrid retrieve → rerank → evidence-sufficiency gate → generate-with-citations → verify → route. Stages one through six and stage nine are deterministic or run on local models; only generate and verify call a hosted LLM.

Retrieval is hybrid by design - semantic vector similarity finds passages that mean the same thing, while a keyword/BM25 leg catches exact identifiers the embedding might blur. The cross-encoder rerank then orders the fused set so the generator sees only the strongest evidence. Verification is the answer's QC pass: it claim-checks each assertion against its citation, confirms the citations resolve, and reassesses confidence before routing decides to answer, clarify, retry or escalate.

Only two stages ever cost money: generate and verify. Embeddings and the cross-encoder rerank run on local open-source models in both modes, so retrieval is free; query understanding is deterministic and free; routing is free. The cost meter therefore only moves in Cloud mode, and only on those two stages.

The cost lever is the evidence-sufficiency gate. A question whose retrieval comes back weak is short-circuited to clarify or escalate before a single token is generated - the system spends on questions it can actually answer, not on hopeless ones. A response cache covers repeats. This is the honest downscale of "intelligent cost control": not a secret routing model, but free deterministic stages plus a gate that refuses to pay for a bad answer.

The two metered stages can run two ways. Cloud uses a hosted model (claude-haiku-4-5) for both the generate and verify roles, cost-capped and fail-closed - at the budget cap it falls back to the $0 OSS path. OSS uses a self-hosted open model (Qwen3-8B) recorded on local M4 hardware and replayed at $0, with embeddings and reranking running locally in both modes.

The honest finding, shown as recorded divergences in the inspector: the cloud model writes more complete, fluent answers and is more consistent about attaching a citation to every claim, while the smaller open model is correct but terser and more conservative about confidence. The deterministic retrieval, evidence gate and routing are identical either way, so the guardrails hold in both modes. One divergence pair is shown as "not captured" rather than inventing an OSS output.

Several production capabilities are deliberately excluded and represented honestly rather than faked. Real document ingestion and live auto-sync at scale are stood in for by a simulated re-index and a freshness indicator. Multi-tier model routing is downscaled to one cloud model plus the recorded OSS model. Production LLM-based query understanding is downscaled to deterministic intent, clarify and rewrite stages. Real escalation, ticketing, SLA timers and work-assignment routing are simulated and labelled. The multilingual surface is narrowed to English, Dutch and German, and enterprise scale and real-time concurrency are downscaled to a synthetic corpus. Model retraining is represented by the audit log, the owner overlay and feedback capture - not live learning.