Review Moderation & Content Quality

Hospitality · 2020-Present

PublicRepresentative · synthetic data
languagefastTextpolicyrules · $0topiclexiconclassifytri-classreasonheld onlyrouter4-bandverdictpublish / hold / rejectmetered · LLMdeterministic · $0
Live diagram - the 7-stage pipeline. Only classify and the held-only reason are metered; the policy, topic and router stages are deterministic at $0.
language$0policy$0topic$0$classifymetered$reasonmeteredrouter$0verdict$0COST LEVER · held-only reason
Live diagram - spend accrues on classify and a held-only reason; the clean majority auto-publishes without a reason call.
ASPECTCLOUD · capable readOSS · recordedBorderline promohold for reviewauto-publishSubtle competitor digreviewerpublishedReason qualityspecificterserMixed-language reviewclassifiednot captured - routed to a human
Live diagram - Cloud (claude-haiku-4-5) vs recorded OSS (Qwen3Guard-Gen-4B + Qwen3-8B); the deterministic policy, topic and router stages are identical in both modes.

The deterministic spine

The pipeline runs seven stages: language detect → policy engine → topic lexicon → classify → reason → router → verdict. The first three are deterministic and free, and they do real work - language routing, hard policy rules and a ten-topic lexicon - so that the cheap, explainable backbone is established before any model runs. The router and the verdict are deterministic too.

Verdicts are Published, Held or Rejected. The four-band router applies the confidence thresholds and downgrades low-confidence automated decisions a band, so the system errs toward a human check rather than a confident mistake.

The metered stages and the cost lever

Two stages can cost money: classify (a tri-class call with confidence) and reason (a short, reviewer-facing rationale). Everything else is deterministic and $0 in both modes.

The cost lever is selectivity. The reason stage is reviewer-bound - it only runs for items that are being held or rejected, never for the clean majority that auto-publish - so the more expensive generation is reserved for the cases a human will actually read. Combined with the deterministic policy and topic stages resolving the obvious cases up front, the metered surface stays small.

Dual approach - cloud vs on-prem OSS

The metered stages run one of two ways. Cloud uses a hosted model (claude-haiku-4-5) for both classify and reason, cost-capped and fail-closed. OSS uses self-hosted open models recorded on local M4 hardware - a safety-tuned guard classifier (Qwen3Guard-Gen-4B) for classify plus Qwen3-8B for the reason - replayed at $0.

The honest finding, shown as recorded divergences in the inspector: on borderline reviews the two paths sometimes reach different verdicts - the cloud model holds a subtle promo that the open model would auto-publish - and the open model's reasons are terser. The deterministic policy, topic and router stages are identical in both modes, so the framework around the disagreement is stable. About four honest borderline verdict-shifts are recorded, and one case is shown as "not captured" rather than fabricating an OSS output.

Out of scope (the real system)

Several production capabilities are deliberately excluded and represented honestly. The full breadth of policy categories, languages and review volume is downscaled to a synthetic catalog and a compact taxonomy. Real reviewer-workflow integrations and notifications are simulated and labelled. Model retraining and online learning are represented by the two-layer rule overlay, the confidence thresholds and the audit log rather than live learning - the system gets better by configuration and human resolution, not by training in place.