SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution

摘要

Hallucination is the reliability bottleneck for LLM-based agents, and fact attribution verifiers are the last line of defense — yet today’s verifiers emit only opaque binary labels, leaving agents unable to self-correct and operators unable to audit. We present Seva, a structured verification agent that emits evidence alignments, step-by-step reasoning chains, calibrated confidence, and a six-category error diagnosis with actionable fixes. Training such an agent with RL is non-trivial: standard binary reward on multi-component output triggers advantage collapse — within-group reward variance vanishes and the GRPO gradient disappears. We resolve this with a process reward that decomposes verification quality into five independent components weighted $70/30$ toward process signals, restoring the gradient and inducing an implicit curriculum — the agent first masters verification behavior (alignment $0.917{\to}0.997$ , format $72\%{\to}100\%$ ), then outcomes (F1 $64.9{\to}69.0$ ). Structured output further enables a Verify $\to$ Reflect $\to$ Probe $\to$ Refine self-evolution loop, which over four rounds on a 7B model surfaces an unexpected structural finding: each round produces a benchmark-specialist, not a generalist ( $+15$ pp on HaluEval, $-10$ to $-14$ pp on TruthfulQA in the same model, persistent at $4\times$ data). On ClearFacts, Seva-3B matches GPT-4o-mini (69.0 vs. 69.8 F1) while producing substantially richer, auditable output — confirming a principle that should generalize: for any RL task with multi-component generation, reward granularity must match output granularity.