arXiv:2606.21724 · 中英对照阅读
Denoising Iterative Self-Correction: Structured Verification Loops for Reliable LLM Reasoning
中文速览
多步推理中的语言模型常会出错,而反复修改又可能把正确答案改坏;这项研究要解决的正是如何修错、同时尽量保住对的答案。作者提出 DISC,让模型先针对答案提出并回答核查问题,再由判断环节确认是否存在明确矛盾,只有发现错误才定点修改,并可重复几轮。实验显示,DISC 在三类任务和多种模型上总体优于两种常见自我修正方法;在一项推理错误基准上,它达到 81.6% 准确率,修好一个错误所伴随的正确答案被改坏情况也显著更少。研究还发现,换用不同模型负责核查和判断能减轻模型自我偏袒,而能力不足的模型即使看出证据冲突,也未必能改对答案;这为更可靠的模型纠错提供了可操作的设计思路。
摘要
Large language models produce fluent but often incorrect multi-step reasoning, and naive correction methods risk degrading already-correct answers. We introduce Denoising Iterative Self-Correction (DISC), a test-time procedure that treats verification question outputs as noisy measurements of where a solution may be corrupted. Using these signals, DISC progressively reduces errors across multiple verify-judge-correct passes, analogous to traditional iterative denoising. A binary judgment gate controls correction precision by blocking rewrites that would damage already-correct answers while the verifier and corrector together repair errors. We evaluate this trade-off using two paired diagnostics: an improvement-to-degradation ratio (precision) and a repair rate (recall). Across three benchmarks (BIG-Bench Mistake, HotpotQA, GPQA Diamond) and four models, DISC dominates Chain-of-Verification and Self-Refine on the precision-recall trade-off, reaching 81.6% accuracy with $13\times$ more improvements per degradation than Chain-of-Verification and $5\times$ more than Self-Refine on BIG-Bench Mistake (Sonnet 4.5). On GPQA Diamond, we identify a capability floor below which judges acknowledge contradictions in evidence but cannot translate that recognition into a correction. We further show that cross-model role allocation—assigning verification and judgment to a model different from the generator—mitigates self-confirmation bias.
术语表
- Large language model (LLM)
- 大语言模型
- Denoising Iterative Self-Correction (DISC)
- 去噪迭代自我纠正
- test-time correction
- 测试时纠正
- verify–judge–correct loop
- 验证–判定–纠正循环
- judgment gate
- 判定门控
- iterative denoising
- 迭代去噪
- verification question
- 验证问题
- improvement-to-degradation ratio (I:D)
- 改进–退化比
- repair rate
- 修复率
- precision
- 精确率
- recall
- 召回率
- Critique Score (CS)
- 批判评分
- Confidence Level (CL)
- 置信水平
- Chain-of-Verification (CoVe)
- 验证链
- Self-Refine
- 自我精炼
- chain-of-thought (CoT)
- 思维链
- self-consistency
- 自洽性
- self-confirmation bias
- 自我确认偏差
- cross-model role allocation
- 跨模型角色分配
- BIG-Bench Mistake (BBM)
- BIG-Bench Mistake 基准
- HotpotQA
- HotpotQA
- GPQA Diamond
- GPQA Diamond
- multi-hop question answering
- 多跳问答
- reinforcement learning (RL)
- 强化学习
- reward model
- 奖励模型
- SCoRe
- SCoRe
- CRITIC
- CRITIC
- verifier bottleneck
- 验证器瓶颈