Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Tian Liang1,3 Zhiwei He Wenxiang Jiao Xing Wang Yan Wang3 Rui Wang2 Yujiu Yang1 Shuming Shi3 Zhaopeng Tu3 1Tsinghua University 2Shanghai Jiao Tong University 3Tencent AI Lab Contributed equally. Work was done when Tian and Zhiwei were interning at Tencent AI Lab. Xing and Yujiu are co-corresponding authors.
Abstract
Modern large language models (LLMs) like ChatGPT have shown remarkable performance on general language tasks but still struggle on complex reasoning tasks, which drives the research on cognitive behaviors of LLMs to explore human-like problem-solving strategies. Along this direction, one representative strategy is self-reflection, which asks an LLM to refine the solution with the feedback generated by itself iteratively. However, our study shows that such reflection-style methods suffer from the Degeneration-of-Thought (DoT) problem: once the LLM has established confidence in its solutions, it is unable to generate novel thoughts later through reflection even if its initial stance is incorrect. To address the DoT problem, we propose a Multi-Agent Debate (MAD) framework, in which multiple agents express their arguments in the state of “tit for tat” and a judge manages the debate process to obtain a final solution. Clearly, our MAD framework encourages divergent thinking in LLMs which would be helpful for tasks that require deep levels of contemplation. Experiment results on two challenging datasets, commonsense machine translation and counter-intuitive arithmetic reasoning, demonstr
中文速览
大型语言模型（LLM）在复杂推理任务上表现欠佳，研究者尝试用"自我反思（self-reflection）"让模型迭代修正答案，但这篇论文发现该方法存在一个根本缺陷——"思维退化（Degeneration-of-Thought, DoT）"：一旦模型对自己的答案建立了信心，即便答案是错的，它也几乎无法通过自我反思产生真正新颖的想法。为此，作者提出了多智能体辩论框架（Multi-Agent Debate, MAD）：让两个模型智能体以"针锋相对"的方式展开辩论，由第三个裁判智能体管理辩论进程并得出最终答案，从而引入外部视角、打破思维定势。在常识性机器翻译和反直觉算术推理两个挑战性任务上的实验表明，MAD显著优于自我反思等基线方法，用GPT-3.5-Turbo搭配MAD甚至能超越GPT-4的表现。这项工作揭示了单一模型自我博弈的内在局限，并为利用多智能体协作提升LLM深度推理能力提供了新思路。
原文 arXiv:2305.19118;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2305.19118v4