Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Maciej Besta1, Nils Blach1††, Ales Kubicek1, Robert Gerstenberger1, Michał Podstawski2, Lukas Gianinazzi1, Joanna Gajda3, Tomasz Lehmann3, Hubert Niewiadomski3, Piotr Nyczyk3, Torsten Hoefler1 Equal contribution
Abstract
We introduce Graph of Thoughts (GoT): a framework that advances prompting capabilities in large language models (LLMs) beyond those offered by paradigms such as Chain-of-Thought or Tree of Thoughts (ToT). The key idea and primary advantage of GoT is the ability to model the information generated by an LLM as an arbitrary graph, where units of information (“LLM thoughts”) are vertices, and edges correspond to dependencies between these vertices. This approach enables combining arbitrary LLM thoughts into synergistic outcomes, distilling the essence of whole networks of thoughts, or enhancing thoughts using feedback loops. We illustrate that GoT offers advantages over state of the art on different tasks, for example increasing the quality of sorting by 62% over ToT, while simultaneously reducing costs by $>$ 31%. We ensure that GoT is extensible with new thought transformations and thus can be used to spearhead new prompting schemes. This work brings the LLM reasoning closer to human thinking or brain mechanisms such as recurrence, both of which form complex networks.
中文速览
让大型语言模型（LLM）更聪明地"思考"一直是个难题——现有的链式思维（Chain-of-Thought）只能让模型沿一条线推理，树状思维（Tree of Thoughts）虽然能分支探索，但依然被树形结构束缚，无法把不同推理路径的优秀想法合并起来。这篇论文提出了"思维图"（Graph of Thoughts，GoT）框架，把模型每一步产生的"想法"当作图的节点，想法之间的依赖关系当作有向边，从而允许任意图结构的推理——既能合并多条推理路径的精华，也能通过反馈循环不断打磨同一个想法。实验表明，在数组排序任务上，GoT比ToT的质量提升约62%，同时将调用成本压低了31%以上；在关键词统计、集合运算、文档合并等任务上也表现出色。这项工作的意义在于，它让提示工程（prompt engineering）首次具备了类似人类复杂联想和大脑递归网络的推理能力，同时还提供了可扩展的模块化架构，方便研究者快速试验新的提示策略。
原文 arXiv:2308.09687;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2308.09687v4