Aha.
正在载入中英对照阅读…

arXiv:2410.10762 · 中英对照阅读

AFlow:自动化智能体工作流生成

AFlow: Automating Agentic Workflow Generation

Jiayi Zhang、Jinyu Xiang、Zhaoyang Yu、Fengwei Teng、Xiong-Hui Chen、Jiaqi Chen、Mingchen Zhuge、Xin Cheng、Sirui Hong、Jinlin Wang、Bingnan Zheng、Bang Liu、Yuyu Luo、Chenglin Wu

中文速览

设计多步大语言模型工作流很费人力,现有自动方法又常受限于搜索范围或效率,难以自己找到好用的流程。AFlow把工作流写成可执行代码,用“集成、检查、修改”等常见操作作积木,再用蒙特卡洛树搜索反复改写流程,并根据运行结果积累经验、挑选更有希望的方案。在六个基准任务上,它比现有最佳方法平均提升5.7%,有些任务中,小模型用不到GPT-4o 5%的推理费用就能取得更好表现。它让工作流设计更有机会摆脱人工调试,也说明精心组织模型调用,可能比单纯换更大的模型更省钱有效。

摘要

大型语言模型(LLMs)已展现出在多个领域解决复杂任务的非凡潜力,通常通过采用遵循详细指令和操作序列的智能体工作流来实现。然而,构建这些工作流需要投入大量人力,限制了其可扩展性和通用性。近期研究致力于实现这些工作流的自动化生成与优化,但现有方法仍依赖初始人工设置,未能实现完全自动化且有效的工作流生成。为应对这一挑战,我们将工作流优化重新表述为在代码表示的工作流上的搜索问题,其中,LLM 调用节点由边连接。我们提出了 AFlow,这是一种自动化框架,利用蒙特卡洛树搜索(MCTS)高效探索这一搜索空间,并通过代码修改、树结构化经验和执行反馈迭代优化工作流。在六个基准数据集上的实证评估表明,AFlow 效果显著,较最先进的基线方法平均提升 5.7%。此外,在特定任务上,AFlow 能够使较小模型以仅占 GPT-4o 推理成本(美元)4.55%的成本超越 GPT-4o。代码见 https://github.com/FoundationAgents/AFlow。

Large language models (LLMs) have demonstrated remarkable potential in solving complex tasks across diverse domains, typically by employing agentic workflows that follow detailed instructions and operational sequences. However, constructing these workflows requires significant human effort, limiting scalability and generalizability. Recent research has sought to automate the generation and optimization of these workflows, but existing methods still rely on initial manual setup and fall short of achieving fully automated and effective workflow generation. To address this challenge, we reformulate workflow optimization as a search problem over code-represented workflows, where LLM-invoking nodes are connected by edges. We introduce AFlow, an automated framework that efficiently explores this space using Monte Carlo Tree Search, iteratively refining workflows through code modification, tree-structured experience, and execution feedback. Empirical evaluations across six benchmark datasets demonstrate AFlow’s efficacy, yielding a 5.7% average improvement over state-of-the-art baselines. Furthermore, AFlow enables smaller models to outperform GPT-4o on specific tasks at 4.55% of its inference cost in dollars. The code is available at https://github.com/FoundationAgents/AFlow.

术语表

Large Language Models (LLMs)
大型语言模型(LLMs)
agentic workflow
智能体工作流
autonomous agents
自主智能体
workflow optimization
工作流优化
automated workflow generation
自动化工作流生成
AFlow
AFlow
Monte Carlo Tree Search (MCTS)
蒙特卡洛树搜索(MCTS)
code-represented workflows
代码表示的工作流
LLM-invoking nodes
LLM 调用节点
code-based edges
基于代码的边
search space
搜索空间
operator
算子
Ensemble
集成
Review & Revise
审查与修订
soft mixed-probability selection
软混合概率选择
node expansion
节点扩展
execution feedback
执行反馈
backpropagation of experience
经验反向传播
prompt optimization
提示词优化
hyperparameter optimization
超参数优化
HumanEval
HumanEval
MBPP
MBPP
MATH
MATH
GSM8K
GSM8K
HotPotQA
HotPotQA
DROP
DROP
cost-performance efficiency
成本效益