arXiv:2410.10762 · 中英对照阅读
AFlow:自动化智能体工作流生成
AFlow: Automating Agentic Workflow Generation
中文速览
设计多步大语言模型工作流很费人力,现有自动方法又常受限于搜索范围或效率,难以自己找到好用的流程。AFlow把工作流写成可执行代码,用“集成、检查、修改”等常见操作作积木,再用蒙特卡洛树搜索反复改写流程,并根据运行结果积累经验、挑选更有希望的方案。在六个基准任务上,它比现有最佳方法平均提升5.7%,有些任务中,小模型用不到GPT-4o 5%的推理费用就能取得更好表现。它让工作流设计更有机会摆脱人工调试,也说明精心组织模型调用,可能比单纯换更大的模型更省钱有效。
摘要
大型语言模型(LLMs)已展现出在多个领域解决复杂任务的非凡潜力,通常通过采用遵循详细指令和操作序列的智能体工作流来实现。然而,构建这些工作流需要投入大量人力,限制了其可扩展性和通用性。近期研究致力于实现这些工作流的自动化生成与优化,但现有方法仍依赖初始人工设置,未能实现完全自动化且有效的工作流生成。为应对这一挑战,我们将工作流优化重新表述为在代码表示的工作流上的搜索问题,其中,LLM 调用节点由边连接。我们提出了 AFlow,这是一种自动化框架,利用蒙特卡洛树搜索(MCTS)高效探索这一搜索空间,并通过代码修改、树结构化经验和执行反馈迭代优化工作流。在六个基准数据集上的实证评估表明,AFlow 效果显著,较最先进的基线方法平均提升 5.7%。此外,在特定任务上,AFlow 能够使较小模型以仅占 GPT-4o 推理成本(美元)4.55%的成本超越 GPT-4o。代码见 https://github.com/FoundationAgents/AFlow。
Large language models (LLMs) have demonstrated remarkable potential in solving complex tasks across diverse domains, typically by employing agentic workflows that follow detailed instructions and operational sequences. However, constructing these workflows requires significant human effort, limiting scalability and generalizability. Recent research has sought to automate the generation and optimization of these workflows, but existing methods still rely on initial manual setup and fall short of achieving fully automated and effective workflow generation. To address this challenge, we reformulate workflow optimization as a search problem over code-represented workflows, where LLM-invoking nodes are connected by edges. We introduce AFlow, an automated framework that efficiently explores this space using Monte Carlo Tree Search, iteratively refining workflows through code modification, tree-structured experience, and execution feedback. Empirical evaluations across six benchmark datasets demonstrate AFlow’s efficacy, yielding a 5.7% average improvement over state-of-the-art baselines. Furthermore, AFlow enables smaller models to outperform GPT-4o on specific tasks at 4.55% of its inference cost in dollars. The code is available at https://github.com/FoundationAgents/AFlow.
术语表
- Large Language Models (LLMs)
- 大型语言模型(LLMs)
- agentic workflow
- 智能体工作流
- autonomous agents
- 自主智能体
- workflow optimization
- 工作流优化
- automated workflow generation
- 自动化工作流生成
- AFlow
- AFlow
- Monte Carlo Tree Search (MCTS)
- 蒙特卡洛树搜索(MCTS)
- code-represented workflows
- 代码表示的工作流
- LLM-invoking nodes
- LLM 调用节点
- code-based edges
- 基于代码的边
- search space
- 搜索空间
- operator
- 算子
- Ensemble
- 集成
- Review & Revise
- 审查与修订
- soft mixed-probability selection
- 软混合概率选择
- node expansion
- 节点扩展
- execution feedback
- 执行反馈
- backpropagation of experience
- 经验反向传播
- prompt optimization
- 提示词优化
- hyperparameter optimization
- 超参数优化
- HumanEval
- HumanEval
- MBPP
- MBPP
- MATH
- MATH
- GSM8K
- GSM8K
- HotPotQA
- HotPotQA
- DROP
- DROP
- cost-performance efficiency
- 成本效益