Large Language Models as Optimizers
Chengrun Yang* Xuezhi Wang Yifeng Lu Hanxiao Liu Quoc V. Le Denny Zhou Xinyun Chen* {chengrun, xuezhiw, yifenglu, {qvl, dennyzhou, Google DeepMind * Equal contribution
Abstract
Optimization is ubiquitous. While derivative-based algorithms have been powerful tools for various problems, the absence of gradient imposes challenges on many real-world applications. In this work, we propose Optimization by PROmpting (OPRO), a simple and effective approach to leverage large language models (LLMs) as optimizers, where the optimization task is described in natural language. In each optimization step, the LLM generates new solutions from the prompt that contains previously generated solutions with their values, then the new solutions are evaluated and added to the prompt for the next optimization step. We first showcase OPRO on linear regression and traveling salesman problems, then move on to our main application in prompt optimization, where the goal is to find instructions that maximize the task accuracy. With a variety of LLMs, we demonstrate that the best prompts optimized by OPRO outperform human-designed prompts by up to $8\%$ on GSM8K, and by up to $50\%$ on Big-Bench Hard tasks. Code at https://github.com/google-deepmind/opro.
中文速览
大规模语言模型（LLM）其实可以充当一个通用优化器——把优化问题用自然语言描述给它，让它根据历史解及其得分迭代生成更好的新解，这就是 OPRO（Optimization by PROmpting）的核心思路。研究者先在线性回归和旅行商问题上验证了这一思路的可行性，随后将其应用到更有实际价值的"提示词优化"场景：自动搜索能让 LLM 在下游任务上准确率最高的指令文本。在多个 LLM（包括 PaLM 2 和 GPT-4 系列）上的实验表明，OPRO 优化出的提示词比人工设计的提示词在 GSM8K 数学推理基准上最多提升 8%，在 Big-Bench Hard 任务上最多提升 50%。这项工作的意义在于：它把"优化"这件原本需要手工设计算法的事情，转化成了一个纯自然语言的迭代对话过程，极大降低了针对新任务定制优化方法的门槛。
原文 arXiv:2309.03409;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2309.03409v3