Automatic Prompt Optimization with “Gradient Descent” and Beam Search
Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, Michael Zeng Microsoft Azure AI
Abstract
Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort. We propose a simple and nonparametric solution to this problem, Prompt Optimization with Textual Gradients (ProTeGi), which is inspired by numerical gradient descent to automatically improve prompts, assuming access to training data and an LLM API. The algorithm uses minibatches of data to form natural language “gradients” that criticize the current prompt, much like how numerical gradients point in the direction of error ascent. The natural language gradients are then “propagated” into the prompt by editing the prompt in the opposite semantic direction of the gradient. These gradient descent steps are guided by a beam search and bandit selection procedure which significantly improves algorithmic efficiency. Preliminary results across three benchmark NLP tasks and the novel problem of LLM jailbreak detection suggest that Automatic Prompt Optimization can outperform prior prompt editing techniques and improve an initial prompt’s performance by up to 31%, by using data to rewrite va
中文速览
手写提示词（prompt）费时费力、效果还难以保证，如何让大语言模型自动找到更好的提示词是当前的核心难题。ProTeGi（用文本梯度做提示词优化）借鉴数值梯度下降的思路，先用一小批训练样本让模型"批评"当前提示词的缺陷，生成自然语言形式的"梯度"，再让模型按照这份批评反向编辑提示词，使其朝更准确的方向演化，整个过程只需调用普通的大模型API、无需访问模型内部参数。搜索效率上，算法结合集束搜索（beam search）与最优臂识别（best arm identification）中的"逐步淘汰"策略，大幅减少了评估次数。在仇恨言论检测、假新闻检测、阿拉伯语讽刺检测以及新提出的大模型越狱攻击检测四项任务上，ProTeGi最高将初始提示词性能提升31%，平均超越现有最优基线4–8%，同时调用API的次数更少，对从业者具有直接的实用价值。
原文 arXiv:2305.03495;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2305.03495v2