Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
Laria Reynolds Kyle McDonell
Abstract
Prevailing methods for mapping large generative language models to supervised tasks may fail to sufficiently probe models’ novel capabilities. Using GPT-3 as a case study, we show that 0-shot prompts can significantly outperform few-shot prompts. We suggest that the function of few-shot examples in these cases is better described as locating an already learned task rather than meta-learning. This analysis motivates rethinking the role of prompts in controlling and evaluating powerful language models. In this work, we discuss methods of prompt programming, emphasizing the usefulness of considering prompts through the lens of natural language. We explore techniques for exploiting the capacity of narratives and cultural anchors to encode nuanced intentions and techniques for encouraging deconstruction of a problem into components before producing a verdict. Informed by this more encompassing theory of prompt programming, we also introduce the idea of a metaprompt that seeds the model to generate its own natural language prompts for a range of tasks. Finally, we discuss how these more general methods of interacting with language models can be incorporated into existing and future bench
中文速览
大型语言模型（如GPT-3）具备的真实能力远比人们通常测试出来的要强，根本原因在于现有的"少样本提示"（few-shot prompting）方式并不是在让模型"现学"任务，而只是在帮它"找到"自己早已学会的能力。作者通过法译英实验证明，精心设计的零样本提示（0-shot prompt）可以超过包含十个例子的少样本提示，由此提出应该用"自然语言编程"的思路来设计提示词——利用叙事框架、文化锚点、强迫模型逐步分解问题等手段，更精准地激活模型潜能；更进一步，他们提出"元提示"（metaprompt）概念，让模型自己为不同任务生成合适的提示词，从而把提示设计工作转包给模型本身。这项研究对语言模型的评测方式和实际应用都有重要启示：现有基准测试很可能系统性地低估了模型能力，重新思考提示设计方法论是释放这类模型真实潜力的关键一步。
原文 arXiv:2102.07350;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2102.07350v1