A Practical Survey on Zero-shot Prompt Design for In-context Learning
Yinheng Li Columbia University / New York City
Abstract
The remarkable advancements in large language models (LLMs) have brought about significant improvements in Natural Language Processing(NLP) tasks. This paper presents a comprehensive review of in-context learning techniques, focusing on different types of prompts, including discrete, continuous, few-shot, and zero-shot, and their impact on LLM performance. We explore various approaches to prompt design, such as manual design, optimization algorithms, and evaluation methods, to optimize LLM performance across diverse tasks. Our review covers key research studies in prompt engineering, discussing their methodologies and contributions to the field. We also delve into the challenges faced in evaluating prompt performance, given the absence of a single ”best” prompt and the importance of considering multiple metrics. In conclusion, the paper highlights the critical role of prompt design in harnessing the full potential of LLMs and provides insights into the combination of manual design, optimization techniques, and rigorous evaluation for more effective and efficient use of LLMs in various NLP tasks.
中文速览
大型语言模型（LLM）能否真正"听懂"指令，关键在于如何设计提示词（prompt），但目前学界对各类提示方法缺乏系统梳理。这篇综述聚焦于上下文学习（in-context learning）框架，将提示词分为离散/连续、零样本/少样本四大类，并系统梳理了手工设计、自动优化算法（如APE、GRIPS、强化学习生成等）以及评估方法（条件概率、任务执行准确率、提示迁移性）三个维度的研究进展，重点介绍了零样本离散提示这一最具实用价值的方向，包括"思维链"等已被证明有效的提示策略。研究发现，精心设计的零样本提示在某些任务上甚至可以超越少样本方法，而当前最大的挑战在于没有统一的"最优提示"评判标准，需要综合多项指标才能衡量提示质量。这一综述为研究者和从业者系统理解提示工程提供了全面参考，也指出了将手工设计与自动优化结合使用才是充分释放LLM潜力的可行路径。
原文 arXiv:2309.13205;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2309.13205v1