Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
Lorenz Kuhn, Yarin Gal, Sebastian Farquhar OATML Group, Department of Computer Science, University of Oxford
Abstract
We introduce a method to measure uncertainty in large language models. For tasks like question answering, it is essential to know when we can trust the natural language outputs of foundation models. We show that measuring uncertainty in natural language is challenging because of ‘semantic equivalence’—different sentences can mean the same thing. To overcome these challenges we introduce semantic entropy—an entropy which incorporates linguistic invariances created by shared meanings. Our method is unsupervised, uses only a single model, and requires no modifications to ‘off-the-shelf’ language models. In comprehensive ablation studies we show that the semantic entropy is more predictive of model accuracy on question answering data sets than comparable baselines.
中文速览
大语言模型在回答问题时，我们往往不知道它的输出到底有多可信，而现有的不确定性度量方法直接在词元（token）层面计算熵，会把"巴黎是法国首都"和"法国首都是巴黎"当作两个不同的答案，从而高估模型的不确定性。针对这一"语义等价"难题，研究者提出了语义熵（semantic entropy）：先让模型多次采样生成若干回答，再用双向蕴含（bidirectional entailment）算法把意思相同的句子聚为一类，最后在意义空间而非词元空间上计算熵值。实验在TriviaQA和CoQA等问答数据集上表明，语义熵比现有无监督基线更准确地预测模型是否答对，随模型规模增大时表现提升也更明显，且无需任何额外训练或模型改动，开箱即用。这项工作为判断何时能够信任大语言模型的输出提供了一个实用且理论上有据可查的工具，对构建更安全、更可靠的AI系统具有重要意义。
原文 arXiv:2302.09664;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2302.09664v3