Teaching models to express their uncertainty in words
Stephanie Lin University of Oxford Jacob Hilton OpenAI Owain Evans University of Oxford
Abstract
We show that a GPT-3 model can learn to express uncertainty about its own answers in natural language – without use of model logits. When given a question, the model generates both an answer and a level of confidence (e.g. “90% confidence” or “high confidence”). These levels map to probabilities that are well calibrated. The model also remains moderately calibrated under distribution shift, and is sensitive to uncertainty in its own answers, rather than imitating human examples. To our knowledge, this is the first time a model has been shown to express calibrated uncertainty about its own answers in natural language.
中文速览
大型语言模型（如GPT-3）在回答问题时经常"一本正经地胡说八道"，用户无法判断模型的回答是否可信。为此，研究者尝试训练GPT-3用自然语言直接说出自己的把握程度——比如"我有90%的把握"——而不依赖模型内部的概率分数（logits）。他们专门构建了一套名为CalibratedMath的数学题测试集，涵盖21种难度各异的算术任务，用来衡量这种"口头概率"（verbalized probability）在分布偏移下的校准表现；实验结果表明，经过微调的GPT-3不仅在训练分布内达到良好校准，在未见过的新任务类型上也保持了合理的校准能力，且性能优于仅靠logits估计不确定性的基线方法。这项工作首次证明语言模型可以用人类可读的方式表达对自身回答的校准置信度，为让AI诚实地传达内部不确定性、减少"幻觉"危害奠定了基础。
原文 arXiv:2205.14334;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2205.14334v2