Factuality Enhanced Language Models for Open-Ended Text Generation
Nayeon Lee † Work done during an internship at NVIDIA. Hong Kong University of Science and Technology Wei Ping Correspondence to: Nayeon Lee Wei Ping NVIDIA Peng Xu NVIDIA Mostofa Patwary NVIDIA Pascale Fung Hong Kong University of Science and Technology Mohammad Shoeybi NVIDIA Bryan Catanzaro NVIDIA
Abstract
Pretrained language models (LMs) are susceptible to generate text with nonfactual information. In this work, we measure and improve the factual accuracy of large-scale LMs for open-ended text generation. We design the FactualityPrompts test set and metrics to measure the factuality of LM generations. Based on that, we study the factual accuracy of LMs with parameter sizes ranging from 126M to 530B. Interestingly, we find that larger LMs are more factual than smaller ones, although a previous study suggests that larger LMs can be less truthful in terms of misconceptions. In addition, popular sampling algorithms (e.g., top- $p$ ) in open-ended text generation can harm the factuality due to the “uniform randomness” introduced at every sampling step. We propose the factual-nucleus sampling algorithm that dynamically adapts the randomness to improve the factuality of generation while maintaining quality. Furthermore, we analyze the inefficiencies of the standard training method in learning correct associations between entities from factual text corpus (e.g., Wikipedia). We propose a factuality-enhanced training method that uses TopicPrefix for better awareness of facts and sentence comp
中文速览
大型语言模型（如GPT系列）在开放式文本生成中频繁产生与事实不符的内容，但此前缺乏针对这一场景的系统性评测与改进方法。研究者构建了名为FactualityPrompts的测评基准，结合命名实体错误率和文本蕴含率两类指标，对参数量从1.26亿到5300亿不等的多个语言模型进行了全面的事实准确性分析，发现模型越大事实性越强——这与此前关于大模型更易产生错误认知的结论并不矛盾，因为两者考察的是不同类型的知识。在此基础上，研究者提出了两项改进方案：一是"事实核采样"（factual-nucleus sampling）解码算法，通过动态调整每步采样的随机性来减少随机捏造信息的问题；二是"事实增强训练"（factuality-enhanced training）方法，借助TopicPrefix提示和句子补全训练目标让模型更好地从维基百科等纯文本语料中学习实体关联。两项方法叠加后，最大规模模型的命名实体事实错误率从33.3%大幅降至14.5%，为在不依赖外部检索系统的前提下从根源上提升语言模型事实可靠性提供了切实可行的路径。
原文 arXiv:2206.04624;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2206.04624v3