Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues?
Yufan Zhao1, Wei Wu2, Can Xu1, 1Microsoft Corporation, Beijing, China 2Meituan, Beijing, China
Abstract
We study knowledge-grounded dialogue generation with pre-trained language models. Instead of pursuing new state-of-the-art on benchmarks, we try to understand if the knowledge stored in parameters of the pre-trained models is already enough to ground open domain dialogues, and thus allows us to get rid of the dependency on external knowledge sources in generation. Through extensive experiments on benchmarks, we find that by fine-tuning with a few dialogues containing knowledge, the pre-trained language models can outperform the state-of-the-art model that requires external knowledge in automatic evaluation and human judgment, suggesting a positive answer to the question we raised.
中文速览
大规模预训练语言模型（如GPT-2和DialoGPT）的参数里其实已经储存了大量世界知识，研究者想弄清楚：在做开放域知识对话时，能不能直接用模型自带的知识来替代外部知识库？实验方法是把Wizard of Wikipedia等三个知识对话基准数据集里的外部知识段落全部丢弃，只用对话文本对预训练模型做少量微调，再与那些依赖外部知识的最先进模型比较。结果发现，微调后的DialoGPT在自动评测指标和人工评估上都能与甚至超过需要检索外部知识的对比模型，说明预训练模型本身就足够"博学"来驱动知识型对话；不过人工评估也揭示了一个明显短板：模型在数字、时间、地点等细节上容易出错，因为它无法实时查阅精确来源。这项工作的价值在于证明了一条更简洁的技术路线——无需复杂的知识检索模块也能做好知识对话，同时也指出将预训练知识与外部知识结合仍是值得深挖的未来方向。
原文 arXiv:2011.09708;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2011.09708v1