Deploying Lifelong Open-Domain Dialogue Learning
Kurt Shuster Jack Urbanek∗ Emily Dinan Arthur Szlam Jason Weston Facebook AI Research * Joint first authors.
Abstract
Much of NLP research has focused on crowdsourced static datasets and the supervised learning paradigm of training once and then evaluating test performance. As argued in de Vries et al. (2020), crowdsourced data has the issues of lack of naturalness and relevance to real-world use cases, while the static dataset paradigm does not allow for a model to learn from its experiences of using language Silver et al. (2013). In contrast, one might hope for machine learning systems that become more useful as they interact with people. In this work, we build and deploy a role-playing game, whereby human players converse with learning agents situated in an open-domain fantasy world. We show that by training models on the conversations they have with humans in the game the models progressively improve, as measured by automatic metrics and online engagement scores. This learning is shown to be more efficient than crowdsourced data when applied to conversations with real users, as well as being far cheaper to collect.
中文速览
研究者们为了让对话AI能像人类一样在真实交流中持续成长，搭建并上线了一款免费的奇幻角色扮演游戏（LIGHT），让玩家在游戏中自然地与AI角色对话，从而以远低于众包标注（约五分之一）的成本大量收集真实、高质量的对话数据。他们设计了"持续学习（lifelong learning）"循环：每积累一批人机对话后，就用其中人类的回复微调模型，再将更新后的模型重新部署到游戏中，如此反复迭代。实验结果表明，模型随着迭代轮次的增加在自动评估指标和玩家"继续游戏率"两个维度上都持续提升，且相较于传统众包数据，来自真实玩家的游戏数据因为更贴近真实分布而效率更高。这项工作证明了将终身对话学习（lifelong dialogue learning）嵌入游戏化场景是一条切实可行的路径，为以低成本、可持续的方式训练更自然的对话系统提供了有力证据。
原文 arXiv:2008.08076;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2008.08076v2