A User Simulator for Task-Completion Dialogues
Xiujun Li† Zachary C. Lipton⋆ Bhuwan Dhingra‡ Lihong Li† Jianfeng Gao† Yun-Nung Chen§ †Microsoft Research, Redmond, WA, USA ⋆University of California, San Diego, CA, USA ‡Carnegie Mellon University, Pittsburgh, PA, USA §National Taiwan University, Taipei, Taiwan
Abstract
Despite widespread interests in reinforcement-learning for task-oriented dialogue systems, several obstacles can frustrate research and development progress. First, reinforcement learners typically require interaction with the environment, so conventional dialogue corpora cannot be used directly. Second, each task presents specific challenges, requiring separate corpus of task-specific annotated data. Third, collecting and annotating human-machine or human-human conversations for task-oriented dialogues requires extensive domain knowledge. Because building an appropriate dataset can be both financially costly and time-consuming, one popular approach is to build a user simulator based upon a corpus of example dialogues. Then, one can train reinforcement learning agents in an online fashion as they interact with the simulator. Dialogue agents trained on these simulators can serve as an effective starting point. Once agents master the simulator, they may be deployed in a real environment to interact with humans, and continue to be trained online. To ease empirical algorithmic comparisons in dialogues, this paper introduces a new, publicly available simulation framework, where our simu
中文速览
任务导向对话系统（task-oriented dialogue system）的强化学习研究长期面临一个尴尬：没法直接用现有语料训练，又很难让真实用户去陪智能体反复试错，而收集标注新数据既费钱又费时。为此，论文提出了一个公开可用的对话仿真框架，核心是针对电影订票场景设计的用户模拟器（user simulator）——它把基于规则的议程式方法（agenda-based approach）与从真实标注数据中学习的序列到序列自然语言生成模块结合起来，既保证用户行为的目标一致性，又具备一定的语言多样性。框架支持"订票"和"查询电影"两项任务，内置自然语言理解、生成、错误模型等完整组件，并提供了多种基线对话智能体（包括规则、监督学习和强化学习版本）的实现与评测流程，方便研究者直接接入自己的算法进行对比实验。这项工作的意义在于，它为对话策略研究提供了一个统一、可复现的测试床，有助于降低领域门槛、推动算法的横向比较与快速迭代。
原文 arXiv:1612.05688;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1612.05688v3