Generative Deep Neural Networks for Dialogue: A Short Review
Iulian Vlad Serban Department of Computer Science and Operations Research, University of Montreal \AndRyan Lowe School of Computer Science, McGill University \AndLaurent Charlin School of Computer Science, McGill University \AndJoelle Pineau School of Computer Science, McGill University
Abstract
Researchers have recently started investigating deep neural networks for dialogue applications. In particular, generative sequence-to-sequence (Seq2Seq) models have shown promising results for unstructured tasks, such as word-level dialogue response generation. The hope is that such models will be able to leverage massive amounts of data to learn meaningful natural language representations and response generation strategies, while requiring a minimum amount of domain knowledge and hand-crafting. An important challenge is to develop models that can effectively incorporate dialogue context and generate meaningful and diverse responses. In support of this goal, we review recently proposed models based on generative encoder-decoder neural network architectures, and show that these models have better ability to incorporate long-term dialogue history, to model uncertainty and ambiguity in dialogue, and to generate responses with high-level compositional structure.
中文速览
对话系统中的生成模型长期面临两个痛点：难以有效利用长对话历史、生成的回复单一且缺乏针对性。研究者在已有的层级循环编解码器模型（HRED）基础上，提出了两种改进方案——引入高维随机潜变量来建模对话中的不确定性与歧义性的 VHRED，以及将回复生成分解为"粗粒度语义序列→自然语言"两级层次结构的多分辨率 RNN（MrRNN）。在 Ubuntu 技术问答对话和 Twitter 对话两个数据集上，通过人工评测和基于活动-实体匹配的自动指标验证，MrRNN 在流畅性、相关性及关键词覆盖率上全面领先，F1 实体分数是其他模型的两倍以上；VHRED 次之，也显著优于基础 HRED 和 LSTM 模型。这一工作表明，精心设计的模型结构本身就能带来更多样、更贴题的回复，有望减少对手工奖励函数或多模型融合的依赖，为生成式对话系统的架构研究指出了一条可行路径。
原文 arXiv:1611.06216;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1611.06216v1