Reinforced Mnemonic Reader for Machine Reading Comprehension
Minghao Hu†, Yuxing Peng†, Zhen Huang†, Xipeng Qiu‡, Furu Wei§, Ming Zhou§ † College of Computer, National University of Defense Technology, Changsha, China ‡ School of Computer Science, Fudan University, Shanghai, China § Microsoft Research, Beijing, China Contribution during internship at Fudan University and Microsoft Research.
Abstract
In this paper, we introduce the Reinforced Mnemonic Reader for machine reading comprehension tasks, which enhances previous attentive readers in two aspects. First, a reattention mechanism is proposed to refine current attentions by directly accessing to past attentions that are temporally memorized in a multi-round alignment architecture, so as to avoid the problems of attention redundancy and attention deficiency. Second, a new optimization approach, called dynamic-critical reinforcement learning, is introduced to extend the standard supervised method. It always encourages to predict a more acceptable answer so as to address the convergence suppression problem occurred in traditional reinforcement learning algorithms. Extensive experiments on the Stanford Question Answering Dataset (SQuAD) show that our model achieves state-of-the-art results. Meanwhile, our model outperforms previous systems by over 6% in terms of both Exact Match and F1 metrics on two adversarial SQuAD datasets.
中文速览
让机器读懂一段文章并准确回答问题，难点在于如何让模型在多轮注意力对齐时既不重复关注同一位置、又不遗漏关键信息，同时训练时强化学习容易因"基线比采样结果更好"而压制模型收敛。针对这两个痛点，研究者提出了"强化记忆阅读器"（Reinforced Mnemonic Reader）：一方面引入再注意力机制（reattention），把历史注意力分布"记"下来，用两个词对相同文本的注意力重叠程度来直接修正当前注意力，避免多轮对齐中的冗余与缺失；另一方面提出动态关键强化学习（Dynamic-Critical Reinforcement Learning，DCRL），每次训练都把随机采样和贪心推断两条路的得分做比较，始终把较高分设为奖励、较低分设为基线，保证归一化奖励恒为正，从根本上消除收敛压制问题。在标准SQuAD数据集上，该模型单模型达到79.5% EM / 86.6% F1的当时最优水平；在两个对抗性SQuAD数据集上更以超过6个百分点的优势领先此前所有方法，说明其鲁棒性尤为突出。
原文 arXiv:1705.02798;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1705.02798v6