A Parallel-Hierarchical Model for Machine Comprehension on Sparse Data
Adam Trischler adam.trischler、Zheng Ye jeff.ye、Xingdi Yuan eric.yuan、Jing He jing.he、Phillip Bachman phil.bachman \ANDKaheer Suleman Maluuba Research Montreal, Québec, Canada
Abstract
Understanding unstructured text is a major goal within natural language processing. Comprehension tests pose questions based on short text passages to evaluate such understanding. In this work, we investigate machine comprehension on the challenging MCTest benchmark. Partly because of its limited size, prior work on MCTest has focused mainly on engineering better features. We tackle the dataset with a neural approach, harnessing simple neural networks arranged in a parallel hierarchy. The parallel hierarchy enables our model to compare the passage, question, and answer from a variety of trainable perspectives, as opposed to using a manually designed, rigid feature set. Perspectives range from the word level to sentence fragments to sequences of sentences; the networks operate only on word-embedding representations of text. When trained with a methodology designed to help cope with limited training data, our Parallel-Hierarchical model sets a new state of the art for MCTest, outperforming previous feature-engineered approaches slightly and previous neural approaches by a significant margin (over 15% absolute).
中文速览
机器阅读理解要让计算机读懂一段短文并回答多项选择题，MCTest 是这一任务中公认的高难度基准数据集，但其训练样本极少，导致此前研究大多依赖大量人工设计特征。本文提出了一种「并行层次化（Parallel-Hierarchical）」神经网络模型，通过多个可训练的视角——从词级相似度、滑动窗口到句子语义——同时比对文章、问题和候选答案，并将问题与答案合并为「假设」进行统一评分。为应对数据稀少的挑战，作者还设计了「训练轮」（training wheels）初始化策略，让网络从模拟启发式规则的合理起点开始学习，而非随机初始化。在 MCTest 上的实验表明，该模型比此前最优的神经网络方法提升逾 15 个百分点，也略优于精心设计的特征工程方法，说明在小数据场景下用层次化可训练视角替代人工特征是行之有效的路线。
原文 arXiv:1603.08884;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1603.08884v1