Iterative Alternating Neural Attention for Machine Reading
Alessandro Sordonif Philip Bachmanf Adam Trischlerf Yoshua Bengiog fMaluuba Research, Montréal, Québec gUniversity of Montréal, Montréal, Québec {alessandro.sordoni, phil.bachman,
Abstract
We propose a novel neural attention architecture to tackle machine comprehension tasks, such as answering Cloze-style queries with respect to a document. Unlike previous models, we do not collapse the query into a single vector, instead we deploy an iterative alternating attention mechanism that allows a fine-grained exploration of both the query and the document. Our model outperforms state-of-the-art baselines in standard machine comprehension benchmarks such as CNN news articles and the Children’s Book Test (CBT) dataset.
中文速览
机器阅读理解（machine comprehension）任务要求模型读懂一篇文章并回答完形填空式问题，难点在于如何同时精细地利用问题和文章中的信息。本文提出了一种"交替迭代注意力"（Alternating Iterative Attention）机制，不像以往模型那样把问题压缩成一个单一向量，而是让模型在每一步推理时先"看"问题的某个部分，再根据这个部分去文章中寻找对应线索，如此交替循环多次，逐步积累答案所需的证据。在 CNN 新闻和儿童图书测试（Children's Book Test）两个标准基准上，该模型超越了当时所有已有方法，达到最优水平。这项工作的意义在于，它证明了对问题进行细粒度、多步骤的动态探索比简单压缩表示更有效，为后续更复杂的阅读理解和自然语言推理模型提供了重要思路。
原文 arXiv:1606.02245;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1606.02245v4