Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
Seonhoon Kim1,2, Inho Kang1, Nojun Kwak2 1Naver Search, 2Seoul National University
Abstract
Sentence matching is widely used in various natural language tasks such as natural language inference, paraphrase identification, and question answering. For these tasks, understanding logical and semantic relationship between two sentences is required but it is yet challenging. Although attention mechanism is useful to capture the semantic relationship and to properly align the elements of two sentences, previous methods of attention mechanism simply use a summation operation which does not retain original features enough. Inspired by DenseNet, a densely connected convolutional network, we propose a densely-connected co-attentive recurrent neural network, each layer of which uses concatenated information of attentive features as well as hidden features of all the preceding recurrent layers. It enables preserving the original and the co-attentive feature information from the bottommost word embedding layer to the uppermost recurrent layer. To alleviate the problem of an ever-increasing size of feature vectors due to dense concatenation operations, we also propose to use an autoencoder after dense concatenation. We evaluate our proposed architecture on highly competitive benchmark d
中文速览
句子匹配（sentence matching）是自然语言推理、复述识别、问答等任务的核心难题，关键在于如何准确捕捉两个句子之间的逻辑与语义关系。现有注意力机制通常用加法融合特征，容易丢失底层原始信息，而简单叠加循环网络又面临梯度消失的困境。受图像领域稠密连接网络 DenseNet 启发，作者提出了 DRCN（Densely-connected Recurrent and Co-attentive Network，稠密连接循环协同注意力网络），让每一层都能直接拼接（而非相加）所有前层的隐状态特征与协同注意力特征，从而将词嵌入层到最顶层的信息完整保留；为了防止拼接导致特征维度爆炸，还引入自编码器作为瓶颈压缩模块，同时起到正则化作用。在 SNLI、MultiNLI、Quora、TrecQA、SelQA 五个权威基准上的实验表明，DRCN 在大多数任务上达到当时最优水平，证明稠密连接与协同注意力的结合能有效提升深层网络对句子语义关系的建模能力。
原文 arXiv:1805.11360;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1805.11360v2