Neural Paraphrase Identification of Questions with Noisy Pretraining
Gaurav Singh Tomar Thyago Duque Oscar Täckström Jakob Uszkoreit Dipanjan Das Google Inc. {gtomar, duque, oscart, uszkoreit,
Abstract
We present a solution to the problem of paraphrase identification of questions. We focus on a recent dataset of question pairs annotated with binary paraphrase labels and show that a variant of the decomposable attention model Parikh et al. (2016) results in accurate performance on this task, while being far simpler than many competing neural architectures. Furthermore, when the model is pretrained on a noisy dataset of automatically collected question paraphrases, it obtains the best reported performance on the dataset.
中文速览
判断两个问题是否语义相同（问题复述识别）是问答社区去重、搜索引擎优化等场景的核心难题。研究者在 Quora 超过 40 万对标注问题对上，对经典的"可分解注意力模型"（Decomposable Attention Model）做了两处关键改进：用字符 n-gram 嵌入替代词嵌入以增强拼写鲁棒性，并在自动采集的大规模噪声复述语料库 Paralex 上对全部模型参数进行预训练，再迁移微调到目标任务。实验结果表明，这一改进后的简单模型不仅超越了使用复杂双向 LSTM 和多视角匹配层的竞争方案，还刷新了 Quora 数据集上的最优成绩。这项工作说明，在领域相近的噪声数据上做全模型预训练，比仅用大规模语料预训练词嵌入更有效，为低成本提升 NLP 任务性能提供了实用思路。
原文 arXiv:1704.04565;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1704.04565v2