Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Alexis Conneau Facebook AI Research、Douwe Kiela Facebook AI Research、Holger Schwenk Facebook AI Research \ANDLoïc Barrault LIUM, Université Le Mans、Antoine Bordes Facebook AI Research
Abstract
Many modern NLP systems rely on word embeddings, previously trained in an unsupervised manner on large corpora, as base features. Efforts to obtain embeddings for larger chunks of text, such as sentences, have however not been so successful. Several attempts at learning unsupervised representations of sentences have not reached satisfactory enough performance to be widely adopted. In this paper, we show how universal sentence representations trained using the supervised data of the Stanford Natural Language Inference datasets can consistently outperform unsupervised methods like SkipThought vectors Kiros et al. (2015) on a wide range of transfer tasks. Much like how computer vision uses ImageNet to obtain features, which can then be transferred to other tasks, our work tends to indicate the suitability of natural language inference for transfer learning to other NLP tasks. Our encoder is publicly available111https://www.github.com/facebookresearch/InferSent.
中文速览
用有监督的自然语言推理（Natural Language Inference, NLI）数据来训练通用句子表示，这件事以前没人系统验证过——大多数方法都走无监督路线。研究者用斯坦福NLI数据集（SNLI）训练了多种句子编码器，重点发现带最大池化的双向LSTM（BiLSTM with max pooling）编码出来的句向量，在情感分析、问题分类、语义相似度等12项迁移任务上全面超越了SkipThought等无监督方法，同时训练速度更快。这项工作还揭示了一个重要规律：NLI任务之所以适合作为预训练目标，是因为它要求模型真正理解句子语义和逻辑关系，就像ImageNet之于计算机视觉一样，能让模型学到可迁移的通用特征。这意味着NLP领域也找到了一个高质量的监督预训练"锚点"，为后续大规模句子表示学习研究奠定了重要基础。
原文 arXiv:1705.02364;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1705.02364v5