Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
Matteo Pagliardini* Iprova SA, Switzerland、Prakhar Gupta* EPFL, Switzerland、Martin Jaggi EPFL, Switzerland
Abstract
The recent tremendous success of unsupervised word embeddings in a multitude of applications raises the obvious question if similar methods could be derived to improve embeddings (i.e. semantic representations) of word sequences as well. We present a simple but efficient unsupervised objective to train distributed representations of sentences. Our method outperforms the state-of-the-art unsupervised models on most benchmark tasks, highlighting the robustness of the produced general-purpose sentence embeddings.
中文速览
词向量（word embeddings）的成功让人们自然想问：能不能用类似的方法为整个句子学出一个好用的语义向量？Sent2Vec 就是为此而生——它把 C-BOW 的训练思路从"用上下文预测单个词"扩展到"用整句话的词和 n-gram 组合来预测句中缺失的词"，通过无监督目标同时学习词向量与句子级别的组合方式。推理时只需对句内所有词/n-gram 向量做平均，计算量极低，可以轻松扩展到海量原始文本。实验结果表明，Sent2Vec 在多项标准测试任务上超过了当时最优的无监督乃至半监督句子表示方法，而训练和推理的速度却与简单平均词向量相当——这说明在句子表示学习中，模型简洁性与大规模数据的结合能带来令人意外的强劲效果。
原文 arXiv:1703.02507;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1703.02507v3