Document Embedding with Paragraph Vectors
Andrew M. Dai Google \AndChristopher Olah Google \AndQuoc V. Le Google
Abstract
Paragraph Vectors has been recently proposed as an unsupervised method for learning distributed representations for pieces of texts. In their work, the authors showed that the method can learn an embedding of movie review texts which can be leveraged for sentiment analysis. That proof of concept, while encouraging, was rather narrow. Here we consider tasks other than sentiment analysis, provide a more thorough comparison of Paragraph Vectors to other document modelling algorithms such as Latent Dirichlet Allocation, and evaluate performance of the method as we vary the dimensionality of the learned representation. We benchmarked the models on two document similarity data sets, one from Wikipedia, one from arXiv. We observe that the Paragraph Vector method performs significantly better than other methods, and propose a simple improvement to enhance embedding quality. Somewhat surprisingly, we also show that much like word embeddings, vector operations on Paragraph Vectors can perform useful semantic results.
中文速览
把文档表示成稠密向量（Paragraph Vectors，段落向量）是近年提出的无监督方法，但它在情感分析之外的任务上表现如何、对超参数是否敏感，此前几乎没有系统评估。作者在Wikipedia（约449万篇文章）和arXiv（约88.6万篇论文）两个语料库上，将段落向量与LDA、TF-IDF词袋等方法进行了全面对比，并提出了一个简单改进：在训练段落向量的同时联合训练词向量，可以明显提升表示质量。实验结果显示，段落向量在文档相似度评测任务上总体优于LDA，对向量维度的变化也更加稳健；更出乎意料的是，段落向量和词向量一样支持语义向量运算，例如用"Lady Gaga的向量 − '美国' + '日本'"就能找到对应的日本明星。这项工作表明，段落向量是大规模语料库语义检索、智能导航等应用的有力工具，为文档表示学习的实用价值提供了更充分的证据。
原文 arXiv:1507.07998;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1507.07998v1