Massively Multilingual Word Embeddings
Waleed Ammar♢ George Mulcaire♡ Yulia Tsvetkov♢ Guillaume Lample♢ Chris Dyer♢ Noah A. Smith♡ ♢School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA ♡Computer Science、Engineering, University of Washington, Seattle, WA, USA
Abstract
We introduce new methods for estimating and evaluating embeddings of words in more than fifty languages in a single shared embedding space. Our estimation methods, multiCluster and multiCCA, use dictionaries and monolingual data; they do not require parallel data. Our new evaluation method, multiqvec-cca, is shown to correlate better than previous ones with two downstream tasks (text categorization and parsing). We also describe a web portal for evaluation that will facilitate further research in this area, along with open-source releases of all our methods.
中文速览
多语言词向量（multilingual word embeddings）的训练和评估一直缺乏系统化方案，这篇论文同时解决这两个问题：作者提出了 multiCluster 和 multiCCA 两种新方法，只需要单语语料和双语词典，无需平行语料，就能把 59 种语言的词向量映射到同一个共享空间。评估方面，他们将单语评估指标 qvec 扩展为 multiqvec-cca，修正了原方法对向量基变换不稳定、分数随维度增加而虚高的缺陷，并通过实验证明新指标与文本分类、依存句法分析两项下游任务的相关性优于已有指标。研究还配套上线了一个开放评估平台，让任何人都能方便地对自己的多语言词向量进行全套测试，大幅降低了该领域后续研究的门槛。
原文 arXiv:1602.01925;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1602.01925v2