Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints
Nikola Mrkšić1,2, Ivan Vulić1, Diarmuid Ó Séaghdha2, Ira Leviant3 Roi Reichart3, Milica Gašić1, Anna Korhonen1, Steve Young1,2 1 University of Cambridge 2 Apple Inc. 3 Technion, IIT
Abstract
We present Attract-Repel, an algorithm for improving the semantic quality of word vectors by injecting constraints extracted from lexical resources. Attract-Repel facilitates the use of constraints from mono- and cross-lingual resources, yielding semantically specialised cross-lingual vector spaces. Our evaluation shows that the method can make use of existing cross-lingual lexicons to construct high-quality vector spaces for a plethora of different languages, facilitating semantic transfer from high- to lower-resource ones. The effectiveness of our approach is demonstrated with state-of-the-art results on semantic similarity datasets in six languages. We next show that Attract-Repel-specialised vectors boost performance in the downstream task of dialogue state tracking (DST) across multiple languages. Finally, we show that cross-lingual vector spaces produced by our algorithm facilitate the training of multilingual DST models, which brings further performance improvements.
中文速览
如何让词向量更好地区分"近义词"与"反义词"，是分布式语义表示的老大难问题——Attract-Repel 算法专门为此而生，它从 WordNet 等词汇资源中抽取同义和反义约束，在训练时把同义词对"吸引"到向量空间的相近位置，同时把反义词对"排斥"到较远位置，从而对已有词向量做语义专门化后处理。更关键的是，该方法能同时利用单语和跨语言词汇资源，把高资源语言的语义知识迁移到资源匮乏的语言，一举生成高质量的多语言对齐向量空间。实验结果表明，Attract-Repel 在六种语言的语义相似度基准（SimLex-999、SimVerb-3500 等）上刷新了当时最优成绩，在对话状态追踪（dialogue state tracking）这一下游任务上也带来了显著提升，多语言联合模型更进一步拉高了性能上限。这项工作的意义在于，它提供了一条低成本、高效率的路径，让中低资源语言也能搭上高质量语义表示的顺风车。
原文 arXiv:1706.00374;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1706.00374v1