XNLI: Evaluating Cross-lingual Sentence Representations
Alexis Conneau, Guillaume Lample, Ruty Rinott, Holger Schwenk, Ves Stoyanov Facebook AI \ANDAdina Williams, Samuel R. Bowman New York University
Abstract
State-of-the-art natural language processing systems rely on supervision in the form of annotated data to learn competent models. These models are generally trained on data in a single language (usually English), and cannot be directly used beyond that language. Since collecting data in every language is not realistic, there has been a growing interest in cross-lingual language understanding (XLU) and low-resource cross-language transfer. In this work, we construct an evaluation set for XLU by extending the development and test sets of the Multi-Genre Natural Language Inference Corpus (MultiNLI) to 15 languages, including low-resource languages such as Swahili and Urdu. We hope that our dataset, dubbed XNLI, will catalyze research in cross-lingual sentence understanding by providing an informative standard evaluation task. In addition, we provide several baselines for multilingual sentence understanding, including two based on machine translation systems, and two that use parallel data to train aligned multilingual bag-of-words and LSTM encoders. We find that XNLI represents a practical and challenging evaluation suite, and that directly translating the test data yields the best pe
中文速览
跨语言自然语言推理（Natural Language Inference，NLI）领域长期缺乏覆盖多语言的大规模评测基准，导致研究人员难以衡量模型在英语以外语言上的泛化能力。为此，研究团队将英语多题材自然语言推理语料库（MultiNLI）的开发集和测试集扩展到包括斯瓦希里语、乌尔都语在内的15种语言，构建了名为XNLI的跨语言评测集，共产生112,500对人工标注的句子对。他们同时提供了多条基线方法，包括用机器翻译将测试数据译回英语再分类，以及利用平行语料对齐多语言词袋和LSTM编码器，结果发现直接翻译测试集的方案在现有基线中表现最优。XNLI填补了句子级跨语言理解评测的空白，为推动多语言文本表示研究提供了一个实用且颇具挑战性的标准平台。
原文 arXiv:1809.05053;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1809.05053v1