Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction
Chunting Zhou Graham Neubig Affiliation: Language Technologies Institute Affiliation: Carnegie Mellon University Email:
Abstract
Labeled sequence transduction is a task of transforming one sequence into another sequence that satisfies desiderata specified by a set of labels. In this paper we propose multi-space variational encoder-decoders, a new model for labeled sequence transduction with semi-supervised learning. The generative model can use neural networks to handle both discrete and continuous latent variables to exploit various features of data. Experiments show that our model provides not only a powerful supervised framework but also can effectively take advantage of the unlabeled data. On the SIGMORPHON morphological inflection benchmark, our model outperforms single-model state-of-art results by a large margin for the majority of languages.11 1 An implementation of our model are available at https://github.com/violet-zct/MSVED-morph-reinflection.
中文速览
形态再变形等带标签的序列转换任务，需要根据输入词和目标标签生成符合要求的输出，但现实中有标签数据稀缺、无标签词形却很多的问题。论文提出多空间变分编码器—解码器（MSVED），用连续潜变量表示词根等隐含信息，用离散潜变量表示形态标签，并通过变分自编码与半监督学习同时利用有标签和无标签数据。实验表明，该模型在SIGMORPHON形态变形基准的大多数语言上大幅超过单模型此前最佳结果，加入无标签数据后还能进一步提升，尤其适合形态变化复杂的语言。它的重要性在于把强大的神经序列生成能力与无标签数据利用结合起来，降低了对昂贵语言标注的依赖。
原文 arXiv:1704.01691;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1704.01691v2