Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction

中文速览

形态再变形等带标签的序列转换任务，需要根据输入词和目标标签生成符合要求的输出，但现实中有标签数据稀缺、无标签词形却很多的问题。论文提出多空间变分编码器—解码器（MSVED），用连续潜变量表示词根等隐含信息，用离散潜变量表示形态标签，并通过变分自编码与半监督学习同时利用有标签和无标签数据。实验表明，该模型在SIGMORPHON形态变形基准的大多数语言上大幅超过单模型此前最佳结果，加入无标签数据后还能进一步提升，尤其适合形态变化复杂的语言。它的重要性在于把强大的神经序列生成能力与无标签数据利用结合起来，降低了对昂贵语言标注的依赖。

摘要

Labeled sequence transduction is a task of transforming one sequence into another sequence that satisfies desiderata specified by a set of labels. In this paper we propose multi-space variational encoder-decoders, a new model for labeled sequence transduction with semi-supervised learning. The generative model can use neural networks to handle both discrete and continuous latent variables to exploit various features of data. Experiments show that our model provides not only a powerful supervised framework but also can effectively take advantage of the unlabeled data. On the SIGMORPHON morphological inflection benchmark, our model outperforms single-model state-of-art results by a large margin for the majority of languages.11 1 An implementation of our model are available at https://github.com/violet-zct/MSVED-morph-reinflection.

术语表

labeled sequence transduction

带标签序列转换

multi-space variational encoder-decoders

多空间变分编码器—解码器

MSVED

MSVED

semi-supervised learning

半监督学习

supervised learning

监督学习

unsupervised learning

无监督学习

discrete latent variable

离散潜变量

continuous latent variable

连续潜变量

generative model

生成模型

recognition model

识别模型

variational autoencoder

变分自编码器

VAE

VAE

variational lower bound

变分下界

marginal log likelihood

边际对数似然

KL divergence

KL 散度

reparameterization trick

重参数化技巧

posterior inference

后验推断

auto-encoder

自编码器

morphological reinflection

形态再屈折

morphological inflection

形态屈折

lemma

词元

affix

词缀

neural sequence-to-sequence model

神经序列到序列模型

attentional encoder-decoder model

注意力编码器—解码器模型

SIGMORPHON

SIGMORPHON

lemma embedding

词元嵌入

multinomial distribution

多项分布