Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction
Chunting Zhou, Graham Neubig Language Technologies Institute Carnegie Mellon University
Abstract
Labeled sequence transduction is a task of transforming one sequence into another sequence that satisfies desiderata specified by a set of labels. In this paper we propose multi-space variational encoder-decoders, a new model for labeled sequence transduction with semi-supervised learning. The generative model can use neural networks to handle both discrete and continuous latent variables to exploit various features of data. Experiments show that our model provides not only a powerful supervised framework but also can effectively take advantage of the unlabeled data. On the SIGMORPHON morphological inflection benchmark, our model outperforms single-model state-of-art results by a large margin for the majority of languages.111An implementation of our model are available at https://github.com/violet-zct/MSVED-morph-reinflection.
中文速览
形态变形（morphological reinflection）任务要求模型把一个已屈折的词形连同目标语法标签一起转换成另一个词形，难点在于有标注的训练数据极为稀缺。作者提出了"多空间变分编码-解码器"（Multi-Space Variational Encoder-Decoder, MSVED），用连续隐变量捕捉词条（lemma）的语义信息，同时用离散隐变量对应各个形态标签，从而把有监督学习和半监督学习统一在同一个生成模型框架下——有标注时直接优化变分下界，无标注时则把标签当作离散隐变量进行推断，让模型也能从大量未标注的表层词形中学习。在 SIGMORPHON 形态变形基准测试上，MSVED 单模型在大多数语言上超越了此前最佳单模型，尤其在形态变化复杂的语言中优势明显，且半监督训练带来了进一步的显著提升。这一工作的意义在于为低资源形态丰富语言提供了一条实用的半监督路径，同时也为更广泛的"带标签序列转换"任务提供了通用的概率建模思路。
原文 arXiv:1704.01691;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1704.01691v2