Segmental Recurrent Neural Networks
Lingpeng Kong, Chris Dyer School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213, USA {lingpenk,、Noah A. Smith Computer Science、Engineering University of Washington Seattle, WA 98195, USA
Abstract
We introduce segmental recurrent neural networks (SRNNs) which define, given an input sequence, a joint probability distribution over segmentations of the input and labelings of the segments. Representations of the input segments (i.e., contiguous subsequences of the input) are computed by encoding their constituent tokens using bidirectional recurrent neural nets, and these “segment embeddings” are used to define compatibility scores with output labels. These local compatibility scores are integrated using a global semi-Markov conditional random field. Both fully supervised training—in which segment boundaries and labels are observed—as well as partially supervised training—in which segment boundaries are latent—are straightforward. Experiments on handwriting recognition and joint Chinese word segmentation/POS tagging show that, compared to models that do not explicitly represent segments such as BIO tagging schemes and connectionist temporal classification (CTC), SRNNs obtain substantially higher accuracies.
中文速览
手写字符识别和中文分词这类任务，既要把输入序列切分成片段，又要给每个片段打标签，传统方法要么用BIO标注绕开显式切分，要么用CTC隐式对齐，都没有直接对"片段"本身建模。论文提出了分段循环神经网络（Segmental RNN，SRNN），用双向LSTM把输入的每一段连续子序列编码成一个向量表示，再通过半马尔可夫条件随机场（semi-Markov CRF）对所有可能的切分和标签联合打分，动态规划保证推断和训练都在多项式时间内完成，并同时支持边界已知的全监督和边界未知的半监督两种训练方式。在在线手写识别和中文分词/词性标注两个任务上，SRNN均显著超过了CTC和BIO基线，证明即便下游任务并不要求输出切分结果，显式对片段建模本身也能带来实质性的精度提升。
原文 arXiv:1511.06018;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1511.06018v2