Transition-Based Dependency Parsing with Stack Long Short-Term Memory
Chris Dyer♣♠ Miguel Ballesteros♢♠ Wang Ling♠ Austin Matthews♠ Noah A. Smith♠ ♣Marianas Labs ♢NLP Group, Pompeu Fabra University ♠Carnegie Mellon University
Abstract
We propose a technique for learning representations of parser states in transition-based dependency parsers. Our primary innovation is a new control structure for sequence-to-sequence neural networks—the stack LSTM. Like the conventional stack data structures used in transition-based parsing, elements can be pushed to or popped from the top of the stack in constant time, but, in addition, an LSTM maintains a continuous space embedding of the stack contents. This lets us formulate an efficient parsing model that captures three facets of a parser’s state: (i) unbounded look-ahead into the buffer of incoming words, (ii) the complete history of actions taken by the parser, and (iii) the complete contents of the stack of partially built tree fragments, including their internal structures. Standard backpropagation techniques are used for training and yield state-of-the-art parsing performance.
中文速览
依存句法分析（dependency parsing）的核心难题在于如何充分表示解析器当时的完整状态，传统方法只能"看到"输入句子和栈顶的少量局部信息。这篇论文提出了一种叫做"栈式长短时记忆网络"（Stack LSTM）的新型神经网络结构，把普通栈数据结构的入栈/出栈操作融入了LSTM，使得模型能在常数时间内随时读取整个栈的连续向量表示。基于这一创新，作者用三个Stack LSTM分别编码待处理词语缓冲区、已建立的部分句法树、以及解析器的完整操作历史，再通过递归神经网络将局部句法片段组合成更大的结构，从而在每一步决策时都能"看全"解析器的全局状态。最终该系统在英语和中文依存句法分析任务上达到了当时的最优性能，既保持了线性时间复杂度的高效性，又大幅扩展了模型所能利用的上下文信息，为后续基于神经网络的句法分析研究奠定了重要基础。
原文 arXiv:1505.08075;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1505.08075v1