When Are Tree Structures Necessary for Deep Learning of Representations?
Jiwei Li1, Minh-Thang Luong1, Dan Jurafsky1 and Eduard Hovy2 1Computer Science Department, Stanford University, Stanford, CA 94305 2Language Technology Institute, Carnegie Mellon University, Pittsburgh, PA 15213
Abstract
Recursive neural models, which use syntactic parse trees to recursively generate representations bottom-up, are a popular architecture. But there have not been rigorous evaluations showing for exactly which tasks this syntax-based method is appropriate. In this paper we benchmark recursive neural models against sequential recurrent neural models (simple recurrent and LSTM models), enforcing apples-to-apples comparison as much as possible. We investigate 4 tasks: (1) sentiment classification at the sentence level and phrase level; (2) matching questions to answer-phrases; (3) discourse parsing; (4) semantic relation extraction (e.g., component-whole between nouns).
中文速览
递归神经网络（recursive neural model）和循环神经网络（recurrent neural model）谁更适合处理自然语言任务，这个问题长期缺乏严格的对比实验。本文选取情感分类、问答匹配、话语分析、语义关系抽取四项任务，在尽量公平的条件下对两类模型进行系统性基准测试。结果显示，依赖句法解析树的递归模型只在少数场景下真正占优——主要是语义关系抽取这类需要跨越较长距离关联关键词的任务，且优势集中于较长的句子；在大多数任务上，循环模型（包括LSTM）表现与之相当甚至更好。进一步地，作者提出一种简单改进：用标点符号将长句切分成类似子句的片段分别编码再合并，使循环模型在长句上逼近递归模型的性能，同时省去对句法解析的依赖。这一发现帮助研究者厘清了两类模型各自的适用边界，也为在不依赖解析器的前提下提升循环模型性能指明了方向。
原文 arXiv:1503.00185;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1503.00185v5