Comparative Study of CNN and RNN for Natural Language Processing
Wenpeng Yin†, Katharina Kann†, Mo Yu‡ Hinrich Schütze† †CIS, LMU Munich, Germany ‡IBM Research, USA
Abstract
Deep neural networks (DNNs) have revolutionized the field of natural language processing (NLP). Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), the two main types of DNN architectures, are widely explored to handle various NLP tasks. CNN is supposed to be good at extracting position-invariant features and RNN at modeling units in sequence. The state-of-the-art on many NLP tasks often switches due to the battle of CNNs and RNNs. This work is the first systematic comparison of CNN and RNN on a wide range of representative NLP tasks, aiming to give basic guidance for DNN selection.
中文速览
选CNN还是RNN一直是NLP研究者绕不开的实践难题，却从未有人系统地做过跨任务的横向比较。这项工作在情感分类、关系分类、文本蕴含、答案选择、问题-关系匹配、路径查询回答和词性标注七类任务上，对CNN、GRU和LSTM进行了统一控制变量的公平对比实验。结果发现，RNN在需要理解整句语义或长距离依赖的任务上更占优势，而CNN在只需识别关键短语的任务（如部分情感检测和答案匹配）中反而更强，两者提供的是互补信息而非谁全面碾压谁。此外实验还揭示了一个实用规律：调整学习率对性能的影响相对平稳，而隐层大小和批量大小的变化会引起性能大幅波动，因此这两个超参数的调优比架构选择本身更为关键。这项系统性比较为从业者在面对具体NLP问题时如何选择基础网络架构提供了有据可查的经验指导。
原文 arXiv:1702.01923;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1702.01923v1