Convolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-level
Rie Johnson RJ Research Consulting, NY, USA Tong Zhang Rutgers University, NJ, USA
Abstract
This paper reports the performances of shallow word-level convolutional neural networks (CNN), our earlier work (2015) [3, 4], on the eight datasets with relatively large training data that were used for testing the very deep character-level CNN in Conneau et al. (2016) [1]. Our findings are as follows. The shallow word-level CNNs achieve better error rates than the error rates reported in [1] though the results should be interpreted with some consideration due to the unique pre-processing of [1]. The shallow word-level CNN uses more parameters and therefore requires more storage than the deep character-level CNN; however, the shallow word-level CNN computes much faster.
中文速览
用浅层词级卷积神经网络（word-level CNN）与超深字符级卷积神经网络（char-level CNN）做文本分类，谁更好？作者把自己2015年提出的浅层词级CNN拿到Conneau等人2016年测试深层字符级CNN所用的八个大规模数据集上重新跑了一遍，发现浅层词级CNN在所有数据集上的错误率均低于深层字符级CNN，加入"双视图嵌入"（tv-embedding）后优势更加明显。虽然词级模型因词汇表庞大而需要更多存储空间，但由于网络层数极少、天然适合并行计算，训练和推理速度远快于那个29层的字符级深网络。这一结果表明，在处理大规模文本分类任务时，结构简单的浅层词级CNN不仅精度上不吃亏，还能在速度上占据显著优势，为实际应用提供了一个高效可行的替代方案。
原文 arXiv:1609.00718;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1609.00718v1