Effective Quantization Methods for Recurrent Neural Networks
Qinyao He, He Wen, Shuchang Zhou, Yuxin Wu, Cong Yao, Xinyu Zhou, Yuheng Zou Megvii Inc. {hqy, wenhe, zsc, wyx, yaocong, zxy,
Abstract
Reducing bit-widths of weights, activations, and gradients of a Neural Network can shrink its storage size and memory usage, and also allow for faster training and inference by exploiting bitwise operations. However, previous attempts for quantization of RNNs show considerable performance degradation when using low bit-width weights and activations. In this paper, we propose methods to quantize the structure of gates and interlinks in LSTM and GRU cells. In addition, we propose balanced quantization methods for weights to further reduce performance degradation. Experiments on PTB and IMDB datasets confirm effectiveness of our methods as performances of our models match or surpass the previous state-of-the-art of quantized RNN.
中文速览
循环神经网络（RNN）的权重和激活值位宽过高，导致存储占用大、计算速度慢，而现有的低位宽量化方案一旦压缩到2-bit就会出现明显的性能下滑。作者针对LSTM和GRU两种主流RNN单元，系统设计了门控结构与层间连接的量化方案，并提出"平衡量化"方法——利用权重绝对值的均值作为自适应阈值，使量化后的权重尽量均匀分布在各离散取值上，从而更充分地利用有限的表达空间。在Penn Treebank语言模型和IMDB情感分类两个任务上，4-bit量化模型的表现几乎与32-bit浮点模型持平，2-bit配置下也显著优于此前最优的量化RNN方法。这项工作填补了RNN量化研究的空白，为在算力受限设备上高效部署序列模型提供了可复现的方案。
原文 arXiv:1611.10176;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1611.10176v1