And the Bit Goes Down: Revisiting the Quantization of Neural Networks
Pierre Stock1,2, Armand Joulin1, Rémi Gribonval2, Benjamin Graham1, Hervé Jégou1 1Facebook AI Research, 2Univ Rennes, Inria, CNRS, IRISA
Abstract
In this paper, we address the problem of reducing the memory footprint of convolutional network architectures. We introduce a vector quantization method that aims at preserving the quality of the reconstruction of the network outputs rather than its weights. The principle of our approach is that it minimizes the loss reconstruction error for in-domain inputs. Our method only requires a set of unlabelled data at quantization time and allows for efficient inference on CPU by using byte-aligned codebooks to store the compressed weights. We validate our approach by quantizing a high performing ResNet-50 model to a memory size of $5$ MB ( $20\times$ compression factor) while preserving a top-1 accuracy of $76.1\%$ on ImageNet object classification and by compressing a Mask R-CNN with a $26\times$ factor.111Code and compressed models: https://github.com/facebookresearch/kill-the-bits.
中文速览
为了在手机、机器人等嵌入式设备上运行高精度卷积神经网络，需要大幅压缩模型体积，而现有方法往往只压缩权重本身，忽视了对网络输出质量的保护。这项工作提出了一种基于乘积量化（Product Quantization, PQ）的压缩方案，关键创新在于优化目标不是让压缩后的权重接近原始权重，而是让压缩后模型在真实输入数据上产生的激活值尽量接近原始模型——这通过一种加权 k-means 算法实现，且全程只需要少量无标注图片。为了避免逐层压缩时误差累积，作者还引入了逐层蒸馏（distillation）流程，让每层都以原始未压缩网络的输出为"老师"来校正量化误差。实验表明，该方法可将 ResNet-50 压缩至仅 5 MB（原模型的 1/20），同时在 ImageNet 上保持 76.1% 的 top-1 准确率，并在 Mask R-CNN 目标检测任务上实现 26 倍压缩，充分证明了这一思路在实际部署中的价值。
原文 arXiv:1907.05686;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1907.05686v5