HAQ: Hardware-Aware Automated Quantization with Mixed Precision
Kuan Wang∗, Zhijian Liu∗, Yujun Lin∗, Ji Lin, and Song Han {kuanwang, zhijian, yujunlin, jilin, Massachusetts Institute of Technology
Abstract
Model quantization is a widely used technique to compress and accelerate deep neural network (DNN) inference. Emergent DNN hardware accelerators begin to support mixed precision (1-8 bits) to further improve the computation efficiency, which raises a great challenge to find the optimal bitwidth for each layer: it requires domain experts to explore the vast design space trading off among accuracy, latency, energy, and model size, which is both time-consuming and sub-optimal. There are plenty of specialized hardware for neural networks, but little research has been done for specialized neural network optimization for a particular hardware architecture. Conventional quantization algorithm ignores the different hardware architectures and quantizes all the layers in a uniform way. In this paper, we introduce the Hardware-Aware Automated Quantization (HAQ) framework which leverages the reinforcement learning to automatically determine the quantization policy, and we take the hardware accelerator’s feedback in the design loop. Rather than relying on proxy signals such as FLOPs and model size, we employ a hardware simulator to generate direct feedback signals (latency and energy) to the RL
中文速览
量化（Quantization）是让神经网络在手机、机器人等设备上跑得更快、更省电的主流方法，但新一代芯片支持每层用不同位宽（混合精度）后，如何给每一层分配最合适的位宽就成了一个搜索空间比宇宙粒子数还大的难题，靠人工经验完全不够用。为此，作者提出了 HAQ（Hardware-Aware Automated Quantization）框架，用强化学习（RL）自动为每一层决定权重和激活值的位宽，并直接把硬件加速器的延迟和能耗数据作为反馈信号传给智能体，而不是用 FLOPs 这类粗糙的替代指标。实验结果表明，与统一 8 位量化相比，HAQ 在精度几乎不损失的情况下将推理延迟降低了 1.4–1.95 倍、能耗降低了约 1.9 倍，而且针对边缘端和云端等不同硬件架构分别生成的最优策略差异显著，证明"一套方案通吃所有硬件"的思路本身就是次优的。这项工作的价值在于：它把量化策略搜索从依赖人类专家的手工活变成了可自动化、可针对具体硬件定制的系统性方法，为实际部署高效神经网络提供了切实可行的工程路径。
原文 arXiv:1811.08886;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1811.08886v3