Rethinking floating point for deep learning
Jeff Johnson Facebook AI Research New York, NY
Abstract
Reducing hardware overhead of neural networks for faster or lower power inference and training is an active area of research. Uniform quantization using integer multiply-add has been thoroughly investigated, which requires learning many quantization parameters, fine-tuning training or other prerequisites. Little effort is made to improve floating point relative to this baseline; it remains energy inefficient, and word size reduction yields drastic loss in needed dynamic range. We improve floating point to be more energy efficient than equivalent bit width integer hardware on a 28 nm ASIC process while retaining accuracy in 8 bits with a novel hybrid log multiply/linear add, Kulisch accumulation and tapered encodings from Gustafson’s posit format. With no network retraining, and drop-in replacement of all math and float32 parameters via round-to-nearest-even only, this open-sourced 8-bit log float is within 0.9% top-1 and 0.2% top-5 accuracy of the original float32 ResNet-50 CNN model on ImageNet. Unlike int8 quantization, it is still a general purpose floating point arithmetic, interpretable out-of-the-box. Our 8/38-bit log float multiply-add is synthesized and power profiled at 28
中文速览
神经网络推理和训练的硬件加速通常依赖整数量化，但这类方案需要大量重新训练和调参，而直接缩减浮点数位宽又会损失关键动态范围。这篇论文提出了一种新型8位对数浮点格式，融合了三项关键设计：用对数域做乘法（避免硬件乘法器）、用Kulisch累加器在线性域做加法（消除累加误差）、以及借鉴Gustafson的posit格式实现"渐变精度"编码（让动态范围和精度自适应分配）。实验表明，不对网络做任何重新训练，只用"就近舍入"替换所有float32参数，该格式在ImageNet上跑ResNet-50的top-1精度仅比原始float32低0.9%；在28nm工艺下，其8位乘加单元功耗仅为同等位宽整数乘加的0.96倍、面积1.12倍，16位版本的功耗更只有IEEE 754 float16的59%。这项工作的价值在于：它在几乎不牺牲精度的前提下，提供了一种即插即用、保留通用浮点语义的高效数值格式，为推理和训练芯片设计提供了一条有别于整数量化的新路径。
原文 arXiv:1811.01721;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1811.01721v1