Fast Sparse ConvNets
Erich Elsen Marat Dukhan Trevor Gale Karen Simonyan DeepMind Google Google DeepMind {eriche, maratek, tgale, These authors contributed equally to this work.Work done as part of the Google AI Residency
Abstract
Historically, the pursuit of efficient inference has been one of the driving forces behind research into new deep learning architectures and building blocks. Some recent examples include: the squeeze-and-excitation module [16], depthwise separable convolutions in Xception [4], and the inverted bottleneck in MobileNet v2 [36]. Notably, in all of these cases, the resulting building blocks enabled not only higher efficiency, but also higher accuracy, and found wide adoption in the field. In this work, we further expand the arsenal of efficient building blocks for neural network architectures; but instead of combining standard primitives (such as convolution), we advocate for the replacement of these dense primitives with their sparse counterparts. While the idea of using sparsity to decrease the parameter count is not new [43], the conventional wisdom is that this reduction in theoretical FLOPs does not translate into real-world efficiency gains. We aim to correct this misconception by introducing a family of efficient sparse kernels for ARM and WebAssembly, which we open-source for the benefit of the community as part of the XNNPACK [30] library. Equipped with our efficient implement
中文速览
在移动端和低功耗设备上，稀疏神经网络在理论上能大幅减少计算量，但长期以来业界普遍认为稀疏运算无法转化为实际推理加速，导致这一方向被忽视。针对这一痛点，研究者专门为ARM和WebAssembly平台设计了一套高效的稀疏矩阵乘法（SpMM）内核，核心思路是利用激活矩阵的稠密性批量处理多个空间位置，并通过L1缓存局部性和输出通道分块来最大化数据复用，相关代码已开源至XNNPACK库。基于这套内核，对MobileNet v1、MobileNet v2和EfficientNet进行梯度幅值剪枝（gradual magnitude pruning）后，稀疏模型在效率-精度曲线上全面超越对应的稠密基准，在骁龙835上推理速度提升1.3–2.4倍、参数量压缩逾2倍、FLOPs最多降低3倍，相当于跨越了MobileNet系列整整一代的进步。这项工作从工程和实验两个层面同时证明了稀疏性作为高效深度学习架构设计工具的可行性，有望推动稀疏方法在工业界的广泛落地。
原文 arXiv:1911.09723;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1911.09723v1