Accelerating Convolutional Neural Network by Exploiting Sparsity on GPUs
Weizhi Xu School of Information Science and Engineering, Shandong Normal UniversityJinanChina Dept. of Electrical、Computer Engineering, University of HoustonHoustonUSA , Yintai Sun School of Information Science and Engineering, Shandong Normal UniversityJinanChina , Shengyu Fan School of Information Science and Engineering, Shandong Normal UniversityJinanChina , Hui Yu School of Information Science and Engineering, Shandong Normal UniversityJinanChina Dept. of Electrical、Computer Engineering, University of HoustonHoustonUSA and Xin Fu Dept. of Electrical、Computer Engineering, University of HoustonHoustonUSA
Abstract
Convolution Neural Network (CNN) is an important deep learning method, which is widely used in many fields. However, it is very time consuming to implement CNN where convolution usually takes most of the time. There are many zero values in feature maps and filters, which leads to redundant calculations and memory accesses if dense methods are used to compute convolution. Many works recently make use of sparsity to skip the calculations for zero values to reduce the inference time of CNN. On the GPU platform, current works cannot fully exploit the sparsity of the feature map and achieve satisfactory performance. Therefore, we design a new parallel strategy to transform the feature map into a new storage format to avoid the redundant computation of zero-values on GPUs. Also considering the sparsity in the feature map, we propose a fused storage format to combine the convolution operation with the following pooling operation, in order to further improve the performance. We carry out experiments with mainstream CNN models and achieve better performance compared with cuDNN and cuSPARSE. For VGG-19, ResNet-50, DenseNet-121 and RegNetX-16GF, $1.97\times$ , $2.23\times$ , $2.74\times$ and
中文速览
卷积神经网络（CNN）推理中卷积运算极为耗时，而经过ReLU激活和网络剪枝之后，特征图（feature map）中往往存在大量零值，用传统稠密方法计算会产生大量无效乘加运算和冗余内存访问。针对这一问题，作者提出了两种新的稀疏存储格式：ECR（扩展压缩行格式）将特征图重新组织，使GPU在一次全局内存访问中就能完成扩展、压缩和稀疏矩阵计算，从而跳过零值运算；在此基础上进一步提出PECR格式，将卷积层与紧随其后的池化层融合到同一个GPU核中执行，减少了片外内存与片上内存之间以及CPU与GPU之间的数据传输开销。在VGG-19、ResNet-50、DenseNet-121和RegNetX-16GF四个主流模型上的实验表明，与cuDNN相比分别获得了1.97×、2.23×、2.74×和1.58×的加速比，也全面优于cuSPARSE。这项工作证明了在GPU上充分利用特征图稀疏性并融合相邻算子是加速CNN推理的有效途径，对工业界广泛使用的GPU深度学习推理优化具有直接参考价值。
原文 arXiv:1909.09927;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1909.09927v6