Convolutional Neural Networks at Constrained Time Cost
Kaiming He Jian Sun Microsoft Research
Abstract
Though recent advanced convolutional neural networks (CNNs) have been improving the image recognition accuracy, the models are getting more complex and time-consuming. For real-world applications in industrial and commercial scenarios, engineers and developers are often faced with the requirement of constrained time budget. In this paper, we investigate the accuracy of CNNs under constrained time cost. Under this constraint, the designs of the network architectures should exhibit as trade-offs among the factors like depth, numbers of filters, filter sizes, etc. With a series of controlled comparisons, we progressively modify a baseline model while preserving its time complexity. This is also helpful for understanding the importance of the factors in network designs. We present an architecture that achieves very competitive accuracy in the ImageNet dataset (11.8% top-5 error, 10-view test), yet is 20% faster than “AlexNet” [14] (16.0% top-5 error, 10-view test).
中文速览
在实际工业场景中，深度卷积神经网络（CNN）虽然精度越来越高，但计算开销也越来越大，常常超出工程师可接受的时间预算。为此，作者在固定计算复杂度的前提下，通过一系列受控对比实验系统地研究了网络深度、宽度（滤波器数量）和滤波器尺寸之间的权衡关系——核心手段是"层替换"，即用复杂度相当的新层组合替换原有层，从而在保持计算量不变的同时探索不同结构对精度的影响。实验表明，在计算量受限时，增加网络深度（同时适当压缩宽度或缩小滤波器）带来的精度提升最为显著，但深度过深后收益趋于饱和。最终得到的模型在 ImageNet 上取得 11.8% 的 top-5 错误率（10视角测试），比 AlexNet 快 20% 的同时错误率低 4.2 个百分点，且只需单块 GPU 训练 3–4 天，为资源受限场景下的网络设计提供了清晰的实证指导。
原文 arXiv:1412.1710;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1412.1710v1