Rethinking Co-design of Neural Architectures and Hardware Accelerators
Yanqi Zhou Xuanyi Dong Berkin Akin Mingxing Tan Daiyi Peng Tianjian Meng Amir Yazdanbakhsh Da Huang Ravi Narayanaswami James Laudon
Abstract
Neural architectures and hardware accelerators have been two driving forces for the progress in deep learning. Previous works typically attempt to optimize hardware given a fixed model architecture or model architecture given fixed hardware. And the dominant hardware architecture explored in this prior work is FPGAs. In our work, we target the optimization of hardware and software configurations on an industry-standard edge accelerator. We systematically study the importance and strategies of co-designing neural architectures and hardware accelerators. We make three observations: 1) the software search space has to be customized to fully leverage the targeted hardware architecture, 2) the search for the model architecture and hardware architecture should be done jointly to achieve the best of both worlds, and 3) different use cases lead to very different search outcomes. Our experiments show that the joint search method consistently outperforms previous platform-aware neural architecture search, manually crafted models, and the state-of-the-art EfficientNet on all latency targets by around 1% on ImageNet top-1 accuracy. Our method can reduce energy consumption of an edge accelerato
中文速览
深度学习的进步依赖神经网络架构与硬件加速器的共同演进，但以往研究要么针对固定硬件搜索最优模型，要么针对固定模型优化硬件，而且主要集中在FPGA等学术级设备上。这篇论文提出了一个名为NAHAS（神经架构与硬件加速器联合搜索）的框架，在工业级边缘加速器（EdgeTPU）上同时搜索神经网络结构参数和硬件配置参数，并针对该加速器特点重新设计了融合IBN（Fused Inverted Bottleneck）等更适配硬件的搜索空间。实验表明，联合搜索在ImageNet图像分类任务上，相比仅做平台感知神经架构搜索的MnasNet、EfficientNet及手工设计模型，在所有延迟目标上均提升约1%的Top-1精度，同等精度下还可将边缘加速器的能耗降低最多2倍。这项工作首次在高度优化的工业级加速器上验证了软硬件协同设计的有效性，为面向自动驾驶、移动端SoC等场景定制专用加速器提供了系统性方法论。
原文 arXiv:2102.08619;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2102.08619v1