MONAS: Multi-Objective Neural Architecture Search
\ANDChi-Hung Hsu,1 Shu-Huan Chang,1 Jhao-Hong Liang,1 Hsin-Ping Chou,1\ANDChun-Hao Liu,1 Shih-Chieh Chang,1 Jia-Yu Pan,2 Yu-Ting Chen,2\ANDWei Wei,2 Da-Cheng Juan2 1National Tsing-Hua University, Hsinchu, Taiwan 2Google, Mountain View, CA, USA {charles1994608, tommy610240, jayveeliang, alan.durant.chou, {jypan, yutingchen, wewei,
Abstract
Recent studies on neural architecture search have shown that automatically designed neural networks perform as good as expert-crafted architectures. While most existing works aim at finding architectures that optimize the prediction accuracy, these architectures may have complexity and is therefore not suitable being deployed on certain computing environment (e.g., with limited power budgets). We propose MONAS, a framework for Multi-Objective Neural Architectural Search that employs reward functions considering both prediction accuracy and other important objectives (e.g., power consumption) when searching for neural network architectures. Experimental results showed that, compared to the state-of-the-arts, models found by MONAS achieve comparable or better classification accuracy on computer vision applications, while satisfying the additional objectives such as peak power.
中文速览
神经网络架构搜索（Neural Architecture Search, NAS）通常只追求预测精度最高，却忽略了功耗、能耗等硬件约束，导致搜索出的模型难以部署在手机等资源受限的设备上。为此，研究者提出了MONAS框架，用强化学习驱动一个循环神经网络自动搜索架构，同时将分类精度和峰值功耗、能量消耗等多个目标共同纳入奖励函数，让搜索过程自动向满足约束的区域聚焦；在需要处理更大搜索空间时，还通过权重共享技术扩展为MONAS-S，搜索空间扩大10²²倍、速度提升3倍。在CIFAR-10图像分类任务上，MONAS搜索出的模型不仅达到甚至超越了人类专家设计的CondenseNet的分类精度，同时还满足了峰值功耗限制，实现了精度与能耗的双重优化。这项工作的意义在于，它提供了一套通用、可定制的多目标架构搜索方案，让深度学习模型的自动化设计能够真正兼顾实际部署环境的资源约束。
原文 arXiv:1806.10332;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1806.10332v2