Generalizing from a Few Examples: A Survey on Few-Shot Learning
Yaqing Wang 0000-0003-1457-1114 Department of Computer Science and EngineeringHong Kong University of Science and Technology Business Intelligence Lab and National Engineering Laboratory of Deep Learning Technology and ApplicationBaidu Research , Quanming Yao 0000-0001-8944-8618 4Paradigm Inc. , James T. Kwok 0000-0002-4828-8248 Department of Computer Science and EngineeringHong Kong University of Science and Technology and Lionel M. Ni 0000-0002-2325-6215 Department of Computer Science and EngineeringHong Kong University of Science and Technology
Abstract
Machine learning has been highly successful in data-intensive applications, but is often hampered when the data set is small. Recently, Few-Shot Learning (FSL) is proposed to tackle this problem. Using prior knowledge, FSL can rapidly generalize to new tasks containing only a few samples with supervised information. In this paper, we conduct a thorough survey to fully understand FSL. Starting from a formal definition of FSL, we distinguish FSL from several relevant machine learning problems. We then point out that the core issue in FSL is that the empirical risk minimizer is unreliable. Based on how prior knowledge can be used to handle this core issue, we categorize FSL methods from three perspectives: (i) data, which uses prior knowledge to augment the supervised experience; (ii) model, which uses prior knowledge to reduce the size of the hypothesis space; and (iii) algorithm, which uses prior knowledge to alter the search for the best hypothesis in the given hypothesis space. With this taxonomy, we review and discuss the pros and cons of each category. Promising directions, in the aspects of the FSL problem setups, techniques, applications and theories, are also proposed to prov
中文速览
小样本学习（Few-Shot Learning，FSL）要解决的是：当训练数据极其有限时，机器学习模型泛化能力急剧下降的问题——而人类只需看几个例子就能掌握新概念。这篇综述从FSL的正式定义出发，指出其核心难点在于样本太少导致经验风险最小化器不可靠，进而以"先验知识如何被利用"为主线，将现有方法归纳为三大类：通过数据增广扩充监督信息、通过模型设计压缩假设空间、通过算法改进优化搜索策略。作者系统梳理了每类方法的优缺点，并展望了问题设定、技术路线、应用场景和理论分析四个层面的未来方向。这项工作为FSL领域提供了迄今最为系统的分类框架，对理解现有方法、指导后续研究具有重要的参考价值。
原文 arXiv:1904.05046;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1904.05046v3