SB-MTL: Score-based Meta Transfer-Learning for Cross-Domain Few-Shot Learning
John Cai Princeton University Bill Cai Massachusetts Institute of Technology Shen Sheng Mei Pensees Pte Ltd
Abstract
While many deep learning methods have seen significant success in tackling the problem of domain adaptation and few-shot learning separately, far fewer methods are able to jointly tackle both problems in Cross-Domain Few-Shot Learning (CD-FSL). This problem is exacerbated under sharp domain shifts that typify common computer vision applications. In this paper, we present a novel, flexible and effective method to address the CD-FSL problem. Our method, called Score-based Meta Transfer-Learning (SB-MTL), combines transfer-learning and meta-learning by using a MAML-optimized feature encoder and a score-based Graph Neural Network. First, we have a feature encoder with specific layers designed to be fine-tuned. To do so, we apply a first-order MAML algorithm to find good initializations. Second, instead of directly taking the classification scores after fine-tuning, we interpret the scores as coordinates by mapping the pre-softmax classification scores onto a metric space. Subsequently, we apply a Graph Neural Network to propagate label information from the support set to the query set in our score-based metric space. We test our model on the Broader Study of Cross-Domain Few-Shot Learn
中文速览
跨域小样本学习（Cross-Domain Few-Shot Learning）要同时应对"样本极少"和"领域差异巨大"两道难题，现有元学习方法在遭遇医学图像、卫星图像等与训练域差距悬殊的场景时会大幅掉点。作者提出了一种名为"基于分数的元迁移学习"（Score-based Meta Transfer-Learning，SB-MTL）的新方法：先用一阶MAML算法为ResNet最后几层找到易于微调的初始化参数，再把微调后输出的预softmax分类分数映射到一个度量空间，最后用图神经网络（Graph Neural Network）在这个"分数度量空间"里把支持集的标签信息传播给查询样本完成再分类——分类分数本身与具体领域特征无关，因此这个度量空间天然具备跨域泛化能力。在涵盖自然图像、卫星图像和医学图像的BSCD-FSL基准上，SB-MTL相比此前最佳迁移学习方法平均提升5.93%，相比最佳元学习方法提升14.28%，证明将"学会微调"与"基于图的度量学习"结合起来是应对大域偏移小样本场景的有效路径。
原文 arXiv:2012.01784;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2012.01784v1