Supplementary Material: Surprising Effectiveness of Few-Image Unsupervised Feature Learning
First Author Institution1 Institution1 address Second Author Institution2 First line of institution2 address
Abstract
We show two plots of the ImageNet linear probes results (Table 2 of the paper) in Figure 1. On the left we plot performance per layer in absolute scale. Naturally the performance of the supervised model improves with depth, while all unsupervised models degrade after conv3. From the relative plot on the right, it becomes clear that with our training scheme, we can even slightly surpass supervised performance on conv1 presumably since our model is trained with sometimes very small patches, thus receiving an emphasis on learning good low level filters. The gap between all self-supervised methods and the supervised baseline increases with depth, due to the fact that the supervised model is trained for this specific task, whereas the self-supervised models learn from a surrogate task without labels.
中文速览
自监督视觉表征学习长期面临一个挑战：如何在不依赖人工标注的情况下，让模型学到接近甚至媲美有监督训练的图像特征。研究者提出了一套基于图像块（patch）增广的自监督训练方案，通过对不同尺度、颜色和形状分布的图像块进行对比学习，引导模型自主发现有意义的视觉表征。实验结果显示，该方法在浅层卷积特征（conv1）上甚至略微超越了有监督基线，原因在于训练时频繁使用极小图像块，迫使模型着重学习优质的低层滤波器；而在更深层特征上，与有监督模型的差距有所扩大，这符合有监督模型针对具体任务专项优化的预期。这一发现表明，精心设计的数据增广策略能够在无需标签的前提下大幅提升自监督模型的特征质量，为降低视觉识别任务对标注数据的依赖提供了有力支撑。
原文 arXiv:1904.13132;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1904.13132v3