Unsupervised Learning of Visual Representations using Videos
Xiaolong Wang, Abhinav Gupta Robotics Institute, Carnegie Mellon University
Abstract
Is strong supervision necessary for learning a good visual representation? Do we really need millions of semantically-labeled images to train a Convolutional Neural Network (CNN)? In this paper, we present a simple yet surprisingly powerful approach for unsupervised learning of CNN. Specifically, we use hundreds of thousands of unlabeled videos from the web to learn visual representations. Our key idea is that visual tracking provides the supervision. That is, two patches connected by a track should have similar visual representation in deep feature space since they probably belong to the same object or object part. We design a Siamese-triplet network with a ranking loss function to train this CNN representation. Without using a single image from ImageNet, just using 100K unlabeled videos and the VOC 2012 dataset, we train an ensemble of unsupervised networks that achieves $~{}52\%$ mAP (no bounding box regression). This performance comes tantalizingly close to its ImageNet-supervised counterpart, an ensemble which achieves a mAP of $~{}54.4\%$ . We also show that our unsupervised network can perform competitively in other tasks such as surface-normal estimation.
中文速览
用视频里的目标跟踪信号来替代人工标注，从而实现卷积神经网络（CNN）的无监督预训练——这正是本文要解决的核心问题。具体做法是：从网络上收集10万段无标注视频，利用跟踪算法提取数百万对"同一物体在不同帧"的图像块，再随机抽取第三个不相关的图像块，构成三元组（triplet），然后用孪生三元组网络（Siamese-triplet network）和排序损失（ranking loss）训练AlexNet，让跟踪对在特征空间中比随机对更相近。最终，在完全不使用ImageNet标注图片的前提下，只借助VOC 2012的标注进行微调，该方法在目标检测任务上达到了约52% mAP，与有ImageNet监督预训练的对应模型（约54.4% mAP）非常接近，在表面法向估计等任务上也表现相当。这项工作首次证明，无监督预训练的CNN可以逼近强监督方案的性能，说明海量语义标注并非学习优质视觉表示的必要条件，为降低深度学习对人工标注的依赖开辟了新路径。
原文 arXiv:1505.00687;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1505.00687v2