Associative embeddings for large-scale knowledge transfer with self-assessment
Alexander Vezhnevets Vittorio Ferrari The University of Edinburgh Edinburgh, Scotland, UK
Abstract
We propose a method for knowledge transfer between semantically related classes in ImageNet. By transferring knowledge from the images that have bounding-box annotations to the others, our method is capable of automatically populating ImageNet with many more bounding-boxes.The underlying assumption that objects from semantically related classes look alike is formalized in our novel Associative Embedding (AE) representation. AE recovers the latent low-dimensional space of appearance variations among image windows. The dimensions of AE space tend to correspond to aspects of window appearance (e.g. side view, close up, background). We model the overlap of a window with an object using Gaussian Processes (GP) regression, which spreads annotation smoothly through AE space. The probabilistic nature of GP allows our method to perform self-assessment, i.e. assigning a quality estimate to its own output. It enables trading off the amount of returned annotations for their quality. A large scale experiment on 219 classes and 0.5 million images demonstrates that our method outperforms state-of-the-art methods and baselines for object localization. Using self-assessment we can automatically ret
中文速览
ImageNet 里只有约10%的图片带有目标框标注，想把剩下那些图片也自动标上框是个很难的问题。这篇论文提出用一种叫"关联嵌入"（Associative Embedding，AE）的低维表示，把图像窗口从高维视觉特征（HOG、词袋）压缩到仅3个维度，让语义相近类别（比如袋鼠和考拉）的外观变化规律得以共享；再用高斯过程（Gaussian Process，GP）回归，把有标注图片上窗口与目标框的重叠程度"平滑传播"到没有标注的图片上，同时天然输出预测的不确定性，实现自我质量评估。在包含219个类别、约50万张图片的大规模实验中，该方法的目标定位精度超过了当时最先进的方法，并且借助自我评估机制可以自动筛选出其中51%的图片返回高质量标注（与人工标注的平均重叠率达71%）。这项工作的意义在于，它提供了一条低成本、可信赖地扩充大规模数据集标注的路径，为检测、跟踪、姿态估计等下游任务提供更多可靠训练数据。
原文 arXiv:1312.3240;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1312.3240v2