MultiGrain: a unified image embedding for classes and instances
Maxim Berman Hervé Jégou Andrea Vedaldi Iasonas Kokkinos Matthijs Douze ESAT–PSI, KU Leuven Facebook AI Research Did this work during an internship at Facebook AI Research.
Abstract
MultiGrain is a network architecture producing compact vector representations that are suited both for image classification and particular object retrieval. It builds on a standard classification trunk. The top of the network produces an embedding containing coarse and fine-grained information, so that images can be recognized based on the object class, particular object, or if they are distorted copies. Our joint training is simple: we minimize a cross-entropy loss for classification and a ranking loss that determines if two images are identical up to data augmentation, with no need for additional labels. A key component of MultiGrain is a pooling layer that takes advantage of high-resolution images with a network trained at a lower resolution.
中文速览
图像识别领域长期存在"分类"、"实例检索"和"版权副本检测"三类任务各自为战、需要独立表示的痛点，MultiGrain 提出用一个统一的紧凑向量嵌入同时搞定这三件事。核心做法是在标准 ResNet 主干之上引入广义均值池化层（Generalized Mean Pooling，GeM），并以交叉熵分类损失和对比排序损失联合训练，其中实例识别信号完全来自数据增强生成的图像对，无需额外标注；同时，每个训练批次包含同一图像的多个增强版本（Repeated Augmentation），进一步提升泛化能力。实验结果显示，仅用 ImageNet 数据从头训练的 ResNet-50，MultiGrain 在 top-1 分类准确率上达到 79.4%，比当时最优的 AutoAugment 方法高出 1.8 个百分点，同时在图像检索基准上也与专门针对检索任务调优的方法持平。这项工作的重要意义在于，它证明分类与检索目标可以相互促进而非对立，用一套嵌入向量完成多粒度识别既节省存储和计算资源，也为构建通用视觉特征提供了新思路。
原文 arXiv:1902.05509;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1902.05509v2