Graph-RISE: Graph-Regularized Image Semantic Embedding
Da-Cheng Juan, Chun-Ta Lu, Zhen Li, Futang Peng, Aleksei Timofeev, Yi-Ting Chen, Yaxi Gao, Tom Duerig, Andrew Tomkins, Sujith Ravi Google AIMountain ViewCA
Abstract
Learning image representations to capture fine-grained semantics has been a challenging and important task enabling many applications such as image search and clustering. In this paper, we present Graph-Regularized Image Semantic Embedding (Graph-RISE), a large-scale neural graph learning framework that allows us to train embeddings to discriminate an unprecedented $\mathcal{O}(40M)$ ultra-fine-grained semantic labels. Graph-RISE outperforms state-of-the-art image embedding algorithms on several evaluation tasks, including image classification and triplet ranking. We provide case studies to demonstrate that, qualitatively, image retrieval based on Graph-RISE effectively captures semantics and, compared to the state-of-the-art, differentiates nuances at levels that are closer to human-perception.
中文速览
如何让机器理解图片语义、并以接近人类直觉的方式判断两张图片"像不像"，一直是图像搜索和聚类领域的核心难题。谷歌研究团队提出了 Graph-RISE（图正则化图像语义嵌入，Graph-Regularized Image Semantic Embedding）框架：先把图像嵌入学习转化为一个超大规模分类任务——以 2.6 亿张图片、约 4000 万个搜索查询词作为类别标签进行训练，再引入神经图学习（neural graph learning），用用户"共同点击"和"相似图片点击"构建图结构，将图像间的相似关系作为正则化信号融入训练，弥补单纯分类标签无法捕捉的细粒度视觉关系。实验结果显示，Graph-RISE 在 ImageNet 上的 kNN Top-1 准确率约为现有最优方法的 2 倍，在 iNaturalist 上更提升超过 5 倍，并在三元组排序等多项评测中全面超越基准。这项工作的重要性在于，它首次在如此大规模的类别空间上验证了分类驱动的嵌入学习范式，并通过图正则化将隐含在用户行为中的人类感知信号系统性地注入模型，使图像检索真正逼近人眼对细微差异的分辨能力。
原文 arXiv:1902.10814;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1902.10814v1