Bilinear CNNs for Fine-grained Visual Recognition
Tsung-Yu Lin Aruni RoyChowdhury Subhransu Maji T.-Y. Lin, A. RoyChowdhury, and S. Maji are with the College of Information and Computer Sciences, University of Massachusetts Amherst, USA. E-mails: {tsungyulin, arunirc,
Abstract
We present a simple and effective architecture for fine-grained visual recognition called Bilinear Convolutional Neural Networks (B-CNNs). These networks represent an image as a pooled outer product of features derived from two CNNs and capture localized feature interactions in a translationally invariant manner. B-CNNs belong to the class of orderless texture representations but unlike prior work they can be trained in an end-to-end manner. Our most accurate model obtains 84.1%, 79.4%, 86.9% and 91.3% per-image accuracy on the Caltech-UCSD birds [67], NABirds [64], FGVC aircraft [42], and Stanford cars [33] dataset respectively and runs at 30 frames-per-second on a NVIDIA Titan X GPU. We then present a systematic analysis of these networks and show that (1) the bilinear features are highly redundant and can be reduced by an order of magnitude in size without significant loss in accuracy, (2) are also effective for other image classification tasks such as texture and scene recognition, and (3) can be trained from scratch on the ImageNet dataset offering consistent improvements over the baseline architecture. Finally, we present visualizations of these models on various datasets usi
中文速览
细粒度视觉识别（fine-grained visual recognition）长期面临一个难题：如何在不依赖昂贵的部件标注的情况下，捕捉对象局部细节特征同时保持对位置和姿态的不变性。本文提出双线性卷积神经网络（Bilinear CNN，B-CNN），用两路CNN分别提取特征，在图像每个位置计算二者的外积（outer product）后再做全局汇聚，从而以无参数、无需部件标注的方式同时建模局部特征间的交互关系与空间不变性。该方法在CUB鸟类、NABirds、FGVC飞机、斯坦福汽车等主流细粒度数据集上分别达到84.1%、79.4%、86.9%和91.3%的准确率，超越了需要部件标注的方法，且在NVIDIA Titan X GPU上可达每秒30帧的实时速度。这项工作的意义在于，它将经典纹理表示统一纳入端到端可训练的深度学习框架，既解决了以往纹理特征无法联合优化的问题，也为材质、场景识别等更广泛任务提供了一种通用、高效的图像表示方案。
原文 arXiv:1504.07889;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1504.07889v6