Deep Perm-Set Net: Learn to predict sets with unknown permutation and cardinality using deep neural networks
Hamid Rezatofighi, Farbod T. Motlagh, Qinfeng Shi、Ian Reid School of Computer Science University of Adelaide, Australia、Roman Kaskman, Daniel Cremers、Laura Leal-Taixé Department of Computer Science TU Munich, Germany
Abstract
Many real-world problems, e.g. object detection, have outputs that are naturally expressed as sets of entities. This creates a challenge for traditional deep neural networks which naturally deal with structured outputs such as vectors, matrices or tensors. We present a novel approach for learning to predict sets with unknown permutation and cardinality using deep neural networks. Specifically, in our formulation we incorporate the permutation as unobservable variable and estimate its distribution during the learning process using alternating optimization. We demonstrate the validity of this new formulation on two relevant vision problems: object detection, for which our formulation outperforms state-of-the-art detectors such as Faster R-CNN and YOLO, and a complex CAPTCHA test, where we observe that, surprisingly, our set based network acquired the ability of mimicking arithmetics without any rules being coded.
中文速览
深度神经网络天生只能处理向量、矩阵等有固定顺序的结构化输出，但现实中大量任务（比如目标检测）的输出本质上是一个"集合"——元素数量不定、彼此之间没有固定顺序。为了解决这个问题，作者提出了"深度排列集合网络"（Deep Perm-Set Network），把集合输出中未知的元素排列方式当作隐变量，在训练过程中通过交替优化来估计其分布，从而让前馈神经网络在训练和推理阶段都能真正处理无序输出。在目标检测任务上，这一框架无需非极大值抑制（NMS）等后处理启发式规则，即可端到端学习，在模拟数据和真实高遮挡场景下均超越了Faster R-CNN和YOLO v2；在一个逻辑型CAPTCHA识别任务中，网络甚至自发学会了模拟算术规则，尽管没有任何显式编码。这项工作的意义在于，它为所有输出天然是集合的问题提供了一个统一的、原则性的深度学习框架，有望替代目前各类任务中依赖启发式规则的临时解决方案。
原文 arXiv:1805.00613;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1805.00613v4