Recurrent Neural Networks for Semantic Instance Segmentation
Anonymous ECCV submission Amaia Salvador1 Míriam Bellver2 Víctor Campos2 Manel Baradad1 Ferran Marques1 Jordi Torres2 Xavier Giro-i-Nieto1
Abstract
We present a recurrent model for semantic instance segmentation that sequentially generates binary masks and their associated class probabilities for every object in an image. Our proposed system is trainable end-to-end from an input image to a sequence of labeled masks and, compared to methods relying on object proposals, does not require post-processing steps on its output. We study the suitability of our recurrent model on three different instance segmentation benchmarks, namely Pascal VOC 2012, CVPPP Plant Leaf Segmentation and Cityscapes. Further, we analyze the object sorting patterns generated by our model and observe that it learns to follow a consistent pattern, which correlates with the activations learned in the encoder part of our network.
中文速览
语义实例分割（semantic instance segmentation）要求给图中每个目标同时标出精确的像素轮廓和类别标签，现有主流方法依赖候选框生成再筛选的两阶段流程，输出后还需非极大值抑制等后处理，整个系统并非真正的端到端训练。本文设计了一个循环神经网络模型，以单张图片为输入，用带跳跃连接的卷积LSTM解码器逐步输出每个目标的二值掩码与类别概率，并用匈牙利算法动态匹配预测与真值、省去任何后处理步骤，从而实现从像素到带类别标签的实例序列的完整端到端训练。在Pascal VOC 2012、植物叶片分割和Cityscapes三个基准上，该方法达到与同类序列化方案相当甚至更优的性能，同时还发现模型自发学会了与场景中目标相对位置高度一致的扫描顺序，揭示了循环解码器内部的空间推理规律。这项工作首次将语义实例分割统一为一个无需中间表示的序列预测问题，为后续端到端感知系统的设计提供了新的思路。
原文 arXiv:1712.00617;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1712.00617v4