Fast Segment Anything
Xu Zhao 1,313{}^{1,3}start_FLOATSUPERSCRIPT 1 , 3 end_FLOATSUPERSCRIPT Wenchao Ding 1,212{}^{1,2}start_FLOATSUPERSCRIPT 1 , 2 end_FLOATSUPERSCRIPT Yongqi An 1,212{}^{1,2}start_FLOATSUPERSCRIPT 1 , 2 end_FLOATSUPERSCRIPT Yinglong Du 1,212{}^{1,2}start_FLOATSUPERSCRIPT 1 , 2 end_FLOATSUPERSCRIPT Tao Yu 1,212{}^{1,2}start_FLOATSUPERSCRIPT 1 , 2 end_FLOATSUPERSCRIPT Min Li 1,212{}^{1,2}start_FLOATSUPERSCRIPT 1 , 2 end_FLOATSUPERSCRIPT Ming Tang 1,212{}^{1,2}start_FLOATSUPERSCRIPT 1 , 2 end_FLOATSUPERSCRIPT Jinqiao Wang 1,2,3,41234{}^{1,2,3,4}start_FLOATSUPERSCRIPT 1 , 2 , 3 , 4 end_FLOATSUPERSCRIPT Institute of Automation, Chinese Academy of Sciences, Beijing, China11{}^{1}start_FLOATSUPERSCRIPT 1 end_FLOATSUPERSCRIPT School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT Objecteye Inc., Beijing, China33{}^{3}start_FLOATSUPERSCRIPT 3 end_FLOATSUPERSCRIPT Wuhan AI Research, Wuhan, China44{}^{4}start_FLOATSUPERSCRIPT 4 end_FLOATSUPERSCRIPT
Abstract
The recently proposed segment anything model (SAM) has made a significant influence in many computer vision tasks. It is becoming a foundation step for many high-level tasks, like image segmentation, image caption, and image editing. However, its huge computation costs prevent it from wider applications in industry scenarios. The computation mainly comes from the Transformer architecture at high-resolution inputs. In this paper, we propose a speed-up alternative method for this fundamental task with comparable performance. By reformulating the task as segments-generation and prompting, we find that a regular CNN detector with an instance segmentation branch can also accomplish this task well. Specifically, we convert this task to the well-studied instance segmentation task and directly train the existing instance segmentation method using only 1/50 of the SA-1B dataset published by SAM authors. With our method, we achieve a comparable performance with the SAM method at 50 $\times$ higher run-time speed. We give sufficient experimental results to demonstrate its effectiveness. The codes and demos will be released at https://github.com/CASIA-IVA-Lab/FastSAM.
中文速览
万物分割模型（Segment Anything Model，SAM）虽然能力强大，但其基于Transformer（ViT）的架构计算开销极大，难以在工业场景中实时部署。为此，研究者提出了FastSAM，将"分割一切"任务拆解为两个阶段：先用基于CNN的YOLOv8-seg完成全实例分割，再通过点、框、文本等提示词筛选出目标区域。FastSAM仅使用SA-1B数据集的2%进行训练，在MS COCO目标候选生成任务上比SAM高出1.2个百分点（AR1000达63.7），同时在单张RTX 3090上运行速度快50倍。这项工作证明了轻量级CNN模型在通用分割基础任务上的可行性，为SAM在实时工业应用中的落地提供了一条高效的替代路径。
原文 arXiv:2306.12156;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2306.12156v1