A Data-scalable Transformer for Medical Image Segmentation: Architecture, Model Efficiency, and Benchmark
Yunhe Gao, Mu Zhou, Di Liu, Zhennan Yan, Shaoting Zhang, and Dimitris N. Metaxas Y. Gao, D. Liu and D. Metaxas are with the Computer Science Department, Rutgers University, Piscataway, NJ, USA. M. Zhou and Z. Yan are with SenseBrain Research, Princeton, NJ, USA.S. Zhang is with Shanghai Artificial Intelligence Laboratory, Shanghai, China.Manuscript received on October 15, 2022. Major revision received on April 4, 2023.
Abstract
Transformers have demonstrated remarkable performance in natural language processing and computer vision. However, existing vision Transformers struggle to learn from limited medical data and are unable to generalize on diverse medical image tasks. To tackle these challenges, we present MedFormer, a data-scalable Transformer designed for generalizable 3D medical image segmentation. Our approach incorporates three key elements: a desirable inductive bias, hierarchical modeling with linear-complexity attention, and multi-scale feature fusion that integrates spatial and semantic information globally. MedFormer can learn across tiny- to large-scale data without pre-training. Comprehensive experiments demonstrate MedFormer’s potential as a versatile segmentation backbone, outperforming CNNs and vision Transformers on seven public datasets covering multiple modalities (e.g., CT and MRI) and various medical targets (e.g., healthy organs, diseased tissues, and tumors). We provide public access to our models and evaluation pipeline, offering solid baselines and unbiased comparisons to advance a wide range of downstream clinical applications.
中文速览
医学图像自动分割对临床诊断和治疗至关重要，但现有视觉Transformer模型要么数据饥渴、需要大规模预训练，要么计算复杂度过高，难以处理三维高分辨率医学影像，泛化能力也不尽如人意。为此，研究者提出了MedFormer——一种专为三维医学图像分割设计的可扩展混合Transformer架构，其核心创新包括：在注意力投影和前馈网络中引入深度可分离卷积以植入有利的归纳偏置、设计双向多头注意力（B-MHA）将自注意力的二次复杂度降至线性级别同时保留全局建模能力、以及一套融合空间与语义信息的多尺度特征融合机制。在涵盖CT和MRI多种模态、跨越极小规模到大规模数据的七个公开数据集上，MedFormer无需任何预训练权重即可从头训练，并在分割精度上全面超越主流CNN和视觉Transformer基线模型。这项工作为医学图像分割提供了一个真正通用、数据高效且可公开复现的强力基准，有望推动广泛的临床下游应用落地。
原文 arXiv:2203.00131;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2203.00131v5