Artificial General Intelligence for Medical Imaging Analysis
Xiang Li*, Lin Zhao*, Lu Zhang*, Zihao Wu, Zhengliang Liu, Hanqi Jiang, Chao Cao, Shaochen Xu, Yiwei Li, Haixing Dai, Yixuan Yuan, Jun Liu, Gang Li, Dajiang Zhu, Pingkun Yan, Quanzheng Li, Wei Liu, Tianming Liu , and Dinggang Shen *These authors contributed equally to this work.(Corresponding authors: Xiang Li, Tianming Liu, Dinggang Shen)Xiang Li and Quanzheng Li are with the Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston 02115, USA. (e-mail: Zhang, Chao Cao, and Dajiang Zhu are with the Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington 76019, USA. (e-mail: and Zhao, Zihao Wu, Zhengliang Liu, Shaochen Xu, Yiwei Li, Haixing Dai and Tianming Liu are with the School of Computing, The University of Georgia, Athens 30602, USA. (e-mail: {zihao.wu1,zl18864,lin.zhao, hd54134, Jiang is with the School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China; College of Engineering, The University of Georgia, Athens 30602, USA. (e-mail: Yuan is with the Department of Electronic Engineering, Chinese University of Hong Kong, Hong Kong. (e-mail: Liu is with the Department of Radiology, Second Xiangya Hospital, Changsha 410011, China. (e-mail: Li is with the Department of Radiology at the University of North Carolina at Chapel Hill, Chapel Hill 27599, USA. (e-mail: Yan is with the Department of Biomedical Engineering at Rensselaer Polytechnic Institute, Troy, New York 12180, USA. (e-mail: Liu is with the Department of Radiation Oncology, Mayo Clinic, Scottsdale 85259, USA. (e-mail: Shen is with the School of Biomedical Engineering, ShanghaiTech University, Shanghai 201210, China; Shanghai United Imaging Intelligence Co., Ltd., Shanghai 200230, China; Shanghai Clinical Research and Trial Center, Shanghai, 201210, China. (e-mail:
Abstract
Large-scale Artificial General Intelligence (AGI) models, including Large Language Models (LLMs) such as ChatGPT/GPT-4, have achieved unprecedented success in a variety of general domain tasks. Yet, when applied directly to specialized domains like medical imaging, which require in-depth expertise, these models face notable challenges arising from the medical field’s inherent complexities and unique characteristics. In this review, we delve into the potential applications of AGI models in medical imaging and healthcare, with a primary focus on LLMs, Large Vision Models, and Large Multimodal Models. We provide a thorough overview of the key features and enabling techniques of LLMs and AGI, and further examine the roadmaps guiding the evolution and implementation of AGI models in the medical sector, summarizing their present applications, potentialities, and associated challenges. In addition, we highlight potential future research directions, offering a holistic view on upcoming ventures. This comprehensive review aims to offer insights into the future implications of AGI in medical imaging, healthcare, and beyond.
中文速览
医疗影像诊断长期依赖专科医生的专业知识，而通用大模型（AGI/LLM）直接应用于该领域时，面临标注数据稀缺、数据隐私限制以及医学图像高度专业化等瓶颈。这篇综述系统梳理了将大型语言模型（Large Language Models, LLMs）、大型视觉模型（Large Vision Models）和大型多模态模型（Large Multimodal Models）迁移到医学影像与医疗健康领域的核心技术路径，涵盖数据增强与去隐私化、领域知识注入、模型结构适配等关键策略。综述显示，这些方法在医学图像分割、疾病诊断、报告生成乃至医学教育等任务上均展现出显著潜力，部分模型已在多个临床场景中验证了超越传统方法的表现。这项工作为研究者提供了一张从技术原理到落地应用的完整路线图，对推动AGI真正融入临床实践、缓解医疗资源短缺具有重要参考价值。
原文 arXiv:2306.05480;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2306.05480v4