Towards Expert-Level Medical Question Answering with Large Language Models
Karan Singhal Google Research, Tao Tu Google Research, Juraj Gottweis Google Research, Rory Sayres Google Research, Ellery Wulczyn Google Research, Le Hou Google Research, Kevin Clark Google Research, Stephen Pfohl Google Research, Heather Cole-Lewis Google Research, Darlene Neal Google Research, Mike Schaekermann Google Research, Amy Wang Google Research, Mohamed Amin Google Research, Sami Lachgar Google Research, Philip Mansfield Google Research, Sushant Prakash Google Research, Bradley Green Google Research, Ewa Dominowska Google Research, Blaise Aguera y Arcas Google Research, Nenad Tomasev DeepMind, Yun Liu Google Research, Renee Wong Google Research, Christopher Semturs Google Research, S. Sara Mahdavi Google Research, Joelle Barral Google Research, Dale Webster Google Research, Greg S. Corrado Google Research, Yossi Matias Google Research, Shekoofeh Azizi Google Research, Alan Karthikesalingam Google Research, Vivek Natarajan Google Research,
Abstract
Recent artificial intelligence (AI) systems have reached milestones in “grand challenges” ranging from Go to protein-folding. The capability to retrieve medical knowledge, reason over it, and answer medical questions comparably to physicians has long been viewed as one such grand challenge.
中文速览
医学问答一直是AI领域的重大挑战，此前最好的模型Med-PaLM在USMLE风格的医学执照考试题上仅得67.2%，与真正医生的水平仍有差距。谷歌研究团队在更强的基础大模型PaLM 2之上，结合医学领域微调和一种新的"集成精炼（ensemble refinement）"提示策略，开发出Med-PaLM 2，将MedQA得分提升至86.5%，超越前代逾19个百分点，并在多个主流医学问答基准上达到或刷新最优水平。更关键的是，在针对1066道消费者健康问题的人工评估中，医生评审者在九项临床实用指标里有八项更偏好Med-PaLM 2的回答而非同行医生的回答，在新构建的对抗性测试集上也远优于前代，低危害率从79.4%升至90.6%。这项工作表明，通用大语言模型经过适当的领域对齐后可以快速逼近甚至局部超越医生水平，为AI辅助医疗问答的实际部署奠定了重要基础。
原文 arXiv:2305.09617;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2305.09617v1