Attend and Diagnose: Clinical Time Series Analysis using Attention Models
Huan Song†, Deepta Rajan‡ , Jayaraman J. Thiagarajan††, Andreas Spanias† † SenSIP Center, School of ECEE, Arizona State University, Tempe, AZ ‡ IBM Almaden Research Center, 650 Harry Road, San Jose, CA †† Lawrence Livermore National Labs, 7000 East Avenue, Livermore, CA The first two authors contributed equally.
Abstract
With widespread adoption of electronic health records, there is an increased emphasis for predictive models that can effectively deal with clinical time-series data. Powered by Recurrent Neural Network (RNN) architectures with Long Short-Term Memory (LSTM) units, deep neural networks have achieved state-of-the-art results in several clinical prediction tasks. Despite the success of RNNs, its sequential nature prohibits parallelized computing, thus making it inefficient particularly when processing long sequences. Recently, architectures which are based solely on attention mechanisms have shown remarkable success in transduction tasks in NLP, while being computationally superior. In this paper, for the first time, we utilize attention models for clinical time-series modeling, thereby dispensing recurrence entirely. We develop the SAnD (Simply Attend and Diagnose) architecture, which employs a masked, self-attention mechanism, and uses positional encoding and dense interpolation strategies for incorporating temporal order. Furthermore, we develop a multi-task variant of SAnD to jointly infer models with multiple diagnosis tasks. Using the recent MIMIC-III benchmark datasets, we demon
中文速览
临床重症监护数据（如MIMIC-III中的ICU多变量时序记录）的智能诊断长期依赖LSTM循环神经网络，但其串行计算结构导致训练效率低下、难以并行化。研究者提出了一种完全基于注意力机制（attention mechanism）的新架构SAnD（Simply Attend and Diagnose），彻底抛弃循环结构，转而用掩码自注意力（masked self-attention）捕捉时序依赖，并结合位置编码与密集插值嵌入（dense interpolation embedding）两种策略保留时间顺序信息。在MIMIC-III四项标准临床预测任务（死亡率预测、生理恶化检测、住院时长预测、表型分类）上的实验表明，SAnD在全部任务中均达到或超越了LSTM的最优水平，同时训练速度更快、可并行化程度更高。这项工作首次将纯注意力模型引入临床时序建模，为电子健康记录的高效智能分析提供了一条更具扩展潜力的新路径。
原文 arXiv:1711.03905;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1711.03905v2