MT-BioNER: Multi-task Learning for Biomedical Named Entity Recognition using Deep Bidirectional Transformers
Muhammad Raza Khan Azure Text Analytics, Microsoft Cloud、AI , Morteza Ziyadi Microsoft Dynamics 365 AI and Mohamed AbdelHady Alexa AI, Amazon
Abstract
Conversational agents such as Cortana, Alexa and Siri are continuously working on increasing their capabilities by adding new domains. The support of a new domain includes the design and development of a number of NLU components for domain classification, intents classification and slots tagging (including named entity recognition). Each component only performs well when trained on a large amount of labeled data. Second, these components are deployed on limited-memory devices which requires some model compression. Third, for some domains such as the health domain, it is hard to find a single training data set that covers all the required slot types. To overcome these mentioned problems, we present a multi-task transformer-based neural architecture for slot tagging. We consider the training of a slot tagger using multiple data sets covering different slot types as a multi-task learning problem. The experimental results on the biomedical domain have shown that the proposed approach outperforms the previous state-of-the-art systems for slot tagging on the different benchmark biomedical datasets in terms of (time and memory) efficiency and effectiveness. The output slot tagger can be u
中文速览
让语音助手（Cortana、Alexa、Siri等）真正听懂医疗健康领域的话，关键难题在于：生物医学命名实体识别（BioNER）既缺乏覆盖所有实体类型的大规模标注数据，又要在内存有限的设备上运行。为此，研究者提出了MT-BioNER——一个把多任务学习（multi-task learning）与预训练语言模型BioBERT相结合的Transformer神经网络框架：底层共享BioBERT编码器，顶层为每个数据集配备独立的任务专属输出层，从而同时利用多个只覆盖部分实体类型的数据集联合训练一个统一的槽位标注器（slot tagger）。在多个生物医学基准数据集上的实验表明，MT-BioNER在准确率和时间、内存效率两方面均超越了此前最优方法。这项工作的意义在于，它为医疗对话系统提供了一条"少量标注数据＋轻量部署"兼顾的实体识别新路径，有望切实提升健康类对话助手识别症状和医学实体的能力。
原文 arXiv:2001.08904;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2001.08904v1