Bidirectional LSTM-CRF Models for Sequence Tagging
Zhiheng Huang Baidu research、Wei Xu Baidu research、Kai Yu Baidu research
Abstract
In this paper, we propose a variety of Long Short-Term Memory (LSTM) based models for sequence tagging. These models include LSTM networks, bidirectional LSTM (BI-LSTM) networks, LSTM with a Conditional Random Field (CRF) layer (LSTM-CRF) and bidirectional LSTM with a CRF layer (BI-LSTM-CRF). Our work is the first to apply a bidirectional LSTM CRF (denoted as BI-LSTM-CRF) model to NLP benchmark sequence tagging data sets. We show that the BI-LSTM-CRF model can efficiently use both past and future input features thanks to a bidirectional LSTM component. It can also use sentence level tag information thanks to a CRF layer. The BI-LSTM-CRF model can produce state of the art (or close to) accuracy on POS, chunking and NER data sets. In addition, it is robust and has less dependence on word embedding as compared to previous observations.
中文速览
序列标注（如词性标注、命名实体识别）长期依赖手工特征与线性统计模型，如何让神经网络更好地同时利用上下文信息和标签间依赖关系是核心难题。研究者系统比较了LSTM、双向LSTM（BI-LSTM）、带条件随机场的LSTM（LSTM-CRF）等多种模型，并首次将双向LSTM与CRF结合成BI-LSTM-CRF模型应用于标准NLP基准数据集：双向LSTM负责从句子两个方向捕捉前后文特征，CRF层则在句子级别建模标签之间的转移约束。实验在词性标注、组块分析和命名实体识别三个任务上均达到了当时最优或接近最优的精度，同时发现该模型对词向量（word embedding）的依赖比此前方法更低，即便不使用预训练词向量也能保持较强性能。这项工作确立了"双向LSTM+CRF"作为序列标注基础框架的地位，对后续NLP研究产生了深远影响。
原文 arXiv:1508.01991;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1508.01991v1