LCSTS: A Large Scale Chinese Short Text Summarization Dataset
Baotian Hu Qingcai Chen Fangze Zhu Intelligent Computing Research Center Harbin Institute of Technology, Shenzhen Graduate School {baotianchina,qingcai.chen,
Abstract
Automatic text summarization is widely regarded as the highly difficult problem, partially because of the lack of large text summarization data set. Due to the great challenge of constructing the large scale summaries for full text, in this paper, we introduce a large corpus of Chinese short text summarization dataset constructed from the Chinese microblogging website Sina Weibo, which is released to the public111http://icrc.hitsz.edu.cn/Article/show/139.html. This corpus consists of over 2 million real Chinese short texts with short summaries given by the author of each text. We also manually tagged the relevance of 10,666 short summaries with their corresponding short texts. Based on the corpus, we introduce recurrent neural network for the summary generation and achieve promising results, which not only shows the usefulness of the proposed corpus for short text summarization research, but also provides a baseline for further research on this topic.
中文速览
中文短文本自动摘要长期受制于大规模标注数据的匮乏，为此研究者从新浪微博抓取了超过240万条真实短文本及其作者撰写的摘要，构建了名为LCSTS的大规模中文短文本摘要数据集，并对其中逾万条样本进行了人工相关性评分，最终向公众开放。在此数据集上，研究者采用基于门控循环单元（GRU）的编码器-解码器（encoder-decoder）框架，分别尝试了有无注意力上下文的两种结构，并对比了字符级与词级两种输入方式。实验结果表明，带有上下文信息的RNN模型在ROUGE指标上表现最佳，且字符级输入优于词级输入，主要原因在于分词会引入大量未登录词。这项工作不仅填补了中文摘要研究在数据层面的空白，还为后续深度学习方法提供了可复现的基准，对推动中文自然语言处理研究具有重要价值。
原文 arXiv:1506.05865;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1506.05865v4