Faithful to the Original: Fact Aware Neural Abstractive Summarization
Ziqiang Cao1,2 Furu Wei3 Wenjie Li1,2 Sujian Li4 1Department of Computing, The Hong Kong Polytechnic University, Hong Kong 2Hong Kong Polytechnic University Shenzhen Research Institute, China 3Microsoft Research, Beijing, China 4Key Laboratory of Computational Linguistics, Peking University, MOE, China {cszqcao, Contribution during internship at Microsoft Research
Abstract
Unlike extractive summarization, abstractive summarization has to fuse different parts of the source text, which inclines to create fake facts. Our preliminary study reveals nearly 30% of the outputs from a state-of-the-art neural summarization system suffer from this problem. While previous abstractive summarization approaches usually focus on the improvement of informativeness, we argue that faithfulness is also a vital prerequisite for a practical abstractive summarization system. To avoid generating fake facts in a summary, we leverage open information extraction and dependency parse technologies to extract actual fact descriptions from the source text. The dual-attention sequence-to-sequence framework is then proposed to force the generation conditioned on both the source text and the extracted fact descriptions. Experiments on the Gigaword benchmark dataset demonstrate that our model can greatly reduce fake summaries by 80%. Notably, the fact descriptions also bring significant improvement on informativeness since they often condense the meaning of the source text.
中文速览
摘要式摘要（abstractive summarization）在压缩改写原文时，常常会捏造原文中并不存在的事实，一项针对主流神经摘要系统的初步统计显示，近30%的输出都存在这一"虚假事实"问题。为此，研究者提出先用开放信息抽取（Open Information Extraction）和依存句法分析从原文中提取真实的事实描述（如"谁对谁做了什么"的三元组），再设计一个双注意力序列到序列模型（dual-attention sequence-to-sequence），让摘要生成同时参照原文和这些事实描述，从而把住"忠实性"这道关。在Gigaword基准数据集上的实验表明，该方法将虚假摘要比例降低了约80%，同时由于事实描述本身高度凝练了原文要义，生成摘要的信息量也得到显著提升。这项工作首次将"忠实性"作为摘要系统的核心评价维度加以系统研究，对推动更可靠的自动摘要技术具有重要意义。
原文 arXiv:1711.04434;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1711.04434v1