Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization
Jun Suzuki Masaaki Nagata NTT Communication Science Laboratories, NTT Corporation 2-4 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0237 Japan {suzuki.jun,
Abstract
This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output words based on the estimation in the decoder. Our method shows significant improvement over a strong RNN-based encoder-decoder baseline and achieved its best results on an abstractive summarization benchmark. 111This is a draft version of EACL-2017
中文速览
循环神经网络编解码模型在生成摘要时频繁出现重复词语的问题，作者提出了一种"词频估计（Word Frequency Estimation, WFE）"子模块来解决这一痛点：在编码阶段预测每个目标词在输出摘要中最多应出现几次，再在解码阶段实时追踪已生成次数，一旦某词达到上限便将其概率强制压至负无穷，从根本上阻断过度重复。在Gigaword和DUC-2004两个主流抽象摘要基准上，加入WFE后的模型在ROUGE各项指标上大幅超越强基线，并刷新了当时的最优成绩。这项工作的意义在于：无需改变训练目标或引入复杂的覆盖约束，仅凭一个轻量级的频率预测组件，就能有效提升生成文本的流畅性与信息密度，为后续序列生成研究提供了简洁实用的解决思路。
原文 arXiv:1701.00138;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1701.00138v2