Text Generation with Exemplar-based Adaptive Decoding
Hao Peng♠∗ Ankur P. Parikh♢ Manaal Faruqui♢ Bhuwan Dhingra♣∗ Dipanjan Das♢ ♠ Paul G. Allen School of Computer Science、Engineering, University of Washington, Seattle, WA ♢ Google AI Language, New York, NY ♣ School of Computer Science, Carnegie Mellon University, Pittsburgh, PA
Abstract
We propose a novel conditioned text generation model. It draws inspiration from traditional template-based text generation techniques, where the source provides the content (i.e., what to say), and the template influences how to say it. Building on the successful encoder-decoder paradigm, it first encodes the content representation from the given input text; to produce the output, it retrieves exemplar text from the training data as “soft templates,” which are then used to construct an exemplar-specific decoder. We evaluate the proposed model on abstractive text summarization and data-to-text generation. Empirical results show that this model achieves strong performance and outperforms comparable baselines.
中文速览
条件文本生成领域长期面临一个矛盾：模板方法生成精确但不灵活，神经网络方法流畅但缺乏结构控制。研究者提出了一种"范例自适应解码"（exemplar-based adaptive decoding）方法，借鉴传统模板生成的思路——用源文本决定"说什么"，用从训练集检索到的相似样本作为"软模板"决定"怎么说"——通过将检索到的范例直接用于重参数化解码器的权重矩阵，为每个输入动态构建一个定制化解码器，从而让范例的结构和风格特征渗透进生成过程，而非仅仅拼接到输入端供注意力机制参考。在文本摘要和数据转文本两类任务的三个基准数据集上，该方法均超越了可比基线，达到了接近当时最优的水平。这一工作的意义在于提供了一种参数高效、即插即用的方式，让神经生成模型在保持流畅性的同时获得更可控的输出结构。
原文 arXiv:1904.04428;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1904.04428v2