Plug and Play Language Models: a Simple Approach to Controlled Text Generation
Sumanth Dathathri CMS, Caltech、Andrea Madotto HKUST、Janice Lan Uber AI、Jane Hung Uber AI \ANDEric Frank Uber AI、Piero Molino Uber AI、Jason Yosinski Uber AI、Rosanne Liu Uber AI \AND {janlan, jane.hung, mysterefrank, piero, yosinski, Work done during internship at Uber AICo-senior authors . ∙ ∙ \mathbin{\vbox{\hbox{\scalebox{0.6}{$\bullet$}}}} Summary of contributions: SD, RL、JY conceptualized PPLMs and led the manuscript writing. SD led the project, implemented the PPLM, set up and ran all modeling experiments, engineered how to obtain workable gradients via the weighted embedding approach, and made the model work. AM helped with preparing datasets for discriminator training, automated evaluation, running experiments, and writing the manuscript. SD, RL、AM ran the external baselines. RL、JL built and oversaw the human evaluation pipeline and computed the statistics. JH ran the story generation with skeleton prefixes. EF assisted with detoxification experiments. PM led efforts to migrate to the new pytorch transformer, helped with code release. JY helped with the annotation pipeline, finding bugs, navigating model and experimental directions, engineering workable gradients, and posing the model mathematically. RL implemented preliminary experiments and multi-attribute control, and cleaned and coordinated release of the code. RL、JY oversaw the project.
Abstract
Large transformer-based language models (LMs) trained on huge text corpora have shown unparalleled generation capabilities. However, controlling attributes of the generated language (e.g. switching topic or sentiment) is difficult without modifying the model architecture or fine-tuning on attribute-specific data and entailing the significant cost of retraining. We propose a simple alternative: the Plug and Play Language Model (PPLM) for controllable language generation, which combines a pretrained LM with one or more simple attribute classifiers that guide text generation without any further training of the LM. In the canonical scenario we present, the attribute models are simple classifiers consisting of a user-specified bag of words or a single learned layer with 100,000 times fewer parameters than the LM. Sampling entails a forward and backward pass in which gradients from the attribute model push the LM’s hidden activations and thus guide the generation. Model samples demonstrate control over a range of topics and sentiment styles, and extensive automated and human annotated evaluations show attribute alignment and fluency. PPLMs are flexible in that any combination of differen
中文速览
大型预训练语言模型（如GPT-2）生成文字能力很强，但想让它按指定主题或情感风格来写作，通常得修改模型结构或重新训练，代价极高。为此，研究者提出了"即插即用语言模型"（Plug and Play Language Model，PPLM）：保持原有语言模型完全不动，只在生成每个词的过程中，用一个极轻量的属性控制器（如关键词列表或参数量仅为语言模型百万分之一的单层分类器）计算梯度，反向推动模型内部的隐藏状态，从而引导生成内容朝目标属性偏转。实验覆盖7个话题控制和情感正负向控制，自动评测与人工评测均显示PPLM在属性贴合度和语言流畅度上与需要大规模重训的CTRL等基线方法持平乃至更优，还能用于减少有毒内容生成。这一方法让任何人都可以像拨旋钮一样灵活组合属性控制器，无需触碰底层大模型，大幅降低了可控文本生成的门槛。
原文 arXiv:1912.02164;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1912.02164v4