Plug and Play Language Models: a Simple Approach to Controlled Text Generation
Sumanth Dathathri CMS, Caltech、Andrea Madotto HKUST、Janice Lan Uber AI、Jane Hung Uber AI \ANDEric Frank Uber AI、Piero Molino Uber AI、Jason Yosinski Uber AI、Rosanne Liu Uber AI \AND {janlan, jane.hung, mysterefrank, piero, yosinski, Work done during internship at Uber AICo-senior authors . ∙ ∙ \mathbin{\vbox{\hbox{\scalebox{0.6}{$\bullet$}}}} Summary of contributions: SD, RL、JY conceptualized PPLMs and led the manuscript writing. SD led the project, implemented the PPLM, set up and ran all modeling experiments, engineered how to obtain workable gradients via the weighted embedding approach, and made the model work. AM helped with preparing datasets for discriminator training, automated evaluation, running experiments, and writing the manuscript. SD, RL、AM ran the external baselines. RL、JL built and oversaw the human evaluation pipeline and computed the statistics. JH ran the story generation with skeleton prefixes. EF assisted with detoxification experiments. PM led efforts to migrate to the new pytorch transformer, helped with code release. JY helped with the annotation pipeline, finding bugs, navigating model and experimental directions, engineering workable gradients, and posing the model mathematically. RL implemented preliminary experiments and multi-attribute control, and cleaned and coordinated release of the code. RL、JY oversaw the project.
Abstract
Large transformer-based language models (LMs) trained on huge text corpora have shown unparalleled generation capabilities. However, controlling attributes of the generated language (e.g. switching topic or sentiment) is difficult without modifying the model architecture or fine-tuning on attribute-specific data and entailing the significant cost of retraining. We propose a simple alternative: the Plug and Play Language Model (PPLM) for controllable language generation, which combines a pretrained LM with one or more simple attribute classifiers that guide text generation without any further training of the LM. In the canonical scenario we present, the attribute models are simple classifiers consisting of a user-specified bag of words or a single learned layer with 100,000 times fewer parameters than the LM. Sampling entails a forward and backward pass in which gradients from the attribute model push the LM’s hidden activations and thus guide the generation. Model samples demonstrate control over a range of topics and sentiment styles, and extensive automated and human annotated evaluations show attribute alignment and fluency. PPLMs are flexible in that any combination of differen
原文 arXiv:1912.02164;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1912.02164v4