GeDi: Generative Discriminator guided Sequence Generation
Ben Krause, Akhilesh Deepak Gotmare††, Bryan McCann, Nitish Shirish Keskar Shafiq Joty, Richard Socher††, Nazneen Fatema Rajani Salesforce Research Equal ContributionWork performed while at Salesforce Research
Abstract
While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of the distribution they generate. This is especially problematic because datasets used for training large LMs usually contain significant toxicity, hate, bias, and negativity. We propose GeDi as an efficient method for using smaller LMs as generative discriminators to guide generation from large LMs to make them safer and more controllable. GeDi guides generation at each step by computing classification probabilities for all possible next tokens via Bayes rule by normalizing over two class-conditional distributions; one conditioned on the desired attribute, or control code, and another conditioned on the undesired attribute, or anti control code. We find that GeDi gives stronger controllability than the state of the art method while also achieving generation speeds more than $30$ times faster. Additionally, training GeDi on only four topics allows us to controllably generate new topics zero-shot from just a keyword, unlocking a new capability that previous controllable generation methods do not have. Lastl
中文速览
大型语言模型（LM）虽然能生成流畅自然的文本，但训练数据中充斥着毒性、偏见与仇恨内容，导致模型很难被引导只生成"好的"内容。为此，研究者提出了GeDi方法：用一个较小的类条件语言模型（class-conditional LM）充当"生成式判别器"，在大模型每步生成时，通过贝叶斯规则同时计算"期望属性"和"非期望属性"两个条件分布，从而高效地为词表中所有候选词打上"是否符合目标属性"的概率分，再以此引导大模型的采样方向。实验表明，GeDi（3.45亿参数）引导GPT-2 XL（15亿参数）时，在情感控制、话题控制和去毒性三项任务上均优于或持平最先进基线，生成速度更是比PPLM快30倍以上；尤其值得一提的是，仅在4个话题上训练的GeDi还能零样本（zero-shot）泛化到全新话题，解锁了此前方法不具备的能力。这项工作为大规模语言模型的安全可控部署提供了一条计算成本低、效果强的实用路径。
原文 arXiv:2009.06367;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2009.06367v2