Societal Biases in Language Generation: Progress and Challenges
Emily Sheng1, Kai-Wei Chang2, Premkumar Natarajan1, Nanyun Peng1,2 1 Information Sciences Institute, University of Southern California 2 Computer Science Department, University of California, Los Angeles
Abstract
Technology for language generation has advanced rapidly, spurred by advancements in pre-training large models on massive amounts of data and the need for intelligent agents to communicate in a natural manner. While techniques can effectively generate fluent text, they can also produce undesirable societal biases that can have a disproportionately negative impact on marginalized populations. Language generation presents unique challenges for biases in terms of direct user interaction and the structure of decoding techniques. To better understand these challenges, we present a survey on societal biases in language generation, focusing on how data and techniques contribute to biases and progress towards reducing biases. Motivated by a lack of studies on biases from decoding techniques, we also conduct experiments to quantify the effects of these techniques. By further discussing general trends and open challenges, we call to attention promising directions for research and the importance of fairness and inclusivity considerations for language generation applications.
中文速览
大型语言模型（如GPT系列）在自动补全、对话、机器翻译等各类文本生成任务中已经相当流行，但它们在生成内容时会对边缘群体产生不成比例的负面偏见，这一问题却缺乏系统梳理。作者对语言生成领域的社会偏见研究进行了首个全面综述，详细分析了训练数据、模型结构以及解码策略（decoding techniques，即控制模型如何从概率分布中采样输出文本的方法）各自如何引入或放大偏见，并专门设计实验量化了不同解码策略对偏见的实际影响。研究发现，贪婪解码、核采样等不同策略会产生程度各异的偏见输出，而现有减偏方法在数据层面和模型层面均有进展但仍存在明显局限。这项工作的重要性在于，语言生成系统直接与真实用户交互、产生新内容，一旦对特定群体持续输出负面或刻板内容，便会形成实质性的社会壁垒，因此呼吁研究者将公平性与包容性纳入语言生成系统开发的核心考量。
原文 arXiv:2105.04054;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2105.04054v3