Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
Po-Sen Huang♠ Huan Zhang♡ Ray Jiang♠ Robert Stanforth♠ Johannes Welbl♠♣ Jack W. Rae♠♣ Vishal Maini♠ Dani Yogatama♠ Pushmeet Kohli♠ ♠DeepMind ♡University of California, Los Angeles ♣University College London
Abstract
Advances in language modeling architectures and the availability of large text corpora have driven progress in automatic text generation. While this results in models capable of generating coherent texts, it also prompts models to internalize social biases present in the training corpus. This paper aims to quantify and reduce a particular type of bias exhibited by language models: bias in the sentiment of generated text. Given a conditioning context (e.g., a writing prompt) and a language model, we analyze if (and how) the sentiment of the generated text is affected by changes in values of sensitive attributes (e.g., country names, occupations, genders) in the conditioning context using a form of counterfactual evaluation. We quantify sentiment bias by adopting individual and group fairness metrics from the fair machine learning literature, and demonstrate that large-scale models trained on two different corpora (news articles, and Wikipedia) exhibit considerable levels of bias. We then propose embedding and sentiment prediction-derived regularization on the language model’s latent representations. The regularizations improve fairness metrics while retaining comparable levels of pe
中文速览
大型语言模型（如GPT-2）在续写文本时，会因输入中涉及的敏感词（如职业、国家、人名）不同而系统性地生成情感倾向差异显著的文本，例如以"会计"为提示时倾向生成负面内容，而以"面包师"为提示时则倾向生成正面内容，这种现象被称为情感偏见（sentiment bias）。作者借鉴公平机器学习领域的个体公平与群体公平指标，通过反事实评估（counterfactual evaluation）将这一偏见量化为Wasserstein-1距离，并在此基础上提出了两种基于正则化的去偏方法：一种对语言模型的隐层表示施加嵌入层正则化，另一种引入情感预测导向的正则化，促使模型对不同敏感属性值产生相近的情感分布。实验表明，两种方法均能在保持文本困惑度和语义相似度基本不变的前提下，显著降低情感偏见指标，且自动评测结果与人工评估高度吻合。这项工作为在不同应用场景（对话系统、新闻生成等）中系统性地审查并约束语言模型的偏见行为提供了可推广的通用框架，具有重要的社会公平意义。
原文 arXiv:1911.03064;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1911.03064v3