Detoxifying Language Models Risks Marginalizing Minority Voices
Albert Xu♢ Eshaan Pathak♢ Eric Wallace♢ Suchin Gururangan♠ Maarten Sap♠ Dan Klein♢ ♢UC Berkeley ♠University of Washington {albertxu3, eshaanpathak, ericwallace, {sg01,
Abstract
Language models (LMs) must be both safe and equitable to be responsibly deployed in practice. With safety in mind, numerous detoxification techniques (e.g., Dathathri et al. 2020; Krause et al. 2020) have been proposed to mitigate toxic LM generations. In this work, we show that these detoxification techniques hurt equity: they decrease the utility of LMs on language used by marginalized groups (e.g., African-American English and minority identity mentions). In particular, we perform automatic and human evaluations of text generation quality when LMs are conditioned on inputs with different dialects and group identifiers. We find that detoxification makes LMs more brittle to distribution shift, especially on language used by marginalized groups. We identify that these failures stem from detoxification methods exploiting spurious correlations in toxicity datasets. Overall, our results highlight the tension between the controllability and distributional robustness of LMs.
中文速览
大量语言模型（language model, LM）在实际部署前会经过"去毒化"（detoxification）处理，以减少模型生成仇恨或冒犯性内容，但这篇论文发现这类处理会对边缘群体造成不公平：去毒化后的模型在非裔美国英语（African-American English, AAE）和少数群体身份词汇（如"gay""Muslim"）上的表现大幅变差，困惑度显著升高，生成文本的话题相关性、流畅度和风格保持能力也明显下滑。研究者通过自动评估和众包人工评估双管齐下加以验证，并追溯根因：去毒化所依赖的毒性标注数据集本身存在偏差，标注者常因不熟悉AAE而将其误判为有毒内容，导致模型把边缘群体的语言模式也一并"遗忘"或规避。这一发现揭示了一个严峻的现实：在不解决底层数据偏见的情况下，越强力的安全干预反而会越深地压制少数群体的语言表达，迫使他们在与NLP系统交互时被迫切换为主流表达方式，既带来心理伤害，也加剧社会污名化。
原文 arXiv:2104.06390;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2104.06390v1