Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Su Lin Blodgett College of Information and Computer Sciences University of Massachusetts Amherst、Solon Barocas Microsoft Research Cornell University \ANDHal Daumé III Microsoft Research University of Maryland、Hanna Wallach Microsoft Research
Abstract
We survey 146 papers analyzing “bias” in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing “bias” is an inherently normative process. We further find that these papers’ proposed quantitative techniques for measuring or mitigating “bias” are poorly matched to their motivations and do not engage with the relevant literature outside of NLP. Based on these findings, we describe the beginnings of a path forward by proposing three recommendations that should guide work analyzing “bias” in NLP systems. These recommendations rest on a greater recognition of the relationships between language and social hierarchies, encouraging researchers and practitioners to articulate their conceptualizations of “bias”—i.e., what kinds of system behaviors are harmful, in what ways, to whom, and why, as well as the normative reasoning underlying these statements—and to center work around the lived experiences of members of communities affected by NLP systems, while interrogating and reimagining the power relations between technologists and such communities.
中文速览
146篇关于NLP系统"偏见"（bias）的论文被系统梳理后，研究者发现这一领域存在根本性的概念混乱：绝大多数论文在指出某种系统行为"有偏见"时，既没有说清楚这对谁有害、如何有害、为何有害，也缺乏必要的规范性推理（normative reasoning），甚至同一任务的不同论文对"偏见"的定义彼此矛盾。更棘手的是，这些论文提出的量化测量或缓解方法往往与其自身动机脱节，也几乎不借鉴社会学、哲学等相关领域的既有研究成果。为此，作者提出三条改进建议：研究者应深入理解语言与社会权力结构之间的关系，明确阐述自己对"偏见"的概念界定，并将受NLP系统影响的社区成员的真实处境置于研究核心。这项工作的重要性在于，它揭示了当前NLP公平性研究在理论基础上的系统性缺陷，为整个领域更严谨、更负责任地推进相关研究指明了方向。
原文 arXiv:2005.14050;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2005.14050v2