Measuring Bias in Contextualized Word Representations
Keita Kurita Nidhi Vyas Ayush Pareek Alan W Black Yulia Tsvetkov Carnegie Mellon University
Abstract
Contextual word embeddings such as BERT have achieved state of the art performance in numerous NLP tasks. Since they are optimized to capture the statistical properties of training data, they tend to pick up on and amplify social stereotypes present in the data as well. In this study, we (1) propose a template-based method to quantify bias in BERT; (2) show that this method obtains more consistent results in capturing social biases than the traditional cosine based method; and (3) conduct a case study, evaluating gender bias in a downstream task of Gender Pronoun Resolution. Although our case study focuses on gender bias, the proposed technique is generalizable to unveiling other biases, including in multiclass settings, such as racial and religious biases.
中文速览
针对BERT等上下文词嵌入模型会悄悄放大训练数据中社会刻板印象这一问题，研究者提出了一种基于模板的新方法——利用BERT本身的掩码语言模型机制，通过对比模型预测目标词（如"he"和"she"）的对数概率来量化偏见，而不再依赖传统的余弦相似度。实验表明，传统余弦方法在检测BERT偏见时结果前后不一，而新方法在性别、种族、宗教等多类偏见场景下均能稳定揭示出统计显著的偏见信号。在性别代词消解的下游任务中，研究进一步证实BERT内嵌的性别偏见会直接影响模型表现——当文本话题偏男性化时，模型更难将女性代词正确指向对应实体。这项工作的意义在于，它为审视和度量大规模预训练模型中的社会偏见提供了一种简便通用的工具，对招聘推荐等高风险实际应用场景中的公平性保障具有直接警示价值。
原文 arXiv:1906.07337;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1906.07337v1