CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Nikita Nangia Clara Vania∗ Rasika Bhalerao∗ Samuel R. Bowman New York University {nikitanangia, c.vania, rasikabh, Equal contribution.
Abstract
Warning: This paper contains explicit statements of offensive stereotypes and may be upsetting. Pretrained language models, especially masked language models (MLMs) have seen success across many NLP tasks. However, there is ample evidence that they use the cultural biases that are undoubtedly present in the corpora they are trained on, implicitly creating harm with biased representations. To measure some forms of social bias in language models against protected demographic groups in the US, we introduce the Crowdsourced Stereotype Pairs benchmark (CrowS-Pairs). CrowS-Pairs has 1508 examples that cover stereotypes dealing with nine types of bias, like race, religion, and age. In CrowS-Pairs a model is presented with two sentences: one that is more stereotyping and another that is less stereotyping. The data focuses on stereotypes about historically disadvantaged groups and contrasts them with advantaged groups. We find that all three of the widely-used MLMs we evaluate substantially favor sentences that express stereotypes in every category in CrowS-Pairs. As work on building less biased models advances, this dataset can be used as a benchmark to evaluate progress.
中文速览
预训练语言模型（如BERT）被广泛用于各类自然语言处理任务，但它们在学习海量文本的同时也悄悄习得了人类社会的刻板印象，可能对弱势群体造成隐性伤害。为了系统衡量这一问题，研究者通过众包方式构建了一个名为CrowS-Pairs的基准测试集，包含1508对句子，每对句子仅在指代群体的词语上存在差异——一句含有刻板印象，另一句则针对对应的优势群体，涵盖种族、宗教、性别等九大偏见类型。他们设计了一种基于伪对数似然的评估指标，通过比较模型对两句话的偏好来量化偏见程度：理想无偏的模型应有50%的概率偏好含刻板印象的句子，而实验结果显示BERT、RoBERTa、ALBERT三个主流模型在所有类别上均显著超过50%，说明它们普遍倾向于生成或认可刻板印象表达。这项工作提供了一个可靠、覆盖面广的偏见测量工具，为后续构建更公平的语言模型奠定了评估基础。
原文 arXiv:2010.00133;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2010.00133v1