Measuring and Reducing Gendered Correlations in Pre-trained Models
Kellie Webster, Xuezhi Wang, Ian Tenney∗, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, Slav Petrov {websterk, xuezhiw, iftenney, alexbeutel, epitler, epavlick, jilinc, edchi, Equal contribution.
Abstract
Pre-trained models have revolutionized natural language understanding. However, researchers have found they can encode artifacts undesired in many applications, such as professions correlating with one gender more than another. We explore such gendered correlations as a case study for how to address unintended correlations in pre-trained models. We define metrics and reveal that it is possible for models with similar accuracy to encode correlations at very different rates. We show how measured correlations can be reduced with general-purpose techniques, and highlight the trade offs different strategies have. With these results, we make recommendations for training robust models: (1) carefully evaluate unintended correlations, (2) be mindful of seemingly innocuous configuration differences, and (3) focus on general mitigations.
中文速览
预训练语言模型（pre-trained language models）在提升NLP任务准确率的同时，也会无意间学到性别刻板印象——比如把某些职业和特定性别挂钩，这种隐患在实际应用中可能造成不公平的模型决策。研究者提出了一套评估框架，包含一种名为DisCo的新指标，用来系统衡量模型在共指消解、语义相似度和职业分类等多个场景下的性别关联程度，结果发现准确率相近的模型在性别关联强度上可以相差悬殊。进一步实验表明，两种通用技术——dropout正则化和反事实数据增强（counterfactual data augmentation, CDA）——都能在基本不损失准确率的前提下显著降低这些关联，而且这种改善在后续微调阶段依然保持，模型也更不容易重新学回有问题的关联。这项工作的价值在于：它证明了无需为每种偏见单独定制方案，在预训练阶段采用通用手段就能一并压制多类不良关联，并为训练更稳健的语言模型提供了可操作的建议。
原文 arXiv:2010.06032;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2010.06032v2