Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification
Daniel Borkan Jigsaw , Lucas Dixon Jigsaw , Jeffrey Sorensen Jigsaw , Nithum Thain Jigsaw and Lucy Vasserman Jigsaw
Abstract
Unintended bias in Machine Learning can manifest as systemic differences in performance for different demographic groups, potentially compounding existing challenges to fairness in society at large. In this paper, we introduce a suite of threshold-agnostic metrics that provide a nuanced view of this unintended bias, by considering the various ways that a classifier’s score distribution can vary across designated groups. We also introduce a large new test set of online comments with crowd-sourced annotations for identity references. We use this to show how our metrics can be used to find new and potentially subtle unintended bias in existing public models.
中文速览
机器学习中的毒性检测模型常常对提及特定身份群体（如"gay""muslim"）的内容打出更高的毒性分数，造成对这些群体的系统性不公平，但现有评测指标大多依赖人为选定的决策阈值，容易掩盖这种偏见的真实面貌。为此，研究者提出了一套无需预设阈值（threshold-agnostic）的偏见评估指标，包括三种基于ROC-AUC的子群体度量和两种平均公平性差距（Average Equality Gap）指标，能从"假正例偏高""假负例偏高"等不同维度分别捕捉模型对不同群体的偏见类型。与此同时，他们还发布了一个近200万条在线评论的大规模标注数据集，其中45万条附有身份提及标注，用于更可靠地测试偏见。将这套指标应用于两个公开可用的Perspective API毒性检测模型后发现，即便是经过偏见缓解训练的模型也仍存在微妙但可测量的偏见，而这些偏见在旧有指标下难以被发现。这项工作为构建更公平的内容审核系统提供了更精细的评估工具，也为整个机器学习公平性领域提供了可复用的方法论框架。
原文 arXiv:1903.04561;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1903.04561v2