Iterative Orthogonal Feature Projection for Diagnosing Bias in Black-Box Models
Julius Adebayo Lalana Kagal CSAIL, MIT, 32 Vassar Street Cambridge, MA 02139 USA.
Abstract
Predictive models are increasingly deployed for the purpose of determining access to services such as credit, insurance, and employment. Despite potential gains in productivity and efficiency, several potential problems have yet to be addressed, particularly the potential for unintentional discrimination. We present an iterative procedure, based on orthogonal projection of input attributes, for enabling interpretability of black-box predictive models. Through our iterative procedure, one can quantify the relative dependence of a black-box model on its input attributes.The relative significance of the inputs to a predictive model can then be used to assess the fairness (or discriminatory extent) of such a model.
中文速览
信用评分、保险核保、招聘筛选等场景越来越多地依赖黑箱机器学习模型做决策，但这些模型可能在无意间对性别、种族等受保护属性产生歧视，而其内部逻辑对外界完全不透明。为此，作者提出了一种基于正交投影（orthogonal projection）的迭代方法：依次将数据集中的每个输入特征"移除"——即令其余特征与该特征正交——再观察黑箱模型的预测性能如何变化，从而量化模型对每个特征的依赖程度并排序。在一家欧洲大型银行的40万客户信用额度数据上进行验证，结果显示该银行模型对性别的依赖程度始终处于较低水平，表明其算法并未过度依赖性别做出信贷决定。这套方法无需了解模型内部结构、也无需修改算法本身，为监管机构和从业者提供了一种可解释、可操作的黑箱模型公平性审计工具。
原文 arXiv:1611.04967;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1611.04967v1