Auditing Black-box Models for Indirect Influence
Philip Adler1, Casey Falk1, Sorelle A. Friedler1, Gabriel Rybeck1, Carlos Scheidegger2, Brandon Smith1, and Suresh Venkatasubramanian3 1 Dept. of Computer Science, Haverford College, Haverford, PA, USA 2 Dept. of Computer Science, University of Arizona, Tucson, AZ, USA 3 Dept. of Computer Science, University of Utah, Salt Lake City, UT, USA
Abstract
Data-trained predictive models see widespread use, but for the most part they are used as black boxes which output a prediction or score. It is therefore hard to acquire a deeper understanding of model behavior, and in particular how different features influence the model prediction. This is important when interpreting the behavior of complex models, or asserting that certain problematic attributes (like race or gender) are not unduly influencing decisions.
中文速览
机器学习模型被广泛用于贷款审批、量刑等影响真实生活的决策，却像黑盒子一样让人看不透内部逻辑，尤其难以判断种族、性别等敏感属性是否通过"代理变量"（proxy variable，比如用邮编间接代表种族）悄悄影响了结果。本文提出一种名为梯度特征审计（Gradient Feature Auditing, GFA）的黑盒审计方法：核心思路是把某个特征的信息从数据中"最小化地抹去"——具体做法是让该特征无法再从其他特征中被预测出来，然后观察模型准确率下降多少，以此量化该特征的"间接影响力"。这一过程无需重新训练被审计的模型，只需通过API调用即可完成，因此适用于真实的第三方审计场景。在多个公开数据集上的实验表明，GFA不仅能检测到特征的直接影响，还能发现其他主流方法（如随机置换法）无法捕捉到的代理变量间接影响，为识别算法决策中隐性的歧视性因素提供了更可靠的工具。
原文 arXiv:1602.07043;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1602.07043v2