Human-in-the-Loop Interpretability Prior
Isaac Lage Department of Computer Science Harvard University、Andrew Slavin Ross Department of Computer Science Harvard University、Been Kim Google Brain、Samuel J. Gershman Department of Psychology Harvard University、Finale Doshi-Velez Department of Computer Science Harvard University
Abstract
We often desire our models to be interpretable as well as accurate. Prior work on optimizing models for interpretability has relied on easy-to-quantify proxies for interpretability, such as sparsity or the number of operations required. In this work, we optimize for interpretability by directly including humans in the optimization loop. We develop an algorithm that minimizes the number of user studies to find models that are both predictive and interpretable and demonstrate our approach on several data sets. Our human subjects results show trends towards different proxy notions of interpretability on different datasets, which suggests that different proxies are preferred on different tasks.
中文速览
机器学习模型的可解释性（interpretability）长期依赖稀疏度、操作数量等间接指标来衡量，但这些替代指标未必真正符合人类的主观感受。本文提出一种将真实用户实验直接嵌入优化循环的框架：先筛选出一批预测精度达标的候选模型，再用贝叶斯优化（model-based optimization / Gaussian Process）以最少的用户研究次数，找到既准确又最被人类认为易懂的模型，其中人类可解释性通过用户回答问题的平均反应时间来量化。在合成数据及蘑菇、人口收入、地表覆盖等多个真实数据集上的实验表明，该方法能有效减少所需的用户研究次数，同时发现不同任务上人类偏好的可解释性代理指标各不相同。这一结果说明，用单一固定指标衡量可解释性存在根本局限，把人直接纳入优化过程才能获得真正贴合具体应用场景的可解释模型。
原文 arXiv:1805.11571;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1805.11571v2