Thermodynamics-inspired Explanations of Artificial Intelligence
Shams Mehdi Biophysics Program and Institute for Physical Science and Technology, University of Maryland, College Park 20742, USA Pratyush Tiwary111Corresponding author. Department of Chemistry and Biochemistry and Institute for Physical Science and Technology, University of Maryland, College Park 20742, USA. University of Maryland Institute for Health Computing, Rockville, MD, USA
Abstract
Abstract In recent years, predictive machine learning methods have gained prominence in various scientific domains. However, due to their black-box nature, it is essential to establish trust in these models before accepting them as accurate. One promising strategy for assigning trust involves employing explanation techniques that elucidate the rationale behind a black-box model’s predictions in a manner that humans can understand. However, assessing the degree of human interpretability of the rationale generated by such methods is a nontrivial challenge. In this work, we introduce interpretation entropy as a universal solution for assessing the degree of human interpretability associated with any linear model. Using this concept and drawing inspiration from classical thermodynamics, we present Thermodynamics-inspired Explainable Representations of AI and other black-box Paradigms (TERP), a method for generating accurate, and human-interpretable explanations for black-box predictions in a model-agnostic manner. To demonstrate the wide-ranging applicability of TERP, we successfully employ it to explain various black-box model architectures, including deep learning Autoencoders, Recur
中文速览
黑箱AI模型（如深度学习）预测准确但难以被人理解，现有解释方法虽能给出特征重要性，却缺乏衡量"人类究竟能不能看懂"这一解释的可靠标准。作者受经典热力学启发，提出"解释熵"（interpretation entropy）这一指标来量化线性代理模型的可解释程度——权重分布越集中，熵越低，人越容易理解——并在此基础上构建了TERP方法：将解释的"不忠实度"类比内能、将解释熵类比热力学熵，通过最小化类自由能的目标函数，自动在准确性与可读性之间找到最优平衡点。作者将TERP应用于分子动力学模拟的自编码器、图像分类的视觉Transformer以及文本分类的双向长短期记忆网络，均获得了比现有方法更准确且更简洁易懂的解释。这项工作为跨领域评估和生成可信AI解释提供了一套有理论保障的通用框架，对需要借助AI辅助决策的科学研究尤具价值。
原文 arXiv:2206.13475;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2206.13475v3