Assessing the Interpretability of Programmatic Policies with Large Language Models
Zahra Bashir, Michael Bowling, Levi H. S. Lelis Department of Computing Science, University of Alberta, Canada Alberta Machine Intelligence Institute(Amii)
Abstract
Although the synthesis of programs encoding policies often carries the promise of interpretability, systematic evaluations were never performed to assess the interpretability of these policies, likely because of the complexity of such an evaluation. In this paper, we introduce a novel metric that uses large-language models (LLM) to assess the interpretability of programmatic policies. For our metric, an LLM is given both a program and a description of its associated programming language. The LLM then formulates a natural language explanation of the program. This explanation is subsequently fed into a second LLM, which tries to reconstruct the program from the natural-language explanation. Our metric then measures the behavioral similarity between the reconstructed program and the original. We validate our approach with synthesized and human-crafted programmatic policies for playing a real-time strategy game, comparing the interpretability scores of these programmatic policies to obfuscated versions of the same programs. Our LLM-based interpretability score consistently ranks less interpretable programs lower and more interpretable ones higher. These findings suggest that our metric
中文速览
想为程序化策略（programmatic policy）的"可解释性"打分，过去一直没有系统化的评估方法，主要原因是人工评估太费时费钱。研究者提出了一种基于大语言模型（LLM）的可解释性度量方法，称为LINT（LLM-based INTerpretability）分数：先让一个LLM把程序"翻译"成自然语言解释，再让另一个LLM仅凭这段解释重新写出程序，最后用行为相似度来衡量重建效果——重建越准，说明原程序越容易被理解。研究者在经典编程题和实时策略游戏MicroRTS的程序化策略上验证了这一方法，将原程序与不同程度的"混淆版"（obfuscated version）对比，发现LINT分数与混淆程度呈强负相关，即越难读懂的程序得分越低、越清晰的程序得分越高。这一结果表明，LINT可以作为一种低成本、可靠的自动化工具，替代昂贵的人工用户研究，用于指导可解释程序化策略的研究与开发。
原文 arXiv:2311.06979;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2311.06979v2