The Counterfactual-Shapley Value: Attributing Change in System Metrics
Amit Sharma Microsoft Research、Hua Li Microsoft Bing Ads、Jian Jiao Microsoft Bing Ads
Abstract
Given an unexpected change in the output metric of a large-scale system, it is important to answer why the change occurred: which inputs caused the change in metric? A key component of such an attribution question is estimating the counterfactual: the (hypothetical) change in the system metric due to a specified change in a single input. However, due to inherent stochasticity and complex interactions between parts of the system, it is difficult to model an output metric directly. We utilize the computational structure of a system to break up the modelling task into sub-parts, such that each sub-part corresponds to a more stable mechanism that can be modelled accurately over time. Using the system’s structure also helps to view the metric as a computation over a structural causal model (SCM), thus providing a principled way to estimate counterfactuals. Specifically, we propose a method to estimate counterfactuals using time-series predictive models and construct an attribution score, CF-Shapley, that is consistent with desirable axioms for attributing an observed change in the output metric. Unlike past work on causal shapley values, our proposed method can attribute a single observ
中文速览
大规模系统（如搜索或广告平台）的某个核心指标突然异常波动时，运营团队最迫切需要知道的是"到底是哪个输入因素造成的"——这正是本文要解决的归因问题。作者的核心思路是：把系统内部的计算流程建模为结构因果模型（Structural Causal Model, SCM），将复杂的整体指标拆解成若干更稳定、更易建模的子模块，再借助时间序列预测模型来估计反事实值（counterfactual），即"如果某个输入保持不变，指标会是多少"。在此基础上，作者提出了满足四条公理的归因分值 CF-Shapley，与现有的因果 Shapley 方法相比，它能针对单次观测到的变化（而非群体平均效应）给出更精准的归因。在真实广告匹配系统上的应用验证了该方法的有效性：CF-Shapley 分值之和精确等于观测到的密度变化量，并成功揭示了"Cheetah Day"等外部事件对广告匹配密度的具体影响，为大型系统的异常诊断和决策优化提供了可落地的分析工具。
原文 arXiv:2208.08399;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2208.08399v1