Pareto Smoothed Importance Sampling
\nameAki Vehtari \addrDepartment of Computer Science Aalto University \AND\nameDaniel Simpson \addrNormal Computing \AND\nameAndrew Gelman \addrDepartments of Statistics and Political Science Columbia University \AND\nameYuling Yao \addrCenter for Computational Mathematics Flatiron Institute \AND\nameJonah Gabry \addrDepartment of Statistics Columbia University
Abstract
Importance weighting is a general way to adjust Monte Carlo integration to account for draws from the wrong distribution, but the resulting estimate can be highly variable when the importance ratios have a heavy right tail. This routinely occurs when there are aspects of the target distribution that are not well captured by the approximating distribution, in which case more stable estimates can be obtained by modifying extreme importance ratios. We present a new method for stabilizing importance weights using a generalized Pareto distribution fit to the upper tail of the distribution of the simulated importance ratios. The method, which empirically performs better than existing methods for stabilizing importance sampling estimates, includes stabilized effective sample size estimates, Monte Carlo error estimates, and convergence diagnostics. The presented Pareto $\hat{k}$ finite sample convergence rate diagnostic is useful for any Monte Carlo estimator.
中文速览
重要性采样（importance sampling）是贝叶斯计算中常见的蒙特卡洛技术，但当提议分布与目标分布差距较大时，少数极端权重会主导估计结果，导致方差爆炸甚至无穷大。作者提出了"帕累托平滑重要性采样"（Pareto Smoothed Importance Sampling，PSIS）：先用广义帕累托分布拟合最大权重的尾部，再用拟合分布的期望次序统计量替换那些极端权重，从而在不引入过大偏差的前提下显著提升估计的稳定性。同时，拟合得到的形状参数 $\hat{k}$ 被发展为一个通用的有限样本收敛诊断指标——当 $\hat{k}<0.7$ 时估计可靠，超过该阈值则自动向用户发出警告。与现有的截断重要性采样方法相比，PSIS 在低维和高维场景下均表现更优，已被集成进下载量超过三百万次的 loo 软件包，广泛用于贝叶斯模型的留一交叉验证，对实际贝叶斯工作流具有重要价值。
原文 arXiv:1507.02646;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1507.02646v9