Optimal Inference After Model Selection
William Fithian111To whom correspondence should be addressed Department of Statistics, University of California Berkeley Dennis L. Sun Department of Statistics, California Polytechnic State University Jonathan Taylor Department of Statistics, Stanford University
Abstract
To perform inference after model selection, we propose controlling the selective type I error; i.e., the error rate of a test given that it was performed. By doing so, we recover long-run frequency properties among selected hypotheses analogous to those that apply in the classical (non-adaptive) context. Our proposal is closely related to data splitting and has a similar intuitive justification, but is more powerful. Exploiting the classical theory of Lehmann and Scheffé (1955), we derive most powerful unbiased selective tests and confidence intervals for inference in exponential family models after arbitrary selection procedures. For linear regression, we derive new selective $z$ -tests that generalize recent proposals for inference after model selection and improve on their power, and new selective $t$ -tests that do not require knowledge of the error variance.
中文速览
做完变量筛选再做假设检验，如果不纠正选择带来的偏差，发表的结论中假阳性率会远高于名义显著性水平——这正是当前科学可重复性危机的根源之一。这篇论文提出用"选择性I型错误（selective type I error）"作为核心控制标准，即在"某个假设被选中进行检验"这一条件下控制犯错概率，用"数据雕刻（data carving）"代替传统的数据分割：不是把数据硬切成两半，而是只把选择过程中真正用掉的信息剥离出去，把剩余信息留给推断阶段，从而减少信息浪费。借助Lehmann和Scheffé关于指数族模型的经典最优理论，作者推导出任意选择程序后的最强无偏选择性检验和置信区间，并针对线性回归的LASSO变量选择场景，给出了比现有方法更强效的选择性z检验和无需已知误差方差的选择性t检验。模拟实验表明，数据雕刻在统计功效上系统性地优于数据分割，为在真实数据驱动的建模流程中实现有效推断提供了兼具理论最优性与实用性的解决方案。
原文 arXiv:1410.2597;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1410.2597v4