Does SLOPE outperform bridge regression?
Shuaiwen Wang Department of Statistics Columbia University NY 10027 USA Haolei Weng∗ Department of Statistics and Probability Michigan State university MI 48824 USA and Arian Maleki Department of Statistics Columbia University NY 10027 USA
Abstract
A recently proposed SLOPE estimator bogdan2015slope has been shown to adaptively achieve the minimax $\ell_{2}$ estimation rate under high-dimensional sparse linear regression models su2016slope . Such minimax optimality holds in the regime where the sparsity level $k$ , sample size $n$ , and dimension $p$ satisfy $k/p\rightarrow 0,k\log p/n\rightarrow 0$ . In this paper, we characterize the estimation error of SLOPE under the complementary regime where both $k$ and $n$ scale linearly with $p$ , and provide new insights into the performance of SLOPE estimators. We first derive a concentration inequality for the finite sample mean square error (MSE) of SLOPE. The quantity that MSE concentrates around takes a complicated and implicit form. With delicate analysis of the quantity, we prove that among all SLOPE estimators, LASSO is optimal for estimating $k$ -sparse parameter vectors that do not have tied non-zero components in the low noise scenario. On the other hand, in the large noise scenario, the family of SLOPE estimators are sub-optimal compared with bridge regression such as the Ridge estimator. Concentration inequality, LASSO, mean square error, noise sensitivity, Ridge, SLOPE
中文速览
SLOPE（排序L1惩罚估计量）此前被证明在稀疏度$k$远小于维度$p$的经典稀疏设定下能达到最优估计速率，但当$k$、样本量$n$与维度$p$同阶线性增长时，不同SLOPE估计量之间孰优孰劣尚不清楚。本文在这一线性增长的高维框架下，利用凸高斯极大极小定理（CGMT）推导出SLOPE均方误差（MSE）的有限样本集中不等式，并通过精细的噪声敏感性分析对各类SLOPE估计量进行系统比较。结果表明：在低噪声场景下，LASSO在所有SLOPE估计量中具有最优的相变边界和噪声敏感性（针对非零分量无重复值的稀疏信号）；而在高噪声场景下，包括LASSO在内的所有SLOPE估计量均劣于岭回归等桥式回归估计量。这一工作首次给出了不同SLOPE估计量MSE的精确刻画与系统对比，为高维稀疏估计中方法选择提供了理论依据。
原文 arXiv:1909.09345;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1909.09345v3