Abstract

We revisit the classic semiparametric problem of inference on a low dimensional parameter $\theta_{0}$ in the presence of high-dimensional nuisance parameters $\eta_{0}$ . We depart from the classical setting by allowing for $\eta_{0}$ to be so high-dimensional that the traditional assumptions, such as Donsker properties, that limit complexity of the parameter space for this object break down. To estimate $\eta_{0}$ , we consider the use of statistical or machine learning (ML) methods which are particularly well-suited to estimation in modern, very high-dimensional cases. ML methods perform well by employing regularization to reduce variance and trading off regularization bias with overfitting in practice. However, both regularization bias and overfitting in estimating $\eta_{0}$ cause a heavy bias in estimators of $\theta_{0}$ that are obtained by naively plugging ML estimators of $\eta_{0}$ into estimating equations for $\theta_{0}$ . This bias results in the naive estimator failing to be $N^{-1/2}$ consistent, where $N$ is the sample size. We show that the impact of regularization bias and overfitting on estimation of the parameter of interest $\theta_{0}$ can be removed by usin