Residual variance and the signal-to-noise ratio in high-dimensional linear models

Abstract: Residual variance and the signal-to-noise ratio are important quantities in many statistical models and model fitting procedures. They play an important role in regression diagnostics, in determining the performance limits in estimation and prediction problems, and in shrinkage parameter selection in many popular regularized regression methods for high-dimensional data analysis. We propose new estimators for the residual variance, the l2-signal strength, and the signal-to-noise ratio that are consistent and asymptotically normal in high-dimensional linear models with Gaussian predictors and errors, where the number of predictors d is proportional to the number of observations n. Existing results on residual variance estimation in high-dimensional linear models depend on sparsity in the underlying signal. Our results require no sparsity assumptions and imply that the residual variance may be consistently estimated even when d > n and the underlying signal itself is non-estimable. Basic numerical work suggests that some of the distributional assumptions made for our theoretical results may be relaxed.