Quantile universal threshold for model selection (1511.05433v3)

Abstract: Efficient recovery of a low-dimensional structure from high-dimensional data has been pursued in various settings including wavelet denoising, generalized linear models and low-rank matrix estimation. By thresholding some parameters to zero, estimators such as lasso, elastic net and subset selection allow to perform not only parameter estimation but also variable selection, leading to sparsity. Yet one crucial step challenges all these estimators: the choice of the threshold parameter~$\lambda$. If too large, important features are missing; if too small, incorrect features are included. Within a unified framework, we propose a new selection of $\lambda$ at the detection edge under the null model. To that aim, we introduce the concept of a zero-thresholding function and a null-thresholding statistic, that we explicitly derive for a large class of estimators. The new approach has the great advantage of transforming the selection of $\lambda$ from an unknown scale to a probabilistic scale with the simple selection of a probability level. Numerical results show the effectiveness of our approach in terms of model selection and prediction.