Structure Adaptive Elastic-Net

Abstract: Penalized linear regression is of fundamental importance in high-dimensional statistics and has been routinely used to regress a response on a high-dimensional set of predictors. In many scientific applications, there exists external information that encodes the predictive power and sparsity structure of the predictors. In this article, we propose the Structure Adaptive Elastic-Net (SA-Enet), which provides a new framework for incorporating potentially useful side information into a penalized regression. The basic idea is to translate the external information into different penalization strengths for the regression coefficients. We particularly focus on group and covariate-dependent structures and study the risk properties of the resulting estimator. To this, we generalize the state evolution framework recently introduced for the analysis of the approximate message-passing algorithm to the SA-Enet framework. We show that the finite sample risk of the SA-Enet estimator is consistent with the theoretical risk predicted by the state evolution equation. Our theory suggests that the SA-Enet with an informative group or covariate structure can outperform the Lasso, Adaptive Lasso, Sparse Group Lasso, Feature-weighted Elastic-Net, and Graper. This evidence is further confirmed in our numerical studies. We also demonstrate the usefulness and the superiority of our method for leukemia data from molecular biology and precision medicine.