Stochastic Lipschitz continuity for high dimensional Lasso with multiple linear covariate structures or hidden linear covariates

Abstract: Two extensions of generalized linear models are considered. In the first one, response variables depend on multiple linear combinations of covariates. In the second one, only response variables are observed while the linear covariates are missing. We derive stochastic Lipschitz continuity results for the loss functions involved in the regression problems and apply them to get bounds on estimation error for Lasso. Multivariate comparison results on Rademacher complexity are obtained as tools to establish the stochastic Lipschitz continuity results.