Linear Models are Most Favorable among Generalized Linear Models

Abstract: We establish a nonasymptotic lower bound on the $L_2$ minimax risk for a class of generalized linear models. It is further shown that the minimax risk for the canonical linear model matches this lower bound up to a universal constant. Therefore, the canonical linear model may be regarded as most favorable among the considered class of generalized linear models (in terms of minimax risk). The proof makes use of an information-theoretic Bayesian Cram\'er-Rao bound for log-concave priors, established by Aras et al. (2019).