Bayesian Smoothing and Feature Selection Using variational Automatic Relevance Determination

Abstract: This study introduces Variational Automatic Relevance Determination (VARD), a novel approach tailored for fitting sparse additive regression models in high-dimensional settings. VARD distinguishes itself by its ability to independently assess the smoothness of each feature while enabling precise determination of whether a feature's contribution to the response is zero, linear, or nonlinear. Further, an efficient coordinate descent algorithm is introduced to implement VARD. Empirical evaluations on simulated and real-world data underscore VARD's superiority over alternative variable selection methods for additive models.