Conjectured bias–variance behavior of Lasso post-selection for random forests

Determine whether applying Lasso post-selection to the individual tree predictions of a random forest (the post-selection boosting random forest of Wang and Wang, 2021) increases prediction variance—depending on the relative magnitude of uncertainties from fitting the base learners—while reducing prediction bias, in comparison to the vanilla random forest that averages tree predictions uniformly.

Background

The paper analyzes the bias–variance tradeoff between vanilla random forests (uniform averaging of tree predictions) and post-selection forests (Lasso regression on individual tree predictions). For the special case without regularization (λ=0), the authors derive closed-form expressions for mean squared error and show exact bias–variance behavior. However, for the Lasso-regularized case (λ>0), closed-form derivations are not available.

In this context, the authors explicitly conjecture the qualitative behavior of Lasso post-selection: that it may increase variance relative to the vanilla forest due to uncertainties in fitting base learners, yet provide benefits via bias reduction. This conjecture calls for a theoretical characterization or proof of these effects under the stated modeling framework.