Sample size requirements for machine learning versus regression to achieve comparable stability

Determine whether tree-based machine learning methods, such as random forests, require substantially larger development sample sizes than penalised or unpenalised logistic regression to achieve comparable stability of individual-level risk estimates, and quantify the extent of any sample size differences.

Background

The authors’ approach is grounded in maximum likelihood theory for logistic regression and yields closed-form uncertainty estimates. They compare empirical uncertainty intervals from random forests and logistic regression in supplementary analyses and observe wider intervals and potential miscalibration for random forests under default settings.

They hypothesize that non-regression machine learning methods may be more data hungry for stability but explicitly state that this needs substantiation through further research.