Another look at inference after prediction (2411.19908v4)

Abstract: From structural biology to epidemiology, predictions from ML models are increasingly used to complement costly gold-standard data to enable faster, more affordable, and scalable scientific inquiry. In response, prediction-based (PB) inference has emerged to accommodate statistical analysis using a large volume of predictions together with a small amount of gold-standard data. The goals of PB inference are two-fold: (i) to mitigate bias from errors in predictions and (ii) to improve efficiency relative to traditional inference using only the gold-standard data. While early PB inference methods focused on bias, their ability to enhance efficiency remains unclear. We revisit a popular PB inference method and show that a simple modification can be applied to guarantee improvements in efficiency beyond yielding valid inferences when the ML predictions are imperfect. The utility of this approach in leveraging prediction-based outcomes to enhance efficiency is demonstrated through extensive simulation studies and an application to the UK Biobank data. We further contextualize the problem of PB inference through historical literature from economics and statistics to highlight perspectives from classical methods in this contemporary problem.