Calibration for fitted prediction functions

Extend the goodness-of-fit calibration for the conditional mean-squared-error invariance test from prediction functions held fixed relative to the evaluation sample to prediction functions fitted using data, while controlling the fitted function’s estimation error relative to the evaluation-sample size.

Background

The paper develops a goodness-of-fit test for whether a fixed prediction function has constant conditional mean squared error given its prediction. Its asymptotic calibration is established when the prediction function is determined independently of the sample used to evaluate the test statistic. The unresolved issue is how to preserve valid calibration when the tested prediction function is estimated from data, particularly when the training and evaluation sample sizes jointly increase.