Universal Inference for Testing Calibration of Mean Estimates within the Exponential Dispersion Family

Abstract: Calibration of mean estimates for predictions is a crucial property in many applications, particularly in the fields of financial and actuarial decision-making. In this paper, we first review classical approaches for validating mean-calibration, and we discuss the Likelihood Ratio Test (LRT) within the Exponential Dispersion Family (EDF). Then, we investigate the framework of universal inference to test for mean-calibration. We develop a sub-sampled split LRT within the EDF that provides finite sample guarantees with universally valid critical values. We investigate type I error, power and e-power of this sub-sampled split LRT, we compare it to the classical LRT, and we propose a novel test statistics based on the sub-sampled split LRT to enhance the performance of the calibration test. A numerical analysis verifies that our proposal is an attractive alternative to the classical LRT achieving a high power in detecting miscalibration.