Learning Coefficients in Semi-Regular Models (2406.02646v4)

Abstract: Recent advances have clarified theoretical learning accuracy in Bayesian inference, revealing that the asymptotic behavior of metrics such as generalization loss and free energy, assessing predictive accuracy, is dictated by a rational number unique to each statistical model, termed the learning coefficient (real log canonical threshold) . For models meeting regularity conditions, their learning coefficients are known. However, for singular models not meeting these conditions, exact values of learning coefficients are provided for specific models like reduced-rank regression, but a broadly applicable calculation method for these learning coefficients in singular models remains elusive. The problem of determining learning coefficients relates to finding normal crossings of Kullback-Leibler divergence in algebraic geometry. In this context, it is crucial to perform appropriate coordinate transformations and blow-ups. This paper introduces an approach that utilizes properties of the log-likelihood ratio function for constructing specific variable transformations and blow-ups to uniformly calculate the real log canonical threshold. It was found that linear independence in the differential structure of the log-likelihood ratio function significantly influences the real log canonical threshold. This approach has not been considered in previous research. In this approach, the paper presents cases and methods for calculating the exact values of learning coefficients in statistical models that satisfy a simple condition next to the regularity conditions (semi-regular models), offering examples of learning coefficients for two-parameter semi-regular models and mixture distribution models with a constant mixing ratio.