- The paper provides a rigorous asymptotic theory for bootstrap calibration of generalized posterior credible sets, differentiating between sampling and posterior Edgeworth corrections.
- The method achieves level-specific coverage calibration via stochastic approximation, yet its scalar learning rate cannot correct shape discrepancies except under proportional covariance.
- Monte Carlo implementation confirms algorithmic convergence under standard conditions while emphasizing the need for multivariate and high-dimensional calibration extensions.
Bootstrap Calibration Theory for Generalized Posterior Credible Sets
Generalized Posterior Credible Sets and Calibration Problem
Generalized posterior distributions, constructed via exponentiated empirical risk criteria, extend Bayesian inference beyond standard likelihood-based contexts. They are utilized for tasks involving estimating equations, pseudolikelihoods, composite likelihoods, or extremum criteria where the sampling model may be misspecified or unavailable. The resultant generalized posterior offers inferential advantages but suffers from frequentist coverage inconsistency: the curvature governing posterior uncertainty differs from the sandwich covariance describing sampling variability. Scalar learning rate adjustments (“temperature” or “risk-based learning rate”) cannot generally correct for shape misspecification, causing nominal posterior coverage probabilities to diverge from frequentist coverage except in the proportional covariance case.
Prior literature—addressing the misspecification gap, sandwich adjustments, and learning rate selection—has established the necessity of credible set calibration, especially for Gibbs posteriors and quasi-Bayesian methodologies [Bissiri et al., 2016; Grünwald & van Ommen, 2017; Holmes & Walker, 2017]. General Posterior Calibration (GPC) [Syring & Martin, 2019] targets the frequentist coverage of posterior credible sets directly, empirically tuning the learning rate to level-specific coverage via bootstrap resampling and posterior reconstruction. Although computationally appealing, theoretical justification and higher-order coverage error analysis for GPC were limited prior to this work.
Higher-order Asymptotics and Edgeworth Expansions
This paper rigorously analyzes GPC under regular fixed-dimensional asymptotics, leveraging Edgeworth expansions for both the sampling and posterior distributions. Assumption 1 formalizes local expansions for the risk criterion and establishes the population minimizer, regular curvature, and existence of uniform Edgeworth expansions for scalar contrasts.
The core theoretical result separates sources of coverage error:
- Sampling Edgeworth corrections: arise from skewness and kurtosis in the estimator’s distribution.
- Posterior Edgeworth corrections: stem from perturbations in credible set boundaries (due to local posterior skewness, kurtosis, and random posterior quantiles).
A key structural limitation is proven: scalar learning rate calibration cannot match both the scale and shape of the frequentist coverage curve for all nominal levels unless the posterior covariance matrix is proportional to the sampling covariance (“sandwich”)—a strong condition rarely satisfied except in correctly specified models.
Bootstrap Equation and Root Consistency
The calibration problem is reformulated as an inverse problem: for each bootstrap sample, the learning rate is chosen so that the bootstrap coverage curve matches the nominal level. Under uniform coverage approximation and local identification (Assumption 2), the bootstrap-calibrated learning rate converges to the correct population root at the bootstrap approximation rate.
Theorem 1 demonstrates that consistency of the root inherits the uniform coverage approximation rate, typically O(n−1) in smooth problems with scalar containment statistics and accurate bootstrap Edgeworth expansions. Practically, this ensures level-specific calibration is valid under standard regularity.
Theorem 2 further explicates higher-order corrections for scalar-boundary credible sets, presenting explicit terms for the expansion of coverage error at orders n−1/2 and n−1. Sampling and posterior corrections manifest distinctly, allowing for fine-grained analysis of how credible set boundaries and empirical distributional properties impair or enhance coverage calibration.
Monte Carlo Implementation and Stochastic Approximation
In practice, GPC relies on stochastic approximation to solve the bootstrap coverage equation, introducing computational error layers from finite bootstrap iterations, posterior Monte Carlo, and credible set boundary estimation. Algorithmic convergence results (Corollary 1) are established under local Lipschitz conditions and martingale noise, confirming that the stochastic recursion converges to the correct root conditional on the data with high probability, given appropriate step sizes and projection intervals.
These results formalize the asymptotic behavior of implemented GPC algorithms, separating algorithmic errors from the statistical calibration target.
Structural Limitations and Future Directions
The paper’s analysis highlights a fundamental structural limitation: scalar learning rates are insufficient for shape calibration except in proportional covariance cases. Thus, GPC can only calibrate marginal intervals or ellipsoidal credible sets for a single level; full shape correction (e.g., for highest posterior density sets in misspecified models) demands higher-order or multivariate calibration methods.
Future developments should address:
- Extension to full multivariate credible sets with arbitrary geometry, requiring more general multivariate Edgeworth expansions.
- Integration of stochastic approximation error analysis, including Monte Carlo and bootstrap fluctuations.
- Development of calibration approaches for non-smooth generalized posteriors—e.g., for Gibbs losses without regular derivatives.
- Adaptation to dependent data via block/bootstrap methods.
- High-dimensional calibration, which necessitates fundamentally new asymptotic regimes due to breakdown of fixed-dimensional expansions.
Implications and Prospects in Statistical Inference
This work provides rigorous foundational theory for bootstrap-based calibration of generalized posterior credible sets. The separation of coverage errors into sampling and posterior components provides actionable insight into the limits and effectiveness of GPC. In practical terms, it justifies the calibration procedure in standard settings but cautions against over-generalization when sandwich structure or multivariate calibration is required.
Theoretically, the results clarify the interplay of curvature, empirical risk, and credible set boundaries in generalized Bayesian posteriors, bridging frequentist and Bayesian perspectives on coverage. The framework sets the stage for more flexible and robust inference in machine learning and statistical applications where likelihood specification is unattainable or unreliable, but also makes clear that certain shape adjustments are out of reach for scalar tuning.
Conclusion
This paper establishes a comprehensive asymptotic theory for bootstrap coverage calibration of generalized posterior credible sets, revealing both its power and structural limitations. The calibration method is theoretically justified for level-specific coverage in regular scalar settings, but cannot correct shape discrepancies except in restrictive scenarios. The separation of higher-order error sources and explicit convergence results for the calibration algorithm enhance understanding and practical reliability. The structural insights underscore the necessity for further exploration of multivariate, non-smooth, and high-dimensional calibration methodologies to extend the utility of generalized posteriors in complex inferential tasks.
