Conformalized Bayesian Inference (CBI)

• Conformalized Bayesian Inference (CBI) is a framework that combines Bayesian analysis with conformal prediction to provide finite-sample, frequentist-style guarantees on credible sets.
• It uses data-derived nonconformity scores and calibration techniques to ensure robust coverage even under model misspecification, covariate shifts, or high-dimensional settings.
• CBI extends to various applications, including supervised learning and clustering, and supports different implementations such as full, split, and online methods for real-time inference.

Conformalized Bayesian Inference (CBI) is an integrative framework at the intersection of Bayesian statistical inference and conformal prediction. CBI systematically attaches finite-sample, frequentist-style guarantees to Bayesian posteriors and predictive sets—even in the presence of model misspecification, nonstandard parameter spaces, covariate shift, or computational constraints. The core mechanism is to use conformal prediction techniques, such as data-derived quantiles of nonconformity scores, to construct credible sets or intervals calibrated to achieve rigorous (often distribution-free) coverage. This approach is unifying, spanning supervised learning, high-dimensional model inference, random partition models, simulation-based inference, online learning, and more.

1. Foundations and Definitional Structure

CBI recasts Bayesian inference as a prediction problem on the space of labels, parameter values, or even structured objects (e.g., partitions, functions). The meta-procedure is to:

• Obtain samples, predictive distributions, or statistics from a Bayesian model or posterior (exact, approximate, or ensemble-based).
• Define a nonconformity (or conformity) score, which measures the “surprise” of an observation or candidate value under the Bayesian predictive or posterior.
• Split data or samples into calibration and inference sets to enable exchangeable ranking.
• Calibrate a threshold (e.g., a quantile of scores) so that, under minimal modeling assumptions (typically only exchangeability), the resulting prediction or credible set achieves guaranteed marginal or conditional coverage at a prescribed nominal level.

Different CBI instantiations (full, split, or online) correspond to different computational and application constraints: full CBI requires recomputation or importance-weighting for each candidate, while split CBI leverages a data split to avoid repeated refitting, and online CBI yields real-time coverage guarantees in streaming or adaptive settings (Bariletto et al., 7 Nov 2025, Deliu et al., 30 Oct 2025, Fong et al., 2021, Zhang et al., 2024).

2. Conformal Prediction Mechanisms within Bayesian Models

Conformal predictions act as wrappers around Bayesian (or generally probabilistic) outputs. In predictive supervised learning and classical parametric models, this reduces to augmenting a Bayesian posterior predictive $p(y\mid x, D)$ with a conformal set construction such as

$$C_{1-\alpha}(x) = \{ y : p(y\mid x, D) \ge q_{1-\alpha} \},$$

where $q_{1-\alpha}$ is an empirical quantile over calibration data conformal scores. The scores can be negative log-likelihoods, residuals, or posterior predictive CDF values (Deliu et al., 30 Oct 2025, Fong et al., 2021). In hierarchical and grouped models, this extends to groupwise conformalization, preserving within-group exchangeability and valid coverage (Fong et al., 2021).

Rather than relying solely on the model's correctness for calibration, CBI interprets the posterior or predictive as a black-box score generator, using exchangeability assumptions and score rankings to determine set boundaries with rigorous error control (Bariletto et al., 7 Nov 2025, Deliu et al., 30 Oct 2025).

3. Extensions to Nonstandard Parameter and Function Spaces

CBI extends naturally to complex combinatorial, functional, or non-Euclidean parameter domains. When the parameter space $\Theta$ is not vectorial (e.g., partitionings, graph structures), CBI leverages kernel density surrogates:

$$s(\theta; D) = \frac{1}{|D|} \sum_{t=1}^{|D|} K\left( \mathcal D(\theta, \theta_t) \right),$$

with $\mathcal D$ a discrepancy or metric and $K$ a smooth kernel. The conformal set is then

$$\mathcal{C}_{1-\alpha} = \{ \theta : s(\theta; D_{\text{train}}) \ge \tau_{1-\alpha} \},$$

where $\tau_{1-\alpha}$ is the $(1-\alpha)$-quantile across calibration samples, yielding a credible “ball” in parameter space (Bariletto et al., 7 Nov 2025).

Density-based clustering algorithms (e.g., Density Peak Clustering) applied to the conformal scores and pairwise discrepancies enable automated detection and summary of posterior modes—even in highly multimodal, nonparametric, or discrete parameter spaces.

4. Assumption-Free Posterior Coverage and Finite-Sample Guarantees

The principal theoretical result underlying CBI is the finite-sample, assumption-free coverage theorem: for any exchangeable sequence of calibration samples from the (possibly misspecified) Bayesian posterior, the conformal region $\mathcal{C}_{1-\alpha}$ constructed as above satisfies

$$\mathbb{P}\left[ \theta^* \in \mathcal{C}_{1-\alpha} \right] \geq 1-\alpha,$$

where $\theta^*$ is an independent posterior draw (Bariletto et al., 7 Nov 2025, Fong et al., 2021, Deliu et al., 30 Oct 2025). When calibration samples are only approximately IID (e.g., MCMC output with thinning), explicit TV bounds control the validity error.

This guarantee is robust to model misspecification, infinite-dimensionality, nonparametric uncertainty, and does not require construction of classical confidence sets or studentization.

5. Point Estimation and Multimodality Analysis

CBI provides a principled procedure for both point and set summarization:

• MAP-like Estimation: The posterior sample $\theta$ maximizing the kernel score $s(\theta; D_{\text{train}})$ is a natural generalization of the modal estimator, even in partition or function spaces.
• Density Peak Clustering: By evaluating $\delta(\theta)$, the minimal distance from $\theta$ to any higher-scoring sample, CBI enables identification of distinct posterior modes as outliers in the $(s, \delta)$ “decision graph,” facilitating exploration and reporting of posterior multimodality (Bariletto et al., 7 Nov 2025).

6. Computational Considerations and Practical Implementation

CBI is highly parallelizable. The kernel density and conformal score evaluations scale as $O(SN)$, while clustering requires $O(N^2)$ pairwise distance computations; both are tractable for thousands of samples using modern compute resources. The framework is agnostic to the choice of $\mathcal D$ and $K$, and supports subsampling or coreset approaches for very large datasets or parameter spaces (Bariletto et al., 7 Nov 2025).

The method is compatible with posterior samples from MCMC, variational inference, or even non-Bayesian generative models, provided the samples are exchangeable or approximately so.

7. Applications: Random Partition Models and Beyond

CBI finds natural application in scenarios where classical Bayesian summarization tools are unavailable or inadequate. In random partition models—such as Dirichlet and Pitman–Yor process mixtures—CBI produces conformal balls of partitions with rigorously calibrated posterior mass, enabling reliable uncertainty quantification in clustering tasks. Empirical results on simulated mixtures, spatial-transcriptomics, and functional data confirm that CBI achieves tight, interpretable credible sets, effective mode detection, and robust calibration—the latter even in instances where classical Bayesian coverage may fail due to model misspecification or domain complexity (Bariletto et al., 7 Nov 2025).

The generality and extensibility of CBI position it as a core methodology for modern Bayesian practice, particularly in high-dimensional or nonstandard statistical domains where classical parametric theory is not applicable.