Posterior Uncertainty Quantification

• Posterior uncertainty quantification is the Bayesian assessment of model uncertainties by analyzing full posterior distributions to derive credible intervals and sets.
• It employs methods like Optional Pólya Tree priors and Bernstein–von Mises theorems to ensure adaptive convergence rates and honest coverage in complex models.
• The technique is pivotal in density estimation, inverse problems, and high-dimensional inference, offering practical tools for uncertainty calibration in advanced applications.

Posterior uncertainty quantification is the rigorous assessment, from a Bayesian perspective, of the uncertainty remaining in model parameters, latent variables, or predictions after conditioning on observed data. By characterizing the full posterior distribution—including marginal or functional summaries rather than just point estimates—posterior UQ provides credible intervals, credible sets, hypothesis tests, and quantifies both aleatoric and epistemic uncertainty. It plays a pivotal role in fields ranging from inverse problems, dynamical system identification, scientific computing, machine learning, and high-dimensional statistical inference.

1. Bayesian Formulation and Posterior Law

At the core of posterior UQ is the Bayesian inferential framework, in which a prior distribution $p(\theta)$ over unknowns $\theta$ is combined with observed data $D$ via the likelihood $p(D|\theta)$. The posterior is given by Bayes’ theorem: $p(\theta|D) \propto p(D|\theta)p(\theta)$ Posterior UQ focuses on the random variable or functional of interest $f(\theta)$ (parameter, prediction, density, or derived statistic) and quantifies the uncertainty by analyzing $p(\theta|D)$ and its induced distribution for $f(\theta)$.

For example:

• In parameter inference, $f(\theta) = \theta$ and one may study marginal CIs or HPD regions.
• In prediction, $f(\theta) = p(y_{\text{new}} | \theta)$ leads to a posterior predictive distribution integrating over uncertainty in $\theta$.
• In density estimation, such as with tree-based priors, $f$ may be a function-valued variable.

Posterior UQ aims to provide both local and global quantification: local credible intervals, global credible regions, and credibly supported features (e.g., in spatial or functional domains).

2. Posterior Rate of Convergence and Adaptive Uncertainty Bands

A central statistical guarantee in posterior UQ is the contraction of the posterior to the true parameter or function at optimal rates, which determines the shrinking of credible intervals/bands as $n\to\infty$. In nonparametric settings, such as density estimation with Optional Pólya Tree (OPT) priors, these contraction rates can adapt to unknown smoothness. Given H\"older regularity $\alpha$, the minimax-optimal sup-norm posterior contraction rate is (up to logarithmic factors): $$\varepsilon_n(\alpha) = \left( n^{-1}\log^2 n \right)^{\alpha/(1+2\alpha)}$$ For $f_0$ in the class of densities of regularity $\alpha$,

$$E_{f_0}\; \Pi \Bigl[\|f-f_0\|_\infty > M_n\,\varepsilon_n(\alpha)\;\big|\;X\Bigr] \to 0$$

ensures credible sets will shrink at this rate. This automatic adaptation arises in OPT priors by randomizing the tree structure and mass splits, so regions with higher sample concentration are partitioned more deeply, providing spatially adaptive bandwidth.

Posterior UQ also extends to cumulative functionals: Bernstein-von Mises-type theorems provide asymptotic normality of the posterior for the empirical distribution function $F(x) = \int_0^x f(t)dt$, enabling Kolmogorov–Smirnov-type credible bands with parametric rate $O(1/\sqrt n)$.

3. Construction and Calibration of Credible Sets

Credible sets in posterior UQ are regions in parameter or function space with prescribed posterior probability. For density estimation with OPT priors:

• The median-tree estimator $f_{T^*}$ (wavelet projection onto majority-supported dyadic tree) serves as a functional posterior center.
• Simple sup-norm credible bands are defined as

$$\mathcal{C}_n = \left\{ f : \|f - f_{T^*}\|_\infty \leq \sigma_n \right\}$$

with $$\sigma_n = v_n \sqrt{ \frac{\log n}{n} 2^{\,d(T^*)/2} }$$, where $d(T^*)$ is the depth of $T^*$. These bands, under mild self-similarity assumptions, achieve honest asymptotic coverage and have adaptive diameter in $\alpha$.

• Intersection with multiscale balls in Banach spaces (e.g., $\mathcal{M}_0(w)$) sharpens the bands. The resulting credible set

$$\mathcal{C}_n^{\text{multi}} = \left\{ f : \|f-f_{T^*}\|_\infty \le \sigma_n\right\} \cap \left\{\|f-f_{T^*}\|_{\mathcal{M}_0(w)} \le R_n/\sqrt n\right\}$$

achieves nominal coverage $1-\gamma$.

• For the distribution function $F$, Kolmogorov–Smirnov bands are derived:

$$\widetilde{\mathcal{C}}_n = \left\{ F : \|F - F_{T^*}\|_\infty \le q_{1-\gamma}/\sqrt n\right\}$$

where $q_{1-\gamma}$ is the empirical $(1-\gamma)$ quantile of the posterior law of $\sqrt n\|F-F_{T^*}\|_\infty$.

All sets above are shown to have asymptotic frequentist coverage matching their posterior credibility.

4. Technical Methodologies: Tree-Based and Multiscale Priors

Posterior UQ relies on explicit construction of priors and posterior laws adequate for functional inference:

• Optional Pólya Tree Priors: The prior is a two-step mixture: a random binary tree structure $T$ (Galton–Watson process), and within $T$, Beta-distributed splits assign mass to subintervals. Conditional on $T$, the posterior for each split is Beta-Binomial conjugate; the posterior on $T$ itself updates via Bayes’ rule incorporating data likelihood in a recursive fashion. Credible bands are constructed around the median-tree projection in Haar or wavelet basis.
• Multiscale Banach Space Techniques: Sup-norm and multiscale norms are used for constructing credible sets that adapt to both local and global regularity.
• Bernstein–von Mises Theorems: Posterior laws for empirical functionals are proven to converge (in distribution) to Gaussian processes (e.g., Brownian bridge for $F$), thereby calibrating credible intervals.

5. Empirical Validation and Simulation Studies

Simulation studies corroborate the theoretical guarantees:

• In structurally diverse scenarios (Lipschitz, random exponentials, mixtures, smooth functions), median-tree estimators adaptively track both rough and smooth features as $n$ increases.
• Empirical credible bands (both simple sup-norm and multiscale-augmented) attain the predicted nominal coverage and adapt diameter according to local smoothness.
• Posterior empirical CDF bands closely follow true distributions even for moderate sample sizes.
• Empirical analyses confirm asymptotic independence between sup-norm credible ball and multiscale ball, as theoretically predicated for Gaussian process priors.

6. Applications and Extensibility

Posterior uncertainty quantification via adaptive Bayesian methods extends naturally to numerous nonparametric and functional settings:

• Structure-learning and feature support in image/functional data
• Testing or inclusion probabilities for hypotheses about local features
• Extending credible set theory to more general Banach spaces and tree/partition models

By furnishing adaptive, honest, and fully probabilistic uncertainty quantification for both parameters and functionals, posterior UQ with structured priors provides a principled statistical foundation for credible inference in complex, high-dimensional, and nonparametric models, as demonstrated in density estimation with Optional Pólya Trees (Castillo et al., 2021).