Ensemble-Based Uncertainty Quantification

• Ensemble-based uncertainty quantification is a methodology that combines predictions from multiple models to quantify and separate uncertainty into aleatoric and epistemic components.
• It employs diverse ensemble construction strategies—such as deep ensembles, bootstrap methods, and Bayesian nonparametric approaches—to enhance prediction reliability and calibration.
• Applications span clinical diagnosis, atmospheric chemistry, and geosteering, among others, providing actionable insights for model averaging and improved decision-making.

Searching arXiv for recent and foundational papers on ensemble-based uncertainty quantification. Searching arXiv for "ensemble-based uncertainty quantification". Ensemble-based uncertainty quantification is a class of methods in which multiple models, hypotheses, realizations, or sampled responses are combined so that the aggregate prediction and the variation across members can be used for model averaging, selection, calibration, and uncertainty quantification. Across the literature, the ensemble object may be a posterior-weighted collection of hypotheses, a deep ensemble of independently trained neural networks, an iterative ensemble smoother, an ensemble of sparse dynamical systems, or a set of predictive distributions induced by a credal set. The resulting uncertainty can be expressed through entropy, variance, quantile intervals, posterior predictive distributions, or membership variability, and many formulations attempt to separate aleatoric and epistemic components explicitly (Shaker et al., 2021, Egele et al., 2021).

1. Formal viewpoints and probabilistic semantics

A general formalization appears in the distinction between probabilistic agents, Bayesian agents, and Levi agents. In this setting, a hypothesis is a mapping $h : \mathcal{X} \to \mathbb{P}(\mathcal{Y})$. Bayesian agents maintain a posterior distribution $p(h \mid \mathcal{D})$ and form predictions by Bayesian model averaging,

$q \defeq \operatorname{bma}(p) = \int_{\mathcal{H}} h(x_q)\, dP(h).$

Levi agents instead use a credal set $Q' \subseteq \mathbb{P}(\mathcal{H})$, which induces a set of predictive distributions

$Q \defeq \big\{ \operatorname{bma}(p) \mid p \in Q' \big\} \subseteq \mathbb{P}(\mathcal{Y}),$

thereby representing uncertainty more conservatively than a single posterior predictive distribution (Shaker et al., 2021).

This Bayesian-versus-credal distinction is important because it clarifies that an ensemble need not correspond to a single averaged predictor. In some frameworks, the ensemble is itself a set-valued predictive object, and uncertainty is attached to the width or non-specificity of that set rather than only to the entropy of one averaged distribution. This suggests that ensemble-based uncertainty quantification is not reducible to “prediction spread” alone.

A further expansion of the probabilistic semantics appears in the Bayesian nonparametric ensemble model. Starting from the standard stacking-style regression model

$Y = \sum_{k=1}^K f_k(x)\theta_k + \epsilon,$

the Bayesian nonparametric ensemble augments the mean with a residual process,

$$Y = \sum_{k=1}^K f_k(x)\theta_k + \delta(x) + \epsilon,$$

and calibrates the full predictive CDF through

$$F^*(y\mid x) = G\!\left[D_{\epsilon}(y\mid x,\mu)\right], \qquad \mu = \sum_{k=1}^K f_k(x)\theta_k + \delta(x).$$

In this construction, posterior uncertainty lives in the joint posterior over $\{\theta,\delta,G\}$, so the ensemble is used to capture parametric uncertainty in $\theta$, structural uncertainty in the prediction function via $p(h \mid \mathcal{D})$0, and structural uncertainty in the distribution function via $p(h \mid \mathcal{D})$1 (Liu et al., 2019).

2. Ensemble construction strategies

Deep ensembles remain a prominent construction. In multi-label thoracic disease diagnosis, a 9-member Deep Ensemble was selected from an initial pool of 14 candidate models by systematic variation across architecture, loss function, and random seed. Diversity was induced by combining DenseNet-121, DenseNet-121 + CBAM attention, EfficientNet-B2, and EfficientNet-B3; by training with both Focal Loss and ZLPR Loss; and by using multiple random seeds. The final ensemble prediction was the uniform average

$p(h \mid \mathcal{D})$2

The authors explicitly contrasted this with Monte Carlo Dropout and treated the ensemble’s disagreement as a direct estimate of epistemic uncertainty (Laksara et al., 24 Nov 2025).

In regression, AutoDEUQ automates ensemble formation by joint neural architecture and hyperparameter search. It uses aging evolution for architecture search, Bayesian optimization for hyperparameter search, trains each candidate with Gaussian negative log-likelihood, and then applies greedy selection to form the final ensemble. The paper explicitly states that AutoDEUQ differs from standard deep ensembles because members are not just different random initializations of the same architecture; instead, both architectures and hyperparameters are automatically varied (Egele et al., 2021).

Other ensemble generators are organized around resampling or optimization trajectories. For neural network interatomic potentials, the comparative study examined bootstrap, dropout, random initialization, and snapshot ensembles, and emphasized that these methods generate spread estimates with different diversity properties and different computational tradeoffs (Kurniawan et al., 8 Aug 2025). In reduced-order atmospheric chemistry, Ensemble SINDy bootstraps both the training observations and the candidate equation terms, so each ensemble member corresponds to a different sparse ODE model fitted on a different bootstrap sample (Guo et al., 2024). In geosteering, uncertainty is represented by a prior ensemble of $p(h \mid \mathcal{D})$3–$p(h \mid \mathcal{D})$4 realizations, which is then iteratively updated by LM-EnRML to form a posterior ensemble over resistivity, density, and boundary locations (Jahani et al., 2021).

Not all ensemble mechanisms average predictions. LUQ-Ensemble computes a sentence-level consistency-based uncertainty score for each LLM and then selects the response from the LLM with the lowest LUQ score for a given question. This is a winner-take-all selection scheme rather than weighted averaging, but uncertainty still functions as the ensemble control variable (Zhang et al., 2024).

At the level of computational infrastructure, embedded ensemble propagation groups several stochastic samples into a single ensemble type so that the solver operates on a group of samples simultaneously. The method replaces repeated scalar simulation with a commuted representation,

$p(h \mid \mathcal{D})$5

thereby reusing sparse graph data, improving memory access patterns, and amortizing communication costs across the ensemble (Phipps et al., 2015).

3. Uncertainty measures and decomposition

A central theme is the decomposition of total uncertainty into aleatoric and epistemic components. For Bayesian ensemble classification, total uncertainty is the entropy of the ensemble-predicted class probabilities,

$p(h \mid \mathcal{D})$6

aleatoric uncertainty is the posterior-weighted average entropy of member predictions,

$p(h \mid \mathcal{D})$7

and epistemic uncertainty is the residual

$p(h \mid \mathcal{D})$8

In the Bayesian interpretation this residual is mutual information, whereas in the credal formulation alternative measures such as generalized Hartley entropy, upper entropy, and lower entropy are used (Shaker et al., 2021).

In deep-ensemble regression, the standard decomposition is based on the law of total variance. AutoDEUQ assumes each member predicts a Gaussian distribution $p(h \mid \mathcal{D})$9, and for an ensemble $q \defeq \operatorname{bma}(p) = \int_{\mathcal{H}} h(x_q)\, dP(h).$0 writes

$q \defeq \operatorname{bma}(p) = \int_{\mathcal{H}} h(x_q)\, dP(h).$1

Its empirical form is

$q \defeq \operatorname{bma}(p) = \int_{\mathcal{H}} h(x_q)\, dP(h).$2

A closely related formulation appears in multi-output aerodynamic regression, where the ensemble predictive variance is written as the sum of averaged within-model variance and between-model mean variance (Yang et al., 2023, Egele et al., 2021).

In classification, the same decomposition is often expressed through predictive entropy. For the thoracic disease ensemble,

$q \defeq \operatorname{bma}(p) = \int_{\mathcal{H}} h(x_q)\, dP(h).$3

$q \defeq \operatorname{bma}(p) = \int_{\mathcal{H}} h(x_q)\, dP(h).$4

$q \defeq \operatorname{bma}(p) = \int_{\mathcal{H}} h(x_q)\, dP(h).$5

The authors interpret aleatoric uncertainty as irreducible data uncertainty and epistemic uncertainty as reducible model uncertainty caused by incomplete knowledge or limited data coverage (Laksara et al., 24 Nov 2025).

Quantile-based ensembles replace variance heads with interval prediction. Ensemble Quantile Regression uses pinball loss,

$q \defeq \operatorname{bma}(p) = \int_{\mathcal{H}} h(x_q)\, dP(h).$6

and interprets within-member quantile interval width as aleatoric uncertainty and disagreement among members’ quantile predictions as epistemic uncertainty. The associated progressive sampling algorithm treats uncertainty type operationally: epistemic regions are those that disappear after targeted data enrichment, whereas aleatoric regions persist (Ansari et al., 2024).

4. Calibration, reliability, and evaluation

Ensemble-based uncertainty quantification is evaluated not only by predictive accuracy but also by calibration. In clinical multi-label classification, the decisive comparison was between Monte Carlo Dropout and a 9-member Deep Ensemble: Monte Carlo Dropout yielded an Expected Calibration Error of $q \defeq \operatorname{bma}(p) = \int_{\mathcal{H}} h(x_q)\, dP(h).$7, Negative Log-Likelihood of $q \defeq \operatorname{bma}(p) = \int_{\mathcal{H}} h(x_q)\, dP(h).$8, and Brier Score of $q \defeq \operatorname{bma}(p) = \int_{\mathcal{H}} h(x_q)\, dP(h).$9, while the Deep Ensemble achieved Mean ECE $Q' \subseteq \mathbb{P}(\mathcal{H})$0, mean NLL $Q' \subseteq \mathbb{P}(\mathcal{H})$1, and mean Brier Score $Q' \subseteq \mathbb{P}(\mathcal{H})$2. The same study reported mean total uncertainty $Q' \subseteq \mathbb{P}(\mathcal{H})$3, mean aleatoric uncertainty $Q' \subseteq \mathbb{P}(\mathcal{H})$4, and mean epistemic uncertainty $Q' \subseteq \mathbb{P}(\mathcal{H})$5, and interpreted the low epistemic term as evidence that remaining uncertainty was dominated by irreducible ambiguity in the data (Laksara et al., 24 Nov 2025).

In high-dimensional inverse problems, calibration is often assessed through coverage. For computational optical form measurements, an ensemble of $Q' \subseteq \mathbb{P}(\mathcal{H})$6 U-Nets used the elementwise standard deviation across member predictions as uncertainty, and coverage probability was defined through the interval $Q' \subseteq \mathbb{P}(\mathcal{H})$7. On the perfectly calibrated test system, the ensemble RMSE was $Q' \subseteq \mathbb{P}(\mathcal{H})$8 nm and the total coverage probability was $Q' \subseteq \mathbb{P}(\mathcal{H})$9, close to the nominal $Q \defeq \big\{ \operatorname{bma}(p) \mid p \in Q' \big\} \subseteq \mathbb{P}(\mathcal{Y}),$0 target (Hoffmann et al., 2021).

Reduced-order atmospheric chemistry employed calibration curves and a calibration $Q \defeq \big\{ \operatorname{bma}(p) \mid p \in Q' \big\} \subseteq \mathbb{P}(\mathcal{Y}),$1 between the empirical calibration curve and the perfect calibration line. With 100 ensemble members, calibration $Q \defeq \big\{ \operatorname{bma}(p) \mid p \in Q' \big\} \subseteq \mathbb{P}(\mathcal{Y}),$2-squared was $Q \defeq \big\{ \operatorname{bma}(p) \mid p \in Q' \big\} \subseteq \mathbb{P}(\mathcal{Y}),$3 among the three latent species on average and $Q \defeq \big\{ \operatorname{bma}(p) \mid p \in Q' \big\} \subseteq \mathbb{P}(\mathcal{Y}),$4 for ozone, indicating that predicted model uncertainty aligned well with actual model error (Guo et al., 2024).

Multi-output regression exposed a different calibration pathology. As the number of deep-ensemble members increased from DE-2 to DE-16, the study observed an underconfidence trend and proposed post-hoc STD scaling,

$Q \defeq \big\{ \operatorname{bma}(p) \mid p \in Q' \big\} \subseteq \mathbb{P}(\mathcal{Y}),$5

with $Q \defeq \big\{ \operatorname{bma}(p) \mid p \in Q' \big\} \subseteq \mathbb{P}(\mathcal{Y}),$6 chosen to minimize validation NLL. The theoretical explanation was expressed through the deviation-from-calibration score

$Q \defeq \big\{ \operatorname{bma}(p) \mid p \in Q' \big\} \subseteq \mathbb{P}(\mathcal{Y}),$7

showing that greater ensemble spread can push the aggregated predictor toward overestimated uncertainty (Yang et al., 2023).

Beyond calibration curves and coverage, other evaluation protocols include accuracy-rejection curves for classification with a reject option, where good uncertainty measures should induce monotonically improving accuracy as increasingly uncertain examples are rejected (Shaker et al., 2021).

5. Scientific and engineering applications

The range of applications is unusually broad. In nuclear mass modeling, ensemble Bayesian model averaging combines mass models as a mixture of normal distributions, whose parameters are optimized against the experimental data by Markov chain Monte Carlo using the No-U-Turn sampler. The reported average size of the best uncertainty estimates of neutron separation energies based on the AME2003 data was $Q \defeq \big\{ \operatorname{bma}(p) \mid p \in Q' \big\} \subseteq \mathbb{P}(\mathcal{Y}),$8 MeV and covered $Q \defeq \big\{ \operatorname{bma}(p) \mid p \in Q' \big\} \subseteq \mathbb{P}(\mathcal{Y}),$9 of new data in the AME2020; the uncertainty estimates were also used to detect outliers with respect to the trend of experimental data and theoretical predictions (Saito et al., 2023).

In geosteering, LM-EnRML performs prior ensemble generation, forecast through forward simulators, and analysis by statistical misfit minimization. The posterior mean is the best estimate and the posterior standard deviation is the uncertainty measure for resistivity, density, and boundary positions. The paper reports about $Y = \sum_{k=1}^K f_k(x)\theta_k + \epsilon,$0–$Y = \sum_{k=1}^K f_k(x)\theta_k + \epsilon,$1 times lower computational time than Metropolis-Hastings Monte Carlo, and emphasizes that the ensemble-based method can run in parallel on multiple CPUs (Jahani et al., 2021).

In atmospheric chemistry, PCA + E-SINDy interprets the selected ensemble as an approximate empirical posterior over plausible reduced-order dynamical systems. The posterior predictive distribution mean improves accuracy over deterministic SINDy, and the $Y = \sum_{k=1}^K f_k(x)\theta_k + \epsilon,$2 confidence interval provides calibrated uncertainty for ozone and latent species (Guo et al., 2024).

Trajectory clustering in ocean ensemble forecasts uses repeated spectral clustering across realizations and then computes ensemble mean membership probabilities and sample standard deviations,

$Y = \sum_{k=1}^K f_k(x)\theta_k + \epsilon,$3

Low standard deviation indicates robust clustering, while high standard deviation appears near cluster boundaries or in regions where different realizations yield different coherent structures (Vieira et al., 2020).

Ensemble-based uncertainty quantification also appears in seismic inversion through weighted deep ensembles with importance sampling, in long-text generation through self-consistency-based model selection, and in generative model evaluation through aggregated precision-recall curves with percentile bands. These cases suggest that ensemble-based UQ applies not only to predictive distributions over outputs, but also to uncertainty in inversion results, long-form factuality, and evaluation metrics themselves (Qu et al., 2024, Zhang et al., 2024, Morales et al., 13 Nov 2025).

6. Limitations, failure modes, and current directions

The literature is explicit that ensemble-based uncertainty quantification is not a universal reliability certificate. In neural network interatomic potentials, uncertainty estimates can plateau or even decrease as predictive errors grow in out-of-distribution settings, and ensemble members can become confidently wrong together because spread measures disagreement among models rather than truth (Kurniawan et al., 8 Aug 2025). In steady-state CFD, iterative ensemble methods with small ensemble sizes do not accurately capture the true posterior distribution, although they can still provide a good estimation of the uncertainties; among EnKF, EnRML, and EnKF-MDA, EnRML was identified as the best approximate Bayesian UQ approach in that comparison (Zhang et al., 2020).

Method-specific failures are equally prominent. Monte Carlo Dropout failed badly in the thoracic disease study, where local stochastic sampling around one set of learned weights was described as a “distribution of similar functions” rather than genuinely diverse predictors (Laksara et al., 24 Nov 2025). In credal inference, the disaggregation measures $Y = \sum_{k=1}^K f_k(x)\theta_k + \epsilon,$4 and $Y = \sum_{k=1}^K f_k(x)\theta_k + \epsilon,$5 performed poorly empirically, even though total uncertainty and some individual epistemic or aleatoric measures were effective (Shaker et al., 2021). Quantile-based separation methods also acknowledge leakage between aleatoric and epistemic components and therefore rely on iterative resampling rather than a purely static decomposition (Ansari et al., 2024).

Computational cost remains a structural issue. Training multiple models, bootstrapping libraries, or repeatedly retraining during progressive sampling is expensive, and several papers treat parallelism, greedy selection, or embedded propagation as necessary implementation responses rather than optional optimizations (Phipps et al., 2015, Egele et al., 2021). A plausible implication is that future work will continue to focus on scalable uncertainty estimation, robust calibration under distribution shift, and uncertainty measures that remain informative when all ensemble members share the same inductive bias.