Uncertainty-Quantified Regression

Updated 25 June 2026

Uncertainty-Quantified Regression is a set of methods that rigorously measures predictive uncertainty by accounting for both inherent data noise and model parameter uncertainty.
It integrates classical, semi-parametric, and deep learning approaches to provide calibrated confidence intervals and robust risk assessments.
Applications span active learning, OOD detection, and model discovery, underpinning advances in reliable prediction and decision-making.

Uncertainty-Quantified Regression encompasses a broad class of methodologies aimed at assigning rigorous, probabilistically valid measures of uncertainty to regression predictions. Unlike point estimation, which yields only a mean or median fit, uncertainty-quantified regression provides information about the spread, confidence, and credibility of predictions, thereby enabling principled statistical inference, model comparison, risk assessment, and robust decision-making. This article delineates key theoretical foundations, practical implementations, evaluative metrics, leading methodologies (both classical and modern, including deep and symbolic paradigms), and ongoing challenges in the rigorous quantification and interpretation of predictive uncertainty for regression.

1. Foundational Concepts and Theoretical Frameworks

Uncertainty quantification (UQ) in regression addresses both aleatoric (irreducible data noise) and epistemic (model-based or specification) uncertainty. Formally, for a regression setup in which response $y$ is modeled as a function of covariate $x$ (possibly high-dimensional) with a specified generative process, predictive uncertainty is characterized through:

First-order distribution: $p(y|x,\theta)$ —capturing inherent stochasticity for given parameters $\theta$ .
Second-order (epistemic) distribution: $q(\theta|\mathcal{D})$ —representing uncertainty over model parameters or structures, conditional on data $\mathcal{D}$ .
Total predictive uncertainty: The uncertainty in $y|x,\mathcal{D}$ integrates both.

This leads to a variety of theoretical frameworks:

Inferential Model (IM) Approaches: Simultaneously valid uncertainty quantification using predictive random sets and belief/plausibility measures, guaranteeing frequentist calibration without prior input (Martin et al., 2014).
Proper Scoring Rules: Measures based on strictly proper scoring rules (e.g., log-score, CRPS, squared error), yielding decompositions into aleatoric and epistemic components with strong theoretical guarantees (Fishkov et al., 30 Sep 2025, Bülte et al., 29 Oct 2025).
Axiomatic Foundations: Recent works introduce a system of axioms (nonnegativity, monotonicity, invariance, etc.) that UQ measures should satisfy (Bülte et al., 25 Apr 2025, Goring et al., 17 Jun 2026), exposing strengths and failures of variance-, entropy-, and volume-based measures.
High-Density Region (HDR)/Volume-Based UQ (QUEST): Uncertainty is characterized by the volume of the highest-density region enclosing most of the predictive mass, yielding robust, mode-attentive uncertainty measures with axiomatic guarantees (Goring et al., 17 Jun 2026).
Empirical/Bootstrap and Conformal UQ: Nonparametric construction of calibrated confidence intervals or bands via partitioning, bootstrapping, or conformal inference, often with distribution-free guarantees (Avanesov, 2021, Duma et al., 29 Dec 2025, Hoppe et al., 2024).

2. Parametric, Semi-Parametric, and Distribution-Free UQ Methods

2.1 Gaussian and Exponential-Family Approaches

Classical regression UQ operates under Gaussian model assumptions, producing analytic posterior and predictive intervals via OLS or Bayesian linear regression with reference priors (Martin et al., 2014, Schmähling et al., 2021). For exponential-family models, aleatoric and epistemic components can be extracted directly from the parameter uncertainty using formulas stemming from the law of total variance or entropy/mutual information decompositions (Bülte et al., 25 Apr 2025, Fishkov et al., 30 Sep 2025).

Closed-form Decompositions:
- Predictive variance: $V[y|x,\mathcal{D}] = E_{q(\theta)}[V_{p(y|x,\theta)}[y]] + V_{q(\theta)}[E_{p(y|x,\theta)}[y]]$ .
- Differential entropy: $H(Y) = H(Y|\theta) + I(Y;\theta)$ .

2.2 Semi- and Non-parametric Techniques

Quantile and Histogram/CR Regression: Quantile Regression (QR) and Classification Restoration (CR) approaches reconstruct the conditional distribution adaptively via quantiles or histogram bins, supporting complex and multi-modal targets (Yan et al., 2022, Jin et al., 23 Jun 2026). These can be efficiently implemented in analytic fashion using closure of variational objectives.
Kernel-Based UQ: Kernel score–based UQ generalizes many approaches by choosing a proper scoring kernel, allowing the design of robust or tail-sensitive UQ per application needs (Bülte et al., 29 Oct 2025).

2.3 Distribution-Free and Data-driven UQ

Conformal Prediction: Provides marginal or locally adaptive coverage guarantees (intervals) using residuals, with straightforward extensions to latent variable models (PLS, PCR, kernelized analogues) (Duma et al., 29 Dec 2025).
Divide-and-Conquer/Bootstrap: For large datasets or distributed computation, confidence bands can be constructed using bootstrapped aggregations of local estimators (e.g., KRR) with rigorous simultaneous coverage (Avanesov, 2021).

3. UQ in Modern Deep Regression

3.1 Deep Bayesian and Evidential Methods

Deep Gaussian/MDN Output Layers: Single- or multi-component Gaussian/MDN predictors with NLL training provide straightforward, low-overhead uncertainty estimation, though the decomposition into aleatoric and epistemic uncertainty is limited in single-component models (Wilkins et al., 2019).
Ensembles and Dropout: Deep ensembles and dropout approximate Bayesian epistemic uncertainty via sample variance of predictions (Schmähling et al., 2021, Lind et al., 2024).
Deep Evidential Regression: Neural networks output high-order evidential parameters (e.g., Normal-Inverse-Gamma for Gaussian, Student- $t$ for quantile regression) to disentangle aleatoric/epistemic uncertainty in a single forward pass; these can adapt to non-Gaussian, heteroscedastic, or multi-modal targets (Hüttel et al., 2023).

3.2 Practical and Post-hoc Approaches

Post-hoc Variance Heads: After fitting a mean predictor, variances are post-hoc regressed from intermediate representations on held-out data, addressing mean-variance feature collapse and overfitting (Jordahn et al., 2 Mar 2026).
Input-Output Conditioned Estimators: Auxiliary post-hoc networks explicitly condition on both $x$ 0 and the frozen model output $x$ 1 to approximate aleatoric plus “quasi-epistemic” uncertainty, with theoretically justified gains in OOD detection and calibration (Bramlage et al., 1 Jun 2025).

4. Metrics, Calibration, and Evaluation of Regression UQ

Evaluating uncertainty estimates requires distinct, often orthogonal, metrics:

Metric	What It Measures	Key Recommendations
Calibration Error	Reliability of predictive intervals/quantiles	Most interpretable/stable (Lind et al., 2024)
Negative Log-Likelihood (NLL)	Proper scoring of full predictive density	Sensitive to both calibration and sharpness; less interpretable (Lind et al., 2024)
Area Under Sparsification Error (AUSE)	Ability of U(x) to identify large errors (risk stratification)	Robust, more meaningful than Spearman; use for selective prediction (Lind et al., 2024)
Structure Metric ( $x$ 2), Distribution Metric (NDIP)	Alignment of predicted uncertainty with squared error, distributional match	Useful for model assessment/selection (Pickering et al., 2022)
Proper Scoring Rule Risk/Excess Risk	Total/epistemic/aleatoric uncertainty decomposition	Use excess-risk for OOD/active learning (Fishkov et al., 30 Sep 2025)

For model comparison and benchmarking, deep UQ methods should be evaluated on benchmark tasks with analytic reference posteriors, as in Bayesian linear regression anchors (Schmähling et al., 2021).

5. UQ in Model Discovery and Symbolic Regression

Symbolic regression (SR) faces unique challenges for uncertainty quantification:

Frequentist UQ: Standard confidence and prediction intervals via delta method, Fisher information, and parameter bootstrap apply when the SR structure is fixed (Reuter et al., 4 Jun 2026).
Bayesian SR: Posterior distributions over both parameters and, with advanced SMC or MCMC techniques, over SR model structures, enable full predictive/posterior UQ and model-structural uncertainty quantification (Bomarito et al., 2022, Reuter et al., 4 Jun 2026).
Model-Evidence–Based Selection: Bayesian marginal likelihood and Occam-penalized (FBF/MDL/AIC/BIC) selection drive population evolution, ensuring robust out-of-sample uncertainty control and reducing overfitting (Bomarito et al., 2022).
Open Problems: Scalable, multimodal posterior inference over symbolic structures, finite-calibration guarantees under model misspecification, and UQ-aware benchmarks for SR remain active research directions.

6. Axiomatic and Volume-Based Perspectives: Beyond Variance and Entropy

Limitation of Classical Measures: Variance and entropy-based UQ may fail basic monotonicity, invariance, or nonnegativity axioms in multimodal or heavy-tailed regimes (Bülte et al., 25 Apr 2025, Goring et al., 17 Jun 2026).
QUEST and HDR Volume Measures: Volume-based UQ measures using highest-density regions (HDRs) provide robust, mode-attentive, and axiomatic-unifying uncertainty quantification, with direct links to Lorenz/Gini curves and selective prediction performance (Goring et al., 17 Jun 2026).
Proper Scoring Rule Generalizations: Kernel scores unify variance, entropy, CRPS, and energy-based UQ—choice of kernel dictates trade-offs in robustness, tail sensitivity, and OOD detection (Bülte et al., 29 Oct 2025).

7. Applications and Impact

Uncertainty-quantified regression underpins credible reasoning in scientific, industrial, and safety-critical systems:

Robust Variable Selection and Post-Selection Inference: IM and related frameworks offer simultaneously valid selection with finite-sample family-wise error control, outperforming classical AIC/BIC/lasso rules in simulation and real data (Martin et al., 2014).
Active Learning and Resource Allocation: Explicit epistemic uncertainty (e.g., excess risk, HDR volume, evidential epistemic parameter, or post-hoc quasi-epistemic) improves label efficiency, OOD detection, and sample efficiency in active label acquisition (Hüttel et al., 2023, Jin et al., 23 Jun 2026, Goring et al., 17 Jun 2026).
Quality Assurance in Image Analysis/Medical Applications: Pixel- or region-level Bayesian uncertainty correlated with downstream performance (Dice, segmentation metrics) enables lightweight, trustworthy triage and human-in-the-loop review (Elfatimi et al., 2024).
Distributed and High-Dimensional Settings: Non-asymptotic, data-driven confidence intervals for high-dimensional and neural estimators bridge theory and practice in imaging and inverse problems (Hoppe et al., 2024).