Density-Informed Pseudo-count EDL

• DIP-EDL is a framework that integrates covariate density with evidential deep learning to decouple epistemic and aleatoric uncertainty.
• It re-parameterizes Dirichlet posteriors as density-weighted pseudo-counts, balancing local data support with class-conditional evidence.
• Empirical results on benchmarks such as MNIST and CIFAR-10 show improved in-distribution accuracy and reliable out-of-distribution detection.

Density-Informed Pseudo-count EDL (DIP-EDL) is a statistical and algorithmic framework for uncertainty-aware classification that enhances Evidential Deep Learning (EDL) by explicitly incorporating covariate density information into the Dirichlet evidence structure. DIP-EDL achieves calibrated separation of epistemic and aleatoric uncertainty, robustifies predictive behavior under distributional shift, and enables reliable out-of-distribution (OOD) detection by re-parametrizing Dirichlet posteriors as density-weighted pseudo-counts, rather than relying solely on global temperature regularization or single-source evidence aggregation (Carlotti et al., 1 Feb 2026).

1. Statistical Foundations: EDL and Hierarchical Bayesian Perspective

Standard EDL constructs a predictive Dirichlet distribution over class probabilities $\pi$ for each input $x$, parameterized as $\mathrm{Dir}(\alpha(x))$, where $\alpha(x) = 1 + h(z(x))$ for some nonnegative activation $h$ applied to neural network logits $z(x)$. The modern interpretation formalizes EDL as amortized variational inference in a hierarchical independent Categorical–Dirichlet (ICD) model with a tempered pseudo-likelihood.

Given class priors $\alpha \in \mathbb{R}_+^K$, and for data $X_i, Y_i$:

• $p_i \sim \mathrm{Dir}(\alpha)$
• $Y_i \mid p_i \sim \mathrm{Cat}(p_i)$

Tempered pseudo-likelihood modifies the standard likelihood $f_p(Y_i) = \prod_k p_k^{1_{Y_i=k}}$ to $f_p(Y_i)^\nu$, introducing a global temperature $\nu > 0$ that drives a Dirichlet posterior $\mathrm{Dir}(\alpha + \nu e_{Y_i})$. EDL’s amortized inference seeks $\phi$ so that $$q_\phi(p_i|X_i) = \mathrm{Dir}(\alpha + NN^\phi(X_i))$$ minimizes the KL divergence against this tempered posterior, leading to the canonical EDL loss: $$\mathcal{L}_{\mathrm{EDL}}^\lambda(\phi) = \sum_i \left\{ -\mathbb{E}_{p \sim \mathrm{Dir}(\alpha + NN^\phi(X_i))}[\log \mathrm{Cat}(Y_i|p)] + \lambda\,\mathrm{KL}(\mathrm{Dir}(\alpha + NN^\phi(X_i))\,\|\,\mathrm{Dir}(\alpha)) \right\}$$ with $\lambda = 1/\nu$ (Carlotti et al., 1 Feb 2026). However, this conflation of epistemic and aleatoric uncertainty via a fixed $\nu$ leads to systematic overconfidence OOD.

2. DIP-EDL Parametrization and Density-Informed Evidence

DIP-EDL introduces a density-informed pseudo-count parametrization that separates conditional (class) and marginal (data) contributions to the Dirichlet evidence: $$\beta(x) = \alpha + n\, \widehat{P}_X(x)\, \widehat{P}_{Y|X}(\cdot | x)$$ where:

• $\widehat{P}_X(x)$ is an explicit or implicit density estimator (DE) approximating the input marginal,
• $\widehat{P}_{Y|X}(\cdot|x)$ is a neural network classifier approximating the conditional label distribution,
• $n$ is an effective sample size scalar,
• $\alpha$ is a non-informative prior.

This decomposition generates Dirichlet pseudo-counts for class $k$ at $x$ as $$c_k(x) = n\, \widehat{P}_X(x)\, \widehat{P}_{Y|X}(k|x)$$. The concentration of the Dirichlet becomes $\alpha_0 + n\, \widehat{P}_X(x)$, ensuring that for low-density (OOD) $x$, the predictive distribution reverts to the uniform prior $\mathrm{Dir}(\alpha)$, while in high-density regions it sharpens toward the inferred conditional, modulated by local sample density. This factorization achieves state-conditioned uncertainty calibration unattainable with global-temperature EDL (Carlotti et al., 1 Feb 2026).

3. Training Objective and Algorithmic Structure

DIP-EDL is architecturally modular, permitting independent density estimation and discriminative model training:

• $DE^\psi(x)$ is trained via maximum likelihood or appropriate density-estimation methods (e.g., normalizing flows in pixel/input space or class-conditional Gaussians on learned features).
• $NN^\phi(x)$ is trained by standard cross-entropy or evidential loss for classification.

The full DIP-EDL loss can be written: $$\mathcal{L}_{\mathrm{DIP}}(\psi, \phi) = \sum_i \left\{ - \mathbb{E}_{p \sim \mathrm{Dir}(\beta(X_i))}[\log \mathrm{Cat}(Y_i|p)] + \mathrm{KL}(\mathrm{Dir}(\beta(X_i))\,\|\,\mathrm{Dir}(\alpha)) \right\}$$ with $$\beta(X_i) = \alpha + n\, DE^\psi(X_i)\, NN^\phi(X_i)$$. In typical use, DE and NN are trained independently, without the need to tune auxiliary regularizers such as the EDL’s $\lambda$ parameter—density naturally controls predictive concentration and calibration (Carlotti et al., 1 Feb 2026).

4. Theoretical Guarantees and Uncertainty Decoupling

DIP-EDL enjoys asymptotic concentration analogously to Bayesian consistency. Suppose the estimators converge in probability to the true densities. Then as $n \to \infty$, for $$p \sim \mathrm{Dir}(\alpha + n DE^\psi(x) NN^\phi(x))$$:

• $\mathbb{E}[p] \to P_{Y|X}(\cdot|x)$,
• $\mathrm{Var}[p] \to 0$,
• After Chebyshev’s inequality, $p$ concentrates on the true conditional.

As a result,

• Epistemic uncertainty (quantified as vacuity, $K/[\alpha_0 + n DE^\psi(x)]$ with $K$ classes) vanishes ID as $n$ increases and remains large for OOD $x$ since $DE^\psi(x)$ is small.
• Aleatoric uncertainty—the irreducible randomness in $P_{Y|X}$—remains intrinsic to the task and is not conflated by density scaling.

This rigorously decouples uncertainty types, avoiding the systematic OOD overconfidence of standard EDL (Carlotti et al., 1 Feb 2026).

5. Empirical Performance and Benchmark Results

DIP-EDL demonstrates superior performance across diverse diagnostics: in-distribution (ID) accuracy, Brier score, AUROC for OOD detection, and OOD calibration. On MNIST $\to$ K-MNIST/Omniglot and CIFAR-10 $\to$ CIFAR-100/SVHN benchmarks:

• On MNIST, DIP-EDL achieves $99.53\%$ accuracy and $0.0081$ Brier score ID; $0.9997$ and $0.9998$ AUROC for K-MNIST and Omniglot OOD, with OOD Brier near $0$ (Carlotti et al., 1 Feb 2026).
• On CIFAR-10, DIP-EDL obtains $91.79\%$ accuracy and $0.1329$ Brier ID, with comparable or superior OOD AUROC to DAEDL and EDL; OOD Brier is higher due to density estimation complexity, but calibration and ranking are preserved.

Empirical ablations confirm that the density estimator (DE) alone controls OOD detection performance, the classifier (NN) alone determines ID accuracy, and sample-size $n$ affects ID sharpness (calibration). These results are validated against recently proposed DAEDL (Yoon et al., 2024) and Posterior Networks.

6. Implementation, Interpretability, and Recommended Practices

DIP-EDL’s modular decomposition renders it practical and interpretable:

• Interpretability: $\beta(x) = \alpha + n\, DE^\psi(x)\, NN^\phi(x)$ exposes the evidential contribution of both marginal density and class-conditional evidence, making the distinction between epistemic and aleatoric uncertainty transparent.
• Practical guidelines:
  • For low-dimensional tasks (e.g., MNIST), use normalizing flows in input space, with logit transform and dequantization.
  • For high-dimensional inputs (e.g., CIFAR-10), perform density estimation in a learned feature space using GDA or class-conditional flows.
  • Use non-informative priors $\alpha$ (e.g., $\alpha_k=1$) for uniform baseline.
  • Train DE and NN independently with standard routines (MLE, cross-entropy), with no need for OOD exemplars or special regularization at training.
  • Normalize log-densities before exponentiation to avoid numerical underflow.

A plausible implication is that, given these modular principles, DIP-EDL can be adapted to diverse architectures and domains with minimal tuning and without the systematic OOD failures of conventional EDL or post-hoc calibration mechanisms.

7. Relation to Broader Density-Based Uncertainty and Pseudo-Count Methods

DIP-EDL is part of a broader lineage of methods employing pseudo-counts derived from explicit or implicit density models to quantify epistemic uncertainty, particularly in exploration and intrinsic motivation frameworks in RL (Ostrovski et al., 2017). Whereas count-based approaches use density-model pseudo-counts as scalar rewards or bonus tools for exploration, DIP-EDL generalizes this notion to the setting of evidential classification. It achieves this by treating the product $n\, DE^\psi(x)\, NN^\phi(x)$ as a full vector-valued pseudo-count, interpreting Dirichlet predictive distributions as locally adaptive mixtures with uncertainty scaling governed by data support. The architectural modularity, theoretical calibration, and empirical robustness distinguish DIP-EDL from earlier unified or one-hot pseudo-count instantiations (Carlotti et al., 1 Feb 2026).

In summary, Density-Informed Pseudo-count EDL provides a statistically principled, interpretable, and empirically validated approach to evidential deep learning in regimes characterized by distributional uncertainty, delivering uncertainty quantification and OOD detection that approaches the theoretical ideal (Carlotti et al., 1 Feb 2026).