Correct-Evidence Regularization (CER)

• The paper introduces a dual regularization approach that separately enforces high vacuity for out-of-distribution samples and high dissonance at class boundaries.
• Its methodology integrates evidential neural network predictions with tailored loss functions to decompose epistemic uncertainty into interpretable measures.
• Experimental evaluations on synthetic data, CIFAR-10, and text classification tasks show enhanced uncertainty calibration and robust OOD detection.

Correct-Evidence Regularization (CER) encompasses a set of regularization techniques designed to promote the use of correct, interpretable, and calibrated evidence in the decision-making process of deep neural networks. There are two prominent strands of CER: one in regularized Evidential Neural Networks (ENN), targeting explicit uncertainty decomposition into vacuity and dissonance for robust out-of-distribution (OOD) handling, and another in rationale-regularized text models, aligning the model's explanations with human-provided rationales to increase credibility and generalization. Both lines emphasize mechanism-level targeting of evidential support, surpassing earlier black-box approaches.

1. Evidential Neural Networks and the Motivation for CER

Evidential Neural Networks parameterize predictions not just as point estimates, but as subjective opinions over Dirichlet distributions, where model outputs $r \in \mathbb{R}_+^K$ represent “evidence” for $K$ classes, forming Dirichlet parameters $\alpha = r+1$. These confer both first-order class probabilities ($\mathbb{E}[p]$) and higher-order uncertainty measures: the total evidence $S = \sum_j \alpha_j$, vacuity $u = K/S$ (where high vacuity denotes ignorance), and dissonance (modeling internal class conflict).

Standard ENNs lack explicit mechanisms to control what kind of uncertainty the network expresses for different sample types. They typically include only a KL regularization term toward a uniform Dirichlet, which does not separately address the orthogonal dimensions of epistemic ignorance (vacuity) versus class conflict (dissonance).

Correct-Evidence Regularization in this context introduces two explicit, data-driven regularizers: one that enforces high vacuity on known OOD samples, and one that drives high dissonance near class boundaries (boundary-of-distribution, BOD). This enables the model to decompose uncertainty appropriately, matching semantic distinctions between ignorance (withholding evidence) and conflict (conflicting evidence near ambiguous decisions) (Zhao et al., 2019).

2. Objective Formulation and Mechanistic Details

Given three data subsets—$D_\text{IN}$ (in-distribution), $D_\text{OOD}$ (explicitly OOD), and $D_\text{BOD}$ (high neighbor-conflict boundary samples)—the CER-augmented ENN loss is

$$\mathcal{L}(\Theta) = \mathbb{E}_{(x,y)\sim D_\text{IN}}\left[\mathcal{L}_\text{class}(\alpha(x;\Theta), y)\right] - \lambda_1\,\mathbb{E}_{x \sim D_\text{OOD}}\left[\text{Vac}(\alpha(x;\Theta))\right] - \lambda_2\,\mathbb{E}_{x \sim D_\text{BOD}}\left[\text{Diss}(\alpha(x;\Theta))\right]$$

The core terms are:

• $\mathcal{L}_\text{class}$: Dirichlet-multinomial squared error, integrating uncertainty into training.
• $\text{Vac}(\alpha) = K / S$, penalized to be high (i.e., force $S$ small) for $D_\text{OOD}$, thus enforcing ignorance.
• $\text{Diss}(\alpha)$: a measure computed from normalized belief masses, designed to peak at class boundaries where evidence is conflicting.

Loss hyperparameters $\lambda_1, \lambda_2$ allow selective emphasis during optimization. Each minibatch interleaves samples from all three partitions to guarantee coverage per gradient step. Optimizer settings (Adam, learning rate 0.01, batch size 1000, dropout 0.9, weight decay 0.005) are as specified in (Zhao et al., 2019).

3. Interpretation and Expressiveness of Uncertainty

CER’s selective regularization strategy imposes an interpretable and decomposable structure on the model’s epistemic uncertainty:

• On $D_\text{OOD}$, high vacuity constrains the model to produce little to no evidence, countering incorrectly overconfident predictions on foreign samples.
• On $D_\text{BOD}$, high dissonance enforces that the model’s belief mass is distributed almost evenly across plausible classes, explicitly encoding the presence of inherent ambiguity.

This dual-factor framework provides more faithful uncertainty quantification and matches desired taxonomies for robust real-world classification, going beyond the undifferentiated uncertainty inferred by Bayesian NNs or traditional ENNs. The effect is tighter control over the model’s evidential behavior, particularly under challenging test-time conditions.

4. Experimental Evaluation and Quantitative Impact

The effectiveness of CER was validated through synthetic and real-world datasets (Zhao et al., 2019):

• Synthetic 2D Toy Data: CER enables vacuity contours to rise predictably as one moves away from in-distribution clusters (with vacuity-only regularization), while dissonance regularization localizes high dissonance at cluster boundaries. The joint model achieves both behaviors.
• CIFAR-10 Classification: Training using 5 in-distribution classes, 2 held-out OOD classes, and 3 for test-time OOD detection, CER resulted in superior OOD detection, as quantified by empirical cumulative distribution functions (CDF) of predictive entropy. ENN-Vac (vacuity regularizer only) outperformed both vanilla ENN and softmax baselines; ENN-Diss alone provided no OOD improvement but correctly modeled class-boundary conflict; the combined model dominated on all metrics.

Evaluation metrics include in-distribution accuracy, OOD detection (measured by entropy CDFs), and qualitative inspection of uncertainty (vacuity and dissonance) plots. These results demonstrate that CER delivers selective uncertainty benefits without degrading classification performance.

5. Ablation Studies and Sensitivity Analysis

Ablation studies compared several regimes: standard softmax, vanilla ENN, ENN+Vac only, ENN+Diss only, and ENN+Vac+Diss. Each regularizer contributed primarily to its targeted uncertainty dimension, and the combination was necessary to simultaneously optimize both OOD robustness and coherent boundary uncertainty. Sensitivity analysis showed that the efficacy of vacuity regularization depends on the representativeness of $D_\text{OOD}$, while dissonance regularization benefits from accurately identified boundary samples via nearest-neighbor label conflict (Zhao et al., 2019).

The influence of hyperparameters $\lambda_1$, $\lambda_2$ was empirically explored, revealing qualitatively similar uncertainty behavior for values near $0.01$, with reduced effect for substantially smaller or larger values.

6. CER in Rationale-Regularized Text Models

A related, distinct line of Correct-Evidence Regularization appears in the context of deep text classification, termed CREX (Du et al., 2019). Here, CER refers to regularizing models so their local explanations (feature attributions) align with human-provided “rationales.” In this setting, the CREX loss incorporates:

• Confident-Explanation Loss: Penalizing attribution mass outside expert rationales.
• Uncertain-Explanation Loss: Ensuring that, after removing rationale features from input, the model’s confidence drops and its explanations become uniform.
• Sparse-Attribution Loss: Encouraging attribution sparsity when rationales are unavailable.

The combined loss enforces that model decisions are grounded in justified, interpretable evidence, and experimental results on Movie Review and Product Review datasets demonstrate improved alignment (measured by KL divergence between model attributions and journaled rationales) and better generalization to new and adversarial domains. CREX is explicitly model-agnostic and differentiable, requiring no changes to architecture or additional networks.

CER Formulation	Domain	Regularizer Targets
ENN-CER	General NN, vision	Vacuity (on OOD), Dissonance (boundary)
CREX	Text classification	Attribution alignment (human rationale), Sparsity

Both approaches reveal the broad applicability of CER as a principle: incorporating domain-specific or epistemic constraints to ensure neural models leverage correct evidence in line with conceptual, semantic, or human-formulated desiderata (Zhao et al., 2019, Du et al., 2019).

7. Limitations and Prospective Directions

The efficacy of CER in ENNs is coupled to the availability and representativeness of known OOD and class-boundary samples during training—a limitation for settings with incomplete or shifting distributions. In CREX, the success is contingent upon the fidelity and consistency of domain expert rationales, with model performance sensitive to noisy annotations. Both lines suggest future work in automating the selection of uncertainty-target samples and extending regularization to additional axes of evidence. Another plausible implication is that integrating weak, partial, or probabilistic rationales with CER could further enhance robustness, especially for large-scale or noisy domains (Zhao et al., 2019, Du et al., 2019).