Frequentist Epistemic Uncertainty

• Frequentist epistemic uncertainty measures are defined using classical probability and cross-entropy, avoiding subjective priors.
• The approach decomposes total uncertainty into aleatoric noise and epistemic uncertainty through feature gap analysis for clear interpretability.
• Bootstrap estimation and deep ensemble methods provide robust, data-driven techniques for practical uncertainty quantification in tasks like contextual QA.

Frequentist measures of epistemic uncertainty aim to quantify a model’s lack of knowledge using classical probability concepts rather than Bayesian priors and posteriors. In this context, uncertainty is viewed through repeated sampling properties, relying on properties of cross-entropy, likelihood functions, and confidence sets. The frequentist perspective carefully separates the irreducible aleatoric noise from the reducible epistemic component and employs internal representation analysis and bootstrap-based estimation to provide interpretable, data-driven uncertainty measures.

1. Formal Definition via Cross-Entropy

Frequentist uncertainty quantification begins by defining total uncertainty at the token level using cross-entropy. Given an unknown true distribution, $P^*$, over LLM outputs and the model’s predictive distribution $P$, the total uncertainty is defined as

$$\mathrm{TU} = -\sum_{y_t \in \mathcal{V}} P^*(y_t \mid \mathbf{y}_{<t}, \mathbf{x}) \ln P(y_t \mid \mathbf{y}_{<t}, \mathbf{x}, \theta)$$

where $\mathcal{V}$ denotes the vocabulary, $\mathbf{x}$ is the input, $\mathbf{y}_{<t}$ are the previously generated tokens, and $\theta$ represents model parameters. This formulation relies purely on observable frequencies and does not invoke any subjective prior, embodying the frequentist principle.

2. Decomposition into Aleatoric and Epistemic Uncertainty

A key insight is the decomposition of total uncertainty into two distinct components:

• Aleatoric Uncertainty: Captures irreducible noise present in the task, represented as the entropy of the underlying true distribution:

$$H\!\left(P^*(y_t \mid \mathbf{y}_{<t}, \mathbf{x})\right)$$

• Epistemic Uncertainty: Reflects model deficiency and lack of knowledge, measured by the Kullback–Leibler divergence between the true distribution and the model’s predictive distribution:

$$\mathrm{KL}\!\left(P^*(y_t \mid \mathbf{y}_{<t}, \mathbf{x}) \,\|\, P(y_t \mid \mathbf{y}_{<t}, \mathbf{x}, \theta)\right)$$

This decomposition ensures that while aleatoric uncertainty is dictated by the data and task characteristics, epistemic uncertainty directly reflects the information gap due to insufficient model knowledge.

3. Approximating the True Distribution

Since the true output distribution $P^*$ is inaccessible, the frequentist approach approximates it by considering an ideally prompted version of the same model. Formally, one assumes

$$P^*(\cdot \mid \mathbf{x}) \approx P(\cdot \mid \mathbf{x}, \theta^*)$$

where $\theta^*$ corresponds to the same model parameters but utilized under a theoretically optimal prompt. The optimal prompt, denoted $\mathbf{s^*}$, is defined as the minimizer of the expected KL-divergence over the data:

$$\mathbf{s^*} = \arg\min_{\mathbf{s}} \mathbb{E}_{\mathbf{x}\sim \mathcal{D}} \Bigl[ \mathrm{KL}\!\Bigl( P^*(\cdot \mid \mathbf{x}) \,\|\, P(\cdot \mid \mathbf{x}, \mathbf{s}, \theta)\Bigr) \Bigr]$$

This frequentist approximation allows practitioners to ground uncertainty estimates on observable model outputs under near-ideal conditions.

4. Bounding Epistemic Uncertainty via Feature Gaps

To circumvent the intractability of exact KL-divergence computation, the frequentist method derives an upper bound based on differences in internal representations. Let $h_t$ and $h_t^*$ denote the last-layer hidden states of the actual and ideally prompted models, respectively, and let $W$ represent the final projection matrix. Then, the epistemic uncertainty is bounded as

$$\mathrm{KL}\!\left(P^*(y_t) \,\|\, P(y_t)\right) \leq 2\, \|W\| \cdot \|h_t^* - h_t\|$$

The difference $\|h_t^* - h_t\|$ is interpreted as the “feature gap” between the current model and its idealized counterpart. Moreover, by expressing the hidden state as a linear combination of semantic feature directions,

$$h_t = \sum_{i} \alpha_i v_i \quad \text{and} \quad h_t^* = \sum_{i} \beta_i v_i,$$

the gap decomposes to

$$\|h_t^* - h_t\| = \left\| \sum_{i} (\beta_i - \alpha_i) v_i \right\|,$$

where each $v_i$ represents a semantic concept and the difference $\beta_i - \alpha_i$ quantifies the deficiency along that axis.

5. Applications in Contextual Question Answering

The framework is applied in the domain of contextual question answering (QA) by hypothesizing that three principal features capture the primary dimensions of epistemic uncertainty:

1. Context Reliance: The degree to which the model draws on the provided context as opposed to relying on parametric (internal) knowledge.
2. Context Comprehension: The ability of the model to extract relevant information from the supplied context.
3. Honesty: The extent to which the model avoids generating intentionally misleading or unsupported responses.

A top-down interpretability approach is used to extract these semantic feature directions. Contrastive prompts (“use context” vs. “use own knowledge” and similar pairs) generate feature activations, and principal component analysis (PCA) is employed to identify the dominant directions. A lightly supervised ensembling mechanism then combines these feature scores into a single uncertainty measure. Empirical evaluations on datasets such as Qasper, HotpotQA, and NarrativeQA have demonstrated improvements in prediction–rejection ratios (PRR) and AUROC, with experiments showing up to a 13-point gain in PRR relative to standard baselines.

6. Bootstrap Estimation and Deep Ensemble Interpretations

Beyond feature-gap methods, a parallel frequentist measure employs the bootstrap to approximate epistemic uncertainty. By resampling the training data and retraining the model for each bootstrap replicate, one obtains a collection of predictive distributions. A bootstrap-based mutual information estimator is then defined as

$$I_b(x_*, D) = H\!\left(\bar{p}\right) - \frac{1}{B}\sum_{b=1}^B H\!\bigl(p^{(b)}\bigr),$$

where $\bar{p}$ is the average prediction over $B$ bootstrap samples and $p^{(b)}$ is the prediction from the $b$th run. Theoretical results, under regularity conditions (as in the Bernstein–von Mises theorem), show that this estimator is asymptotically equivalent to the Bayesian mutual information. This equivalence provides a frequentist interpretation for the epistemic uncertainty measured by deep ensemble methods, with the dominant source of uncertainty in practice often arising from algorithmic randomness (e.g., different initialization seeds).

7. Theoretical Implications and Practical Considerations

Frequentist measures of epistemic uncertainty offer several advantages:

• They avoid reliance on subjective priors by basing all uncertainty on observable data and repeated sampling.
• The decomposition into aleatoric and epistemic components is data-driven, ensuring that the aleatoric term is invariant to model architecture and training methods.
• Bounding uncertainty via internal feature gaps provides interpretability, linking uncertainty directly to deficiencies in semantic representations.
• Bootstrap estimators, which are computationally accessible, further bridge the gap between Bayesian methods and frequentist guarantees, ensuring that methods such as deep ensembles have a solid theoretical foundation.

However, practitioners should be aware that approximating the true distribution via idealized prompting and feature extraction requires careful design and a sufficient number of labeled examples for supervision. Despite these challenges, empirical evidence in tasks such as contextual QA supports the effectiveness and data efficiency of these approaches (Bakman et al., 3 Oct 2025, Jain et al., 24 Oct 2025).

By unifying cross-entropy based formulations, internal representation analysis, and resampling-based uncertainty estimation, frequentist measures of epistemic uncertainty provide robust, interpretable, and theoretically grounded tools for uncertainty quantification in modern machine learning.