Credal Transformer

• The Credal Transformer replaces Softmax with a Credal Attention Mechanism, allowing the model to capture and express uncertainty via evidential Dirichlet distributions.
• It represents attention as a credal set—a convex set of plausible distributions—enabling ambiguity quantification and a principled measure of epistemic uncertainty.
• Empirical evaluations show improved out-of-distribution detection and reduced confident hallucinations with modest computational overhead in real-world applications.

The Credal Transformer is a Transformer variant proposed to quantify and mitigate hallucinations in LLMs by replacing standard Softmax attention with a Credal Attention Mechanism (CAM) grounded in evidential theory. Its central claim is that ordinary attention creates an “Artificial Certainty” by collapsing ambiguous attention scores into a single probability distribution, thereby discarding uncertainty information at each layer. CAM instead represents attention as a credal set—a convex set of plausible distributions—parameterized by a Dirichlet distribution, so that the model represents both where it attends and how certain it is about that choice. In this formulation, sufficient evidence recovers standard attention-like behavior, whereas insufficient evidence yields a diffuse distribution whose size directly measures uncertainty (Ji et al., 14 Oct 2025).

1. Architectural motivation: Softmax and “Artificial Certainty”

In the standard Transformer, attention between query $\mathbf{Q}_i$ and keys $\mathbf{K}_j$ is computed from raw compatibility scores $s_{ij}$, then normalized with Softmax: $$\mathbf{a}_i = \text{Softmax}(\mathbf{s}_i), \quad a_{ij} = \frac{\exp(s_{ij})}{\sum_{k=1}^{L} \exp(s_{ik})}.$$ A central consequence of this normalization is that even when scores are noisy, conflicting, or weak, Softmax still outputs a single probability vector whose entries sum to $1$. The model is therefore forced to distribute full confidence somewhere. The Credal Transformer identifies this as a structural source of hallucination: the attention mechanism cannot directly express ignorance.

The proposal is not merely that some downstream confidence score should be added after generation. Rather, the criticism targets the attention operator itself. Once a definite attention vector is formed at many layers, that certainty can propagate through the model and contribute to confident but false outputs. This reframes hallucination as partly architectural, not only a consequence of data quality or decoding strategy.

This also changes the epistemic interpretation of attention. Standard attention always makes a definite choice, whereas the Credal Transformer is designed to distinguish evidential support from uncertainty. A plausible implication is that hallucination mitigation is pursued at the level where token interactions are formed, rather than only at the output layer.

2. Credal Attention Mechanism

The Credal Attention Mechanism (CAM) changes the semantics of attention by treating raw scores as evidence rather than quantities that must be collapsed into a single normalized vector. The transformation begins with a nonnegative evidence mapping: $e_{ij} = \exp(s_{ij}).$ The paper notes that any nonnegative mapping could be used, but adopts the exponential form to ensure positive evidence.

These evidence values are then converted into concentration parameters of a Dirichlet distribution: $\alpha_{ij} = e_{ij} + 1.$ For a fixed query $i$, this yields

$$\text{Dir}(\mathbf{p}_i \mid \boldsymbol{\alpha}_i),$$

where $$\boldsymbol{\alpha}_i = (\alpha_{i1}, \dots, \alpha_{iL})$$. The resulting object is not a single attention vector but a distribution over possible attention vectors. The corresponding family of plausible categorical attention distributions is the credal set.

The intended behavior is explicitly bifurcated. Strong, consistent evidence should produce a sharp, confident attention distribution. Weak or conflicting evidence should produce a diffuse distribution and explicit uncertainty. This means that CAM does not force an exact answer to “where to attend?” when the evidential basis is inadequate; it can instead represent ambiguity directly.

For computation, the architecture does not sample from the Dirichlet. It uses the expected value of the Dirichlet as the actual attention weights, with total concentration

$\mathbf{K}_j$0

This preserves end-to-end differentiability and practical tractability while retaining uncertainty information in the concentration parameters.

3. Dirichlet semantics, credal sets, and vacuity

The Dirichlet distribution is used because it is the conjugate prior for categorical distributions. Since attention over keys is categorical, the Dirichlet gives a principled representation of uncertainty over the attention probability vector. Large $\mathbf{K}_j$1 values for one or a few keys correspond to strong evidence and a sharply concentrated Dirichlet; small and similar $\mathbf{K}_j$2 values correspond to weak evidence and a broad distribution.

A central uncertainty quantity in CAM is vacuity, defined as

$\mathbf{K}_j$3

where $\mathbf{K}_j$4 is the sequence length and $\mathbf{K}_j$5 is the sum of Dirichlet parameters. If total evidence is large, then $\mathbf{K}_j$6 is large and $\mathbf{K}_j$7 is small. If total evidence is small, $\mathbf{K}_j$8 is large. Uncertainty is therefore inversely related to accumulated evidence.

This gives a differentiable uncertainty signal at every attention head. The model can represent not only what it attends to, but also how reliable that attention is. The paper further emphasizes two regimes. Under sufficient evidence, CAM behaves like a confident Transformer: evidence is large, the Dirichlet is sharply concentrated, and expected attention resembles ordinary Softmax attention. Under insufficient evidence, the evidence values are small and more uniform, the Dirichlet is diffuse, the credal set is large, and vacuity is high.

In the paper’s framing, the size of the credal set corresponds directly to uncertainty: a larger set means more plausible attention distributions and hence less certainty about where to attend. This is the main epistemic difference from standard attention. Standard attention outputs one normalized vector and cannot directly express ignorance; credal attention outputs a credal set via a Dirichlet distribution and makes uncertainty an explicit architectural object.

4. Relation to evidential deep learning and abstention

The Credal Transformer adapts Evidential Deep Learning (EDL) from output-layer uncertainty estimation to the attention mechanism itself. In standard EDL, a network predicts evidence for class labels, and that evidence is transformed into Dirichlet parameters representing uncertainty over class probabilities. In CAM, the same logic is applied to attention weights: keys play the role of classes, attention scores become evidence, and Dirichlet parameters represent uncertainty over attention distributions.

This makes uncertainty a first-class citizen inside the Transformer, rather than an after-the-fact confidence estimate. The significance is architectural: uncertainty is integrated into the model’s internal token-to-token routing rather than appended only to the final prediction.

The excerpt does not provide a detailed custom loss derivation, but it states that the model is trained end-to-end with standard optimization. Its downstream use of uncertainty is clearest in abstention. When uncertainty is high, the model can choose not to produce a confident answer. This is especially relevant for unanswerable question-answering examples, where a standard LLM may fabricate an answer. The conceptual decision rule is therefore: if evidence is sufficient, answer normally; if evidence is insufficient and uncertainty is high, abstain.

This suggests a shift from calibration by post hoc score adjustment to selective prediction grounded in internal evidence accounting. In the paper’s framing, the model is not simply more cautious; it is able to represent when the evidence does not justify a specific answer.

5. Empirical behavior and computational profile

The empirical evaluation reported for the Credal Transformer emphasizes three behaviors: out-of-distribution detection, ambiguity quantification, and reduction of confident hallucinations on unanswerable questions (Ji et al., 14 Oct 2025).

For out-of-distribution analysis, the model is evaluated on three synthetic input types: In-Distribution (ID), Out-of-Distribution (OOD), and Nonsense. The reported uncertainty scores increase as inputs move farther from the training distribution.

Input type	Uncertainty score
ID	0.0415
OOD	0.1378
Nonsense	0.1953

These values support the claim that internal uncertainty can be used to detect unfamiliar inputs. The same section argues that when inputs are inherently ambiguous, CAM naturally produces a larger credal set, allowing the model to represent ambiguity rather than forcing a single arbitrary interpretation. This is relevant because many hallucinations arise not only from missing facts but also from ambiguous prompts.

A key practical result concerns question answering with unanswerable questions. Standard models often fabricate answers with high confidence; CAM is described as detecting insufficient evidence and enabling abstention rather than hallucination. The result is framed specifically as a reduction of confident errors rather than a generic reduction of all errors.

The reported computational overhead is modest.

Quantity	Reported value
GFLOPs	25.77 G
Inference overhead	+4.4%
Training step overhead	+11.6%

The reported GFLOPs are identical to standard attention, while the inference and training overheads are limited to approximately $\mathbf{K}_j$9 and $s_{ij}$0, respectively. The broader significance claimed for these results is that uncertainty benefits are obtained without large computational penalties.

6. Terminological distinction from the Credibility Transformer

The Credal Transformer should not be conflated with the Credibility Transformer, despite the similarity in naming. The latter is a tabular-data Transformer from actuarial modeling that introduces a credibility mechanism through a special CLS token and a credibility-weighted blend of prior information and observation-based information, explicitly drawing on Bühlmann and Bühlmann–Straub credibility theory (Richman et al., 2024). It is therefore not “credal” in the sense of credal sets or imprecise probability.

A later extension, the In-Context Learning Enhanced Credibility Transformer, augments that actuarial architecture with a context batch of similar instances, an outcome token decorator, an ICL Transformer layer, and a frozen decoder. Its purpose is to enrich CLS token representations with in-context information and improve predictive accuracy and generalization on tabular risk prediction problems such as the French MTPL setting (Padayachy et al., 9 Sep 2025).

Model	Central mechanism	Primary setting
Credal Transformer	CAM, Dirichlet distribution, credal set	LLM hallucination mitigation
Credibility Transformer	Credibility-weighted CLS token	Tabular actuarial prediction
ICL-Credibility Transformer	Context batch, decorated CLS tokens, frozen decoder	In-context tabular prediction

This distinction is substantive rather than terminological. The Credal Transformer is uncertainty-centric: it replaces Softmax with an evidential mechanism that preserves ambiguity. The Credibility Transformer is credibility-centric: it regularizes tabular representation learning through a blend of prior and instance-specific information. A plausible implication is that “credal” and “credibility” label two different research programs that both modify Transformer internals, but for different theoretical reasons and application domains.

7. Significance and design paradigm

The Credal Transformer’s broader contribution is to argue that hallucination is not only a problem of bad data or bad decoding, but also partly a consequence of how Transformers are built. If attention is forced through Softmax, the model cannot express “I’m unsure.” CAM addresses this by making uncertainty intrinsic to attention itself.

The architectural contribution can be summarized in four points. Uncertainty is modeled at the layer level; evidence is preserved rather than collapsed; a credal set replaces a single point estimate; and the model can abstain when knowledge is insufficient. In one sentence, the Credal Transformer is a Transformer whose attention mechanism uses evidence, Dirichlet distributions, and credal sets to represent uncertainty explicitly, replacing Softmax’s forced certainty with a principled measure of epistemic uncertainty that improves out-of-distribution detection and reduces hallucinated confident answers.

This framing positions the model as a design paradigm for more reliable AI. Rather than appending uncertainty to a finished architecture, it integrates uncertainty quantification directly into the model’s internal mechanism for allocating attention.