Softmax Energy (SME) as Confidence Score

• Softmax Energy (SME) is a logit-derived confidence score that quantifies uncertainty through the log-partition function of neural classifiers.
• It is positioned among other logit-based measures like MSP and log-softmax margins, emphasizing metacognitive ranking for model performance.
• Empirical studies on CIFAR-10 demonstrate that SME effectively captures correctness-aligned information despite the presence of more complex energy probes.

Searching arXiv for papers on softmax energy / energy-based OOD detection and related confidence scoring. Softmax Energy (SME) is a scalar confidence or uncertainty score derived from a classifier’s logits, typically defined as

$E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$

or, in temperature-scaled form,

$$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$

In the OOD and uncertainty literature, SME belongs to the broader family of logit-based scores that also includes maximum softmax probability (MSP) and softmax or log-softmax margins. A 2026 analysis of discriminative predictive coding networks (PCNs) places SME in a broader structural context by showing that an apparently richer K-way energy probe reduces approximately to a log-softmax-based signal plus a residual term that is not trained to correlate with correctness (Cacioli, 13 Apr 2026). In that regime, SME and related logit functions are not merely convenient heuristics; they capture most of the correctness-aligned information available to the model.

1. Formal definition and placement among logit-based scores

SME is computed directly from the logits $z(x) \in \mathbb{R}^K$ of a $K$-class classifier. The score uses the log-partition function,

$E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$

with a temperature-scaled variant

$$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$

The data also identifies closely related logit-based quantities: $\mathrm{MSP}(x) = \max_k \mathrm{softmax}(z(x))_k,$

$$\mathrm{margin}(x) = \mathrm{softmax}(z)_{(1)} - \mathrm{softmax}(z)_{(2)},$$

and the log-softmax margin

$$M_{\text{logsm}}(x) = \log p_{(1)}(x) - \log p_{(2)}(x), \quad \log p_k(x) = \log \mathrm{softmax}(z(x))_k.$$

These quantities differ in parameterization but share the same basic input: the logit vector. The 2026 PCN study treats them as a family of “simple functions of logits” and argues that, for metacognitive ranking, monotone transforms among them are often operationally close (Cacioli, 13 Apr 2026). That framing is important because it shifts attention from the superficial complexity of a score to the question of whether it accesses information beyond the logits.

2. SME as a confidence signal for metacognition

Within the source material, metacognition is defined in the Type-2 signal detection theory sense: a model’s ability to discriminate its own correct from incorrect predictions independently of Type-1 task accuracy. The principal metric is Type-2 AUROC, denoted AUROC$_2$, obtained by treating correctness versus incorrectness as the binary label and a confidence score as the predictor (Cacioli, 13 Apr 2026).

In this setting, SME is not isolated from other confidence scores. The same study compares a structural confidence signal from a predictive-coding energy probe against softmax-based confidence, typically the margin between the top-1 and top-2 softmax probabilities or, equivalently, the corresponding log-softmax margin. Because AUROC is invariant under monotone transformations of the confidence signal, a monotone function of logits can serve the same ranking role even if its numerical scale differs. This is the core sense in which SME, MSP, and log-softmax margins are treated as part of one logit-based confidence family (Cacioli, 13 Apr 2026).

A common misconception is that any score involving an “energy” function must access epistemically richer information than a score derived from logits. The predictive-coding analysis directly contests that intuition. It distinguishes between structural richness in the computation and discriminative richness in the correctness signal: a probe may depend on more internal quantities than SME while still reducing, for ranking purposes, to a perturbation of the same logit-based information (Cacioli, 13 Apr 2026).

3. Reduction of K-way predictive-coding energy to a softmax-based form

The 2026 paper studies a standard discriminative PCN in the Pinchetti-style formulation. For input $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$0, latent hierarchy $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$1, top-down generative maps $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$2, and one-hot target $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$3, the energy is

$$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$4

with

$$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$5

The K-way energy probe fixes each candidate class as a target, runs inference to settling, computes the per-hypothesis settled energies $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$6, predicts by $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$7, and defines a structural confidence margin as the gap between the smallest and second-smallest class energies (Cacioli, 13 Apr 2026).

The central technical claim is an approximate decomposition: $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$8 where $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$9 are the feedforward logits, $z(x) \in \mathbb{R}^K$0 is class-independent, and $z(x) \in \mathbb{R}^K$1 is a residual collecting lower-layer contributions and small inference corrections. Consequently,

$z(x) \in \mathbb{R}^K$2

The paper presents this as an approximate reduction under assumptions A1–A5, not as a formal upper bound (Cacioli, 13 Apr 2026).

The significance of the reduction is twofold. First, it identifies the layer-$z(x) \in \mathbb{R}^K$3 term directly beneath the output clamp as the part aligned with correctness through the log-softmax. Second, it isolates the remaining structural contributions into a residual that is not trained to rank correct hypotheses above incorrect ones. This is why the analysis predicts that the structural probe should “track softmax from below”: it inherits the correctness-aligned softmax term and then adds perturbations without a correctness-aware objective (Cacioli, 13 Apr 2026).

4. Why SME can match or exceed structurally richer probes

The predictive-coding analysis is explicit that the K-way energy probe appears richer than SME because it reads all layerwise prediction errors, depends on the full generative chain, and includes whatever dynamics arise from iterative inference. Nevertheless, under target-clamped CE-energy training with effectively feedforward latent dynamics, that appearance is misleading (Cacioli, 13 Apr 2026).

The mechanism is architectural and objective-driven. Training aligns the generative chain with the encoder’s activations for the true class at the penultimate layer, which produces the softmax-linked term. By contrast, the lower-layer residual contributions do not receive a training signal of the form “reduce energy if $z(x) \in \mathbb{R}^K$4 is correct, increase if $z(x) \in \mathbb{R}^K$5 is incorrect” for arbitrary clamped labels. The result is not random noise in a colloquial sense, but a structured residual that is not correctness-calibrated (Cacioli, 13 Apr 2026).

This has direct consequences for SME. If a more complex probe reduces to a monotone function of log-softmax plus an untrained residual, then a simpler logit-derived score can be as good as or better than the structural alternative for misclassification metacognition. The paper therefore supports a narrow but consequential conclusion: in standard discriminative PCNs with CE output energy and near-feedforward inference, SME-style scores are already extracting the principal correctness-aligned signal (Cacioli, 13 Apr 2026).

A plausible implication is methodological. When a model’s training objective and test-time dynamics privilege discriminative logits, additional energy structure should not be assumed to confer additional metacognitive content. Complexity of readout and informativeness of readout are separable properties.

5. Empirical behavior on CIFAR-10

The empirical study evaluates six conditions on CIFAR-10 and reports that, in every condition, the K-way probe sat below softmax. The reported regime is explicitly small: a single seed, a 2.1M-parameter network, and 1280 test images; the result is framed as a preprint inviting replication (Cacioli, 13 Apr 2026).

In extended deterministic discriminative-PC training, the epoch-25 results were: softmax AUROC$z(x) \in \mathbb{R}^K$6 $z(x) \in \mathbb{R}^K$7 with accuracy $z(x) \in \mathbb{R}^K$8, versus K-way probe AUROC$z(x) \in \mathbb{R}^K$9 $K$0 with accuracy $K$1, a gap of $K$2 AUROC$K$3. Across checkpoints, the gap ranged from $K$4 to $K$5 AUROC$K$6, always negative (Cacioli, 13 Apr 2026). The paper interprets this as the predicted “tracks softmax from below” pattern.

A latent-movement diagnostic measured inference updates of only $K$7–$K$8 per element, an energy decrease of $K$9 or $E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$0, and final MSE between settled and feedforward latents of $E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$1–$E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$2 (Cacioli, 13 Apr 2026). This supports the effective-feedforward assumption used in the reduction and suggests that the iterative dynamics add little new signal in the tested setup.

A post-hoc fairness control on a backpropagation network further tightened the connection to softmax. With a frozen encoder and a trained decoder mirroring the PCN generative chain, the BP softmax score achieved accuracy $E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$3 and AUROC$E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$4 $E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$5, while the BP+decoder K-way probe achieved accuracy $E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$6 and AUROC$E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$7 $E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$8, for a gap of $E_{\text{SM}}(x) = -\log \sum_{i=1}^K e^{z_i(x)},$9 (Cacioli, 13 Apr 2026). The paper interprets this near-collapse as evidence that when encoder–decoder alignment is strongly enforced, the residual becomes nearly class-invariant.

Noise injection did not reverse the pattern. Under a five-point Langevin inference temperature sweep, the softmax baseline on the model was accuracy $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$0 and AUROC$$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$1 $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$2. K-way AUROC$$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$3 values were $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$4 at $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$5, $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$6 at $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$7, $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$8 at $$S_{\text{SME}}(x) = -\frac{1}{T}\log \sum_i e^{z_i(x)/T}.$$9, $\mathrm{MSP}(x) = \max_k \mathrm{softmax}(z(x))_k,$0 at $\mathrm{MSP}(x) = \max_k \mathrm{softmax}(z(x))_k,$1, and $\mathrm{MSP}(x) = \max_k \mathrm{softmax}(z(x))_k,$2 at $\mathrm{MSP}(x) = \max_k \mathrm{softmax}(z(x))_k,$3, with the best K-way performance at deterministic evaluation and monotonic degradation as noise increased (Cacioli, 13 Apr 2026).

Trajectory-integrated MCPC training also left the basic conclusion intact. Final-state versus MCPC softmax AUROC$\mathrm{MSP}(x) = \max_k \mathrm{softmax}(z(x))_k,$4 was $\mathrm{MSP}(x) = \max_k \mathrm{softmax}(z(x))_k,$5 versus $\mathrm{MSP}(x) = \max_k \mathrm{softmax}(z(x))_k,$6, while deterministic K-way AUROC$\mathrm{MSP}(x) = \max_k \mathrm{softmax}(z(x))_k,$7 was $\mathrm{MSP}(x) = \max_k \mathrm{softmax}(z(x))_k,$8 versus $\mathrm{MSP}(x) = \max_k \mathrm{softmax}(z(x))_k,$9, a difference of $$\mathrm{margin}(x) = \mathrm{softmax}(z)_{(1)} - \mathrm{softmax}(z)_{(2)},$$0. The abstract summarizes this by noting that final-state and trajectory-integrated training produced probes whose AUROC$$\mathrm{margin}(x) = \mathrm{softmax}(z)_{(1)} - \mathrm{softmax}(z)_{(2)},$$1 values differed by less than $$\mathrm{margin}(x) = \mathrm{softmax}(z)_{(1)} - \mathrm{softmax}(z)_{(2)},$$2 at deterministic evaluation (Cacioli, 13 Apr 2026).

6. Scope conditions, limits, and open directions

The reduction to softmax-based confidence is explicitly scoped. The paper lists several settings in which one or more assumptions A1–A5 may fail and where a structural energy signal could, in principle, go beyond SME-style logits (Cacioli, 13 Apr 2026).

These exceptions include generative PC at test time, where there is no target clamping and no CE term at output; bidirectional PC, where the CE-plus-quadratics structure is altered; prospective configuration, where inference fixed points differ substantially from feedforward initialization; non-CE or more general energy formulations such as Hamiltonian PC or learned observation noise; architectures with skip connections or attention, which violate the simple deterministic top-down chain; and non-classification tasks, where the K-way clamping construction is not directly defined (Cacioli, 13 Apr 2026).

The importance of these caveats is conceptual. The paper does not claim that SME is universally sufficient, nor that all energy-based confidence measures collapse to softmax. Rather, it isolates a regime in which discriminative training, target clamping, and near-feedforward inference constrain the effective confidence signal to be logit-dominated (Cacioli, 13 Apr 2026). This suggests that surpassing SME may require a genuinely different source of uncertainty information rather than a more elaborate functional of the same discriminative architecture.

The paper also identifies “productive structural probing” directions that its analysis does not foreclose: inference protocols that break the effective-feedforward assumption, joint generative and discriminative training that makes the residual a learned correctness signal, and probes based on trajectory-level quantities such as rate of energy decay or variance under stochastic inference rather than K-way clamping alone (Cacioli, 13 Apr 2026).

7. Interpretation of SME in current research practice

Within the evidential scope of the 2026 study, SME should be understood as more than a baseline convenience. In standard discriminative PCNs, the structural probe that seems to leverage the full generative hierarchy is shown to reduce approximately to a softmax-derived term plus a residual that tends to degrade rather than improve correctness ranking (Cacioli, 13 Apr 2026). In that sense, SME occupies a privileged position because it is already closely matched to the discriminative objective.

This does not mean SME and related scores are identical in all applications. The source material distinguishes among SME, MSP, top-class negative log probability, and log-softmax margins, while also emphasizing that they are all simple functions of logits and can be effectively equivalent for ranking-based metacognition under monotone transformations (Cacioli, 13 Apr 2026). The practical implication is that numerical differences among these scores need not correspond to substantive differences in the information they extract.

For researchers using discriminative PC models, the immediate conclusion is conservative: simple logit-based confidence measures, including SME, do not appear to leave substantial metacognitive performance untapped in the tested regime. For researchers seeking gains beyond SME, the same analysis points toward different objectives, different inference regimes, or different architectures rather than a more elaborate energy readout on top of the same discriminative backbone (Cacioli, 13 Apr 2026).