Logit Spectroscopy

• Logit spectroscopy is the systematic analysis of pre-softmax activations to understand learning dynamics, confidence allocation, and model stability.
• In reinforcement learning, it tracks the L₂ norm of logit updates to reveal self-regulating behavior and guide exploration–exploitation transitions.
• In adversarial settings, it examines maximum logits and logit gaps to quantify robustness and diagnose shifts in model confidence.

Logit Spectroscopy is the systematic study of the vector of pre-softmax activations (“logits”) in machine learning models, both to understand and to diagnose the spectral structure of learning dynamics and confidence allocation. In reinforcement learning, logit spectroscopy involves tracking the L₂ norm of the logit update vector to reveal the underlying self-regulating behavior of softmax policy gradient methods (Li, 15 Jun 2025). In supervised learning and adversarial settings, it refers to detailed analysis of the logit distribution, including scalar summaries such as maximum logit values and logit gaps, to characterize how confidence and robustness are internalized by deep neural networks (Seguin et al., 2021). Logit spectroscopy thus provides a unified empirical and theoretical lens for quantifying learning stability, exploration–exploitation transitions, and the impact of robust training protocols.

1. Foundations in Softmax Policy Gradient Methods

In softmax policy gradient methods, the fundamental signal is not the explicit parameter vector but the change induced on the logits $z(s;\theta)$ for a given state $s$, chosen action $a_c$, and advantage $A$. The canonical update for each logit coordinate, derived from the score function $$\partial \log \pi(a_i|s)/\partial z_j = \delta_{ij} - P_j$$, yields

$$\Delta z_c = \eta\,A\,(1 - P_c), \qquad \Delta z_o = -\eta\,A\,P_o\ \ (o \neq c)$$

where $P_c$ is the probability assigned by the policy to the chosen action, $C(P) = \sum_a P_a^2$ is the collision probability, and $\eta$ is the learning rate. The squared L₂ norm of the logit update vector is

$$\|\Delta z\|_2^2 = (\eta A)^2 [ (1 - P_c)^2 + \sum_{o \neq c} P_o^2 ] = (\eta A)^2 [ 1 - 2P_c + C(P) ]$$

Thus, the L₂ norm of the logit update is

$s$0

This formula exposes a “sensitivity factor” that couples the magnitude of the update directly to the policy’s confidence and entropy, with self-regulating consequences for policy learning (Li, 15 Jun 2025).

2. Self-Regulating Dynamics and Stability

Logit spectroscopy uncovers an implicit, data-dependent learning rate schedule: as the policy becomes more confident ($s$1, $s$2), the L₂ norm of the logit update $s$3 automatically vanishes. This “coasting” effect prevents overshooting and oscillation, even with a fixed $s$4. The following proposition formalizes this self-annealing property:

• Under softmax logit updates, as the policy approaches a deterministic regime ($s$5), the expected update magnitudes decay to zero at rate $s$6.

This self-regulation offers foundational insight into the stability and convergence criteria for softmax policy gradient models, supplying a necessary condition for the sequence of logits to converge (Li, 15 Jun 2025).

3. Logit Spectroscopy as an Empirical Diagnostic Framework

The logit spectroscopy framework prescribes the online tracking of the scalar $s$7, together with $s$8 and $s$9, across training. These quantities reveal key learning phases:

• High $a_c$0, low $a_c$1: Exploration phase with large updates.
• Low $a_c$2, high $a_c$3: Exploitation, marked by diminished updates as the policy concentrates on favored actions.

Diagnosing update vigor through visualization (e.g., time-series and phase-space plots of magnitude vs. $a_c$4 or $a_c$5) allows identification of exploration–exploitation transitions, the collapse of exploration (early rise of $a_c$6), or suboptimal learning-rate settings (persistently large or small $a_c$7). Example protocols involve bandit models or tabular Markov Decision Processes, plotting key metrics such as collision probability, chosen-action probability, and update norm over episodes (Li, 15 Jun 2025).

Experimental Protocol Elements

Quantity	Description
$a_c$8	L₂ norm of logit update; measures learning vigor
$a_c$9	Probability of chosen action under current policy
$A$0	Collision probability (concentration measure), $A$1

4. Logit Spectroscopy in Adversarially-Trained Deep Networks

In the context of adversarial robustness, logit spectroscopy is leveraged to analyze how adversarial training (AT) reshapes the spectral structure of pre-softmax activations in classification networks (Seguin et al., 2021). Adversarial training leads to three pivotal shifts in logit statistics compared to standard training (ST):

1. Compression of Maximum Logits and Logit Gaps: AT produces distributions with much smaller means and positive skew for both $A$2-logit and logit gap ($A$3). In contrast, ST yields symmetric, high-mean distributions.
2. Altered Confidence Ordering: Sample-wise confidence rankings change systematically; more than 10% of examples shift by over half the dataset in confidence rank between ST and AT models, while such shifts across AT seeds remain under 2%.
3. Critical Role of Non-Max Logit Values: The tail of the logit spectrum encodes essential robustness information. Distillation studies show that encoding only the top-$A$4 logits of a robust teacher suffices to reconstruct almost all adversarial robustness by $A$5, while omitting tail information fatally degrades robustness.

This systematic compression and reallocation of confidence, revealed through logit spectroscopy, highlights that adversarial robustness requires not just shrinking gradients or smoothing labels, but encoding a distinct spectral fingerprint throughout the logit vector (Seguin et al., 2021).

5. Mathematical Characterizations and Key Quantities

Key scalar summaries and statistical measures in logit spectroscopy for classification problems include:

• Maximum Logit: $A$6
• Logit Gap: $A$7
• Distribution Functions: $A$8 (probability distribution over $A$9); $$\partial \log \pi(a_i|s)/\partial z_j = \delta_{ij} - P_j$$0 (distribution of gaps)
• Collision Probability: $$\partial \log \pi(a_i|s)/\partial z_j = \delta_{ij} - P_j$$1, used as a concentration entropy surrogate. Satisfies $$\partial \log \pi(a_i|s)/\partial z_j = \delta_{ij} - P_j$$2 for $$\partial \log \pi(a_i|s)/\partial z_j = \delta_{ij} - P_j$$3-class models, with $$\partial \log \pi(a_i|s)/\partial z_j = \delta_{ij} - P_j$$4 at maximal entropy (uniform distribution), $$\partial \log \pi(a_i|s)/\partial z_j = \delta_{ij} - P_j$$5 at minimal entropy (one-hot).

For policy gradient methods, the core equation is

$$\partial \log \pi(a_i|s)/\partial z_j = \delta_{ij} - P_j$$6

For temperature-scaled softmax, the magnitude further scales as $$\partial \log \pi(a_i|s)/\partial z_j = \delta_{ij} - P_j$$7 (Li, 15 Jun 2025).

6. Extensions, Robust Distillation, and Practical Implications

Logit spectroscopy directly informs the design of training protocols and diagnostic tools for both reinforcement and supervised learning. In reinforcement learning, monitoring $$\partial \log \pi(a_i|s)/\partial z_j = \delta_{ij} - P_j$$8 offers efficient criteria to detect premature policy collapse, lack of sufficient exploration, or excessive update magnitudes that indicate hyperparameter misconfiguration (Li, 15 Jun 2025).

In supervised settings, the process of robust distillation demonstrates that not only do the largest logit values matter but the entire tail of the spectrum is critical for transferring adversarial robustness. Specifically, distilling a robust teacher’s full logit vector to a student on clean data results in substantial adversarial accuracy. If only the top-1 (largest) logit is preserved, adversarial accuracy collapses for a vanilla student; recovering the tail ($$\partial \log \pi(a_i|s)/\partial z_j = \delta_{ij} - P_j$$9) is sufficient to reclaim almost all of a teacher's adversarial robustness (Seguin et al., 2021).

Empirically, logit spectroscopy reveals:

• AT models yield highly compressed gap histograms (tall spike near zero), signaling overall lower class-separating confidence.
• ST models develop broad, high-mean gap distributions.
• Visualizations such as ranking-scatter plots expose class-specific and sample-specific confidence reallocations due to adversarial training.

7. Summary and Significance

Logit spectroscopy bridges theoretical analysis and practical diagnostics for both policy-gradient reinforcement learning algorithms and adversarially-trained supervised models. In RL, it formalizes the intrinsic self-regulation of softmax updates; in adversarial settings, it uncovers how robustness is embedded spectrally in the neural activation space. As a monitoring and debugging tool, logit spectroscopy enables early detection of training pathologies and supplies the empirical basis for validating exploration strategies, diagnosing robustness shortfalls, and understanding the spectral allocation of model confidence (Li, 15 Jun 2025, Seguin et al., 2021). Its methodology yields precise empirical and theoretical insights transferable across model classes and domains.