Entropy and Attention Dynamics in Small Language Models: A Trace-Level Structural Analysis on the TruthfulQA Benchmark

Abstract: Small LLMs (SLMs) have been increasingly deployed in edge devices and other resource-constrained settings. However, these models make confident mispredictions and produce unstable output, making them risky for factual and decision-critical tasks. Current evaluation methodology relies on final accuracy or hallucination rates without explaining how internal model behavior affects outputs. Specifically, how entropy evolves during decoding, how attention is distributed across layers, and how hidden representations contribute to uncertainty, logical inconsistencies, and misinformation propagation are often overlooked. Consequently, this study introduces a trace-level analysis of entropy and attention dynamics in SLMs evaluated with the TruthfulQA dataset. Four models with parameter ranges of 1B-1.7B parameters were examined via token-level output entropy, attention entropy, head dispersion, and hidden-state representation. The results reflect three model classifications by entropy patterns. Deterministic models (DeepSeek-1.5B and LLaMA-1B): output entropy decreases over time. Exploratory models (Gemma-1B): with increasing entropy, and balanced models (Qwen-1.7B): have moderate and stable entropy. Also, each group has distinctively different hidden-state movement and attention dispersion patterns. The analysis demonstrates that truthfulness in SLMs emerges from structured entropy and attention dynamics. Monitoring and optimizing these internal uncertainty patterns can guide the design of a more reliable, hallucination-aware, and application-specific edge SLMs.

• The paper reveals that trace-level metrics of entropy, attention, and representational drift distinguish deterministic, exploratory, and balanced regimes in small language models.
• Using a uniform evaluation framework with greedy decoding and human-validated annotations, the study isolates internal dynamics linked to epistemic reliability and hallucination risks.
• The analysis highlights that optimizing SLMs for factual outputs requires entropy regularization and precise attention tuning to mitigate hallucinations.

Entropy and Attention Dynamics in Small LLMs: Structural Insights from Trace-Level Analysis on TruthfulQA

Introduction

The study "Entropy and Attention Dynamics in Small LLMs: A Trace-Level Structural Analysis on the TruthfulQA Benchmark" (2604.03589) provides a rigorous dissection of Small LLM (SLM) behavior by integrating entropy dynamics, attention dispersion, and representational geometry across token and layer-time axes. By analyzing four transformer-based SLMs (1B–1.7B parameters; LLaMA-1B, DeepSeek-1.5B, Gemma-1B, Qwen-1.7B) on the TruthfulQA benchmark, the paper bridges the methodological gap between static outcome metrics and the evolving, multivariate internal dynamics that underpin generative truthfulness, epistemic reliability, and hallucination propensity in edge-scale LLMs.

Methods and Experimental Protocol

A uniform, trace-level evaluation framework is implemented, extracting probabilistic and geometric signals (token-level entropy, layer-wise attention entropy, Head Dispersion Index, hidden-state L2, KV cache memory) at each decoding step. Greedy decoding is used to eliminate sampling stochasticity and expose intrinsic model behaviors. Human evaluation against TruthfulQA’s diverse factuality classes ensures annotation validity (inter-rater reliability: $\kappa=0.81$, $\alpha=0.79$). Architectural scaling, attention protocol, and trace measurement are standardized to isolate structural differences. Layer-wise, temporal, and cross-metric analyses are executed, providing comprehensive associations between entropy, attention, and representational drift.

Cross-Model Entropy and Attention Dynamics

The models cluster into three behavioral regimes by entropy drift:

• Deterministic (DeepSeek-1.5B, LLaMA-1B): Output entropy decreases during generation, reflecting sharp probabilistic peaking and decisive token selection (DeepSeek mean output entropy 0.124, Top-1–Top-2 gap 0.957). Deterministic models consolidate confidence, but risk confident mispredictions and hallucinations in ambiguous cases.
• Exploratory (Gemma-1B): Displays increasing entropy with volatile and high-magnitude representational drift (output entropy 0.678, SD 0.920; mean Delta L2 1600.649), indicating probabilistic uncertainty and unstable reasoning—directly correlating with higher hallucination rates in factual error categories.
• Balanced (Qwen-1.7B): Intermediate, stable entropy profile (output entropy 0.374, SD 0.527; moderate representational drift), with dynamic attention and strong cross-layer specialization.

Component-level trends between outcome types affirm that hallucination rates spike in error-prone generations, especially when internal entropy is high or attention volatility is pronounced.

Figure 1: Model-wise trends for identification, classification, and hallucination incidence across TruthfulQA outcome types.

Temporal and Structural Evolution: Entropy and Attention

Entropy and attention are not static; they evolve characteristically along the decoding trajectory:

• Attention entropy universally increases, denoting context diffusion as the sequence lengthens.
• Deterministic models (DeepSeek, LLaMA) reduce output entropy over time, signifying confidence consolidation.
• Exploratory/balanced models (Gemma, Qwen) increase output entropy, reflecting growing epistemic uncertainty, amplified in long-form reasoning.
• Structural decoupling is observed: increased attention diffusion does not necessarily parallel increased output uncertainty.

  Figure 2: Temporal drift of output and attention entropy across early and late generation steps.

Layer-level analysis reveals that Qwen exhibits the highest vertical specialization (HDI 0.951), with distinct layers serving as broad context integrators versus refinement layers, as opposed to DeepSeek, which is vertically uniform. Diffuse early layers are primarily context gatherers; concentrated later layers perform focused final decision-making.

Geometric Representation: Internal State Magnitude and Drift

Hidden-state L2 norms and between-step L2 drift further stratify model behavior:

• Gemma operates in a high-magnitude, high-drift regime, paralleling increased uncertainty and volatile generative behavior.
• DeepSeek shows minimal drift (Delta L2 67.8), with subdued but highly stable internal geometry.
• LLaMA and Qwen reside in moderate regimes, balancing stability, and adaptability.

There is a consistent, positive association between representational drift and output entropy—models with the most representational instability demonstrate the greatest probabilistic uncertainty. However, magnitude and entropy are partially independent; scale normalization is crucial, and raw L2 metrics are not directly comparable across architectures.

Figure 3: Joint distributions and interactions between output entropy, Top-1 confidence, KV cache memory, representational drift, and attention dispersion of four SLMs.

Structural Interactions: Correlational Synthesis

Quantitative correlation analysis integrates all structural signals. Output entropy and Top-1 confidence are negatively correlated, confirming the expected inverse relationship between uncertainty and decisiveness. Entropy and representational drift are strongly positively correlated, highlighting that geometric instability underpins regions of probabilistic ambiguity and hallucination risk. Attention entropy and KV memory show weaker, architecture-dependent relationships. No single structural component dictates truthfulness or error modes: generative reliability is an emergent property of their interaction.

Figure 4: Correlation heatmap of entropy, confidence, attention dispersion, representational drift, and memory metrics.

Theoretical and Practical Implications

This trace-level structural taxonomy advances theoretical understanding of SLMs:

• Truthfulness is a function of temporally regulated entropy, layered attention allocation, and representational geometry—not merely output accuracy or static confidence.
• Deterministic architectural motifs minimize uncertainty but risk brittle failure; exploratory structures adapt to complex context at the expense of instability and hallucination.
• Optimizing SLMs for factual and decision-critical applications demands internal uncertainty regularization, not just surface-level performance tuning.
• Structural metrics (stepwise entropy/dispersion, representational drift) should inform adaptive decoding protocols, hallucination-aware training, and system design for edge SLM deployment.

Conclusion

Integrating probabilistic and geometric analysis, this paper charts a rigorous framework for evaluating SLM reliability through trace-level internal signals, demonstrating that entropy evolution, attention allocation, and representational dynamics together shape generative truthfulness. This methodology enables principled architectural comparisons, real-time uncertainty monitoring, and the development of entropy-regularized, application-specific SLMs—a direction with substantial implications for both trustworthy edge AI and the interpretability/diagnosability of transformer-based generative models. Future work addressing sampling temperature effects, multi-hop reasoning scenarios, and scaling to larger/heterogeneous models will extend these foundational insights.