From Syntax to Emotion: A Mechanistic Analysis of Emotion Inference in LLMs

Abstract: LLMs are increasingly used in emotionally sensitive human-AI applications, yet little is known about how emotion recognition is internally represented. In this work, we investigate the internal mechanisms of emotion recognition in LLMs using sparse autoencoders (SAEs). By analyzing sparse feature activations across layers, we identify a consistent three-phase information flow, in which emotion-related features emerge only in the final phase. We further show that emotion representations comprise both shared features across emotions and emotion-specific features. Using phase-stratified causal tracing, we identify a small set of features that strongly influence emotion predictions, and show that both their number and causal impact vary across emotions; in particular, Disgust is more weakly and diffusely represented than other emotions. Finally, we propose an interpretable and data-efficient causal feature steering method that significantly improves emotion recognition performance across multiple models while largely preserving language modeling ability, and demonstrate that these improvements generalize across multiple emotion recognition datasets. Overall, our findings provide a systematic analysis of the internal mechanisms underlying emotion recognition in LLMs and introduce an efficient, interpretable, and controllable approach for improving model performance.

• The paper demonstrates a three-phase progression in LLMs from syntax, through semantics, to emotion-specific features using sparse autoencoders.
• The study employs phase-stratified causal tracing to identify and intervene on features that selectively drive emotion logits, enhancing model accuracy.
• Targeted feature steering improves underrepresented emotion recognition while preserving overall language competence, offering pathways for safer AI.

Mechanistic Dissection of Emotion Inference in LLMs

Introduction

"From Syntax to Emotion: A Mechanistic Analysis of Emotion Inference in LLMs" (2604.25866) investigates the representational hierarchy and causal pathways underlying emotion inference in LLMs using feature-level tools from mechanistic interpretability, particularly sparse autoencoders (SAEs). The study addresses three core research questions: (1) How emotion signals are encoded and evolve within model layers; (2) which internal representations have direct causal influence on emotion predictions; and (3) whether targeted intervention in these representations can reliably improve emotion inference without degrading linguistic competence. The work spans multiple LLMs (Gemma-2-2B, Gemma-2-9B, Llama-3.1-8B) and utilizes carefully selected datasets with verified emotion labels. It combines feature analysis, causal interventions, and controlled steering experiments to expose the structure, deficits, and malleability of emotion circuits in LLMs.

Phase Structure of Emotion Representation

The study demonstrates a robust three-phase progression in the internal representations supporting emotion inference. By extracting and analyzing SAE-derived feature activations across all layers, and grouping them via topic modeling into syntactic, conceptual, and emotional categories, the authors document a sequential internal computation:

1. Phase 1: Syntax-related features, encoding surface structures, dominate the early hidden states.
2. Phase 2: Semantic or conceptual features emerge and peak in mid-layers, supporting event and situational inference.
3. Phase 3: Emotion-related features arise in late layers and are selectively activated in line with the targeted emotion class.

The delineation of these phases is empirically supported by category-level activation statistics. Notably, emotion-specific features display consistent and often co-activated patterns, particularly in later layers. The early-to-late evolution underlines that emotional inference is a high-level computation building on progressively more abstracted representations, and that direct targets for intervention are concentrated in the final phase.

Figure 1: A subset of feature topics and their mean activation across layers for instances with different emotion labels, illustrating temporal emergence and co-activation structure.

Figure 2: Average activation (smoothed) per category across layers, highlighting phase boundaries in representational emergence.

This observation recasts previous layerwise probe results, clarifying that mid-layer probe accuracy arises from correlated semantic constructs, with emotion disentanglement only occurring at later computational stages.

Causal Pathways and Feature Specialization

To localize and assess the features that directly drive emotion predictions, the study employs phase-stratified causal tracing. The backbone of this methodology is the ablation of individual SAE features and quantification of logit impact per emotion class, followed by hierarchical backtracking to earlier phases to reconstruct minimal causal pathways. Features are filtered by logit effect magnitude and activation salience.

Key findings include:

• Only a restricted set of late-phase features exert strong, selective causal influence on emotion logits.
• Many syntactic and conceptual features (phases 1 and 2) exert diffuse, shared influence across emotions, but late-phase features are substantially more emotion-specific.
• There is marked unevenness in the internal representation of emotions: Disgust is notably weak and distributed, with few strongly causal features, while Surprise possesses few but highly specialized features. In contrast, Anger, Joy, and Fear exhibit richer, denser causal feature sets.

  Figure 3: Influence of causal sparse features on emotion logits, revealing the emotion selectivity and impact distribution across phases.

The study also illustrates, via case-specific causal graphs, the sequence by which inference traverses from syntax through highly compositional semantic concepts to tightly emotion-specific features, confirming the hierarchical organization of emotion pathways.

Figure 4: Subgraph of the sparse feature causal graph for a fear-prompt, mapping the flow from syntactic, through situational/semantic, to emotion-specific activations.

Targeted Steering and Controllability

The authors introduce a causal feature steering approach, optimizing a sparse, interpretable scaling vector over the set of causally identified features. This intervention is shown to:

• Substantially improve emotion recognition accuracy in Gemma and Llama-3 models, especially for previously underrepresented emotions (notably Disgust and Surprise).
• Preserve general language modeling ability, reflected in negligible change in perplexity on standard corpora.
• Generalize robustly across external emotion datasets, outperforming zero-shot, few-shot, and non-causal feature selection baselines.

The steering vector only modifies a small subset of features (26 for Gemma-2-2B), mainly phase-3, directly mapping to interpretable concepts (e.g., social interaction, power dynamics, personal crises). Data efficiency experiments show that only a handful of examples per emotion are required to achieve near-maximal improvements, underscoring the transferability and reusability of the discovered features.

Theoretical and Practical Implications

This study demonstrates the practicality of using SAEs as an analytic and intervention tool for affective circuit dissection within LLMs. The robust three-phase structure and the identification of emotion-specific, strongly causal features provide a rational, localized basis for targeted model control. The empirical revelation of severe inter-emotion imbalance is especially salient; it points to latent sources of bias and unreliability in real-world deployments even for high-performing LLMs.

Future work may focus on extending these circuit-discovery principles to more complex, compositional affect constructs and to larger, state-of-the-art models. The findings imply the feasibility of incorporating sparse, monosemantic feature-based controls into safety-critical LLM systems for controllable and auditable emotional behavior. Furthermore, the revealed structure can inform dataset design by highlighting emotional blind spots and guiding data curation towards underrepresented phenomena.

Conclusion

The paper delivers an authoritative mechanistic analysis of emotion inference in LLMs. By leveraging sparse, interpretable features and causal intervention, it exposes a structured computational flow from syntax to emotion, identifies key features driving emotion classification, and demonstrates that targeted, interpretable steering is not only feasible but effective at correcting deficits without loss of global performance. The results set a new standard for causal mechanistic analysis in affective NLP, providing both methodological advances and actionable diagnostic insights for the future development and deployment of emotionally intelligent AI systems.