Decomposing Theory of Mind: How Emotional Processing Mediates ToM Abilities in LLMs (2511.15895v1)

Abstract: Recent work shows activation steering substantially improves LLMs' Theory of Mind (ToM) (Bortoletto et al. 2024), yet the mechanisms of what changes occur internally that leads to different outputs remains unclear. We propose decomposing ToM in LLMs by comparing steered versus baseline LLMs' activations using linear probes trained on 45 cognitive actions. We applied Contrastive Activation Addition (CAA) steering to Gemma-3-4B and evaluated it on 1,000 BigToM forward belief scenarios (Gandhi et al. 2023), we find improved performance on belief attribution tasks (32.5\% to 46.7\% accuracy) is mediated by activations processing emotional content : emotion perception (+2.23), emotion valuing (+2.20), while suppressing analytical processes: questioning (-0.78), convergent thinking (-1.59). This suggests that successful ToM abilities in LLMs are mediated by emotional understanding, not analytical reasoning.

• The paper reveals that steering emotional processes boosts ToM accuracy by 14.2%, as evidenced by a rise from 32.5% to 46.7%.
• The paper applies a mechanistic interpretability approach using 45 linear probes to identify critical activation shifts in mid-model layers.
• The paper demonstrates that enhancing emotional and generative faculties results in a suppression of analytical reasoning in LLMs.

Decomposing ToM in LLMs: Emotional Processing as a Mediator

Introduction

This paper presents a mechanistic interpretability approach to dissect Theory of Mind (ToM) abilities in LLMs, specifically focusing on the role of emotional processing versus analytical reasoning. The authors apply Contrastive Activation Addition (CAA) steering to the Gemma-3-4B model and utilize a suite of 45 linear cognitive action probes. Evaluation on 1,000 BigToM belief attribution scenarios demonstrates that steering not only improves ToM task performance (an increase in accuracy from 32.5% to 46.7%) but also induces pronounced changes in the model’s internal activation structure. Notably, these changes reflect a systematic enhancement of emotional and generative cognitive processes and a concurrent suppression of analytical processes during successful perspective-taking.

Methods

The methodology integrates activation steering, probe-based interpretability, and controlled evaluation on ToM tasks. The paper defines a taxonomy of 45 cognitive actions, spanning Metacognitive, Analytical, Creative, Emotional, and Memory-related processes, constructed from cognitive science frameworks and literature.

Synthetic narratives exemplifying each action serve as training data for one-vs-rest linear probes, which are trained to decode action presence from activations captured across all 30 layers of Gemma-3-4B. The CAA steering vectors are trained using contrastive triplets from BigToM’s forward belief scenarios, ensuring that the vectors target representational differences underlying correct versus incorrect mental state attributions.

For evaluation, the model's answers to belief attribution questions are scored via logit-based ranking, and probe activations are measured at three critical timepoints: upon question presentation, after the correct answer, and after the incorrect answer. Statistical analysis focuses on difference of activation (steered minus baseline) for each probe and category, thereby elucidating which cognitive processes become more or less prominent under improved ToM performance.

Probe Performance and Layer Diagnostics

Validation of the linear probes indicates robust classification, with an average AUC-ROC of 0.78 across all cognitive actions and layers. Layer-wise inspection reveals a pronounced peak in probe discriminability within mid-layers (layers 5-24), with optimal cognitive action abstraction at layer 9 (AUC-ROC 0.948). Early layers are biased toward superficial linguistic features, while the final layers demonstrate dominance by next-token prediction dynamics, diminishing probe fidelity.

Figure 1: Cognitive action probe performance across all 30 layers, peaking at layer 9, with mid-layers encoding the highest-level cognitive abstractions.

Furthermore, action-wise ranking exposes considerable heterogeneity. Actions such as suspending_judgment (AUC-ROC 0.988) and counterfactual_reasoning (0.984) manifest highly localized, easily detectable activation patterns. In contrast, emotion_responding (0.778) and understanding (0.837) appear more contextually or representationally distributed.

Figure 2: Top and bottom cognitive actions by probe discriminability, indicating which mental faculties are most and least clearly represented.

Cognitive Shifts Induced by Steering

Application of CAA vectors produces a marked increase in ToM task accuracy, with a 14.2% improvement driven by a shift in the model's internal processing. Detailed probe analysis shows that this shift is characterized by increased activation of emotional and creative processes—specifically emotion_perception (Δ=+1.73), emotion_valuing (Δ=+0.85), emotion_understanding (Δ=+0.77), and hypothesis_generation (Δ=+1.63)—across all timepoints. Simultaneously, analytical processes decline: questioning (Δ=-1.24), convergent_thinking (Δ=-1.13), and understanding (Δ=-0.77).

The radar comparison and longitudinal probe assessment (Figure 3) illustrate the magnitude and consistency of this divergence.

Figure 3: Cognitive action category comparison demonstrating the amplification of emotional and creative processes in steered models relative to baseline.

Complete timecourse analyses confirm these findings: emotional actions maintain elevations at question, post-correct answer, and post-incorrect answer stages, while analytical actions decrease across all phases.

Figure 4: Time-resolved steering effects indicating systematic modulation of action categories, with persistent emotional activation increases and analytical decreases.

Category-level aggregation further corroborates that ToM improvement in LLMs is not an effect of increased overall cognition, but reflects a trade-off: enhanced emotional and creative faculties at the expense of classical analytical faculties.

Figure 5: Category-wise quantification and distribution of steering effects at the answer timepoint, reinforcing the observed polarity shift.

Analysis of the most and least affected cognitive actions pinpoints the processes most radically altered by ToM steering.

Figure 6: Top cognitive actions by activation change between baseline and steering, highlighting emotional and generative processes as primary drivers of successful ToM.

Visualization of probe differences across all actions and timepoints, as a heatmap, demonstrates the global and consistent nature of these shifts.

Figure 7: Heatmap of cognitive action differences induced by steering, confirming categorical patterns.

Theoretical and Practical Implications

The results directly contradict the hypothesis that LLM-based ToM is predominantly driven by analytical or chain-of-thought instantiation. Instead, the findings substantiate a model wherein emotional processing forms the mechanistic substrate for belief attribution and successful perspective-taking in LLMs. This aligns with select neuroscience results suggesting that cognitive and affective ToM are tightly coupled in human neural architectures, especially in temporoparietal regions—though no direct correspondence is claimed here.

Practically, the mechanistic decomposition approach grants fine-grained interpretability, which may inform the design of safer, more reliable AI with explicit cognitive scaffolding. It further suggests that future advances in social reasoning tasks may depend more on representing emotional context and generative faculties, rather than optimizing deliberative, analytical routines.

The protocol offers a scalable template for cognitive decomposition: applying interpretability tools (probes, steering) to dissect and modulate internal faculties relevant for any complex AI capability.

Expanding this framework to larger, more diverse model architectures and broader benchmarks will be crucial for future mechanistic transparency. It also opens questions about the transferability of steering effects and whether analogous patterns are a universal property of neural architectures trained on linguistic data.

Conclusion

This paper provides a rigorous, probe-driven decomposition of the internal mechanisms through which activation steering improves ToM in LLMs. The analyses reveal that enhanced performance is mediated by increased engagement of emotional and generative cognitive processes and suppression of analytical reasoning, suggesting that the essence of successful perspective-taking in LLMs is fundamentally affective rather than deliberative. These insights challenge prevailing interpretive frameworks and pave the way for more nuanced investigations of high-level cognition in artificial agents.