Emotion2Skill: Model-Internal Emotion Signals for Adaptive Skill Selection and Evolution

Abstract: Skill-based LLM agents select reusable procedures from an external library to solve complex tasks, yet their routing decisions rely entirely on text-level signals such as task descriptions, verbal reflections, and experience-derived rules, while the model's own internal representational state remains unobserved. Recent interpretability work has shown that LLMs maintain linear emotion representations that causally influence behavior; however, these representations have been exploited only for post-hoc analysis or direct output steering, and have not been used to inform agent-level decision-making. We propose Emotion2Skill, a framework that extracts LLM-internal emotion vectors and incorporates them into both skill selection and skill evolution. At each decision step, a 27-dimensional emotion state is extracted from the residual stream and mapped to a confidence-gated summary injected into the routing prompt. Beyond online selection, emotion trajectories are analyzed for abrupt internal-state shifts to pinpoint problematic skill invocations, guiding targeted SOP rewriting that replaces the coarse binary outcome signal of prior methods. On WebShop and ALFWorld, Emotion2Skill with Qwen3-8B improves over the Zero-Shot baseline by +26.9% success rate and +25.5% average success respectively, outperforming all baselines on both benchmarks with consistent gains on Qwen3-14B. Co-activation analysis further reveals semantically coherent emotion--skill pairings, confirming that the routing improvements reflect meaningful internal-state signals rather than opaque statistical correlations. These results establish LLM-internal emotion representations as an effective decision-level signal for orchestrating agent skill systems, extending their utility beyond interpretability and output steering. The code is available at https://github.com/BoHan-LIN04/Emotion2Skill.

• The paper introduces a confidence-gated framework that extracts 27-dimensional emotion states from LLM residual streams for adaptive skill selection and trajectory-guided SOP rewriting.
• The method improves Qwen3-8B success by 25.5 percentage points on ALFWorld and 26.9 points on WebShop versus zero-shot routing, while also improving Qwen3-14B, MATH, and MBPP results.
• Ablations show that contrastive emotion extraction, learned state encoding, confidence gating, and emotion-guided evolution provide distinct benefits, although calibration, causal attribution, and cross-model robustness remain open challenges.

Emotion2Skill: Model-Internal Emotion Signals for Adaptive Skill Selection and Evolution

Research Problem and Central Thesis

“Emotion2Skill: Model-Internal Emotion Signals for Adaptive Skill Selection and Evolution” (2608.09248) investigates whether latent affective representations in an LLM can serve as actionable control signals for skill-based agents. Existing skill-routing systems typically condition decisions on task descriptions, observations, retrieved skill descriptions, verbal reflections, or trajectory-level outcomes. These signals expose the agent’s textual environment and explicit history, but not the model’s contemporaneous internal state. Emotion2Skill argues that this omission is consequential: two externally similar situations may require different procedures depending on the model’s accumulated uncertainty, confidence, frustration, or perceived progress.

The paper builds on evidence that high-level concepts are represented approximately linearly in LLM activation spaces (Zou et al., 2023, Li et al., 2024) and, more specifically, that fine-grained emotion concepts can be decoded from residual-stream activations and causally influence model behavior (Sofroniew et al., 9 Apr 2026). Its central claim is that these representations are not merely objects of interpretability analysis or mechanisms for output steering. They can also provide an interpretable, task-independent signal for decision-level orchestration, including both online skill selection and offline skill evolution.

The proposed system has two coupled functions. First, it extracts a 27-dimensional emotion state from the model’s residual stream and converts it into a prompt-compatible summary for skill routing. Second, it detects abrupt changes in the emotion trajectory and uses them to localize failure-prone segments for targeted rewriting of skill SOPs.

Figure 1: Emotion2Skill treats residual-stream emotion representations as an internal decision signal that complements external task and environment observations.

Framework Architecture

Emotion2Skill operates with a skill library $\mathcal{S}$ whose elements are reusable procedures represented by natural-language SOPs. At each decision step, the agent receives the conventional textual context: task instructions, dialogue history, observations, and candidate SOPs. The method adds an auxiliary emotion summary only when the extracted state is judged sufficiently informative.

The framework contains three principal stages:

1. Offline emotion extraction: emotion directions are estimated from residual-stream activations elicited by synthetic stories corresponding to the 27 GoEmotions categories [demszky2020goemotions]. Contrastive averaging, global-mean subtraction, and PCA-based denoising produce a linear extractor.
2. Online emotion-augmented routing: the extracted state is passed through a small MLP encoder. The encoder selects one of 12 natural-language templates and estimates a confidence score. If confidence exceeds a threshold, the template and confidence are appended to the skill-selection prompt; otherwise, the agent falls back to text-only routing.
3. Emotion-guided evolution: stepwise emotion vectors are logged during episodes. Large cosine-distance transitions are treated as internal-state change points. For underperforming skills, local windows surrounding these transitions are supplied to an LLM rewriter, which revises the corresponding SOP.

This design preserves the LLM as the primary routing policy. Emotion information is not used as a hard-coded controller or direct action policy; it is translated into text and reintroduced through the same prompt channel as the task context.

Figure 2: The method combines offline residual-stream probing, confidence-gated prompt augmentation, and trajectory-localized SOP rewriting.

Residual-Stream Emotion Extraction

For each GoEmotions category, the authors generate 100 short stories, yielding 2,700 stories in total. The target emotion is not explicitly named in the stories. Activations are mean-pooled after an initial prefix and averaged within each category. The category-specific direction is obtained by subtracting the global mean activation, thereby removing shared linguistic structure. The top principal components of neutral-story activations are then projected out to reduce stylistic variance.

At inference time, the activation at the Assistant delimiter token is projected onto the 27 denoised directions and normalized. The resulting vector is interpreted as the model’s current internal emotion state. This state is not equivalent to a human affective report; rather, it is a linear probe-derived representation whose dimensions are aligned with named emotion concepts.

The paper’s layer analysis is important because the emotion signal is strongly layer-dependent. For Qwen3-8B, classification accuracy across candidate layers is non-monotonic and peaks at layer 24, reaching 37.2% under the paper’s hit-any evaluation criterion, compared with an 8.2% random baseline. Qwen3-14B achieves 39.4% at its selected layer, layer 26. The result supports the claim that emotion information is concentrated in a late-middle region rather than increasing monotonically toward the output layer.

Figure 3: Emotion decoding is layer-specific and non-monotonic, with Qwen3-8B reaching 37.2% GoEmotions accuracy at layer 24.

The representational geometry also exhibits broad semantic organization. Positive-valence emotions cluster separately from negative-valence emotions, while cognitive states such as confusion and curiosity occupy intermediate regions. However, fine-grained categories remain substantially overlapping. This observation motivates the learned encoder: simply injecting the top few emotion labels would discard activation magnitude and interactions among simultaneously active dimensions.

The encoder is a three-layer MLP with fewer than 27,000 parameters. It maps the 27-dimensional state into a 128-dimensional representation, selects among 12 template prototypes, and predicts a scalar confidence value. Training uses warm-up trajectories from a baseline agent and combines supervised contrastive learning with template classification. Positive contrastive pairs correspond to states associated with the same selected skill and successful outcomes. Consequently, the encoder is not trained solely to reproduce emotion labels; it is optimized to organize emotion states according to their routing utility.

Confidence-Gated Skill Selection

The online routing mechanism injects the selected template and confidence score into the skill-selection prompt only when $c_t \geq \tau$. The default threshold is $\tau=0.3$. This gate is a central design choice because an internal signal that is weakly decoded or poorly calibrated can degrade routing by adding irrelevant semantic content.

The gate therefore implements a conservative hybrid policy. When the emotion representation is informative, the agent receives both textual context and internal-state information. When the confidence score is low, the method reverts to the baseline text-only prompt. In the reported WebShop distribution, 72% of decision steps are gate-admitted and 28% are suppressed, indicating that the method uses emotion information frequently but not universally.

The threshold sensitivity results suggest moderate robustness. Performance remains relatively stable for thresholds between 0.1 and 0.6, whereas $\tau=0.9$ produces a substantial degradation. This is consistent with a precision–recall trade-off: an overly high threshold suppresses useful internal-state information, while an overly low threshold admits noisier signals.

The paper presents the gate as an approximate utility filter. That interpretation is plausible but depends on a strong calibration assumption: the encoder’s confidence must correlate with the probability that emotion augmentation improves the selected skill. The empirical results support the operational value of gating, but they do not fully establish calibrated estimates of conditional routing utility.

Emotion Trajectories and Skill Evolution

The second contribution extends emotion signals from online selection to skill-library maintenance. Conventional skill evolution usually receives an episode-level binary outcome and perhaps a textual reflection. Such supervision identifies that a trajectory failed but not necessarily where the failure emerged or which SOP instruction became ineffective.

Emotion2Skill computes the cosine distance between consecutive normalized emotion states. A transition point is declared when the distance exceeds the episode mean by more than one standard deviation. For a flagged skill, the method extracts a three-step diagnostic window containing observations, selected skills, emotion templates, and shift magnitudes. A verbalized transition summary is then supplied to the SOP rewriter along with the current skill description and episode outcome.

The resulting procedure resembles a lightweight change-detection system. It is computationally inexpensive and agnostic to which emotion dimensions changed. In principle, it can detect transitions from exploratory to confused states, from confidence to apprehension, or between different cognitive modes. Its principal limitation is that a large activation-direction shift is not necessarily a causal failure signal. Internal changes may reflect legitimate task progress, prompt boundaries, or benign strategy changes. The paper’s criterion is therefore best understood as a diagnostic heuristic rather than a validated causal attribution mechanism.

The case studies illustrate the intended behavior. In WebShop, a transition into confusion causes the agent to select QueryRephrase instead of repeatedly invoking ProductSearch. During evolution, recurring confused states expose the absence of a rephrasing budget and escalation condition, leading to an SOP that imposes a three-query limit and a fallback path. The example demonstrates how trajectory-localized internal signals can be converted into concrete procedural revisions.

Figure 4: In the WebShop case study, confidence, template changes, and a large emotion-state transition jointly identify a routing correction and an SOP-revision target.

Main Experimental Results

The evaluation covers WebShop and ALFWorld using Qwen3-8B and Qwen3-14B. Baselines include Zero-Shot, ReAct (Patil et al., 2023), Reflexion (Shinn et al., 2023), ExpeL (Lin et al., 2024), and MASA (Yu et al., 29 May 2026). The reported system uses greedy decoding and non-thinking mode, with skill libraries held fixed across the principal comparisons.

On Qwen3-8B, Emotion2Skill obtains:

• 47.4% average success on ALFWorld, compared with 21.9% for Zero-Shot, a gain of 25.5 percentage points.
• 29.7% WebShop success, compared with 2.8% for Zero-Shot, a gain of 26.9 percentage points.
• The strongest performance among the listed baselines on both benchmarks.

On Qwen3-14B, it obtains:

• 52.3% average ALFWorld success, compared with 23.4% for Zero-Shot, a gain of 28.9 percentage points.
• 30.7% WebShop success, compared with 9.4% for Zero-Shot, a gain of 21.3 percentage points.

The method’s standard deviations are also relatively small, at no more than 1.6 percentage points for the principal Emotion2Skill results. This contrasts with higher variability for some baselines, including Reflexion and ExpeL. The authors interpret this as evidence that the internal signal supplies a stable routing advantage rather than an occasional improvement caused by prompt stochasticity.

The ALFWorld breakdown is particularly informative. On Qwen3-8B, Heat improves from 9.6% to 56.9%, a gain of 47.3 percentage points, while Pick2 improves from 4.4% to 31.3%. These are procedural tasks with frequent precondition failures and recovery requirements, making them plausible settings in which latent uncertainty could precede explicit environmental failure messages. However, the benefit is not uniform: MASA remains stronger on Cool for Qwen3-8B. This counterexample limits the claim that emotion signals universally improve routing and suggests that stereotyped tasks may offer less opportunity for adaptive internal-state conditioning.

Figure 5: Emotion–skill co-activation patterns show interpretable associations such as confusion with query reformulation and approval with purchase confirmation.

Interpretability and Generalization

The co-activation analysis is intended to establish that the performance gains do not result from an entirely opaque correlation between arbitrary hidden features and skills. Four associations are highlighted:

• curiosity and desire with ProductSearch;
• confusion and nervousness with QueryRephrase;
• approval and optimism with PurchaseConfirm;
• annoyance and disappointment with PriceCompare.

In a sample of 200 events, GPT-4o judged 76.5% of emotion–skill pairings semantically coherent, with Cohen’s $\kappa=0.81$ against a 50-event human-annotated subset. Most inconsistent cases involved low-intensity activations, which the authors argue may make the estimate conservative.

This analysis provides qualitative and correlational support, but it should not be conflated with causal validation. Co-activation can arise because both the emotion representation and skill choice respond to the same textual context. Stronger evidence would require interventions on the emotion directions, counterfactual routing tests, or mediation analyses showing that the extracted state contributes beyond observations and history.

The out-of-domain results are nevertheless notable. Without re-extracting emotion vectors, the Qwen3-8B system improves:

• MATH accuracy from 54.8% to 69.2%, a gain of 14.4 percentage points.
• MBPP pass@1 from 60.0% to 71.8%, a gain of 11.8 percentage points.

These results suggest that the extracted state may encode domain-general uncertainty or strategy-change information rather than WebShop- or ALFWorld-specific semantics. The strongest gains occur on multi-step mathematical problems and compositional programming tasks, where subskill decomposition and verification are useful.

Figure 6: Reusing the same Qwen3-8B emotion vectors improves routing on MATH and MBPP, supporting transfer beyond interactive environments.

Ablation Evidence

The ablations support the non-redundancy of the main components. Removing contrastive emotion extraction reduces WebShop success by 4.7 percentage points and ALFWorld average success by 8.6 points. Replacing the learned encoder with raw top-three emotion labels reduces performance by 3.1 and 5.5 points, respectively. Removing emotion-guided evolution produces smaller but consistent losses of 0.9 points on WebShop and 3.1 points on ALFWorld.

The largest ablation effect is associated with the extraction procedure, indicating that the representation quality is more important than the prompt realization alone. The encoder result further shows that the method does not gain merely from adding emotion words to the prompt. Magnitude-sensitive encoding, prototype selection, and confidence gating collectively matter.

The evolution ablation is comparatively modest on WebShop but larger on ALFWorld. This discrepancy is theoretically plausible: WebShop often permits immediate routing corrections, whereas ALFWorld’s longer procedural dependencies may benefit more from improving the underlying SOPs. The results also imply that selection and evolution are partially separable sources of improvement.

Theoretical Claims and Methodological Qualifications

The appendix provides analyses linking contrastive direction extraction to Fisher-style discriminant structure, confidence gating to expected utility, transition detection to sequential change detection, and emotion augmentation to conditional mutual information. These analyses clarify the intended statistical interpretation of the method, but several claims require careful qualification.

The Fisher-LDA connection depends on approximate shared and isotropic within-class covariance. The paper reports a residual covariance deviation below 0.12 relative to the Frobenius norm, but this diagnostic does not establish that the resulting classifier is Bayes-optimal in the full 27-class, multi-label setting. Similarly, the proposed layer-selection proposition applies under Gaussian and isotropic assumptions that are unlikely to hold exactly in transformer residual spaces.

The information-theoretic argument that adding emotion states cannot reduce mutual information is formally correct for the true joint variable $(X_t,E_t)$. It does not guarantee that a finite, noisy, textualized estimate of $E_t$ improves an LLM’s routing decision. The confidence gate addresses this problem operationally, but its claimed utility interpretation remains dependent on calibration and on the relationship between template text and the backbone’s decision boundary.

The theoretical convergence result for SOP evolution is conditional on a consistent rewriter that strictly incorporates the diagnosed root cause. Real LLM rewriting is stochastic, non-monotonic, and capable of introducing regressions. The empirical three-round evolution results are encouraging, but they cannot establish the stated monotonic convergence behavior without stronger controls and repeated evolution trajectories.

Practical Implications and Future Directions

Emotion2Skill has several practical implications for white-box agent systems. It provides a low-parameter control interface between mechanistic interpretability and agent orchestration. The extracted state can be computed without modifying the backbone, and the encoder requires less than 27,000 parameters and under ten minutes of training in the reported setup. The confidence gate also supplies a direct fallback mechanism, which is useful when the residual signal is poorly decoded or when a task has a highly deterministic action policy.

The principal systems limitation is the requirement for residual-stream access. API-only models cannot directly expose the necessary activations. Possible alternatives include training a smaller observer model on text and behavior to predict the internal state, using hidden-state distillation from an accessible teacher, or replacing emotion vectors with other interpretable latent variables such as uncertainty, planning mode, or error likelihood.

Future work should test whether emotion directions remain stable across instruction tuning, quantization, decoding modes, and model families. The reported drop from 37.2% emotion accuracy on the base model to 28.1% on an instruction-tuned variant indicates that extraction cannot be assumed to transfer unchanged. Per-task or context-conditioned gating may also address the observed non-uniformity across ALFWorld task types.

A more substantial direction is closed-loop representation learning. The current architecture maps internal state to skills, but executed skills do not explicitly update the emotion representation or its encoder. A bidirectional system could learn a state-transition model in which skill outcomes revise both routing beliefs and latent-state interpretations. Combining internal signals with external reward, trajectory retrieval, or reinforcement learning could produce hybrid routers that use affective representations for fast local adaptation and outcome signals for long-horizon optimization.

Conclusion

Emotion2Skill presents a technically coherent method for converting model-internal emotion representations into actionable signals for skill-based agents. Its main empirical claim is substantial: on Qwen3-8B, it improves Zero-Shot performance by 25.5 percentage points on ALFWorld and 26.9 percentage points on WebShop, while also producing gains on Qwen3-14B, MATH, and MBPP. The ablations indicate that contrastive extraction, learned encoding, confidence gating, and trajectory-guided evolution each contribute distinct value.

The work’s strongest contribution is conceptual as well as empirical: it treats interpretability-derived latent features as components of an agent-control architecture rather than solely as retrospective explanations. The evidence supports emotion representations as useful routing correlates, but causal attribution, calibration, cross-model stability, and robustness to distribution shift remain open issues. Further progress will depend on validating these signals under interventions and integrating them with externally grounded feedback in adaptive, closed-loop skill systems.