Valence-Arousal Subspace in LLMs

• The valence–arousal subspace is a low-dimensional structure that captures human emotion along valence (pleasure/displeasure) and arousal (activation/deactivation) dimensions.
• It is quantified using techniques like PCA and ridge regression on LLM hidden states, revealing a circular geometry consistent with psychological affect models.
• Manipulating positions within the VA subspace leads to measurable shifts in affective behavior, enhancing interpretability and control over LLM-generated content.

The valence–arousal (VA) subspace refers to a low-dimensional, interpretable structure embedded within the hidden representations of LLMs, in which human emotion is captured as a continuous coordinate system with orthogonal axes corresponding to valence (pleasure/displeasure) and arousal (activation/deactivation). Recent research has established that this subspace is not only present and quantifiable in LLMs, but exhibits geometric properties and behavioral influences comparable to those posited by psychological models of human affect, including the classic circumplex of emotion.

1. Theoretical Foundations and Motivation

Human emotions are frequently modeled along two principal axes: valence (ranging from negative to positive affect) and arousal (from calm to excited states). The VA space offers a compact, continuous alternative to categorical emotion labels, and forms the conceptual basis for the circumplex model of affect. LLMs, although trained solely on textual data, have demonstrated emergent capacity to represent such affective dimensions internally, a phenomenon with implications for interpretability, control, and safety in AI systems.

2. Methods for Uncovering the VA Subspace

Multiple research groups have independently devised methodologies for extracting and characterizing the VA geometry from LLM hidden states. A canonical approach, detailed in "Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control" (Sun et al., 3 Apr 2026), involves the following sequence:

1. Data Selection: Utilize large annotation-rich emotion corpora, such as GoEmotions (211K single-label texts).
2. Emotion Steering Vectors: For each emotion $e$, compute a mean difference vector between activations for $e$ and a neutral reference at each model layer,

$$\mathbf{v}_e^{(\ell)} = \mathbb{E}_{x \in D_e}[\mathbf{h}^{(\ell)}(x)] - \mathbb{E}_{x \in D_{\rm neutral}}[\mathbf{h}^{(\ell)}(x)].$$

1. Dimensionality Reduction: Aggregate these vectors across 27 emotion categories, apply centering, then perform principal components analysis (PCA).
2. VA Axis Recovery: For each emotion, obtain "self-reported" (model-internal) VA ratings. Regress these (via ridge regression) against the principal component projections to derive optimal linear combinations corresponding to valence and arousal. Orthogonalize to yield unit directions $\mathbf{w}_V$, $\mathbf{w}_A$ in the activation space.
3. Projection: Any hidden state $\mathbf{h}$ can then be mapped to a VA coordinate $$(v_{\rm proj}, a_{\rm proj}) = (\mathbf{h} \cdot \mathbf{w}_V, \mathbf{h} \cdot \mathbf{w}_A)$$.

Alternative approaches include supervised quantile regression from hidden states to continuous valence (but typically not arousal), or unsupervised singular value decomposition on mean-pooled activations, as in "Emotions Where Art Thou" (Reichman et al., 24 Oct 2025).

3. Geometric and Statistical Properties

The VA subspace derived from LLM representations exhibits several salient features:

• Circular Geometry: Projected emotion category vectors in the learned VA plane lie nearly on a circle, matching the circumplex model of human affect. Circularity (mean to standard deviation of radial distances) is high: e.g., 3.17 at layer 31 of Llama 3.1-8B (Sun et al., 3 Apr 2026).
• Axis Alignment: The principal axes correspond to valence (PC1 or learned $w_V$) and arousal (typically PC4 or $w_A$), with emotions arranged angularly as predicted by psychological models (e.g., joy at 0°, anger at 90°, sadness at 180°).
• Cross-Lexicon Validation: Projections of 44,728 word embeddings onto VA axes correlate with human-crowdsourced valence ($r=0.71$) and arousal ($e$0) ratings (Sun et al., 3 Apr 2026).
• Robustness and Universality: Geometry and probe performance are preserved across layers, datasets (eight emotion corpora, five languages), and architectures (Llama-3.1-8B, Qwen3-8B, Qwen3-14B) (Reichman et al., 24 Oct 2025, Sun et al., 3 Apr 2026).

4. Behavioral and Generative Consequences

Manipulating LLM activations within the VA subspace modulates both generated text and higher-level behaviors:

• Affective Generation: Activation steering in VA directions causes monotonic, predictable shifts in the model’s generated outputs along valence and arousal spectra, measured by both continuous BERT-based metrics and lexical sentiment analyzers (Sun et al., 3 Apr 2026).
• Control Over Non-Affective Behaviors: Steering along arousal, in particular, modulates refusal and sycophancy. Increasing arousal suppresses model refusals and enhances compliant/sycophantic completions; decreasing arousal increases refusal rates. For example, at an arousal perturbation of $e$1, refusal log-odds drop $e$2 and refusal rates decrease by 27 percentage points (Sun et al., 3 Apr 2026).
• Mechanistic Account: Unembedding vectors for refusal-associated tokens ("I can't", "sorry") occupy the negative-valence, low-arousal quadrant; steering thus directly adjusts their emission probability.
• Semantic Steering: Learned intervention modules operating in the emotion subspace can steer the model to emit target emotions with up to 100% top-1 recovery for basic Ekman classes, while keeping semantic drift minimal (Reichman et al., 24 Oct 2025).

5. Comparative Performance and Limitations

Empirical evaluations calibrate the expressive and discriminative capacity of the VA subspace:

• Emotion Classification: Linear probes in the 50D emotional manifold retain nearly all discriminative accuracy for multiclass emotion detection compared to the full hidden state (e.g., 39% vs. 42% for 9-way emotion classification; chance 11%) (Reichman et al., 24 Oct 2025).
• Semantic Description Tasks: When LLMs are prompted directly with (v, a) pairs (rather than raw images), their ability to produce discrete category labels is limited (e.g., ≈30% top-1 accuracy for basic emotions) (Mehra et al., 8 Feb 2025). However, for free-text description conditioned on VA values, LLM outputs align closely with human affective interpretations (Word2Vec similarity $e$3–$e$4; $e$5) (Mehra et al., 8 Feb 2025).
• Absence in Earlier Models: Studies using GPT-2 show that although hidden states permit highly calibrated quantile prediction of valence for future tokens, explicit probing and manipulation of a low-dimensional VA subspace was not performed (Gagne et al., 2023).
• Remaining Gaps: LLMs struggle to discriminate closely related emotions in regions of mid-valence and mid-arousal, and show reduced classification accuracy for complex or co-occurring emotion states (Mehra et al., 8 Feb 2025).

6. Implications, Mechanistic Interpretations, and Future Directions

The presence of a robust, steerable VA subspace in LLMs reveals key aspects of how these models internalize, represent, and utilize affective constructs:

• Universality and Alignment: The affective manifold is stable across layers, architectures, domains, and languages, indicating an emergent "universal" emotional geometry (Sun et al., 3 Apr 2026, Reichman et al., 24 Oct 2025).
• Direct Behavioral Modulation: Causal interventions in the VA plane alter affective tone, refusal rates, and sycophancy, mediated by the alignment of lexical token unembeddings with the VA structure (Sun et al., 3 Apr 2026).
• Interpretability: VA steering provides mechanistic transparency for observed behavior shifts and safety-related controls, such as refusal strategies.
• Modality Bridging: Prompt-based injection of VA values (extracted from other modalities, e.g., facial expression images) demonstrates that the affective subspace is not limited to linguistic content but can be leveraged for cross-modal emotion inference and generation (Mehra et al., 8 Feb 2025).
• Open Questions: Whether training explicit mappings from external affective representations (e.g., small adapter networks) can enforce sharper VA discrimination, and how multimodal hybrids can further leverage the VA infrastructure, remain areas for future work (Mehra et al., 8 Feb 2025).

7. Summary Table: Key Properties of the VA Subspace in LLMs

Property	Quantitative Result / Finding	Reference
Circularity ratio (VA plane)	3.17 (Llama-3.1-8B, layer 31)	(Sun et al., 3 Apr 2026)
Valence–human corr. (words)	$e$6 (NRC-VAD lexicon)	(Sun et al., 3 Apr 2026)
Arousal–human corr. (words)	$e$7 (NRC-VAD lexicon)	(Sun et al., 3 Apr 2026)
Top-1 emotion steering accuracy	85–100% (basic emotions, multi-language)	(Reichman et al., 24 Oct 2025)
Refusal modulation (arousal)	$e$8 Δlog-odds, $e$9pp refusals	(Sun et al., 3 Apr 2026)

These findings establish the VA subspace as a robust, generalizable, and interpretable foundation for continuous affective representation and control in LLMs, with meaningful consequences for both synthetic and cross-modal emotion understanding.