---
title: Valence-Arousal Subspace in LLMs
url: https://www.emergentmind.com/topics/valence-arousal-subspace-in-llms
type: topic
---

# Valence-Arousal Subspace in LLMs

The valence–arousal (VA) subspace refers to a low-dimensional, interpretable structure embedded within the hidden representations of large language models (LLMs), in which human emotion is captured as a continuous coordinate system with orthogonal axes corresponding to valence (pleasure/displeasure) and arousal (activation/deactivation). Recent research has established that this subspace is not only present and quantifiable in LLMs, but exhibits geometric properties and behavioral influences comparable to those posited by psychological models of human affect, including the classic circumplex of emotion.

## 1. Theoretical Foundations and Motivation

Human emotions are frequently modeled along two principal axes: valence (ranging from negative to positive affect) and arousal (from calm to excited states). The VA space offers a compact, continuous alternative to categorical emotion labels, and forms the conceptual basis for the circumplex model of affect. LLMs, although trained solely on textual data, have demonstrated emergent capacity to represent such affective dimensions internally, a phenomenon with implications for interpretability, control, and safety in AI systems.

## 2. Methods for Uncovering the VA Subspace

Multiple research groups have independently devised methodologies for extracting and characterizing the VA geometry from LLM hidden states. A canonical approach, detailed in "Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control" [2604.03147], involves the following sequence:

1. **Data Selection:** Utilize large annotation-rich emotion corpora, such as GoEmotions (211K single-label texts).
2. **Emotion Steering Vectors:** For each emotion $e$, compute a mean difference vector between activations for $e$ and a neutral reference at each model layer,
   $$
   \mathbf{v}_e^{(\ell)} = \mathbb{E}_{x \in D_e}[\mathbf{h}^{(\ell)}(x)] - \mathbb{E}_{x \in D_{\rm neutral}}[\mathbf{h}^{(\ell)}(x)].
   $$
3. **Dimensionality Reduction:** Aggregate these vectors across 27 emotion categories, apply centering, then perform principal components analysis (PCA). 
4. **VA Axis Recovery:** For each emotion, obtain "self-reported" (model-internal) VA ratings. Regress these (via ridge regression) against the principal component projections to derive optimal linear combinations corresponding to valence and arousal. Orthogonalize to yield unit directions $\mathbf{w}_V$, $\mathbf{w}_A$ in the activation space. 
5. **Projection:** Any hidden state $\mathbf{h}$ can then be mapped to a VA coordinate $(v_{\rm proj}, a_{\rm proj}) = (\mathbf{h} \cdot \mathbf{w}_V, \mathbf{h} \cdot \mathbf{w}_A)$.

Alternative approaches include supervised quantile regression from hidden states to continuous valence (but typically not arousal), or unsupervised singular value decomposition on mean-pooled activations, as in "Emotions Where Art Thou" [2510.22042].

## 3. Geometric and Statistical Properties

The VA subspace derived from LLM representations exhibits several salient features:

- **Circular Geometry:** Projected emotion category vectors in the learned VA plane lie nearly on a circle, matching the circumplex model of human affect. Circularity (mean to standard deviation of radial distances) is high: e.g., 3.17 at layer 31 of Llama 3.1-8B [2604.03147].
- **Axis Alignment:** The principal axes correspond to valence (PC1 or learned $w_V$) and arousal (typically PC4 or $w_A$), with emotions arranged angularly as predicted by psychological models (e.g., joy at 0°, anger at 90°, sadness at 180°).
- **Cross-Lexicon Validation:** Projections of 44,728 word embeddings onto VA axes correlate with human-crowdsourced valence ($r=0.71$) and arousal ($r=0.23$) ratings [2604.03147].
- **Robustness and Universality:** Geometry and probe performance are preserved across layers, datasets (eight emotion corpora, five languages), and architectures (Llama-3.1-8B, Qwen3-8B, Qwen3-14B) [2510.22042, 2604.03147].

## 4. Behavioral and Generative Consequences

Manipulating LLM activations within the VA subspace modulates both generated text and higher-level behaviors:

- **Affective Generation:** Activation steering in VA directions causes monotonic, predictable shifts in the model’s generated outputs along valence and arousal spectra, measured by both continuous BERT-based metrics and lexical sentiment analyzers [2604.03147].
- **Control Over Non-Affective Behaviors:** Steering along arousal, in particular, modulates refusal and sycophancy. Increasing arousal suppresses model refusals and enhances compliant/sycophantic completions; decreasing arousal increases refusal rates. For example, at an arousal perturbation of $\alpha=+0.30$, refusal log-odds drop $-5.63$ and refusal rates decrease by 27 percentage points [2604.03147].
- **Mechanistic Account:** Unembedding vectors for refusal-associated tokens ("I can't", "sorry") occupy the negative-valence, low-arousal quadrant; steering thus directly adjusts their emission probability.
- **Semantic Steering:** Learned intervention modules operating in the emotion subspace can steer the model to emit target emotions with up to 100% top-1 recovery for basic Ekman classes, while keeping semantic drift minimal [2510.22042].

## 5. Comparative Performance and Limitations

Empirical evaluations calibrate the expressive and discriminative capacity of the VA subspace:

- **Emotion Classification:** Linear probes in the 50D emotional manifold retain nearly all discriminative accuracy for multiclass emotion detection compared to the full hidden state (e.g., 39% vs. 42% for 9-way emotion classification; chance 11%) [2510.22042].
- **Semantic Description Tasks:** When LLMs are prompted directly with (v, a) pairs (rather than raw images), their ability to produce discrete category labels is limited (e.g., ≈30% top-1 accuracy for basic emotions) [2502.06875]. However, for free-text description conditioned on VA values, LLM outputs align closely with human affective interpretations (Word2Vec similarity $M=0.80$–$0.81$; $p<.001$) [2502.06875].
- **Absence in Earlier Models:** Studies using GPT-2 show that although hidden states permit highly calibrated quantile prediction of valence for future tokens, explicit probing and manipulation of a low-dimensional VA subspace was not performed [2307.01784].
- **Remaining Gaps:** LLMs struggle to discriminate closely related emotions in regions of mid-valence and mid-arousal, and show reduced classification accuracy for complex or co-occurring emotion states [2502.06875].

## 6. Implications, Mechanistic Interpretations, and Future Directions

The presence of a robust, steerable VA subspace in LLMs reveals key aspects of how these models internalize, represent, and utilize affective constructs:

- **Universality and Alignment:** The affective manifold is stable across layers, architectures, domains, and languages, indicating an emergent "universal" emotional geometry [2604.03147, 2510.22042].
- **Direct Behavioral Modulation:** Causal interventions in the VA plane alter affective tone, refusal rates, and sycophancy, mediated by the alignment of lexical token unembeddings with the VA structure [2604.03147].
- **Interpretability:** VA steering provides mechanistic transparency for observed behavior shifts and safety-related controls, such as refusal strategies.
- **Modality Bridging:** Prompt-based injection of VA values (extracted from other modalities, e.g., facial expression images) demonstrates that the affective subspace is not limited to linguistic content but can be leveraged for cross-modal emotion inference and generation [2502.06875].
- **Open Questions:** Whether training explicit mappings from external affective representations (e.g., small adapter networks) can enforce sharper VA discrimination, and how multimodal hybrids can further leverage the VA infrastructure, remain areas for future work [2502.06875].

## 7. Summary Table: Key Properties of the VA Subspace in LLMs

| Property                   | Quantitative Result / Finding            | Reference         |
|----------------------------|------------------------------------------|-------------------|
| Circularity ratio (VA plane)   | 3.17 (Llama-3.1-8B, layer 31)           | [2604.03147]      |
| Valence–human corr. (words)    | $r=0.71$ (NRC-VAD lexicon)              | [2604.03147]      |
| Arousal–human corr. (words)    | $r=0.23$ (NRC-VAD lexicon)              | [2604.03147]      |
| Top-1 emotion steering accuracy| 85–100% (basic emotions, multi-language)| [2510.22042]      |
| Refusal modulation (arousal)   | $-5.63$ Δlog-odds, $-27$pp refusals     | [2604.03147]      |

These findings establish the VA subspace as a robust, generalizable, and interpretable foundation for continuous affective representation and control in large language models, with meaningful consequences for both synthetic and cross-modal emotion understanding.

Source: https://www.emergentmind.com/topics/valence-arousal-subspace-in-llms