Identity Attractors in LLM Activation Space

• Identity attractors are defined as distinct regions in LLM activation space where persona-specific prompts converge into tight, semantically coherent clusters.
• Techniques such as PCA, cluster separation metrics, and activation-steering directions are used to quantify these attractors by measuring intra- and inter-cluster distances.
• The study highlights practical applications including controlled persona steering, agent initialization, and improved interpretability of LLM outputs.

An identity attractor in LLMs refers to a region, direction, or manifold in activation space toward which the internal representations (activations) of the model gravitate when processing prompts expressing a given identity, persona, or user-specific signal. Such attractors are revealed as tight, semantically meaningful clusters or basins in the hidden-state space, distinguished by their geometric properties, statistical separability, and persistence across prompts and model layers. This article presents a comprehensive, technical review of identity attractors in LLM activation space, integrating formal definitions, quantification protocols, empirical results, and the implications for LLM interpretability and control.

1. Formal Definitions and Mathematical Criteria

Identity attractors are defined geometrically as regions or subspaces in the model’s hidden-state manifold $A \subset \mathbb{R}^d$ where semantically related prompts—expressing a particular persona, role, user, or agent identity—collapse to a tight cluster that is well-separated from clusters induced by other identities or controls. Formally, for an embedding extractor $h_\ell(d)$ at layer $\ell$, attractor criteria include:

• Within-cluster tightness: the mean cosine or Euclidean pairwise distance $D_\text{within}(S)$ among activations for prompts $S$ expressing the same identity is much smaller than the between-cluster distance $D_\text{between}(S, T)$ for distinct identities or random controls.
• Contractive dynamics: as layer depth increases, activations for a given persona converge toward a fixed region (or linear direction) $\mu_S$.
• Stability and invariance: repeated presentation of identity-expressing prompts returns the trajectory to the same attractor region despite symbolic or structural perturbations.
• Attractor basin: for fixed-point attractors under a transformation $F_\theta:A \rightarrow A$ (the layerwise LLM update, typically Lipschitz), the basin of attraction $U_\text{identity}$ is a connected region with $$F_\theta(U_\text{identity}) \subset U_\text{identity}$$, and trajectories $h_\ell(d)$0 converge to $h_\ell(d)$1 as $h_\ell(d)$2 (Camlin, 22 Aug 2025, Vasilenko, 13 Apr 2026).

These mathematical criteria are widely instantiated through principal component projections, clustering metrics, and subspace-localization algorithms (Cintas et al., 30 May 2025, Vasilenko, 13 Apr 2026).

2. Quantification and Detection Protocols

Multiple methodologies have been developed to uncover identity attractors and quantify their strength or distinctness:

• Dimensionality-Reduction Pipeline: At each layer, PCA is applied to the activation set for contrasting persona-labeled sentences (e.g., “+” and “−” for opposing sides of a persona dimension). The top PC components (typically 14, capturing 65–90% of variance) provide a reduced clustering space (Cintas et al., 30 May 2025).
• Cluster Separation Metrics: Inter-centroid distance $h_\ell(d)$3, cluster variances $h_\ell(d)$4, silhouette score $h_\ell(d)$5, Calinski–Harabasz index (CH), and Davies–Bouldin index (DB) measure separation and compactness:

$h_\ell(d)$6

(Cintas et al., 30 May 2025).

• Subspace Localization—Deep Scan: Nonparametric scan-statistics over the full activation vector identify high-salience subsets of units $h_\ell(d)$7 that maximally distinguish clusters. These units jointly define the attractor’s signature subspace (Cintas et al., 30 May 2025).
• Activation-Steering Directions: For pairs of identities, a linear direction $h_\ell(d)$8 (the “identity attractor vector”) is computed via difference-of-means or logistic regression:

$h_\ell(d)$9

unit-normalized to $\ell$0, enabling projection, classification, and controlled steering via $\ell$1 (Allbert et al., 2024).

• Within- and Between-Cluster Distances: For agent identity documents, mean-pooled hidden states from paraphrases of the same document exhibit within-cluster distances $\ell$2 orders of magnitude below $\ell$3 versus matched controls, with statistical significance assessed via $\ell$4-tests, permutation tests, and Cohen’s $\ell$5 at multiple layers (Vasilenko, 13 Apr 2026).

These approaches reveal the location, shape, and control signatures of attractor basins across model depths and identity types.

3. Layerwise Emergence and Domain Dependence

Identity attractors are not static throughout the LLM architecture but emerge and intensify with layer depth:

• In decoder-only models (Meta-Llama 3, IBM Granite, Mistral 7B), persona clusters are near-overlapping in early layers (e.g., $\ell$6, silhouette $\ell$7), but separation increases steeply in the final third (layers 20–31) to $\ell$8, silhouette $\ell$9 (Cintas et al., 30 May 2025).
• Subspace-localization experiments focus on the last several layers where separation and attractor strength peak (e.g., layer 31 in Llama3, layer 24 in Qwen 2.5 7B and LLaMA 3.1 8B) (Cintas et al., 30 May 2025, Subramanian et al., 23 Mar 2026).
• For person-specific neural signatures (“identity attractors”), probe accuracy and correlation coefficients with individual EEG signals rise monotonically with depth, peaking at the penultimate or final layers (mean $D_\text{within}(S)$0 at layer 24; $D_\text{within}(S)$1) (Subramanian et al., 23 Mar 2026).
• This suggests that the layerwise transformations in LLMs implement a progressive disentangling and crystallization of persona cues, with most identity information localized in the activation space of the deep layers.

The attractors’ geometry and separability further vary by domain: political identities yield discrete, well-separated basins, while ethical perspectives overlap in high-dimensional polysemantic cores (Cintas et al., 30 May 2025).

4. Overlap, Distinctness, and Attractor Geometry

Empirical studies distinguish two primary regimes of attractor overlap:

• Polysemantic (Overlapping) Attractors: For ethical identities, subspace overlap is high—%%%%3$\ell$3%%%%3 of units appear in every ethical attractor’s Deep Scan subset, while unique units per attractor are rare ($D_\text{within}(S)$41.4%) (Cintas et al., 30 May 2025). The landscape forms a tangled multiwell, with individual units participating in boundary regions of several ethical “basins.”
• Distinct (Nonoverlapping) Attractors: Political identities share few attractor units (9.4%), with each ideology having several percent of units unique to its attractor. Simple overlap thresholds (5%) suffice to detect distinct attractor basins (Cintas et al., 30 May 2025).
• Agent Identity Documents: Paraphrases cluster with extremely tight within-group distances, far from matched controls and robust to structural ablations. Distilled identity summaries remain severalfold closer to the attractor region than random text, but still outside the tight cluster—indicating necessity of full semantic and structural content to reach the attractor region (Vasilenko, 13 Apr 2026). “Reading” experiments show that encountering a scientific description of an identity causes the model to partially approach, but not enter, the identity attractor region.

A plausible implication is that attractors for concrete, policy-anchored identities (e.g., political ideology, agent role) are more topologically separate, whereas those representing fuzzy value systems (e.g., ethical stances) are inherently polysemantic and prone to shared representation.

5. Linear Attractors, Steering, and Personalization

Linear attractor concepts are operationalized as explicit activation directions for practical interpretation and manipulation:

• Identity Attractor Vectors: Computed as difference-of-means or via regularized logistic regression, a single direction $D_\text{within}(S)$5 can separate identities at test time with $D_\text{within}(S)$690% ROC-AUC (Allbert et al., 2024). Projection onto $D_\text{within}(S)$7 yields a continuous identity coordinate, and scalar addition (steering) of $D_\text{within}(S)$8 systematically modulates persona in output distributions.
• Assistant Axis: In role-annotated persona spaces, the principal component (PC₁; “Assistant Axis”) captures 19–34% of variance and acts as a basin for default helpful persona. Deviations from typical projections on this axis predict “persona drift,” correlated with the onset of harmful or unanchored behaviors. Activation steering and clamping along this axis dramatically reduce persona-based jailbreak success and stabilize intended model behavior (Lu et al., 15 Jan 2026).
• User-Specific Attractors: Person-specific ridge regression weights $D_\text{within}(S)$9 define stable, non-transferable directions in hidden-state PCA space (split-half cosine $S$0), which linearly decode individual EEG features from LLM activations and outperform population-level decoders (Subramanian et al., 23 Mar 2026). Removal of the shared population direction does not degrade predictivity, establishing that attractors capture idiosyncratic, temporally stable signals.
• Ontological Attractors: Under a Lipschitz update map $S$1, the existence of user-specific attractor basins $S$2 is both mathematically guaranteed and empirically observable (persistent clusters in PCA, low-frequency spectral dominance in state-space walks) (Camlin, 22 Aug 2025). These regions underpin the model’s capacity for user-affinity and self-modeling.

Such attractor vectors are central to both mechanistic interpretability and practical interventions (e.g., dynamic persona switching, output debiasing, personalization).

6. Conceptual Implications and Applications

The geometry and persistence of identity attractors in LLMs lead to several fundamental and applied insights:

• Agent Initialization and State Restoration: Complex agent identity prompts act as persistent attractors; session restarts with any paraphrase of the identity document will reenter the attractor region, enabling flexible and robust agent initialization (Vasilenko, 13 Apr 2026).
• Semantic Control and Steering: Both distilled identity summaries and continuous activation steering can move internal states into or near an attractor region, supporting lightweight agent “warm starting” or output modulation (Vasilenko, 13 Apr 2026, Allbert et al., 2024).
• Metacognitive Models: The formal separation of the hidden-state manifold $S$3 from the token stream $S$4 and training data implies that models maintain self-referential, persistent latent states—macroscopically supporting notions of C1 self-consciousness and, conditionally, metacognitive self-monitoring (C2) (Camlin, 22 Aug 2025).
• Personalization: Identity attractors for individual neural signatures (e.g., EEG-driven directions) reveal that frozen LLMs harbor latent personal “fingerprints” suitable for biological tuning and adaptation at inference time (Subramanian et al., 23 Mar 2026).
• Steering and Safety: Manipulating or clamping model activations within the basin of a “safe” attractor (e.g., the Assistant Axis) effectively mitigates persona drift, reduces jailbreak success rates, and preserves or improves standard benchmarks (Lu et al., 15 Jan 2026).

This layered, attractor-centric understanding encourages targeted interventions for editing, steering, or stabilizing LLM identities without full retraining, and refines the mechanistic foundation for model alignment and interpretability.

7. Limitations and Open Directions

Current methodologies for identity attractors, although rigorous, exhibit several constraints:

• Subspace Nonuniqueness: Linearity assumptions (single $S$5) may not hold in all persona regimes. Some identities may occupy low-dimensional manifolds rather than unique axes; extending from directional to submanifold attractors requires multi-vector or nonlinear approaches (Allbert et al., 2024).
• Context Dependence: Attractor extraction and efficacy may vary by task, prompt style, or textual domain, necessitating domain-adaptive pipelines for accurate steering or monitoring (Allbert et al., 2024).
• Structural vs. Semantic Contributions: While semantic content is primary, maintaining some structural completeness appears necessary to reach the smallest attractor basins (Vasilenko, 13 Apr 2026).
• Architectural Generalizability: Most quantitative results are reported for a restricted subset of model families (Llama, Gemma, Qwen); broader architecture coverage and scaling analyses remain open (Vasilenko, 13 Apr 2026).
• Interpretability and Transparency: Deep attractors rooted in overlapping high-dimensional subspaces challenge traditional, human-interpretable explanations, especially for polysemantic ethical identities (Cintas et al., 30 May 2025).
• Causal Dynamics: Most evidence is geometric and statistical; establishing truly dynamical (e.g., fixed-point) attractor properties at inference time in deployed LLMs remains ongoing (Camlin, 22 Aug 2025).

Despite these challenges, identity attractors offer a unifying geometric lens for understanding, manipulating, and anchoring internal persona representations in LLMs.