From Directions to Regions: Decomposing Activations in Language Models via Local Geometry

Abstract: Activation decomposition methods in LLMs are tightly coupled to geometric assumptions on how concepts are realized in activation space. Existing approaches search for individual global directions, implicitly assuming linear separability, which overlooks concepts with nonlinear or multi-dimensional structure. In this work, we leverage Mixture of Factor Analyzers (MFA) as a scalable, unsupervised alternative that models the activation space as a collection of Gaussian regions with their local covariance structure. MFA decomposes activations into two compositional geometric objects: the region's centroid in activation space, and the local variation from the centroid. We train large-scale MFAs for Llama-3.1-8B and Gemma-2-2B, and show they capture complex, nonlinear structures in activation space. Moreover, evaluations on localization and steering benchmarks show that MFA outperforms unsupervised baselines, is competitive with supervised localization methods, and often achieves stronger steering performance than sparse autoencoders. Together, our findings position local geometry, expressed through subspaces, as a promising unit of analysis for scalable concept discovery and model control, accounting for complex structures that isolated directions fail to capture.

• The paper introduces a novel region-based decomposition that models LM activations as Gaussian regions, emphasizing local covariance structure over traditional global directions.
• It employs Mixtures of Factor Analyzers to partition high-dimensional activations into low-dimensional regions, enhancing interpretability and enabling targeted interventions.
• Empirical evaluations demonstrate significant improvements in localization and causal steering, outperforming sparse autoencoders in both qualitative and quantitative benchmarks.

Decomposing LM Activations: A Local-Geometry Perspective

The paper "From Directions to Regions: Decomposing Activations in LLMs via Local Geometry" (2602.02464) advances activation analysis in LMs by shifting from global direction-based decompositions to region-based, low-dimensional modeling grounded in local geometry. Through large-scale unsupervised training and comprehensive empirical evaluation, the authors demonstrate that parcels of activation space encoded as Gaussian regions with local covariance structure more accurately reflect the nonlinear and high-dimensional representation of concepts in large LMs.

Motivations and Limitations of Global Direction Analysis

Existing interpretability and feature discovery approaches in LMs primarily rely on finding global directions—linear axes in activation space—corresponding to interpretable or causally relevant features. These methods, including sparse autoencoders (SAEs) and related dictionary-learning techniques, implicitly assume that salient features are linearly separable and adequately captured by a global, sparse basis. Empirically, this assumption fails for concepts with complex, nonlinear, or multi-dimensional structure: such features appear diffusely across many global directions, with little built-in geometric linkage among related axes. Recovering such distributed concepts with global direction methods typically requires post hoc heuristics or manual feature selection, limiting scalability and robustness.

Furthermore, the local manifold hypothesis—supported by recent probing results—suggests that activation clusters corresponding to semantically related contexts have much lower intrinsic dimension than global space, and that these clusters exhibit structure-specific, context-sensitive variation unaccounted for by global methods. Thus, a region-based analytic framework is necessitated to accommodate the true geometry of neural representations in LMs.

Mixture of Factor Analyzers for Activation Decomposition

The authors propose modeling LM activations using Mixtures of Factor Analyzers (MFA), a classical latent variable model that partitions the activation space into Gaussian regions, each associated with its own centroid and local low-dimensional subspace. Within each region, structured variation is captured by a low-rank factor analysis model, providing both an absolute region anchor (the centroid) and directions of local variation (the factor loading subspace). This enables a compositional decomposition: for each activation, assignment to a region via posterior responsibilities, and decomposition into a centroid contribution and local subspace offset.

Key algorithmic details:

• MFA components and ranks are chosen to conservatively approximate local intrinsic dimension while avoiding ill-conditioned covariance estimation.
• Initialization is performed via K-means clustering, which empirically provides more diverse and well-segregated centroids than random selection.
• All parameters (means, loadings, covariances, mixing weights) are learned by maximizing the log marginal likelihood via gradient descent across large batches of residual activations sampled from model layers.

The decomposition allows for both localization—assigning an activation to a semantic region—and steering—manipulating generation by interventions toward centroids or along local axes.

Structural Findings in Activation Space

Through large-scale MFA training (up to 32K components) on Llama-3.1-8B and Gemma-2-2B across multiple layers, the analysis reveals two primary classes of Gaussian regions:

• Narrow Gaussians: Constrain tightly to lexical patterns or specific tokens; variation is predominantly syntactic.
• Broad Gaussians: Span thematically coherent concepts; variation is primarily semantic.

As the number of components increases, MFA produces more narrow Gaussians and a higher proportion of semantic local directions, especially pronounced in Gemma-2-2B. Neighboring components often correspond to related semantics, forming constellations over complex concepts, implying that multifaceted themes are realized not as single components but as neighborhoods of locally linear regions.

Comparative Evaluation: MFA vs. Sparse Autoencoders

A rigorous side-by-side comparison with state-of-the-art SAE dictionary learning (Llamascope/Gemmascope) demonstrates several advantages of MFA:

• Decomposition in MFA is highly interpretable: assignments to region centroids and within-region axes are semantically meaningful for the vast majority of activations (average interpretability fraction 0.96 ± 0.2), whereas only a minority of SAE component activations are directly interpretable from local context (0.29 ± 0.2).
• Reconstructions in MFA proceed in two natural stages—locating the activation at a centroid (region) followed by refinement via local subspace—contrasted with SAE's incremental assembly from multiple global directions.
• MFA supports clean, targeted interventions; centroid interpolations induce broad concept shifts, while local axes provide fine-grained, context-sensitive control, covering both coarse and granular feature manipulation.

Strong Numerical Results on Localization and Steering

Quantitative benchmarks substantiate these qualitative findings. On localization tasks (RAVEL, MCQA from MIB suite), MFA:

• Strongly outperforms unsupervised baselines (PCA, SAEs) by 3–16 points.
• Exceeds supervised desiderata-based masking in 5 of 8 settings.
• Is competitive with, or outperforms, state-of-the-art supervised localization (DAS) on multiple tasks.

Disaggregation indicates that coarse-grained variables (e.g., continent) are accounted for at the centroid (region) level, while finer variables (e.g., pointer positions) require local axis variation.

For causal steering, when manipulating activations during inference, MFA-based interventions using centroids achieve:

• Significantly higher conceptual alignment and coherence compared to SAE and supervised DiffMeans, with approximately 2× gain on Gemma-2-2B and ~33% improvement on Llama-3.1-8B.
• No degradation in fluency compared to baselines, and no systematic benefit conferred by further increasing the number of MFA components.

These results highlight local region-based geometry—rather than global sparse dictionaries—as an effective and scalable analytic and control mechanism.

Implications and Future Directions

This work substantiates a paradigm shift toward region-based and subspace-based interpretability in LMs. Theoretically, it aligns with geometrical characterizations of representation learning (e.g., manifold and local linear hypotheses) and empirically accounts for the emergence of multi-dimensional, context-dependent concepts. Practically, it enables more interpretable, scalable, and causally robust analysis and steering, with immediate applications to model auditing, concept localization, and behavior editing.

Key open questions and directions for future research include:

• Incorporating label or task-supervised information into the MFA framework for more targeted or hierarchical decomposition.
• Adaptive selection of latent rank per component to better characterize intrinsic dimension variations.
• Refinement of initialization or annealing strategies to avoid suboptimal local minima at large scale.
• Deeper characterization of the relationship between region geometry and non-local emergent properties such as compositionality, abstraction, or model vulnerability.

Conclusion

By decomposing LM activations according to local geometry via Mixture of Factor Analyzers, this paper demonstrates a highly interpretable, scalable, and effective method for understanding and controlling large neural models. Empirical results across diverse benchmarks indicate clear superiority over both unsupervised and supervised baselines. The local-geometry-based view has impactful theoretical and practical implications, positioning region-and subspace-based approaches as foundational for the next generation of mechanistic interpretability and model control in deep learning (2602.02464).