Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior

Abstract: Neural representations carry rich geometric structure; but does that structure causally shape behavior? To address this question, we intervene along paths through activation space defined by different geometries, and measure the behavioral trajectories they induce. In particular, we test whether interventions that respect the geometry of activation space will yield behaviors close to those the model exhibits naturally. Concretely, we first fit an activation manifold $M_h$ to representations and a behavior manifold $M_y$ to output probability distributions. We then test the link $M_h \leftrightarrow M_y$ via interventions: we find that steering along $M_h$, which we term manifold steering, yields behavioral trajectories that follow $M_y$, while linear steering -- which assumes a Euclidean geometry -- cuts through off-manifold regions and hence produces unnatural outputs. Moreover, optimizing interventions in activation space to produce paths along $M_y$ recovers activation trajectories that trace the curvature of $M_h$. We demonstrate this bidirectional relationship between the geometry of representation and behavior across tasks and modalities. In LLMs, we use reasoning tasks with cyclic and sequential geometries as well as in-context learning tasks with more complex graph geometries. In a video world model, we use a task with geometry corresponding to physical dynamics. Overall, our work shows that geometry in neural representation is not merely incidental, but is in fact the proper object for enabling principled control via intervention on internals. This recasts the core problem of steering from finding the right direction to finding the right geometry.

• The paper demonstrates that manifold steering along fitted activation manifolds produces smooth, ordered probability transitions compared to erratic outputs from linear steering.
• The authors employ PCA-based manifold fitting and spline interpolation to reveal a near-isometric relationship between neural representations and output behavior.
• Their framework has significant implications for controllability, interpretability, and safe behavioral modulation across structured and complex visual tasks.

Manifold Steering and the Shared Geometry of Neural Representation and Behavior

Introduction and Motivation

This paper ["Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior" (2605.05115)] systematically analyzes the geometric structure of neural network activation spaces and its causal relationship to model output behavior. The central claim is that the intrinsic geometry of neural activation manifolds is tightly coupled to behavioral manifolds in probabilistic output space. The authors contend that interventions respecting this geometric coupling—specifically, "manifold steering" along fitted activation manifolds—yield natural behavioral transitions, while conventional linear steering methods produce unnatural, off-manifold behaviors.

Figure 1: Overview of geometric steering paradigms; only interventions following the intrinsic geometry of fitted activation or behavioral manifolds yield natural output transitions, whereas linear steering induces unnatural behavioral trajectories.

Empirical Characterization of Activation and Behavior Manifolds

The authors introduce a protocol for fitting low-dimensional manifolds both to internal representations (activation manifolds, $\mathcal{M}_h$) and to the corresponding output behavioral distributions (behavior manifolds, $\mathcal{M}_y$). The manifold fitting is performed using PCA dimensionality reduction, centroid aggregation, and spline-based interpolation, with Euclidean and Hellinger geometries for activations and output probabilities, respectively. This approach is systematically applied across Llama 3.1 8B on reasoning tasks involving cyclic (e.g., days, months) and sequential (e.g., letters, ages) conceptual domains.

Crucially, the results show that both activation and output distributions recapitulate the conceptual structure of the underlying domain: activations and outputs for days of the week lie on a cyclic manifold, and those for letters or ages on a linear manifold.

Figure 2: Cyclic conceptual tasks—activation and behavior manifolds (PCA), with isometry confirmed by high correlation in on-manifold distances and MDS embeddings.

Figure 3: Sequential conceptual tasks—activation and behavior manifolds (PCA), again exhibiting strong isometry via geodesic distance comparisons.

Strong quantitative evidence is presented for approximate isometry between $\mathcal{M}_h$ and $\mathcal{M}_y$ across all four conceptual tasks: correlation coefficients for pairwise on-manifold distances are in the range $r = 0.89$ to $0.999$, significantly higher than those for naive Euclidean (linear) interpolation in activation space.

Causal Interventions: Manifold vs Linear Steering

The authors move beyond correlational observations by executing causal interventions: interpolating between activation centroids corresponding to different concepts, either along the fitted activation manifold (manifold steering) or via straight lines in activation space (linear steering), and measuring the resulting behavioral (output) trajectories.

Figure 4: Behavioral trajectories induced by linear vs. manifold steering; only manifold steering produces smooth, ordered transitions matching human intuition and task structure.

Manifold steering yields ordered probability transitions between adjacent concepts, faithfully tracking the fitted behavior manifold. In contrast, linear steering demonstrates erratic "teleportation" behavior: probability mass jumps to non-adjacent concepts and intermediate outputs become unnatural.

Quantitatively, cumulative trajectory energy (Bhattacharyya-based energy to the behavior manifold) is significantly lower for manifold steering: for instance, on the "letters" task $E_{BC}=2.42\pm0.07$ (manifold) vs $6.95\pm0.27$ (linear), a $2.8\times$ reduction on average across tasks ($p<0.001$ in all comparisons).

Bidirectional Geometry: Pullback and Isometry

To further test the bidirectionality, the authors optimize for activation space paths whose interventions yield output trajectories closely following the behavior manifold—defining the "pullback geometry." They find that these paths recover the same curved structure as the original activation manifold, with $\mathcal{M}_y$0 values (matching between pullback and manifold paths) from $\mathcal{M}_y$1 to $\mathcal{M}_y$2 (vs baselines as low as $\mathcal{M}_y$3), again confirming the shared, nearly isometric structure.

Figure 5: Trajectories from manifold (black) and pullback (teal) strategies are nearly coincident in both activation and behavior spaces, evidencing a strong isometry.

Generalization to Multi-Dimensional and Visual Domains

The analysis extends beyond one-dimensional structured domains. In the ICLR synthetic in-context learning (ICLR) tasks with grid and cylindrical geometry, the fitted activation and behavior manifolds exhibit a two-dimensional surface structure. Factored manifold steering along activation-intrinsic coordinates (obtained via thin-plate splines) achieves independent control of each conceptual subspace—demonstrating compositionality and lack of interference between dimensions.

Figure 6: Isometry and factored control on 2D grid conceptual spaces; manifold steering enables independent modulation of each grid axis, in contrast to distorted transitions under linear steering.

The technique's robustness is also validated in a recurrent visual world model (Mountain Car). Manifold steering induces smooth, interpretable transitions in perceptual output—moving a car's position naturally through visual space—whereas linear steering produces ambiguous or "blended" intermediate images.

Figure 7: In vision tasks, manifold steering yields coherent movement in the latent spatial domain, while linear steering causes ambiguous, off-manifold outputs.

A Geometric Framework for Steering Strategies

The paper formalizes steering as navigation under a Riemannian metric in activation space. The three paradigms are:

• Linear Steering: Flat (Euclidean) metric; disregards the density and structure of actual activations, producing straight-line interpolations.
• Manifold Steering: Density-adaptive metric scaling movement cost by the probability density of observed activations, making off-manifold movement costly, so geodesics align with empirical manifolds.
• Pullback Steering: Metric pulled back from behavior space, mapping the geometry of output distributions into activation space via the Jacobian of the mapping from representations to behavior.

Notably, the metrics derived from activation density and from behavior pullback yield almost identical geodesic paths—empirically demonstrating that the representation and behavior manifolds are dual images of the same underlying conceptual geometry.

Implications and Discussion

The strong, bidirectional isometry between activation and behavior manifolds empirically invalidates the assumption underlying common linear steering methods for all but degenerate cases where representation geometry is truly linear. Instead, respecting the intrinsic low-dimensional geometry of neural representations enables smooth, interpretable, and controllable behavioral transitions. This geometric coupling was also observed in synthetic in-context learning tasks and recurrent visual models, not merely in text-based, highly-structured conceptual domains.

Theoretical and Practical Implications

• Causal Interpretability: The paper demonstrates that the appropriate causal mediators for behavioral control are the intrinsic coordinates of the learned manifold, not arbitrary directions in the high-dimensional activation space. This unifies causality-grounded intervention protocols with recent insights into representation geometry.
• Control and Alignment: For behavior modulation (both for interpretability/safety and for practical modulation of model outputs), this suggests a paradigm shift: steering should proceed along fitted representation manifolds, not in arbitrary directions.
• Extension to Complex Domains: While the experiments are primarily on structured tasks, the approach sets the groundwork for disentangling and navigating conceptual spaces even in more abstract or high-level domains; extensions will rely on robust unsupervised manifold discovery and intrinsic coordinate identification.

Quantitative Statement of Contradiction

A strong, empirically-backed contradiction to tradition is asserted: the commonly employed Euclidean geometry and linear steering through activation space produces unnatural, off-manifold, and incoherent outputs; this is neither necessary nor intrinsic to activation-based interventions, but a consequence of ignoring the learned manifold structure.

Future Directions

The authors note several directions for extension:

• Application to complex, in-the-wild conceptual domains requiring unsupervised or semi-supervised manifold fitting.
• Expansion from token-level to sequence- or trajectory-level behavioral controls, necessitating more sophisticated belief- or outcome-space geometries.
• Incorporation of manifold-based steering protocols for manipulation of intermediate algorithmic variables or latent features, particularly in modular or compositional architectures.

Conclusion

This paper provides a rigorous, causal, and geometric account of the relationship between neural network internal representations and output behavior, robustly demonstrating that the correct locus for activation-based interventions is along empirically-fitted representation manifolds, not arbitrary linear trajectories. The nearly isometric relationship between activation and behavior manifolds across diverse models and domains defines both a unifying interpretability paradigm and a practical blueprint for principled behavioral steering via internal manipulation.

(2605.05115)