How are linear representations learned? Exact solutions to the dynamics of abstraction

Abstract: In artificial and biological neural networks, concepts are often encoded as consistent linear directions in representation space. In deep learning, this idea is known as the linear representation hypothesis and underpins many interpretability and control methods based on linear probes, from concept detection to activation steering. Yet while prior work has studied whether such directions should exist $\textit{after}$ training, the dynamics of how they emerge $\textit{during}$ training remain poorly understood. Here, we develop a framework to study the alignment of concept directions during training - a process we call "abstraction". In a minimal linear network setting, we obtain exact solutions for the full trajectory of abstraction. These solutions reveal key analytic principles governing abstraction: (i) data and target geometry jointly determine abstraction at the end-of-learning, (ii) abstraction improves with network depth, and (iii) initialization scale controls the maximum abstraction reached during training. Extending our theory to nonlinear networks, we analyze how the choice of nonlinearity affects abstraction dynamics: erf networks approximate the linear theory, while abstraction in ReLU networks depends less on target geometry and more on input geometry. Across both, we prove a striking attenuation law: both nonlinearities weaken abstraction in activations relative to preactivations. We find evidence for this law in open models (DINOv3, Gemma 4) and apply our theory to improve linear probe generalization in LLMs. Together, our results provide a dynamical theory of abstraction with implications for interpretability and control.

• The paper provides rigorous exact solutions for abstraction dynamics using a minimal two-layer linear model, revealing transient overshoots and metastable plateaus.
• It extends the analysis to deep and nonlinear architectures, demonstrating how network depth and weight initialization shape abstraction and probe generalization.
• Empirical validations across convolutional nets, transformers, and primate data confirm the theoretical predictions, emphasizing practical implications for interpretability.

Exact Solutions to Abstraction Dynamics in Linear and Nonlinear Networks

The paper "How are linear representations learned? Exact solutions to the dynamics of abstraction" (2607.08843) provides a rigorous analytical and empirical investigation of the emergence and dynamics of abstract, linearly-aligned concept directions in artificial and biological neural networks. The authors develop exact solutions for the learning trajectory of abstraction in minimal linear models, extend the analysis to deep and nonlinear network settings, and validate theoretical predictions across diverse architectures and empirical domains, including deep convolutional networks, vision transformers, LLMs, and primate neural data.

---

Defining Abstraction and Theoretical Framework

Abstraction is quantified as the alignment (cosine similarity) between context-specific concept vectors, e.g., the "shape" direction in different color contexts. The core quantity of interest is the abstraction score $\alpha$, which equals 1 for perfect abstraction (parallel vectors). The theoretical motivation is two-fold: such linear representations underlie many interpretability/control methods in AI (e.g., linear probes, activation steering) and are closely related to cross-contextual "abstract" representations observed in higher-level brain regions.

The authors introduce a minimal, but analytically tractable, model: a two-layer linear network trained to minimize MSE with a projected (optimized) readout. The analysis assumes two latent binary concepts (shape, color) yielding a $2 \times 2$ design and leverages a two-factor symmetry (2FS) that permits simultaneous diagonalization of all relevant kernels (input, target, and feature).

---

Exact Dynamics and Terminal Laws in Linear Networks

Main Theoretical Results:

• The gradient flow dynamics reduce, under 2FS, to independent ODEs for each eigenmode (signal, noise, and others), permitting an exact parametric solution for the abstraction trajectory $\alpha(t)$.
• Terminal Abstraction Law (Theorem 1): The asymptotic abstraction attained after initialization is

$$\alpha_\infty = \frac{1 - \sqrt{\nu(\Sigma_x)\nu(\Sigma_y)}}{1 + \sqrt{\nu(\Sigma_x)\nu(\Sigma_y)}}$$

where $\nu(\cdot)$ is the eigenmode ratio (interaction/"noise" to signal), and the product involves both input and target geometry. Perfect abstraction ($\alpha_\infty = 1$) corresponds to noiseless inputs or targets.

• Initialization-Dependent Maximum: The maximum abstraction achieved during training (even if transient) can substantially overshoot $\alpha_\infty$, especially for small initializations ("rich" regime), leading to metastable plateaus of nearly perfect abstraction.
• Depth Effects: In deep linear networks under a balanced scaling regime, abstraction interpolates linearly (in $\arctanh$ space) from the data to the target abstraction across layers. Terminal abstraction increases monotonically with depth, provided the target is more abstract than the input.

  Figure 1: Analytic and numerical trajectories of abstraction in the minimal model; dependence on initialization, depth, and asymptotic behavior.

The correspondence between abstraction and probe generalization error is formalized by showing that transfer of a linear probe across contexts depends monotonically on $\alpha$ (Proposition 1).

---

Empirical Validation Beyond Minimal Models

The derived theoretical predictions are robustly supported in empirical settings where assumptions are grossly violated (e.g., convolutional ResNets on 3dshapes tasks). The observed maximum abstraction decreases systematically with weight initialization scale. Moreover, as predicted, final abstraction increases with network depth, and deeper layers within a network exhibit greater abstraction.

Figure 2: Empirical validation in ResNets trained on 3dshapes; abstraction increases with depth and decreases with initialization scale.

---

Extensions: Nonlinear Dynamics and the Attenuation Law

The theory is extended to infinite-width nonlinear networks (erf, ReLU, Leaky ReLU). The key structural result is that large-width nonlinearities maintain the 2FS structure for both preactivations and features, enabling an explicit kernel map analysis.

• For erf nonlinearities with weak regularization, abstraction dynamics and terminal values are quantitatively close to the linear case (Figure 4A,B, Proposition 2).
• For ReLU/Leaky ReLU, abstraction becomes fundamentally more sensitive to input geometry than target geometry. Notably, even when targets are noiseless, perfect abstraction is attained only when the input is sufficiently "clean" (low-interaction-noise), in sharp contrast to the linear regime (Figure 4C,D, Proposition 3).
• Attenuation Law (Theorem 3): For both erf and Leaky ReLU, abstraction in features can never exceed abstraction in preactivations, i.e., nonlinearities strictly attenuate abstraction:

$$|\alpha_{\mathbf{K}}(t)| \leq |\alpha_{\mathbf{Q}}(t)|$$

The attenuation factor is fully characterized in terms of the NNGP/NTK kernel maps and context correlations.

Figure 3: Phase diagrams showing the impact of nonlinearities on terminal abstraction. ReLU transitions exhibit sharp dependence on input SNR.

---

Applications: Improving Abstractness in Frontier Models

1. Transformer Models: Applying a "local GELU ablation" (replacing GELU activations with identity locally in the MLP of a particular layer) increases abstraction and linear probe generalization in both vision transformers (DINOv3) and LLMs (Gemma 4). Across layers and concepts, abstraction scores post-ablation are consistently higher (Figure 4), supporting the attenuation law's practical implication that some nonlinearities can be bypassed to extract more universally generalizable directions.

   Figure 4: Local GELU ablation increases abstraction and probe generalization in DINOv3 and Gemma 4 models.

2. Neuroscience: Analyses on macaque ventral-stream data reveal abstraction increases along the V4 → IT hierarchy, consistent with depth interpolation predictions (Theorem 2), and higher abstraction correlates with better generalization of linear cross-condition probes (Figure 5).

   Figure 5: Abstraction is higher in macaque IT compared to V4, yielding superior cross-context probe transfer.

---

Theoretical and Practical Implications

Theoretical implications:

• The framework generalizes the linear representation hypothesis, linking dynamics of abstraction directly to data and target geometry.
• Network initialization and depth jointly shape the achievable and observed abstraction, with rich-to-lazy dynamics naturally explaining non-monotonic abstraction trajectories.
• Nonlinearities introduce a fundamental upper bound on abstraction in features, constraining the effectiveness of linear probes for interpretability.

Practical implications:

• Reliable extraction of steerable or interpretable directions should operate on the most "linear" representations, potentially requiring explicit removal or bypassing of nonlinearities during analysis or deployment.
• Model scaling via depth enhances the capacity for abstraction transfer, particularly when downstream target structure is abstract.

Future directions include relaxing the symmetry hypotheses, extending analytical tools to deep nonlinear networks, and leveraging the derived laws for more robust and generalizable interpretability/control procedures in state-of-the-art models.

---

Conclusion

This work delivers a comprehensive dynamical theory of abstraction in supervised and self-supervised neural networks, with analytical control in both linear and nonlinear regimes, and strong cross-domain empirical grounding. The explicit characterization of abstraction dynamics provides a bridge between mechanistic interpretability, practical probe reliability, and the computational principles underlying abstract representation in both artificial and biological systems.