As Language Models Scale, Low-order Linear Depth Dynamics Emerge

Abstract: LLMs are often viewed as high-dimensional nonlinear systems and treated as black boxes. Here, we show that transformer depth dynamics admit accurate low-order linear surrogates within context. Across tasks including toxicity, irony, hate speech and sentiment, a 32-dimensional linear surrogate reproduces the layerwise sensitivity profile of GPT-2-large with near-perfect agreement, capturing how the final output shifts under additive injections at each layer. We then uncover a surprising scaling principle: for a fixed-order linear surrogate, agreement with the full model improves monotonically with model size across the GPT-2 family. This linear surrogate also enables principled multi-layer interventions that require less energy than standard heuristic schedules when applied to the full model. Together, our results reveal that as LLMs scale, low-order linear depth dynamics emerge within contexts, offering a systems-theoretic foundation for analyzing and controlling them.

• The paper develops a 32-dimensional, prompt-conditioned linear surrogate that predicts transformer activation-steering gain profiles with 0.99–1.00 Spearman correlation across ten GPT-2-large classification tasks.
• The paper finds that surrogate identifiability improves with model scale, with mean Spearman agreement rising from about 0.77 in GPT-2 to 0.995 in GPT-2-large at the same reduced order.
• The paper derives minimum-energy multi-layer steering schedules that transfer effectively to full nonlinear models, typically using two to five times less energy than uniform intervention and substantially less than last-layer-only control.

Overview

This paper, "As LLMs Scale, Low-order Linear Depth Dynamics Emerge" (2603.12541), develops a systems-theoretic account of transformer depth dynamics and reports an empirical regularity with direct consequences for activation steering. The authors—Nettasinghe and Joseph—treat transformer depth as discrete time and the last-token hidden state as the system state, then identify a prompt-conditioned reduced-order linear surrogate (the Linear Layer Variant, LLV) of how additive perturbations propagate through subsequent blocks to a final concept readout. Two claims carry the paper: first, a 32-dimensional linear surrogate reproduces full layerwise steering-gain profiles of GPT-2-large with Spearman correlations of 0.99–1.00 across ten binary NLP tasks; second, at fixed reduced order, identifiability improves monotonically with model size across the GPT-2 family, with mean Spearman agreement rising from roughly 0.77 (GPT-2) to 0.81 (GPT-2-medium) to 0.995 (GPT-2-large). The surrogate also yields closed-form minimum-energy multi-layer intervention schedules that outperform heuristic baselines when validated in the full nonlinear model.

A local systems framework for depth dynamics

The framework begins by defining the state $x_\ell(p) = h_\ell(p)[t(p),:] \in \mathbb{R}^H$ as the last non-padding token's hidden representation at depth $\ell$. For a fixed operating prompt, all non-last-token rows are frozen, inducing a prompt-conditioned frozen-context map $x_{\ell+1} = f_\ell(x_\ell; p)$. Linearizing this map around the unperturbed operating trajectory yields a local Jacobian $A_\ell(p)$, computed via Jacobian–vector products or central finite differences. Steering is modeled as additive actuation along a per-layer concept direction $v_\ell$, estimated from class-conditional mean differences on a separate concept split.

To obtain a tractable surrogate, the dynamics are projected onto a concept-anchored basis $P_\ell \in \mathbb{R}^{H \times d}$ whose first column is exactly $v_\ell$ and whose remaining columns form a reachability-informed Krylov complement built from mean Jacobian actions propagated forward through depth. This yields the reduced model

$$r_{\ell+1} \approx \bar{A}_\ell(p)\, r_\ell + \bar{B}_\ell(p)\, u_\ell,$$

with $\bar{A}_\ell = P_{\ell+1}^\top A_\ell P_\ell$ and $\bar{B}_\ell = P_{\ell+1}^\top A_\ell v_\ell$, at reduced order $\ell$0 in all main experiments. Two quantities follow immediately: the predicted layerwise gain curve $\ell$1, where $\ell$2 is the reduced transition product and $\ell$3 is the final concept readout; and, because the stacked output shift is linear in the control vector, the minimum-norm multi-layer control achieving a target shift $\ell$4,

$\ell$5

where $\ell$6 is the vector of predicted single-layer gains.

Three disjoint balanced splits per dataset—concept (400/class), operating (200/class), and held-out evaluation (200/class)—ensure that predicted gains are never scored on prompts used for identification. The task suite spans sentiment (Amazon Polarity, Yelp Polarity, SST-2, IMDB), question answering (BoolQ), entailment (binary GLUE MNLI), toxicity (Civil Comments), and social-media classification (TweetEval irony, hate, offensive).

Gain-profile prediction across depth and tasks

The primary identifiability test compares the surrogate-predicted gain profile against empirical gains measured in the full transformer by adding $\ell$7 before block $\ell$8 and taking central finite differences of the final concept score on held-out prompts. On GPT-2-large at $\ell$9 and $x_{\ell+1} = f_\ell(x_\ell; p)$0, predicted and empirical curves track each other closely across all ten datasets, with Spearman correlations of 0.99 or 1.00 in every displayed case. Crucially, the surrogate captures the full shape of the depth response—not merely the identity of the best layer—including smooth late-depth ramps, broad middle-to-late plateaus, and non-monotone gain landscapes.

Two methodological choices deserve note. First, optimal intervention depth is clearly task-dependent, which argues directly against universal heuristics such as always steering at the final block. Second, the authors treat Pearson and Spearman agreement as primary metrics and top-$x_{\ell+1} = f_\ell(x_\ell; p)$1 layer overlap only as a secondary diagnostic, because broad high-gain plateaus make exact set overlap unstable even when the entire profile is predicted accurately—an honest concession about metric sensitivity. Robustness sweeps over $x_{\ell+1} = f_\ell(x_\ell; p)$2 show high agreement throughout, indicating operation in a stable local-response regime rather than at a finely tuned perturbation scale. Replacing a random basis complement with the Krylov construction systematically improves prediction, consistent with steering effects concentrating in a low-dimensional reachable subspace.

Scaling of identifiability

The paper's most striking claim is that identifiability itself improves with scale. Under an identical pipeline, fixed reduced order $x_{\ell+1} = f_\ell(x_\ell; p)$3, and identical evaluation magnitude, aggregate agreement increases monotonically from GPT-2 to GPT-2-medium to GPT-2-large:

Model	Mean Spearman	Mean Pearson
GPT-2	~0.77	~0.68
GPT-2-medium	~0.81	~0.74
GPT-2-large	~0.995	~0.997

By GPT-2-large, the low-order surrogate nearly saturates the gain-prediction task. The authors emphasize that this is stronger than the statement that larger models are easier to steer: it asserts that larger models are more cleanly identifiable through a compact local dynamical abstraction under the same reduced-order budget. The implication is that scale may improve not only capability but also the fidelity of compact mechanistic descriptions—a claim that runs counter to the common framing of scale as increasing opacity.

The authors offer a plausible mechanism—width and representational redundancy stabilizing local Jacobian responses and reducing estimation variance in projected dynamics—but state plainly that their results do not prove it. They also concede that the monotone trend is strongest at the aggregate level rather than uniformly task-by-task: some datasets are already predictable in smaller models while others improve sharply only at larger scale. Whether this regularity extends beyond the GPT-2 family remains an open question, and the paper does not test it.

Minimal-energy multi-layer control

Because the reduced output shift is linear in the stacked control sequence, the minimum-energy direction is available in closed form and proportional to the predicted gain vector. The policy is evaluated in the full transformer by scaling with a single amplitude found via bisection for targets $x_{\ell+1} = f_\ell(x_\ell; p)$4, with energy measured as squared $x_{\ell+1} = f_\ell(x_\ell; p)$5 norm of injection coefficients. Across all datasets and target shifts on GPT-2-large, the LLV-optimal schedule is consistently lowest- or tied-lowest-energy. Uniform all-layer injection—the strongest heuristic baseline—typically requires two to five times more energy; last-layer-only interventions are frequently an order of magnitude worse, and early-layer or random single-layer policies worse by one to several orders of magnitude.

The qualitative pattern is interpretable through the gain curves: when gain concentrates in a narrow depth band, the optimum focuses actuation there; when gain spreads across a plateau, the optimum distributes control over multiple layers. Uniform-all partially hedges but wastes effort on layers the surrogate correctly identifies as inefficient. This closes the loop from prediction to action: the same identified depth response that predicts sensitivity also determines how control should be distributed, and the resulting policy transfers back to the full nonlinear model.

Limitations and open questions

The paper is explicit that the surrogate is deliberately local: it is identified around prompt-conditioned operating trajectories and is not a global law of transformer computation. All headline results concern the GPT-2 family; the scaling regularity is established within one architecture lineage at three sizes, and its extension to other families, training regimes, and much larger scales is untested. The proposed mechanism for improved identifiability—Jacobian stabilization and better alignment of concept directions with dominant reachable modes—is offered as intuition, not demonstrated causally. Evaluation is restricted to binary classification tasks with concept-only actuation along mean-difference directions, leaving open whether multi-concept or generative settings admit comparably faithful low-order surrogates. Finally, the scaling analysis uses three random seeds, and per-task trends vary, so finer-grained statistical characterization of the scaling law remains open.

Conclusion

This paper reframes activation-based intervention as a consequence of an identifiable low-order linear structure in transformer depth dynamics rather than a heuristic search over layers. Its quantitative results are strong: near-perfect gain-profile prediction on GPT-2-large at 32 dimensions, monotone improvement of identifiability with scale, and two-to-five-fold energy reductions over the best heuristic schedules. The central open question the work poses is whether increasing scale systematically improves the quality of compact mechanistic abstractions across model families—if so, identifiability may serve as a systems-level quantity for comparing architectures and intervention strategies.