Evidence for feature-specific error correction in LLMs

Abstract: Understanding the features of LLMs is a central goal of interpretability. LLMs are commonly assumed to use superposition to represent more features than they have dimensions. They may not only represent features in superposition but also perform computation in superposition. Theory predicts that computing in superposition requires error correction that privileges feature directions over generic ones, but this prediction has not been tested empirically. We propose an empirical test of error correction in LLMs based on activation perturbations. Perturbing residual-stream activations, we find that they are robust to small perturbations--forming activation plateaus consistent with error correction--but less robust along candidate feature directions ("pure" directions, constructed from contrastive prompt pairs) than along mixtures of two such directions, indicating that the pure directions are privileged. We quantify this privilegedness by modeling the perturbation effect as a function of the $L^p$-norm of its decomposition into feature components. For $p=2$ the response is a quadratic form with at most as many nonzero eigenvalues as the residual-stream dimension, which cannot privilege the many feature directions superposition requires. $p>2$ lifts this constraint and is consistent with feature-specific error correction. We find $p>2$ for contrastive, MELBO, and SAE-decoder directions, and $p\approx2$ for random and PCA directions (controls). These results replicate across Gemma-2-9B, Qwen3-1.7B, Llama-3.1-8B, Mistral-7B-v0.3, Aya-Expanse-8B, and Yi-1.5-9B. We further validate our method on a toy model of error correction with known ground-truth features, recovering $p>2$ for true feature directions, degrading toward $2$ as we rotate away from them.

• The paper demonstrates that LLMs privilege feature directions in internal activations, with empirical p-values exceeding 2 indicating feature-specific error correction.
• A robust perturbation protocol applied to early-layer activations reveals directional sensitivity that decays as activations rotate away from candidate feature directions.
• Results across multiple LLM families and toy models validate theoretical predictions, advancing interpretability and controlled generation in language models.

Evidence for Feature-Specific Error Correction in LLMs

Introduction

This work presents the first direct empirical evidence for feature-specific error correction (FSEC) in LLMs, a core prediction of theories positing that LLMs both represent and compute many more features than the dimensionality of their activations via superposition. The central hypothesis is that when LLMs operate in regimes of superposition, they must correct for noise associated with feature interference by differentially privileging specific feature directions in their internal activations—a form of error correction not previously measured empirically.

Theoretical Context and Motivation

It is a prevailing yet only indirectly substantiated assumption that LLMs engage in superposition (cf. [Elhage et al., 2022]) to represent more concepts than model dimension; specifically, evidence for computation in superposition (CiS) remains largely theoretical. A necessary theoretical condition for robust CiS is the presence of feature-specific error correction: the model's dynamics should be robust to perturbations along mixtures of features (that is, non-feature directions), but sensitive along directions aligned to model features. Existing theoretical work (e.g., [Hänni et al., 2024]) posits that such mechanisms are mandatory for maintaining separability of features during computation in superposition. However, this property had yet to be directly measured in LLMs.

Empirical Framework and Methodology

The authors introduce a robust perturbation-based protocol operating on the residual stream of various LLMs. Their core empirical diagnostic is the response of the model to controlled perturbations of the early-layers’ activations. By constructing candidate feature directions (contrastive pairs, MELBO directions, and columns from sparse autoencoder [SAE] decoders) and a set of control (non-feature) directions (PCA, random, random-difference), the study quantifies how the downstream residual stream responds to perturbations in these directions, adjusting for perturbation magnitude using a norm-matched framework.

A key analytic construct is the LP-norm parametrization ($p$-norm) of the response surface measured as the angle at which the downstream model’s activations cross a plateau-breaking threshold. For $p = 2$, model sensitivity reduces to a quadratic form; importantly, only $p>2$ can realize the stringent direction-selectivity required for feature-privileging in high-superposition settings. The fitted exponent $p$ thus serves as an empirical witness to FSEC: privilege for candidate feature directions is indicated by $p > 2$.

Results

Main Findings

1. Evidence for Privileging of Feature Directions: Across six LLM families—Gemma-2-9B, Qwen3-1.7B, Llama-3.1-8B, Mistral-7B-v0.3, Aya-Expanse-8B, and Yi-1.5-9B—the superellipse exponents for contrastive, SAE-decoder, and MELBO candidate directions all satisfy $p > 2$ ($p \approx 2.2$–$2.4$), well above the baseline controls ($p \approx 2.0–2.05$ for PCA/random/random-difference). This is consistent with these directions being aligned to the putative features encoded and manipulated by the networks.
2. Directional Sensitivity as Feature Alignment: The sensitivity along these candidate directions decays monotonically toward $p = 2$ as one rotates away from them (or from ground-truth features in a toy model), supporting the interpretation that these directions are genuinely privileged by the internal computations relevant for feature-specific error correction.
3. Robustness Across Metrics and Setups: This effect is invariant across multiple ablations including residual stream layer, measurement layer, response metric (L2, cosine, KL-divergence), threshold, anchor data source, and perturbation scheme.
4. Toy Model Validation: In an idealized two-layer denoising network with known ground-truth features, the fitted exponent reaches significantly higher values ($p = 2$0), decaying toward $p = 2$1 as perturbations are rotated away from feature axes. This validates the methodology and demonstrates that imperfect direction alignment in LLM probes induces an underestimation of the true exponent.

Quantitative Summary Table

Direction Type	Mean $p = 2$2	95% CI	Median $p = 2$3	Controls $p = 2$4
Contrastive	2.42	[2.29, 2.62]	2.30	2.03–2.05
MELBO	2.21	[2.01, 2.42]	2.16	—
SAE-Decoder	2.28	[2.14, 2.47]	2.19	—
PCA/Random	2.03	[1.94, 2.12]	1.97–2.02	—

These results are consistently above the isotropic (elliptical) reference $p = 2$5 only for candidate feature directions.

Interpretation and Theoretical Implications

The data shows that LLMs consistently privilege perturbations along candidate feature directions (manifest as higher $p = 2$6), supporting the hypothesis that superposition with feature-specific error correction is actively employed. Theoretically, with $p = 2$7, a network can realize selective responsiveness to a large number of feature directions, a necessity under high-dimensional superposition. In contrast, a quadratic ($p = 2$8) response cannot realize this selectivity due to rank constraints. The empirical values of $p = 2$9 in LLMs likely underestimate the selectivity present for genuine features due to imperfect direction approximation.

The authors also discuss alternative interpretations of their findings, notably that activation plateaus and direction-dependent sensitivity could emerge from mechanisms unrelated to error correction (e.g., general robustness, geometry of the activation manifold). However, the discriminative power of the $p>2$0 exponent provides a clear tool for tracking feature alignment, independent of mechanistic subtleties.

Practical and Future Implications

Practically, this work advances the interpretability of LLMs by providing a direct test for feature alignment—directions exhibiting $p>2$1 are more likely to correspond to meaningful features rather than arbitrary bases. This test can inform the construction of more interpretable and steerable LLMs, e.g., for controlled generation or model editing.

Future research directions include the search for unsupervised feature directions maximizing the exponent $p>2$2 itself, investigation into deeper and more realistic toy models matching measured exponents, and fuller integration with monosemanticity extraction methods (e.g., large-scale SAEs).

Conclusion

This study delivers direct empirical substantiation for feature-specific error correction as a computational primitive in LLMs. By developing and validating a perturbation-based method to quantify privilege in candidate directions—a necessity for robust computation in superposition—the work significantly advances the empirical interpretability toolkit. The separation between feature-aligned and control directions marked by $p>2$3 is robust, model-general, and informative about the internal computational geometry of LLMs. These results clarify and constrain theoretical models of LLM computation and furnish quantitative targets for both neuroscientific analogy and future mechanistically interpretable model design.