Progressive Activation Drift

• Progressive activation drift is the gradual change in neural activations or embedded representations over time, even when task performance remains stable.
• It results from stochastic optimization, hardware faults, and evolving input dynamics, and is quantified by measuring norm fluctuations and angular diffusion.
• Mitigation strategies include regularizing representation magnitudes, activation clipping, and online steering to preserve stability and enhance robustness.

Progressive activation drift denotes a family of phenomena in which internal activations or learned representations change gradually as optimization, inference, or interaction proceeds. Related work uses the labels representational drift, task drift, agent drift, and layer-wise semantic drift, depending on whether the object of study is a neural representation, a prompted task specification, an agent trajectory, or a steering direction. Across these settings, the common object is a trajectory in activation space whose evolution may preserve task performance, degrade long-horizon behavior, or alter robustness unless explicitly constrained (Pashakhanloo et al., 2023, Abdelnabi et al., 2024, Sui et al., 7 May 2026, Jiang et al., 12 Mar 2026).

1. Conceptual scope and formal definitions

In the most explicit theoretical formulation, representational drift is the over-time change in a hidden representation for fixed stimuli after task performance has converged. In a two-layer linear network trained continually with SGD, the hidden representation is

$\mathbf{h}(t;\mathbf{x})=\mathbf{U}(t)\mathbf{x},$

and drift is the change of $\mathbf{h}(t;\mathbf{x})$ over time for fixed $\mathbf{x}$, with the input–output map remaining effectively stable because motion occurs along a minimum-loss manifold (Pashakhanloo et al., 2023).

A broader continual-learning perspective defines representational drift as non-mean-reverting changes in neural representations of a task not associated with changes in performance, sensory inputs or motor outputs. In that formulation, drift is distinguished both from forgetting, where performance changes, and from noise, which is mean-reverting rather than cumulative (Veldt et al., 26 Dec 2025).

Other literatures operationalize the same phenomenon at different interfaces. In classification networks, AL2 defines the relevant representation as the vector $\phi(x)$ formed by the activations of all neurons in the last feature layer before the final classification layers, and treats uncontrolled growth of $\|\phi(x)\|_2$ as the quantity to regularize (Helou et al., 2020). In prompt-injection detection, task drift is measured through task activation residuals,

$$\widetilde{\mathrm{Act}}^{x}=\mathrm{Act}^{x}-\mathrm{Act}^{x_{\mathrm{pri}}},$$

which quantify how hidden states change when untrusted external data is added to the primary task specification (Abdelnabi et al., 2024). In coding agents, drift is defined behaviorally as long-horizon degradation into overthinking and overacting, but the paper locates this drift in the residual stream at the </think> token and models it as movement away from a calibrated region along two low-dimensional axes (Sui et al., 7 May 2026).

2. Mechanistic accounts

A central mechanistic account treats drift as a consequence of stochastic optimization in overparameterized models. In the linear two-layer setting, the loss landscape contains a manifold of equivalent minima. SGD noise produces mean-reverting fluctuations in directions normal to this manifold and, through normal–tangent coupling, an effective diffusion along tangent directions. The result is bounded fluctuation of representation norms together with angular diffusion of hidden representation directions, with drift rate controlled by learning rate $\eta$, regularization $\gamma$, network dimensions, and input statistics; more frequently presented stimuli drift more slowly (Pashakhanloo et al., 2023).

A distinct mechanism arises during early training with positively biased activations. Under MSE or cross-entropy loss, the gradient with respect to positive pre-activations is non-negative in expectation at initialization, which drives downstream weights toward negative values. The paper identifies this as a negative weight drift intrinsic to optimization rather than data, persisting across MLP, ResNet, ViT, GPT-nano, and MP-SENet, and across ReLU, GELU, and SiLU. Coupled with ReLU, this produces activation sparsity reaching up to 90% in GPT-nano; with squared activations it can also amplify intermediate-layer spikes (Shvetsov et al., 17 May 2026).

Several application-specific mechanisms instantiate progressive activation drift as accumulation of perturbations. In hardware-fault settings, random bit flips in weights, biases, and threshold parameters create erroneous activations that are amplified as the signal propagates layer by layer, producing a progressive activation drift across layers unless activations are clipped (Mousavi et al., 2024). In quantized diffusion models, the denoiser is reused over many timesteps, so quantization error at one step perturbs the latent trajectory seen by subsequent steps. PCR formalizes the ultimate sample error as approximately a linear combination of the per-timestep quantization errors $\{\|\Delta_t\|\}$, which makes activation drift a timestep-accumulated effect (Tang et al., 2023).

In continual learning, the broader interpretation is that drift reflects a mixture of homeostatic turnover and learning-related synaptic plasticity. This suggests that drift is compatible with approaches that preserve past-task behavior through functional constraints rather than by freezing parameters, but that drift is a byproduct rather than a solution to continual learning (Veldt et al., 26 Dec 2025).

3. Quantification and diagnostics

Theoretical work quantifies drift through both norm fluctuation and directional diffusion. For covariance eigenvector $\mathbf{v}_i$, the hidden representation $\mathbf{h}(t;\mathbf{x})$0 has norm fluctuation variance

$\mathbf{h}(t;\mathbf{x})$1

while pairwise angular diffusion is summarized by coefficients $\mathbf{h}(t;\mathbf{x})$2 defined from the mean-squared angular increment. This separates drift into bounded fluctuations of representation magnitude and persistent diffusion of representation direction (Pashakhanloo et al., 2023).

In supervised representation learning, AL2 uses the activation norm itself as the primary diagnostic. The total loss is

$\mathbf{h}(t;\mathbf{x})$3

with a progressively increasing $\mathbf{h}(t;\mathbf{x})$4. The paper then diagnoses generalization via randomized-label tests, cumulative ablation curves, and CCA of learned representations. Under 75% label corruption on MNIST, adding AL2 increases test accuracy from about 25.84% to about 68.46% in the bare baseline at epoch 700, and the cumulative-ablation AUC for the bare model rises from about 15.19/100 to about 47.65/100, indicating that representation compactness and ablation robustness change together (Helou et al., 2020).

In LLMs, activation-space probes have become a direct diagnostic. "Get my drift? Catching LLM Task Drift with Activation Deltas" uses the residual

$\mathbf{h}(t;\mathbf{x})$5

at the last token across layers and shows that a simple linear classifier detects drift with near-perfect ROC AUC on an out-of-distribution test set. The same work tracks token-prefix evolution and shows that the embedding distance from the primary-task representation rises sharply at the onset of an injected instruction, making drift progressively observable as more tokens are ingested (Abdelnabi et al., 2024).

Coding-agent work introduces a closely related diagnostic geometry. TACT labels steps as overthinking, overacting, or calibrated, extracts residual-stream states at the </think> token, and defines two drift axes from calibrated behavior toward each failure mode. The centered scalar projection

$\mathbf{h}(t;\mathbf{x})$6

measures how far the current hidden state has drifted along axis $\mathbf{h}(t;\mathbf{x})$7, and the paper reports AUC $\mathbf{h}(t;\mathbf{x})$8 for linear separability along these axes (Sui et al., 7 May 2026).

Quantized diffusion models require a different diagnostic regime. PCR argues that FID to COCO is distorted by a distribution gap and replaces it with FID-to-FP32 and CLIP score, which more directly expose accumulated activation drift relative to the full-precision trajectory (Tang et al., 2023). At the system level, multi-agent LLM work introduces the Agent Stability Index (ASI) over rolling windows of 50 interactions as a behavioral surface measure of drift; although it is not an activation metric, it tracks the progressive degradation of the policy induced by evolving context (Rath, 7 Jan 2026).

4. Control and mitigation strategies

One control strategy is to regularize representation magnitude directly during training. AL2 applies an $\mathbf{h}(t;\mathbf{x})$9 penalty only to the last feature representation and increases its weight monotonically with an approximately exponential schedule, beginning from $\mathbf{x}$0. The stated rationale is that neural networks learn general patterns first and then overfit to data-specific patterns, so late training should be progressively deprived of memorization capacity while early training remains comparatively unconstrained (Helou et al., 2020).

Another strategy is to bound activation ranges to suppress fault amplification. ProAct introduces the hybrid clipped activation function HyReLU, which uses layer-wise thresholds in early layers and neuron-wise thresholds only in the last layer, and trains those thresholds progressively from the last hidden layer backward. This yields a large reduction in threshold-parameter overhead relative to full neuron-wise clipping while reducing the average $\mathbf{x}$1 distance between faulty and fault-free activations from 65.6 to 45.9 in AlexNet, from 80.3 to 72.4 in VGG-16, and from 93.7 to 86.5 in ResNet-50 (Mousavi et al., 2024).

For diffusion-model quantization, PCR performs progressive calibration backward through timesteps, calibrating the quantizer for timestep $\mathbf{x}$2 using inputs already perturbed by quantized steps $\mathbf{x}$3. It complements this with activation relaxing, which assigns higher activation bitwidth only to a small subset of sensitive timesteps. This is an explicit attempt to calibrate against drifted activation distributions rather than against full-precision activations (Tang et al., 2023).

LLM work emphasizes test-time steering. TACT applies online activation steering at each step’s </think> token, pulling out-of-band projections back toward a calibrated band along orthogonalized overthinking and overacting axes (Sui et al., 7 May 2026). GER-Steer addresses the related problem of layer-wise semantic drift in activation engineering by extracting a global evolutionary direction from tangent dynamics and rectifying raw per-layer steering vectors to align with that global signal, thereby reducing high-dimensional noise and cross-layer inconsistency (Jiang et al., 12 Mar 2026).

Continual-learning work moves from parameter regularization to functional invariance in activation space. InTAct stores percentile-based activation hypercubes for selected layers and constrains future updates so that, for inputs in the cumulative protected region $\mathbf{x}$4, the pre-activation change satisfies

$\mathbf{x}$5

Interval arithmetic turns this into a differentiable loss, supplemented by compactness, alignment, and masked feature-distillation terms, so that stability is imposed on past-task activation regions without freezing parameters or storing past data (Krukowski et al., 21 Nov 2025).

5. Empirical manifestations and consequences

In supervised classification, progressive control of activations is tightly linked to generalization under label noise. AL2 consistently reduces overfitting across MNIST, Fashion-MNIST, and CIFAR-10, prevents full memorization of randomized labels, and can raise test accuracy by up to 60 percentage points when added to weight decay in the most extreme MNIST condition reported. Its representations also retain more accuracy under cumulative activation ablations, which the paper interprets as stronger generalization (Helou et al., 2020).

In coding agents, activation drift appears as a long-horizon deterioration of think–act allocation. TACT identifies overthinking and overacting as two recurrent residual-stream directions and reports average resolve-rate gains of $\mathbf{x}$6 percentage points on Qwen3.5-27B and $\mathbf{x}$7 percentage points on Gemma-4-26B-A4B-it, while cutting steps-to-resolve by up to 26% (Sui et al., 7 May 2026). A related multi-agent analysis, framed behaviorally rather than at the activation level, finds that systems with severe drift show task success dropping from 87.3% to 50.6%, human interventions per task rising from 0.31 to 0.98, and inter-agent conflicts increasing from 0.08 to 0.47 per task (Rath, 7 Jan 2026).

In text-to-image diffusion, uncontrolled activation drift severely degrades both fidelity and text alignment under PTQ. PCR is the first method in the cited set to achieve quantization for Stable Diffusion XL while maintaining performance, and in the W8A8 SDXL setting it reduces FID-to-FP32 from about 38 for Q-diffusion/PTQ4DM to 12.00 while improving CLIP score from about 15.98 to 24.05 (Tang et al., 2023).

In fault-tolerant CNNs, activation drift across layers under persistent bit flips directly translates into accuracy loss. ProAct reduces the drop relative to FT-ClipAct by 1.36× to 6.4× and relative to FitAct by 1.07× to 1.72×, while requiring far less threshold memory than full neuron-wise clipping (Mousavi et al., 2024). In training-dynamics studies, the same concept appears as a sparsity–performance tradeoff: negative weight drift can drive activation sparsity up to 90% in GPT-nano, but there is a sharp accuracy cliff above approximately 70% activation sparsity (Shvetsov et al., 17 May 2026).

In rehearsal-free continual learning, activation drift is strongly associated with forgetting under domain shift. InTAct reports Average Accuracy gains of up to 8 percentage points over state-of-the-art baselines and, on DomainNet and ImageNet-R, substantially lowers Average Forgetting when added to prompt-based methods such as L2P, DualPrompt, and CODA-Prompt (Krukowski et al., 21 Nov 2025).

6. Interpretation, misconceptions, and limitations

A recurrent misconception is that drift is necessarily synonymous with failure. Theoretical and neuroscience-inspired accounts instead treat drift as compatible with stable task performance when motion occurs along flat or functionally equivalent directions of the loss landscape (Pashakhanloo et al., 2023, Veldt et al., 26 Dec 2025). Conversely, several applied works study drift precisely because it precedes or predicts failure: task drift under prompt injection, overthinking and overacting in coding agents, or coordination breakdown in multi-agent systems (Abdelnabi et al., 2024, Sui et al., 7 May 2026, Rath, 7 Jan 2026).

Another misconception is that controlling drift requires freezing parameters. The cited interventions show several alternatives: direct norm penalties on representations, activation clipping, timestep-wise calibration, activation steering, and interval-based functional constraints. This suggests that the practical question is not whether activations move, but where, when, and along which directions movement remains compatible with generalization, robustness, or continual adaptation (Helou et al., 2020, Tang et al., 2023, Krukowski et al., 21 Nov 2025).

The concept remains heterogeneous. The most explicit analytic theory assumes a two-layer linear network, squared-error loss, small learning rate, and stationary input distributions (Pashakhanloo et al., 2023). LLM steering methods require access to hidden states and are therefore limited to open-weight models; TACT also assumes that overthinking and overacting are well captured by a small number of linear directions (Sui et al., 7 May 2026). GER-Steer relies on a global cross-layer direction being recoverable from tangent dynamics, while InTAct approximates protected activation regions by axis-aligned hypercubes, which can over-approximate complex manifolds (Jiang et al., 12 Mar 2026, Krukowski et al., 21 Nov 2025). PCR adds offline calibration cost, and ProAct’s evidence is currently restricted to CNNs on CIFAR-10 and CIFAR-100 (Tang et al., 2023, Mousavi et al., 2024).

Taken together, these works suggest that progressive activation drift is best understood not as a single pathology but as a general dynamical property of internal representations. Depending on context, it may appear as diffusion on a minimum-loss manifold, uncontrolled growth of feature norms, accumulation of quantization or hardware error, semantic deviation in residual streams, or functional overwriting in continual learning. The central technical problem is therefore the same across domains: to distinguish benign or even useful representational motion from the specific forms of drift that undermine calibration, robustness, or memory (Veldt et al., 26 Dec 2025).