SingularClip: Preventing Spectral Collapse to Maintain Plasticity in Continual and Reinforcement Learning

Abstract: Neural networks trained on nonstationary tasks frequently lose the ability to fit new targets, a phenomenon referred to as loss of plasticity. We identify a novel source of plasticity loss due to the growing anisotropy of weight matrices' singular values during training, and analyze this phenomenon both empirically and theoretically. To mitigate this issue, we introduce SingularClip, a procedure that periodically clips the singular values of all weight matrices. We show that SingularClip performs strongly against baselines across a range of tasks in both continual supervised learning and deep reinforcement learning.

• The paper introduces SingularClip, a method that mitigates anisotropy-induced plasticity loss (AIPL) in continual and reinforcement learning by clamping singular values of weight matrices, maintaining network adaptability, and improving representational capacity.
• SingularClip features an implementation that outperforms standard methods, including LayerNorm and periodic resetting, across various continual learning benchmarks and demonstrates superior stability in reinforcement learning histories.
• The empirical validation and theoretical bounding reveal that SingularClip achieves these results through spectral tuning adequate to keep matrices near the optimal singular value intervals.

Mechanisms and characterization of anisotropy-induced plasticity loss

Continual learning and reinforcement learning algorithms routinely suffer from a degradation in network adaptability over extended training horizons, commonly termed loss of plasticity. While existing literature attributes plasticity loss to phenomena such as dormant units, stale optimizer momentum buffers, and parameter norm inflation driving effective learning rate decay, Kastner et al. identify a distinct structural failure mode: anisotropy-induced plasticity loss (AIPL) (2608.18319). In standard gradient-based optimization, empirical gradients frequently concentrate within low-dimensional subspaces. When trained on a sequence of distinct objectives, neural network weight matrices accumulate parameter mass predominantly along these historical update directions. Consequently, the singular value spectra of intermediate weight matrices become heavily skewed, leading to exponential growth in matrix condition numbers across hidden layers.

Figure 1 illustrates this phenomenon during training on the Random Label CIFAR benchmark, demonstrating that every intermediate layer except the output classification head undergoes severe condition number divergence.

Figure 1: Evolution of the condition numbers for each layer in a network training on Random Label CIFAR task. Error bars show 95\% bootstrapped confidence intervals across 10 runs.

The spectral distortion underlying this condition number explosion is detailed in Figure 2, which tracks the empirical singular value distribution of an early convolutional layer. Even when parameter magnitude is constrained via weight decay, the largest singular value increases threefold while the trailing singular values collapse toward zero.

Figure 2: Evolution of the singular value distribution for the first convolutional layer in a run of Random Label CIFAR. Note that the largest singular value increases by a factor of roughly 3, as the parameter norm growth is controlled by weight decay.

The mechanistic consequence of this spectral collapse manifests during backpropagation. For a linear layer parameterized by $W = U \Sigma V^\top$, the downstream gradient signal transmitted to preceding layers given output error gradient $g$ is $W^\top g = \sum_{i} \alpha_i \sigma_i v_i$, where $\alpha_i$ represents the coordinate projections of $g$ onto the left singular vectors $U$. In nonstationary environments where new task objectives require gradient updates orthogonal to the historical dominant subspace, the corresponding components $\alpha_i$ align with the collapsed singular values $\sigma_i \ll \sigma_1$. As a direct result, gradient signals passed through the layer along novel directions undergo severe magnitude attenuation, severely stalling representation learning in earlier layers.

Figure 3 establishes the empirical causal link between weight anisotropy and optimization slowdown. By artificially manipulating the initialization condition number of network weights while keeping the Frobenius norm strictly constant, the number of optimization steps required to achieve perfect training accuracy exhibits an inverse monotonic relationship with the dimensionality of the populated singular subspace. Furthermore, tracking sequential tasks on Random Label MNIST demonstrates a direct negative correlation between initial layer condition numbers and average online task accuracy.

Figure 3: Measuring the number of steps to reach perfect train accuracy as a function of initialization anisotropy, across 20 independent runs (left). Measuring initial weight condition number against average task accuracy in the Random Label MNIST task; datapoints are collected in a sequence of 35 tasks, across 10 independent runs (right).

Theoretical bounds on adaptation slowdown

To formally isolate AIPL from confounding factors such as parameter norm growth, the theoretical analysis focuses on a two-layer linear network $f_W(x) = W_2 W_1 x$ optimizing cross-entropy loss under continuous-time projected gradient flow on the unit Frobenius hypersphere, where $\|W_1(t)\|_F = \|W_2(t)\|_F = 1$ for all $g$0. This projected geometry corresponds to the continuous dynamics induced by scale-invariant normalization layers without affine parameters, removing effective learning rate decay from the optimization dynamics.

For any hidden subspace $g$1 with orthogonal projector $g$2, the subspace parameter mass is formalized as:

$g$3

Under projected gradient flow driven by cross-entropy loss against a teacher distribution, the growth rate of this subspace mass satisfies the differential inequality:

$g$4

where $g$5 represents the data-dependent coupling constant bounding the sample leverage. Applying Grönwall's inequality yields an exponential upper bound on subspace mass growth:

$g$6

When adapting to a new task requiring a minimum subspace energy $g$7 where initial allocation is deficient ($g$8), the minimum adaptation time $g$9 required to solve the task is strictly lower bounded:

$W^\top g = \sum_{i} \alpha_i \sigma_i v_i$0

where $W^\top g = \sum_{i} \alpha_i \sigma_i v_i$1 and $W^\top g = \sum_{i} \alpha_i \sigma_i v_i$2 denote the maximal singular values of the initial weights restricted to $W^\top g = \sum_{i} \alpha_i \sigma_i v_i$3, and $W^\top g = \sum_{i} \alpha_i \sigma_i v_i$4 denote the respective subspace ranks. If $W^\top g = \sum_{i} \alpha_i \sigma_i v_i$5 aligns with the weakest singular directions of the network, the adaptation time bound becomes:

$W^\top g = \sum_{i} \alpha_i \sigma_i v_i$6

where $W^\top g = \sum_{i} \alpha_i \sigma_i v_i$7 and $W^\top g = \sum_{i} \alpha_i \sigma_i v_i$8 are the condition numbers of $W^\top g = \sum_{i} \alpha_i \sigma_i v_i$9 and $\alpha_i$0. This establishes that the lower bound on adaptation latency scales logarithmically with the condition number of the weight matrices, proving that spectral collapse provably slows feature learning in downstream layers.

The SingularClip algorithm and projection optimality

To mitigate AIPL without incurring the excessive information loss of full network resets, Kastner et al. propose SingularClip (2608.18319). The method applies a periodic spectral transformation to all linear and reshaped convolutional weight matrices:

$\alpha_i$1

where $\alpha_i$2 is the singular value decomposition and $\alpha_i$3 clamps all singular values into the interval $\alpha_i$4 for a chosen hyperparameter $\alpha_i$5.

SingularClip acts as the exact Frobenius norm projection of the weight matrix $\alpha_i$6 onto the set of matrices whose singular values are bounded within $\alpha_i$7:

$\alpha_i$8

for any matrix $\alpha_i$9 with spectrum contained in $g$0. Because the operator norm is upper bounded by the Frobenius norm, SingularClip directly minimizes the worst-case layer-wise feature distortion:

$g$1

thereby preserving maximal learned representational structure while enforcing bounded anisotropy.

$U$1

As demonstrated in Figure 4, two-sided clipping is strictly necessary. Lower-clipping alone fails to prevent parameter norm expansion, leading to effective learning rate decay. Upper-clipping alone prevents norm explosion but permits singular value collapse, failing to curb AIPL.

Figure 4: Ablation of one-sided clipping across a subset of continual learning tasks. The baseline curve corresponds to a network with no intervention applied.

Figure 5 confirms that while spectral regularization limits the maximal singular value, it permits unconstrained condition number growth across layers. SingularClip strictly bounds condition numbers throughout training.

Figure 5: Comparison of condition number growth between SingularClip and spectral regularization on the Random Label CIFAR task, across different network layers.

Empirical evaluation in continual supervised learning

The empirical performance of SingularClip was benchmarked across five standard continual supervised learning protocols: Permuted MNIST, Random Label MNIST, Random Label CIFAR, Random Label ImageNet, and Continual ImageNet. Evaluations were conducted across 30 independent runs measuring average online training accuracy.

Figure 6: Comparison of average online accuracy on continual supervised learning tasks. Shaded regions represent bootstrapped estimates of 95\% confidence intervals. Each experiment is repeated over 30 independent runs.

Figure 6 demonstrates that SingularClip consistently outclasses all competing baselines, including LayerNorm, periodic parameter resetting, Normalize and Project (NaP), spectral regularization, Shrink and Perturb, and DASH. On complex memorization tasks such as Random Label CIFAR and Random Label ImageNet, periodic resetting fails due to complete destruction of historical feature representations. Conversely, methods that solely constrain the top singular value or parameter norm (such as NaP and spectral regularization) suffer from gradual performance degradation as anisotropy accumulates. SingularClip maintains high online accuracy across all 100 sequential tasks without exhibiting catastrophic plasticity decay.

Figure 7 explores the sensitivity of the clipping hyperparameter $g$2 across continual learning benchmarks, demonstrating that moderately tight bounds ($g$3) yield optimal trade-offs between anisotropy suppression and feature preservation.

Figure 7: Ablation of c over a subset of continual learning tasks.

Figure 8 demonstrates the applicability of SingularClip when combined with modern orthogonalized optimizers such as Muon. Although Muon utilizes Newton-Schulz iterations to orthogonalize updates, numerical accumulation errors still induce condition number drift and subsequent plasticity loss on Continual ImageNet. Augmenting Muon with SingularClip suppresses spectral drift, yielding sustained online classification gains.

Figure 8: Comparison of adding SingularClip to Muon on the Continual ImageNet task. We plot average weight condition number (left) and online accuracy (right). Error bars represent bootstrapped 95\% confidence intervals across 10 runs.

Empirical evaluation in deep reinforcement learning

In deep reinforcement learning, plasticity loss is exacerbated by nonstationary temporal-difference targets and high update-to-data (UTD) training regimes. SingularClip was evaluated within the DeepMind Control (DMC) suite across Soft Actor-Critic (SAC) and Bigger, Regularized, Optimistic (BRO) architectures.

Figure 9: Performance of SAC-based agents aggregated across DMC suite. Performance is aggregated across 10 runs per task. Error bars indicate 95\% bootstrapped confidence intervals (left). Performance curves of SAC-based agents in the humanoid-stand environment averaged across 10 independent runs. Shaded regions correspond to 95\% bootstrapped confidence intervals (right).

Figure 9 highlights interquartile mean (IQM) scores across the DMC suite under varying UTD ratios. While standard SAC collapses at higher replay ratios (UTD 4 and 8), SingularClip scales favorably, outperforming periodic resetting across all evaluation points. In sparse, high-dimensional control tasks such as humanoid-stand (Figure 9, right), periodic resetting erases policy progress before the agent reaches reward states, resulting in zero return throughout training. SingularClip preserves sub-network competence, enabling steady optimization and superior asymptotic returns.

Figure 10 displays hyperparameter sweeps over $g$4 for RL tasks, indicating that a slightly wider interval ($g$5) is preferred in high-UTD regimes to prevent excessive policy disruption while controlling condition number divergence.

Figure 10: Ablation of c over a subset of RL settings.

Figure 11 details the evaluation on the BRO agent across the DMC suite over 20 seeds. Replacing the native periodic reset mechanism of BRO with SingularClip yields statistically significant sample efficiency gains up to 800,000 environment steps, confirming that minimal spectral projection retains task-relevant policy features far more effectively than state re-initialization.

Figure 11: Performance of BRO-based agents across the DMC suite, across 20 independent runs. Shaded areas indicate bootstrapped 95\% confidence intervals.

Limitations and open questions

While SingularClip demonstrates empirical robustness and theoretical optimality as a Frobenius projection, several practical and theoretical considerations remain open:

• Computational complexity of exact SVD: Direct singular value decomposition scales cubically with layer dimension $g$6. While negligible when performed intermittently (e.g., every task transition or every $g$7 environment steps), exact SVD introduces computational bottlenecks if applied at high frequencies or to extreme-scale foundation models.
• Approximation stability: Iterative polynomial approximations such as Newton-Schulz iterations reduce runtime on tensor accelerators but incur non-trivial numerical truncation errors that permit residual condition number growth. Developing stable, low-overhead spectral clipping algorithms for distributed multi-GPU training remains unresolved.
• Non-convexity of the spectral constraint set: Because the lower singular value bound $g$8 defines a non-convex matrix manifold, the Frobenius projection is not uniquely defined when weight matrices contain degenerate zero singular values, though rotational invariance preserves the achieved distance.
• Hyperparameter scheduling: The clipping ratio $g$9 is currently maintained as a static hyperparameter. Determining whether adaptive layer-wise or dynamics-aware schedules for $U$0 can optimize feature retention across heterogeneous task sequences remains an open theoretical question.

Conclusion

Anisotropy-induced plasticity loss represents a fundamental failure mode in continual and reinforcement learning, wherein gradient concentration along dominant historical subspaces causes intermediate weight spectra to collapse, severely attenuating learning signals along novel parameter directions. By formulating weight spectral clipping as an exact minimal Frobenius norm projection, SingularClip prevents unbounded condition number growth while preserving maximal learned representational capacity. Across extensive continual supervised benchmarks and high-UTD reinforcement learning environments, SingularClip consistently outperforms full resetting and upper-spectrum regularization methods, providing a principled approach to preserving model trainability in nonstationary optimization.