The Stable Recovery Manifold: Geometric Principles Governing Recoverability in Continual Learning

Abstract: Catastrophic forgetting is often viewed as the destruction of previously learned knowledge during sequential learning. Building on the Accessibility Collapse framework, we investigate the geometric structure of recoverability in continual learning. Using Split CIFAR-100 and a sequentially trained ResNet-18, we analyze recoverability, representational drift, and recovery complexity across ten tasks. We introduce Recovery Subspace Dimensionality (k_t), a measure of the minimum number of singular directions required to preserve 90 percent of full probe performance. Contrary to our Recoverability Diffusion hypothesis, recovery dimensionality remains stable throughout training (mean k_t = 8.0) despite substantial representational drift. Principal-angle drift strongly predicts recoverability (r = -0.862), and a simple geometric model explains 82.2 percent of recoverability variance. These findings support the Stable Recovery Manifold hypothesis, suggesting that forgotten knowledge remains compactly decodable despite representational reorganization. The results indicate that catastrophic forgetting is primarily an accessibility and manifold-alignment problem rather than information destruction.

• The paper falsifies the assumption of recoverability diffusion by showing that task-specific information remains in a stable 8-dimensional subspace.
• It reveals that catastrophic forgetting is an accessibility collapse due to geometric misalignment rather than permanent loss of representations.
• Experiments using ResNet-18 on Split CIFAR-100 confirm that recovery metrics like principal-angle drift strongly predict recoverability.

Stable Recovery Manifold and Geometric Recoverability in Continual Learning

Introduction

The investigation addresses the geometric mechanisms underlying catastrophic forgetting in continual learning systems. Contrary to prevailing assumptions that forgetting arises from the permanent obliteration of acquired representations, the paper posits that forgotten task information persists within a stable, compact subspace, and what is lost is accessibility via the classifier, not the underlying representational substrate. The study leverages representational probing—specifically linear probe evaluation—to reveal high recoverability of past tasks even following extensive sequential adaptation, ultimately reframing forgetting as a geometric accessibility collapse.

Experimental Design and Metrics

The core empirical studies employ a ResNet-18 backbone trained on Split CIFAR-100 across ten sequential tasks, devoid of memory replay or regularization. The representations are systematically interrogated using:

• Linear Probe Accuracy (LPt)
• Recoverability (Rt): Accuracy after classifier reset/retraining
• Accessibility Gap (AGt): Difference between recoverability and active accuracy
• Projection Energy (PEt): Variance aligned with the original task subspace
• Principal-Angle Drift (Dt): Mean angular difference between recovery subspaces
• Subspace Dimensionality (kt): Rank of probe matrix preserving 90% accuracy
• Participation Ratio (PR): Effective dimensionality per layer (covariance-based)

The full protocol quantifies how recoverable knowledge evolves, which geometric parameters predict recoverability, and whether subspace structure shifts as more tasks are introduced.

Key Findings

Falsification of Recoverability Diffusion

The initial hypothesis posited monotonic growth of recovery subspace dimensionality as tasks accumulate (Recoverability Diffusion). However, kt remains remarkably constant at ~8 across all tasks ($\sigma = 0.82$), signifying that task-specific information does not scatter, but is maintained within a stable, low-rank subspace. This result is fundamentally negative and sharply contrasts with assumptions in the continual learning literature that dimensionality of the manifold would increase.

Stable Recovery Manifold Hypothesis

The SRM hypothesis asserts that catastrophic forgetting is characterized by accessibility collapse. Task knowledge is preserved within approximately 8 principal directions (out of 512), and recoverability is governed by geometric orientation (Dt) rather than volume. Empirically, principal-angle drift emerges as the primary predictor ($r = -0.862$), with a combined regression of LPt, PEt, and Dt explaining 82% of the variance in recoverability ($R^2 = 0.82$). This places geometric misalignment—not information loss—at the center of the forgetting phenomenon.

Depth-Stratification and Layer Dynamics

Experiments reveal systematic depth stratification in representations: early layers (L1, L2) become more distributed and general as tasks progress, while late layers (L3, L4) become increasingly specialized and compressed. Layer 1 achieves a retention rate near 100%, functioning as an invariant universal feature extractor, whereas Layer 4 undergoes substantial reorganization (CKA final score 0.49), intensifying the accessibility gap.

Recoverability Timeline

Recoverability persists at ∼70% across tasks even as active classifier accuracy collapses to almost random levels by the third task. There is a significant and persistent accessibility gap, demonstrating the network’s loss of classifier alignment rather than destruction of the representational substrate.

Implications and Theoretical Considerations

Reframing Forgetting

The results substantiate a paradigm shift: catastrophic forgetting should be construed as accessibility failure, not representational destruction. The network retains sufficient structure for linear recovery but loses access due to classifier misalignment. The geometric orientation of the recovery manifold becomes the central object of analysis.

Efficacy of Existing Mitigation Strategies

Classical methods penalizing parameter drift (e.g., EWC, SI, MAS) do not explicitly target subspace orientation preservation. Replay-based strategies and gradient projection approaches indirectly maintain manifold alignment; findings suggest that retaining only the top 8 singular directions could suffice for comprehensive recoverability, implying significant storage and computational efficiency gains.

Role of Architectural Inductive Bias

The resilience and compactness of the recovery manifold raise questions regarding architecture universality. The 8-dimensional result may reflect constraints imposed by residual connectivity in ResNet, meriting investigation into whether analogous properties manifest in ViT, MLP-Mixer, or conventional convolutional architectures.

Manifold Anchor Regularization

Future mitigation strategies should prioritize penalizing rotation of the recovery subspace. The proposed anchor regularizer operates by extracting the top-8 singular directions after an initial task, subsequently enforcing alignment in future tasks. This mechanism would preserve recoverability without impeding plasticity—a critical requirement for scalable continual learning.

Limitations

Several constraints delimit the generalizability and scope of results:

• Only ResNet-18 and Split CIFAR-100 are examined; the stability and dimensionality of the SRM remain untested in long-horizon regimes and alternate architectures.
• The functional form of recoverability decay as a function of principal-angle drift cannot be precisely specified with limited data points.
• The universality of the 8-dimensional compactness is unresolved.

Potential Future Directions

• Empirical validation across diverse architectures and datasets is needed to establish universality.
• Long-horizon continual learning (tasks t > 10) will confirm or challenge manifold stability beyond short sequences.
• Minimal-replay protocols targeting only the recovery manifold could substantively reduce exemplar or checkpoint storage requirements.

Conclusion

The paper delivers substantive advances in the geometric analysis of catastrophic forgetting. Forgotten task knowledge is retained within a compact, stable, low-dimensional recovery manifold whose orientation in representational space governs recoverability. The principal-angle drift between original and current subspaces dominates recoverability, nullifying the assumption that information is scattered. These findings redirect continual learning research to focus on manifold orientation preservation as the key bottleneck and intervention target. If architectural and task-scale generalisation holds, manifold-based regularization and efficient replay could enable scalable yet robust continual learning systems.