- The paper analyses the mechanistic causes of catastrophic forgetting in LLMs during continual fine-tuning, identifying three key factors: gradient interference, representational drift, and phase flattening. With 35,000 GPU-hours of experimentation, this study's insights centre on these mechanism details, such as early epoch attention disruption, intermediate layer drift, and the inversion of gradients due to conflicts.
- The decimation of 4 lifiscale retrention models due to increased intertask similarity was found to be a crucial driver of forgetting.
- Forgetting was correlated with loss landscape flattening preceding its effects on long-term memory management.
Overview and scope
This paper presents a mechanistic analysis of catastrophic forgetting in LLMs subjected to sequential fine-tuning, moving beyond behavioral characterization to identify parameter-level causes (2601.18699). The study examines six models—Llama 4 Scout (109B total/17B active parameters), Llama 4 Maverick (400B total/17B active), DeepSeek-V3.1 (671B), GPT-5.1 (~1.5T estimated), Claude Opus 4.5, and Gemini 2.5 Pro (~1T estimated)—across twelve task sequences built from twenty-four NLP tasks spanning seven domains. The central claim is that forgetting decomposes into three interacting mechanisms: gradient interference disrupting attention weights, representational drift in intermediate layers, and loss landscape flattening around prior task minima.
Experimental framework
The authors constructed high-, medium-, and low-similarity task sequences (four to six tasks each, 3–5 epochs per task) and collected checkpoints every 500 steps, preserving optimizer states and gradient statistics. Behavioral forgetting was measured as accuracy degradation on held-out test sets of prior tasks; mechanistic measurements included attention weight distances, attention entropy, CKA similarity of hidden representations, per-parameter gradient cosine similarities, and Hessian eigenvalue spectra via stochastic Lanczos quadrature. Experiments consumed roughly 35,000 GPU-hours on H100/H200 hardware.
Behavioral findings
Forgetting was substantial and systematically modulated by task similarity and model scale. On Llama 4 Maverick, first-task accuracy on high-similarity sequences fell from 89.3% to 67.2% after three subsequent tasks (24.8 points absolute); low-similarity sequences produced worse forgetting (31.7 points) than medium-similarity ones (18.3 points). Counterintuitively, higher inter-task similarity correlated with more severe forgetting (Pearson r=0.87, p<0.001). Temporally, forgetting was non-linear: minimal during the first 1–2 epochs, then accelerating sharply as the new task converged. Mixture-of-experts architectures showed heterogeneous patterns, with some experts retaining task knowledge while others were overwritten.
Attention disruption as the dominant early mechanism
Component-freezing experiments identified attention as the primary locus of early forgetting: freezing attention layers reduced forgetting by 64%, versus 23% for feedforward layers and 8% for output layers. Approximately 15–23% of attention heads underwent severe reorganization (weight-space displacement exceeding 2.5 standard deviations), disproportionately concentrated in lower layers. Disrupted heads exhibited entropy increases of 1.8–2.4 bits and specialization-index decreases averaging 0.43. Notably, only 31% of severely disrupted heads were highly activated on the new task, indicating that disruption is largely non-selective—a strong claim implying current fine-tuning wastes capacity by overwriting heads irrelevant to the target objective. Causally, ablating the 20% most disrupted heads post-hoc restored 47% of lost original-task performance at only 8% cost on the new task.
Intermediate layers showed the largest drift, with CKA decreasing 0.32–0.47 compared to 0.15–0.23 in lower layers and 0.08–0.14 in upper layers. PCA revealed rotation of leading principal components by 35–52 degrees while higher-order components rotated only 8–15 degrees, indicating preferential disruption of dominant representational structure. Drift magnitude was uncorrelated with task relevance (r=0.12, p=0.24): both task-relevant and irrelevant dimensions drifted comparably. Learned affine realignment of intermediate representations recovered 38% of lost performance; combined with head restoration, interventions recovered 71% of original-task performance, establishing causal contributions of both mechanisms.
Gradient interference as a predictive signature
Gradient cosine similarity between current-task and previous-task gradients strongly predicted retention: when similarity dropped below −0.3, forgetting rates increased 2.4-fold relative to positively aligned conditions. Interference localized to attention query/key projections (67% conflict rate) versus value projections (34%) and feedforward weights (29%). First-epoch gradient alignment predicted final forgetting magnitude at r=−0.79, providing an actionable early-warning signal. Severely disrupted parameters also exhibited 2.8× larger gradient magnitudes; aggressive gradient clipping (norm 0.5) reduced forgetting by only 18% while slowing new-task learning by 27%.
Loss landscape flattening
Hessian analysis showed maximum eigenvalues for the first task's loss falling from 147.3 at convergence to 34.2 after three subsequent tasks, while curvature along recent-task directions remained high (112.7). Linearity indices rose from 0.28 to 0.71, indicating erosion of well-defined minima. Critically, curvature reduction preceded behavioral forgetting by 1–2 epochs, making it a leading mechanistic indicator. A curvature regularization penalty reduced forgetting by 34% with 12% slower convergence—a markedly better trade-off than L2 regularization (14% reduction, 31% slower convergence).
Integrated mechanistic model
The three mechanisms operate on distinct timescales and interact synergistically: gradient interference disrupts lower-layer attention during epochs 1–2; representational drift accumulates over epochs 3–5 as intermediate layers adapt; landscape flattening emerges after epoch 4+, removing restoring forces toward prior minima and rendering forgetting progressively irreversible. Larger models exhibit less gradient interference (attributed to higher-dimensional parameter spaces) but comparable representational drift, suggesting drift reflects intrinsic properties of transformer representations rather than capacity limits. The counterintuitive similarity–forgetting relationship is explained by localized query/key conflicts masked by globally aligned gradients.
Limitations and open questions
The paper concedes several constraints explicitly. Mechanisms were characterized under supervised fine-tuning with discrete task boundaries; continual adaptation under RLHF or instruction tuning, and gradual distribution shift, may involve additional mechanisms. Task sequences contained only 4–6 tasks, far short of realistic deployment horizons involving hundreds of tasks. Although proprietary models up to ~1.5T parameters were included via API, the deepest mechanistic analyses (full gradients, Hessians) were feasible mainly up to the 400B open-weight scale, leaving open whether mechanisms change qualitatively beyond a trillion parameters. Alternative architectures (state-space models, encoder-decoder variants) were not analyzed in depth, so architecture-generality remains unverified. Finally, results describe average behavior across runs; individual variability and rare failure modes are uncharacterized.
Conclusion
The paper provides a systematic decomposition of catastrophic forgetting in transformer LLMs into attention disruption, representational drift, and loss landscape flattening, supported by causal intervention experiments and predictive early-training signatures. Its most consequential findings—that freezing attention reduces forgetting by 64%, that first-epoch gradient alignment forecasts final forgetting at r=−0.79, and that curvature regularization outperforms generic regularizers—suggest mechanism-targeted continual learning strategies are feasible. Whether these mechanisms hold across longer task horizons, non-supervised adaptation regimes, and architectures beyond decoder-only transformers remains unresolved.