What governs catastrophic forgetting and why RL vs. SFT differ

Determine the underlying mechanism that governs catastrophic forgetting in foundation models and explain why supervised fine-tuning (SFT) and on-policy reinforcement learning (RL) exhibit different forgetting behavior despite achieving similar new-task performance.

Background

The paper studies catastrophic forgetting when adapting foundation models via supervised fine-tuning (SFT) and reinforcement learning (RL). Empirically, RL reaches comparable new-task performance while preserving prior capabilities better than SFT. Prior approaches targeted symptoms (e.g., constraining parameter changes) but did not explain the cause of forgetting or the algorithmic differences. The authors’ results motivate identifying a principled mechanism that explains when and why forgetting occurs and why RL behaves differently than SFT.

References

Consequently, it remains unclear what truly governs forgetting or why different training algorithms behave so differently.

— RL's Razor: Why Online Reinforcement Learning Forgets Less  (2509.04259 - Shenfeld et al., 4 Sep 2025) in Section 1, Introduction

Yet whether parameter drift itself causes forgetting remains unverified, as existing evidence is limited to specific tasks and small training sets.

— Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO  (2608.27351 - Ba et al., 27 Aug 2026) in Section 1, Introduction

Instead, mitigation approaches now aim to minimize change in pre-trained networks (Xiong & Xie, 2025; Nayak et al., 2025; Wang et al., 2023; Liang & Li, 2024), yet the exact definition of change, and how to prevent it, remain open problems.

— Local Support Learning  (2610.02126 - Ben-Kish et al., 1 Oct 2026) in Section 1, Introduction, p. 2

This motivates separating transient functional degradation from persistent capability loss, but their dynamical relationship remains unclear.

— Collapse, Repair, and Drift: Dissecting Fine Tuning Dynamics in Speech Language Models  (2610.18192 - Wang et al., 13 Oct 2026) in Section 2, subsection “Transient Degradation and Forgetting”