STABLE: Gated Continual Learning for Large Language Models (2510.16089v1)

Abstract: LLMs increasingly require mechanisms for continual adaptation without full retraining. However, sequential updates can lead to catastrophic forgetting, where new edits degrade previously acquired knowledge. This work presents STABLE, a gated continual self editing framework that constrains forgetting during sequential updates using parameter efficient fine tuning via Low Rank Adaptation (LoRA; see arXiv:2106.09685). Each candidate edit is evaluated against a stability budget using one of three metrics: (i) Exact Match (EM) drop, capturing factual accuracy loss; (ii) bits increase, reflecting reduced model confidence; and (iii) KL divergence, quantifying distributional drift between the base and adapted models. If a threshold is exceeded, the LoRA update is rescaled through a clipping procedure or rejected. Experiments on the Qwen-2.5-7B model show that gating effectively mitigates forgetting while preserving adaptability. EM based gating achieved the highest cumulative performance in short continual learning sequences. Our results show that different gating strategies can achieve comparable distribution shift (measured by KL divergence) while producing different accuracy outcomes, highlighting the importance of gating design in continual adaptation. This approach offers a principled method for continual model editing, enabling LLMs to integrate new knowledge while maintaining reliability. Code: https://github.com/Bhoy1/STABLE

• The paper introduces a gating mechanism to constrain parameter updates, effectively mitigating catastrophic forgetting in continual learning scenarios.
• It leverages parameter-efficient fine-tuning via Low Rank Adaptation (LoRA) and evaluates edits using metrics like Exact Match drop, bits increase, and KL divergence.
• Empirical results show that EM-based gating delivers the highest cumulative performance gain, balancing factual retention and model adaptability.

Gated Continual Learning for LLMs: The STABLE Framework

Introduction

The STABLE framework addresses the challenge of catastrophic forgetting in LLMs during continual adaptation. As LLMs are increasingly deployed in dynamic environments requiring incremental updates, the risk of degrading previously acquired knowledge becomes acute. STABLE introduces a gating mechanism for parameter-efficient fine-tuning (PEFT) via Low Rank Adaptation (LoRA), constraining sequential updates to preserve factual accuracy and model confidence. The framework evaluates candidate edits against a stability budget using three alternative metrics—Exact Match (EM) drop, bits increase, and KL divergence—and either rescales or rejects updates that exceed the threshold. This essay provides a technical analysis of STABLE’s methodology, empirical results, and implications for robust continual learning in LLMs.

Methodology

Gated Continual Self Editing

STABLE extends the SEAL framework by integrating a gating layer into the self-editing loop. Each LoRA adapter merge is treated as a candidate update, which must pass through a gate enforcing a user-defined budget on forgetting. The gate evaluates the edit using one of three metrics:

• Exact Match (EM) Drop: Measures the reduction in factual accuracy on anchor questions, using a binary LLM grader.
• Bits Increase: Quantifies the increase in bits per token required to encode answers, reflecting reduced model confidence.
• KL Divergence: Computes the average per-token KL divergence between the base and adapted models, capturing distributional drift.

If the computed forgetting score $f$ exceeds the budget $\epsilon$, a binary search is performed over the LoRA scaling factor $\alpha$ to find the largest value satisfying $f(\alpha \cdot W_{\text{LoRA}}) \leq \epsilon$. If no feasible $\alpha$ exists above a minimum threshold, the edit is rejected.

Figure 1: Performance and drift comparison across gating methods. EM based gating achieves the highest cumulative performance gain (+0.397).

Formalization

The gating constraint is expressed as:

$f \leq \epsilon$

where $f$ is the forgetting measure (EM drop, bits increase, or KL divergence) and $\epsilon$ is the user-defined budget. The LoRA update is accepted, rescaled, or rejected based on this criterion.

Implementation Details

• LoRA Configuration: Rank 32, scaling factor $\alpha=64$, 10 epochs, learning rate $1\text{e-}3$.
• Batching: Batch size 1.
• Binary Search: 5 evaluations per edit for LoRA clipping.
• Evaluation: 12 passes across 8 datapoints per run.
• Safety Mode: Minimum scale factor 0.1.
• Compute: Experiments conducted on two NVIDIA A100 GPUs (80GB each).

Empirical Results

Experiments were conducted on the Qwen-2.5-7B model using SQuAD-style factual editing tasks. Each gating strategy (EM, bits, KL) was evaluated independently across 12 runs, with each run sampling 8 datapoints. Anchor questions from previous edits were used to measure forgetting.

Performance Comparison

EM-based gating achieved the highest cumulative performance gain (+0.397), outperforming bits-based (+0.216) and KL-based (+0.154) gating at their respective thresholds. Notably, all three gating strategies resulted in comparable cumulative KL divergence, indicating similar levels of distributional drift despite differing accuracy outcomes.

Figure 2: Cumulative performance improvement across sequential updates for the three gating methods. EM based gating shows best results per step and cumulative.

Drift Analysis

Cumulative KL divergence increased steadily across sequential updates for all gating methods. The dashed horizontal lines in the analysis indicate the overall KL from baseline to final adapted model, which differs from the sum of per-step KL values due to non-additivity.

Figure 3: Cumulative KL divergence across sequential updates for the three gating methods. Dashed horizontal lines indicate the overall KL from baseline to final adapted model, which differs from the sum of per step KL values.

Acceptance and Scaling Behavior

• EM Gating: Most edits were accepted or scaled, with mean scale factors ranging from 0.768 to 0.930.
• Bits Gating: Higher proportion of scaled merges, mean scale factors 0.644–0.937.
• KL Gating: Majority of edits accepted with minimal scaling, mean scale factors 0.906–0.995.

Outlier Handling

KL divergence calculations were robustified via outlier detection, excluding values exceeding five standard deviations from the median. This mitigates the impact of rare tokens with near-zero probability under the reference model, which can disproportionately inflate KL estimates.

Theoretical and Practical Implications

Constrained Optimization Perspective

The gating mechanism in STABLE is analogous to trust region methods in RL, where updates are constrained by KL divergence to ensure policy stability. By projecting LoRA merges onto the feasible region defined by the forgetting budget, STABLE enforces bounded drift and mitigates catastrophic forgetting.

Trade-offs

• Strictness vs. Adaptability: Tighter gating thresholds reduce forgetting but may reject beneficial edits, limiting adaptability.
• Computational Overhead: Gating introduces additional evaluation cost per candidate edit, which may be significant in large-scale or real-time deployments.
• Metric Selection: EM-based gating yields superior factual retention, but bits and KL-based gating offer alternative trade-offs between confidence and distributional alignment.

Limitations

• Evaluation focused on SQuAD-style factual tasks; broader benchmarks are needed for generality.
• Fixed forgetting budgets may not optimally balance stability and adaptation across diverse contexts.

Future Directions

• Adaptive Budgets: Dynamic adjustment of forgetting thresholds based on task importance or context.
• Streaming and Real-Time Adaptation: Efficient gating mechanisms for continuous, high-throughput environments.
• Integration with Self-Directed Editing: Unified frameworks combining autonomous edit generation and acceptance control.

Conclusion

STABLE provides a principled approach to gated continual learning in LLMs, leveraging PEFT via LoRA and explicit gating to constrain catastrophic forgetting. Empirical results demonstrate that gating strategy significantly impacts the trade-off between knowledge retention and adaptability, with EM-based gating achieving the highest cumulative performance. The framework’s constrained optimization perspective and robust empirical validation position it as a practical solution for long-lived, incrementally updated LLMs. Future work should explore adaptive gating, broader task coverage, and efficient deployment strategies to further enhance the reliability of continual learning in LLMs.