Relearning Convergence Delay

• The paper introduces RCD as a tool to measure the integral of excess error during retraining, capturing both immediate degradation and persistent forgetting.
• RCD is theoretically grounded using gradient descent analysis under strong convexity and smoothness, with exponential convergence in its discrete approximation.
• The influence eliminating unlearning (IEU) framework leverages gradient ascent with noise injection to balance retaining-set performance and maximizing forgetting resistance.

Searching arXiv for the cited papers to ground the article in the latest available records. Relearning convergence delay (RCD) is a machine-unlearning metric that measures how slowly an unlearned model can recover performance on a forgetting set when retraining is resumed from the unlearned weights. In its formal definition, RCD is the integral over retraining time of the excess evaluation error on the forgetting set relative to the optimal model for that set, so it quantifies not only immediate post-unlearning degradation but also the persistence of forgetting under relearning. In the formulation introduced for unlearning, RCD is intended to capture both changes in weight space and prediction space, and to assess the risk that forgotten data can be recovered from the unlearned model (Tran et al., 10 Apr 2026).

1. Formal definition and notation

The standard setup distinguishes a full training set $\mathcal{D}^{train} = \{x_i,y_i\}$, a forgetting set $\mathcal{D}_f$, and a retaining set $$\mathcal{D}_r = \mathcal{D}^{train}\setminus \mathcal{D}_f$$. A model is written as $f(x,\theta)$ with parameters $\theta \in \mathbb{R}^d$, and an iterative training algorithm is denoted by $\mathcal{T}(\theta_0,\mathcal{D},t)$, meaning that training starts from $\theta_0$ and runs for $t$ steps on dataset $\mathcal{D}$. The optimal parameters for a dataset $\mathcal{D}$ are denoted by

$\mathcal{D}_f$0

and the evaluation error on $\mathcal{D}_f$1 is denoted by $\mathcal{D}_f$2.

With this notation, the continuous form of relearning convergence delay for an unlearned model $\mathcal{D}_f$3 is

$\mathcal{D}_f$4

The corresponding discrete $\mathcal{D}_f$5-step approximation is

$\mathcal{D}_f$6

This definition makes RCD an area-under-the-relearning-curve quantity rather than a pointwise score. A model that briefly performs poorly on $\mathcal{D}_f$7 but rapidly relearns can have substantially smaller RCD than a model whose error remains elevated over many updates. A common simplification in unlearning evaluation is to focus only on model predictions; the introduction of RCD is motivated by the claim that prediction-only metrics limit insight into the model’s true underlying data characteristics (Tran et al., 10 Apr 2026).

2. Theoretical characterization

The theory for RCD is developed under three assumptions. First, the training procedure converges in the sense that $\mathcal{D}_f$8 for any $\mathcal{D}_f$9. Second, the condition number of the Hessian is assumed to be monotone along training: if $$\mathcal{D}_r = \mathcal{D}^{train}\setminus \mathcal{D}_f$$0 has ordered eigenvalues $$\mathcal{D}_r = \mathcal{D}^{train}\setminus \mathcal{D}_f$$1, then $$\mathcal{D}_r = \mathcal{D}^{train}\setminus \mathcal{D}_f$$2 is nonincreasing in $$\mathcal{D}_r = \mathcal{D}^{train}\setminus \mathcal{D}_f$$3. Third, under $$\mathcal{D}_r = \mathcal{D}^{train}\setminus \mathcal{D}_f$$4-strong convexity and $$\mathcal{D}_r = \mathcal{D}^{train}\setminus \mathcal{D}_f$$5-smoothness, the eigenvalues satisfy

$$\mathcal{D}_r = \mathcal{D}^{train}\setminus \mathcal{D}_f$$6

with $$\mathcal{D}_r = \mathcal{D}^{train}\setminus \mathcal{D}_f$$7 bounding the initial condition number.

Under gradient descent with step size $$\mathcal{D}_r = \mathcal{D}^{train}\setminus \mathcal{D}_f$$8, RCD admits the upper bound

$$\mathcal{D}_r = \mathcal{D}^{train}\setminus \mathcal{D}_f$$9

Under $f(x,\theta)$0-strongly convex and $f(x,\theta)$1-smooth losses, this yields the corollary

$f(x,\theta)$2

The proof sketch uses the descent lemma together with the PL-style bound

$f(x,\theta)$3

The discrete approximation converges exponentially fast: $f(x,\theta)$4 Accordingly, only $f(x,\theta)$5 steps are needed to approximate RCD within $f(x,\theta)$6 (Tran et al., 10 Apr 2026).

These results establish two points of principle. First, RCD is controlled jointly by excess loss and the local condition number, so it is not reducible to post-unlearning accuracy alone. Second, because the discrete approximation converges exponentially, the metric is computationally tractable in regimes where exact infinite-horizon retraining would be impractical.

3. Influence Eliminating Unlearning

The principal algorithmic framework built around RCD is Influence Eliminating Unlearning (IEU). Its high-level objective is to produce unlearned parameters $f(x,\theta)$7 that simultaneously minimize retained-set loss and maximize RCD on the forgetting set. In the paper’s formulation, the forgetting objective is realized through a combination of retaining-set gradient descent, forgetting-set gradient ascent, and noisy regularization (Tran et al., 10 Apr 2026).

The retaining-set utility term is the standard descent direction $f(x,\theta)$8. Forgetting pressure is introduced through gradient ascent on the forgetting set,

$f(x,\theta)$9

with forgetting-set step size $\theta \in \mathbb{R}^d$0 and $\theta \in \mathbb{R}^d$1. The third component is an iterative re-initialization mechanism: $\theta \in \mathbb{R}^d$2 This noise injection is described as raising the condition number on $\theta \in \mathbb{R}^d$3 and slowing convergence.

The full IEU update rule is

$\theta \in \mathbb{R}^d$4

where $\theta \in \mathbb{R}^d$5, $\theta \in \mathbb{R}^d$6, and $\theta \in \mathbb{R}^d$7. The hyperparameters have distinct roles: $\theta \in \mathbb{R}^d$8 controls the strength of forgetting-set ascent, $\theta \in \mathbb{R}^d$9 controls the speed of re-initialization noise, and $\mathcal{T}(\theta_0,\mathcal{D},t)$0 is the base learning rate on $\mathcal{T}(\theta_0,\mathcal{D},t)$1.

Theoretical control of retaining-set utility is provided by a convergence bound. Under $\mathcal{T}(\theta_0,\mathcal{D},t)$2-Lipschitz, $\mathcal{T}(\theta_0,\mathcal{D},t)$3-strongly convex, $\mathcal{T}(\theta_0,\mathcal{D},t)$4-smooth $\mathcal{T}(\theta_0,\mathcal{D},t)$5, step size $\mathcal{T}(\theta_0,\mathcal{D},t)$6, and $\mathcal{T}(\theta_0,\mathcal{D},t)$7, the expected retaining-loss error satisfies

$\mathcal{T}(\theta_0,\mathcal{D},t)$8

The first term decays exponentially in $\mathcal{T}(\theta_0,\mathcal{D},t)$9, while the remaining terms quantify the penalty induced by gradient ascent and noise injection. This makes explicit that maximizing forgetting resistance is coupled to a utility cost whose magnitude is shaped by $\theta_0$0 and $\theta_0$1.

4. Empirical behavior

The empirical study spans both classification and generative unlearning. In classification, the evaluation includes CIFAR-10, CIFAR-100, and TinyImageNet; architectures include ResNet-50 and ViT; baselines include FT, RL, SCRUB, and SALUN; and reported metrics include $\theta_0$2, $\theta_0$3, test accuracy, MIA ASR, Avg. Gap vs retrain, and $\theta_0$4 (Tran et al., 10 Apr 2026).

For TinyImageNet with ResNet under random 30% forgetting, the reported values are:

Method	Avg.Gap	$\theta_0$5
Retraining	0	$\theta_0$6
FT	$\theta_0$7	$\theta_0$8
RL	$\theta_0$9	$t$0
SCRUB	$t$1	$t$2
IEU w/GA	$t$3	$t$4
IEU w/Noisy	$t$5	$t$6
IEU w/GA+Noisy	$t$7	$t$8

The associated trade-off plot places IEU variants near the retrain corner, meaning high RCD together with low Avg.Gap. This indicates that strong resistance to relearning need not require large degradation on the retaining distribution.

The generative setting studies Stable Diffusion NSFW unlearning with base model SD v1.4. ESD and SALUN are evaluated with and without GA and Noisy variants; I2P nudity prompts are used to measure Nudity-ratio, unrelated ImageNette classes are used to measure FID, and relearning with Adam yields $t$9.

Framework	Nudity / FID	$\mathcal{D}$0
ESD	0.45 / 27.3	$\mathcal{D}$1
ESD+Noisy	0.12 / 29.4	$\mathcal{D}$2
SALUN	0.58 / 25.8	$\mathcal{D}$3
SALUN+Noisy	0.15 / 26.1	$\mathcal{D}$4
ESD+GA+Noisy	0.10 / 27.0	$\mathcal{D}$5

In these results, Noisy-only and GA+Noisy consistently reduce Nudity and raise $\mathcal{D}$6 while preserving or improving FID. The ablation studies further report that smaller step sizes for $\mathcal{D}$7 calculation marginally increase RCD but leave method rankings stable; that $\mathcal{D}$8 values that are too low lead to model collapse and values that are too high lead to weak forgetting; and that $\mathcal{D}$9–$\mathcal{D}$0 balances retention and forgetting on the reported ViT CIFAR-100 setting. The Pareto curve indicates an explicit privacy-utility trade-off through tuning $\mathcal{D}$1 (Tran et al., 10 Apr 2026).

5. Related notions of interference and delay

A related but distinct precursor appears in recurrent-network learning, where sequential updates can be obstructed by destructive interference. In the analysis of LMS and FORCE for memory and pattern-generation tasks, LMS is described as constantly obstructed by forgetting, manifested as destruction of dynamical objects from previous trials, whereas FORCE uses a whitened update through

$\mathcal{D}$2

and thereby protects previously learned directions (Beer et al., 2018). Trial-trial overlap is measured by

$\mathcal{D}$3

and higher overlap produces slower net convergence and stronger one-step-back forgetting. The reported scale difference is substantial: FORCE uses approximately one trial per new target, whereas LMS can require $\mathcal{D}$4 trials for two highly overlapping targets and can easily exceed $\mathcal{D}$5–$\mathcal{D}$6 total trials when memorizing five values. This suggests a conceptual continuity between RCD in unlearning and interference-driven relearning delay in sequential online learning, even though the formal metrics differ.

The word “delay” also has a separate meaning in optimization and dynamical-systems analysis. In asynchronous optimization, delay denotes stale information, with updates of the form

$\mathcal{D}$7

based on $\mathcal{D}$8, and delay-adaptive rules choose $\mathcal{D}$9 from the remaining budget

$\mathcal{D}_f$00

so that convergence depends on actual time-varying delays rather than conservative worst-case bounds (Wu et al., 2022). In delayed gradient descent, fixed delay $\mathcal{D}_f$01 still permits non-ergodic linear convergence under strong convexity or the PL condition, with admissible step sizes improved to $\mathcal{D}_f$02 for $\mathcal{D}_f$03, $\mathcal{D}_f$04 for $\mathcal{D}_f$05, and $\mathcal{D}_f$06 for large $\mathcal{D}_f$07 under PL (Choi et al., 2023). In continuous-time linear systems, fixed delay can even improve convergence rate when the critical eigenvalues lie in the sector $\mathcal{D}_f$08, with maximal gain bounded by $\mathcal{D}_f$09 (Moradian et al., 2018). These results are not about machine unlearning, but they clarify that “relearning convergence delay” is a specific metric of resistance to recovery, not a generic synonym for time-delayed optimization.

6. Interpretation, misconceptions, and open questions

RCD is designed to measure residual influence rather than only prediction mismatch. The underlying claim is that evaluation based solely on model predictions offers limited visibility into whether the forgotten data remain easily recoverable from the weights. Because the upper bound on RCD depends on both excess loss and the condition number, a high immediate forgetting-set loss alone does not certify high resistance to relearning; conversely, low Avg.Gap on the retaining set does not by itself establish that forgetting is deep (Tran et al., 10 Apr 2026).

The framework also has clear current limits. The theory is built on gradient descent, whereas practical systems often use Adam or other optimizers. The reported behavior of $\mathcal{D}_f$10 is informative but is explicitly noted to behave differently, and not strictly slower with IEU-GA. Beyond vision, scalability to NLP, graphs, and lifelong unlearning remains to be studied. Fairness, security, and legal compliance aspects of unlearning algorithms are likewise identified as open areas.

Within its present scope, RCD provides a technically grounded way to evaluate whether an unlearned model merely suppresses immediate evidence of a forgetting set or whether it also delays reacquisition of that set under renewed training. That distinction is the central contribution of the metric and the organizing principle behind IEU.