Curriculum Unlearning Methods

• Curriculum unlearning is a staged approach that gradually schedules data forgetting to mitigate catastrophic forgetting while preserving overall model utility.
• It employs diverse strategies such as sequential training on balanced user groups, confidence-based partitioning, and adversarial gating to regulate unlearning intensity.
• This method has been applied in recommendation systems, image classification, and large language models, demonstrating improved unlearning efficiency and robust retention.

Searching arXiv for papers on curriculum unlearning and closely related machine unlearning work. arxiv_search(query="curriculum unlearning machine unlearning", max_results=10, sort_by="relevance") Curriculum unlearning denotes a family of machine unlearning procedures in which forgetting is staged progressively rather than imposed uniformly from the outset. In the supplied literature, this progression appears in several forms: sequential training over balanced user groups in erasable recommendation, easy-to-hard scheduling of forgetting samples in approximate unlearning, and gradient-norm-based gating of adversarial pressure in LLM unlearning (Li et al., 2022, Miao et al., 18 Sep 2025, Yang et al., 21 Aug 2025, Li et al., 2 Feb 2026). Across these settings, the common motivation is explicit: aggressive unlearning can induce catastrophic forgetting or degrade recommendation performance, whereas conservative interventions can leave residual knowledge or approximate erasure that remains insufficiently robust (Li et al., 2 Feb 2026, Miao et al., 18 Sep 2025).

1. Conceptual scope and recurring motivation

Machine unlearning is presented as the task of removing the influence of a specified subset while preserving performance on the complementary data. In LLMs, the goal is to eliminate traces of a specified “forget” dataset $\mathcal{D}_f$ while preserving performance on a complementary “retain” dataset $\mathcal{D}_r$ (Li et al., 2 Feb 2026). In CUFG, the forgetting set is denoted $D_n$ and the retained complement is $D_r = D \setminus D_n$, with approximate unlearning seeking parameters that maintain low loss on $D_r$ and “push away” from $D_n$ (Miao et al., 18 Sep 2025). In session-based recommendation, CAU defines $D_{\text{unlearn}} \subset D$ and $D_{\text{train}} = D \setminus D_{\text{unlearn}}$, with the explicit two-fold goal of removing the influence of $D_{\text{unlearn}}$ while retaining as much of the original model’s performance on $D_{\text{train}}$ as possible (Yang et al., 21 Aug 2025).

The rationale for a curriculum is also stated directly. CUFG argues that some forgetting samples are easier to “erase” because the model has lower confidence on them, and that it is destabilizing to present all forgetting instructions at once (Miao et al., 18 Sep 2025). CAU states that existing studies fail to consider the ordering of unlearning samples when simultaneously processing multiple unlearning requests, leading to sub-optimal recommendation performance and unlearning effect (Yang et al., 21 Aug 2025). AGT$\mathcal{D}_r$0 formulates the issue as a stability-robustness dilemma: aggressive forgetting can collapse model capabilities, while conservative forgetting may merely “mask” the unwanted knowledge, leaving it recoverable under adversarial conditions (Li et al., 2 Feb 2026).

This suggests that “curriculum unlearning” is not a single algorithmic template. Rather, it is a scheduling principle applied to different control variables: user groups, forgetting samples, mini-batches, or adversarial perturbations.

2. Early curriculum structure in erasable recommendation

A clear precursor appears in LASER, a general erasable recommendation framework for recommender systems that consists of Group module and SeqTrain module (Li et al., 2022). The Group module partitions users into balanced groups based on the similarity of collaborative embedding learned via hypergraph, and the SeqTrain module trains the model sequentially on all groups with curriculum learning (Li et al., 2022).

The framework is built on a hypergraph $\mathcal{D}_r$1 in which $\mathcal{D}_r$2 corresponds one-to-one to users, each hyperedge $\mathcal{D}_r$3 is defined from the set of users reachable from $\mathcal{D}_r$4 by any path of length $\mathcal{D}_r$5 in the original user-item bipartite graph, and the weight $\mathcal{D}_r$6 is the average rating that $\mathcal{D}_r$7 gave to any item in the bipartite $\mathcal{D}_r$8-neighborhood of $\mathcal{D}_r$9; in practice $D_n$0 (Li et al., 2022). Random walks over this hypergraph produce user sequences, after which Word2Vec is applied to obtain a user-embedding matrix $D_n$1, with experiments using $D_n$2, $D_n$3, and hidden-dim $D_n$4 (Li et al., 2022).

Balanced grouping minimizes total within-group embedding-distance,

$D_n$5

subject to $D_n$6 (Li et al., 2022). The curriculum schedule is then defined through collaborative cohesion,

$D_n$7

with groups sorted in descending order of $D_n$8, producing the training sequence $D_n$9 (Li et al., 2022).

The theoretical analysis states that if groups with higher $D_r = D \setminus D_n$0 tend to receive higher $D_r = D \setminus D_n$1, then curriculum with $D_r = D \setminus D_n$2 strictly increases the utility (Li et al., 2022). The same paper also links balanced grouping to unlearning-time complexity: for large $D_r = D \setminus D_n$3, the expected unlearning cost is about $D_r = D \setminus D_n$4 versus $D_r = D \setminus D_n$5 if no grouping, yielding a $D_r = D \setminus D_n$6 speedup in expectation when $D_r = D \setminus D_n$7 (Li et al., 2022).

Empirically, on MovieLens 1M and Amazon Digital Music, LASER variants outperform C-SISA in recommendation utility for the reported setting $D_r = D \setminus D_n$8, $D_r = D \setminus D_n$9, model=DMF (Li et al., 2022). On ML with DMF and $D_r$0, wall-clock time to unlearn $D_r$1 of the last group is reported as 135 s for Retrain, 12 s for C-SISA, and 13 s for L-CBKM, so LASER matches C-SISA speed while retaining higher utility (Li et al., 2022). Under the ablation “Curriculum vs. reverse,” Easy$D_r$2Hard SeqTrain yields NDCG$D_r$3 whereas Hard$D_r$4Easy yields NDCG$D_r$5 (Li et al., 2022).

3. Forgetting-gradient curricula in approximate unlearning

CUFG presents curriculum unlearning explicitly as a framework for fine-tuning-based approximate unlearning (Miao et al., 18 Sep 2025). Its starting point is the saddle-type objective

$D_r$6

with retain gradient $D_r$7 and forgetting-set gradient $D_r$8 (Miao et al., 18 Sep 2025). The paper defines the forgetting gradient

$D_r$9

that is, the direction in which loss on the to-be-forgotten data increases most rapidly (Miao et al., 18 Sep 2025).

CUFG’s first mechanism is the Gradient Corrector. Let $D_n$0 be the angle between $D_n$1 and $D_n$2, and let $D_n$3 be a threshold. The corrected unlearning gradient is

$D_n$4

followed by the update $D_n$5 (Miao et al., 18 Sep 2025). The stated purpose is to adjust the unlearning gradient only when the current fine-tuning step has too large a component in the “wrong” direction, toward the to-be-forgotten distribution (Miao et al., 18 Sep 2025).

Its second mechanism is an easy-to-hard curriculum over the forgetting set. For each $D_n$6, the difficulty score is

$D_n$7

the softmax probability assigned to the true label by the trained model (Miao et al., 18 Sep 2025). The forgetting set is sorted in ascending order of $D_n$8, from lowest confidence to easiest, and then sliced into $D_n$9 contiguous groups $D_{\text{unlearn}} \subset D$0 processed sequentially (Miao et al., 18 Sep 2025). In the end-to-end algorithm, epoch budget is divided across these subsets, and each inner update combines the current forgetting gradient on $D_{\text{unlearn}} \subset D$1 with retain mini-batch gradients on $D_{\text{unlearn}} \subset D$2 (Miao et al., 18 Sep 2025).

The paper’s interpretation is explicit. Retrain from scratch on $D_{\text{unlearn}} \subset D$3 is the gold standard: stable but extremely costly. Naïve fine-tuning on $D_{\text{unlearn}} \subset D$4 is stable but forgets too weakly. Aggressive methods such as gradient ascent, random-labeling, and influence functions can over-correct, causing gradient explosion or vanishing or loss of generalization (Miao et al., 18 Sep 2025). CUFG is designed to bridge these extremes by steering each update “away” from the forgotten distribution only when necessary and by spreading the forgetting task over a curriculum from easy to hard samples (Miao et al., 18 Sep 2025).

On CIFAR-10, CIFAR-100, and SVHN with ResNet-18 and VGG-16, the reported metrics are UA, RA, TA, MIA, RTE, and Avg.Gap (Miao et al., 18 Sep 2025). Table 1 is summarized as showing that CUFG achieves the smallest Avg.Gap in all settings. For Random 10\% on ResNet-18, Retrain has UA$D_{\text{unlearn}} \subset D$5, RA$D_{\text{unlearn}} \subset D$6, TA$D_{\text{unlearn}} \subset D$7, MIA$D_{\text{unlearn}} \subset D$8, Avg.Gap$D_{\text{unlearn}} \subset D$9, while CUFG has UA$D_{\text{train}} = D \setminus D_{\text{unlearn}}$0, RA$D_{\text{train}} = D \setminus D_{\text{unlearn}}$1, TA$D_{\text{train}} = D \setminus D_{\text{unlearn}}$2, MIA$D_{\text{train}} = D \setminus D_{\text{unlearn}}$3, Avg.Gap$D_{\text{train}} = D \setminus D_{\text{unlearn}}$4 (Miao et al., 18 Sep 2025). For Random 50\% on ResNet-18, Retrain has UA$D_{\text{train}} = D \setminus D_{\text{unlearn}}$5, RA$D_{\text{train}} = D \setminus D_{\text{unlearn}}$6, TA$D_{\text{train}} = D \setminus D_{\text{unlearn}}$7, MIA$D_{\text{train}} = D \setminus D_{\text{unlearn}}$8, Avg.Gap$D_{\text{train}} = D \setminus D_{\text{unlearn}}$9, while CUFG has UA$D_{\text{unlearn}}$0, RA$D_{\text{unlearn}}$1, TA$D_{\text{unlearn}}$2, MIA$D_{\text{unlearn}}$3, Avg.Gap$D_{\text{unlearn}}$4 (Miao et al., 18 Sep 2025). Efficiency for Random 10\% on ResNet-18 is reported as Retrain RTE$D_{\text{unlearn}}$5 min and CUFG$D_{\text{unlearn}}$6 min, described as a $D_{\text{unlearn}}$7 speedup over scratch (Miao et al., 18 Sep 2025).

4. Pareto-based curriculum approximate unlearning for session-based recommendation

CAU extends curriculum unlearning to session-based recommendation and makes the scheduling problem part of a multi-objective optimization formulation (Yang et al., 21 Aug 2025). The model $D_{\text{unlearn}}$8 may be GRU4Rec, SASRec, or BERT4Rec, trained on full dataset $D_{\text{unlearn}}$9, with a subset $D_{\text{train}}$0 of interactions to be forgotten and remaining training data $D_{\text{train}}$1 (Yang et al., 21 Aug 2025).

The unlearning term is defined as

$D_{\text{train}}$2

which, when minimized by gradient descent, corresponds to gradient ascent on the negative log-likelihood of the unlearning item (Yang et al., 21 Aug 2025). CAU uses two retaining terms: the normal prediction loss

$D_{\text{train}}$3

and the consistency loss

$D_{\text{train}}$4

combined in

$D_{\text{train}}$5

For implementation, the scalarized sum is

$D_{\text{train}}$6

but the paper emphasizes that CAU seeks a Pareto-Optimal solution via MGDA rather than fixing $D_{\text{train}}$7 by hand (Yang et al., 21 Aug 2025).

MGDA solves

$D_{\text{train}}$8

where $D_{\text{train}}$9, $\mathcal{D}_r$00, and $\mathcal{D}_r$01 are the gradients of the three objectives (Yang et al., 21 Aug 2025). This is the mechanism by which CAU states that it seeks the Pareto-Optimal solution, achieving effective unlearning with trivial sacrifice on recommendation performance (Yang et al., 21 Aug 2025).

The curriculum itself is built from two difficulty metrics. The gradient unlearning difficulty is

$\mathcal{D}_r$02

with lower $\mathcal{D}_r$03 meaning easier sample (Yang et al., 21 Aug 2025). The embedding unlearning difficulty is

$\mathcal{D}_r$04

where a larger dot-product means the model is very confident in $\mathcal{D}_r$05 and hence the sample is hard to forget; scheduling proceeds ascendingly by $\mathcal{D}_r$06 (Yang et al., 21 Aug 2025).

CAU offers two curriculum strategies. Hard-sampling sorts all of $\mathcal{D}_r$07 ascendingly by difficulty, partitions it into consecutive batches, and processes the batches from easiest to hardest (Yang et al., 21 Aug 2025). Soft-sampling instead defines a time-dependent selection probability

$\mathcal{D}_r$08

so that early steps emphasize low-difficulty samples and late steps emphasize high-difficulty samples (Yang et al., 21 Aug 2025).

The reported metrics are Recall@K, NDCG@K, Hit$\mathcal{D}_r$09@K, and the combined $\mathcal{D}_r$10 score (Yang et al., 21 Aug 2025). CAU is reported to preserve Recall and NDCG within $\mathcal{D}_r$11–$\mathcal{D}_r$12 of the original full model, whereas exact-unlearning baselines SISA and SRU drop by $\mathcal{D}_r$13–$\mathcal{D}_r$14 (Yang et al., 21 Aug 2025). In unlearning effect, CAU achieves Hit$\mathcal{D}_r$15 nearly as low as Retrain but with only $\mathcal{D}_r$16–$\mathcal{D}_r$17 of the computation, approximately $\mathcal{D}_r$18 faster than SISA/SRU and approximately $\mathcal{D}_r$19 faster than retrain (Yang et al., 21 Aug 2025). Soft-sampling usually edges out hard-sampling by $\mathcal{D}_r$20–$\mathcal{D}_r$21 in $\mathcal{D}_r$22 (Yang et al., 21 Aug 2025).

5. Curriculum as adversarial gating in LLM unlearning

AGT$\mathcal{D}_r$23 generalizes the notion of curriculum unlearning beyond data ordering by deferring and conditionally enabling adversarial pressure in latent space (Li et al., 2 Feb 2026). The method combines Adaptive Orthogonality (AO) and Adversarial Gating Training (AGT) in a unified framework for robust and stabilized LLM unlearning (Li et al., 2 Feb 2026).

AGT casts unlearning as the bi-level problem

$\mathcal{D}_r$24

where

$\mathcal{D}_r$25

(Li et al., 2 Feb 2026). The inner loop seeks $\mathcal{D}_r$26 by PGD,

$\mathcal{D}_r$27

and the outer loop updates

$\mathcal{D}_r$28

(Li et al., 2 Feb 2026).

The curriculum is implemented through a binary gate $\mathcal{D}_r$29:

$\mathcal{D}_r$30

During the first $\mathcal{D}_r$31 steps there is no adversary, and afterward the adversary is engaged only when the landscape has sufficiently flattened (Li et al., 2 Feb 2026). The paper states that this Gradient-Norm-Based Gating prevents early “gradient explosion” and enables a controlled ramp-up of adversarial difficulty (Li et al., 2 Feb 2026).

AO addresses gradient conflict between forgetting and retention. With

$\mathcal{D}_r$32

AO defines

$\mathcal{D}_r$33

When $\mathcal{D}_r$34 and $\mathcal{D}_r$35 are non-conflicting, $\mathcal{D}_r$36; when they conflict, the term grows and softly rotates the net update toward an orthogonal decomposition that minimizes harm to retained knowledge (Li et al., 2 Feb 2026).

The empirical validation explicitly attributes a major role to the curriculum schedule. On TOFU, removing the gating is reported to lead to training instability, higher KUR ($\mathcal{D}_r$37 vs. $\mathcal{D}_r$38) and lower utility (Model Utility $\mathcal{D}_r$39 vs. $\mathcal{D}_r$40), while eliminating warm-up similarly degrades performance (Li et al., 2 Feb 2026). Sensitivity experiments identify a warm-up of one epoch and $\mathcal{D}_r$41 as striking the best balance (Li et al., 2 Feb 2026). The abstract reports a superior trade-off between unlearning efficacy, with KUR $\mathcal{D}_r$42, and model utility, with MMLU $\mathcal{D}_r$43, and the benchmarks listed are TOFU, MUSE, and WMDP (Li et al., 2 Feb 2026).

A notable feature of this formulation is that the curriculum variable is not sample order but adversarial intensity. This suggests that curriculum unlearning can include staged optimization regimes in which the model first stabilizes and only later confronts stronger recovery attempts.

6. Comparative patterns, evaluation, and recurring controversies

The supplied literature uses different operationalizations of “difficulty,” and the differences are substantive rather than terminological.

Work	Difficulty or curriculum signal	Progression
LASER (Li et al., 2022)	collaborative cohesion $\mathcal{D}_r$44	groups sorted in descending order of $\mathcal{D}_r$45
CUFG (Miao et al., 18 Sep 2025)	$\mathcal{D}_r$46	lowest confidence $\mathcal{D}_r$47 easiest
CAU (Yang et al., 21 Aug 2025)	$\mathcal{D}_r$48 or $\mathcal{D}_r$49	ascending difficulty; hard- or soft-sampling
AGT$\mathcal{D}_r$50 (Li et al., 2 Feb 2026)	warm-up and $\mathcal{D}_r$51	no adversary early, adversary later

One common misconception is that curriculum unlearning is merely a heuristic for sample ordering. The evidence in the supplied papers is narrower and more heterogeneous. LASER uses curriculum at the level of balanced user groups (Li et al., 2022). CUFG uses a confidence-based partition of forgetting samples (Miao et al., 18 Sep 2025). CAU uses per-sample difficulty plus MGDA-based weighting of multiple objectives (Yang et al., 21 Aug 2025). AGT$\mathcal{D}_r$52 uses gating over adversarial perturbation rather than over examples (Li et al., 2 Feb 2026).

Another recurring controversy concerns the relation between forgetting strength and retained utility. AGT$\mathcal{D}_r$53 states this as a “fundamental dilemma”: aggressive unlearning often induces catastrophic forgetting, whereas conservative strategies risk superficial forgetting and adversarial recovery (Li et al., 2 Feb 2026). CUFG describes the same tension by contrasting stable but weak naïve fine-tuning against destabilizing aggressive methods such as gradient ascent and influence functions (Miao et al., 18 Sep 2025). CAU reports that removing the curriculum slows convergence and lowers ultimate $\mathcal{D}_r$54 by $\mathcal{D}_r$55–$\mathcal{D}_r$56, while removing $\mathcal{D}_r$57 and using pure GA collapses recommendation performance (Yang et al., 21 Aug 2025). LASER frames the issue through the collaborative structure of recommendation: existing MU methods ignore collaborative information across users and items, so they cannot be directly applied into recommendation (Li et al., 2022).

The evaluation protocols also differ sharply by domain. CUFG reports UA, RA, TA, MIA, RTE, and Avg.Gap on image classification benchmarks (Miao et al., 18 Sep 2025). CAU uses Recall@K, NDCG@K, Hit$\mathcal{D}_r$58@K, and $\mathcal{D}_r$59 for session-based recommendation (Yang et al., 21 Aug 2025). LASER uses NDCG@10 and HR@10 along with unlearning time (Li et al., 2022). AGT$\mathcal{D}_r$60 reports KUR, Model Utility, and MMLU, with additional discussion of robustness under latent perturbations (Li et al., 2 Feb 2026). This suggests that “curriculum unlearning” is best understood as a training principle whose empirical signature depends on the benchmark family and threat model, rather than as a single metric-defined paradigm.

7. Research trajectory and implications

Within the supplied papers, curriculum unlearning develops from recommendation-specific sequencing to more explicit and general formulations. LASER already uses curriculum learning in SeqTrain and gives a theorem linking prior-weighted utility to curriculum ordering by collaborative cohesion (Li et al., 2022). CUFG elevates the phrase “curriculum unlearning” to the central method name and states that the concept has substantial research potential and offers forward-looking insights for the development of the MU field (Miao et al., 18 Sep 2025). CAU shows that curriculum can be coupled to Pareto-based multi-objective optimization and to stochastic sampling rules that vary over time (Yang et al., 21 Aug 2025). AGT$\mathcal{D}_r$61 shows that curriculum can regulate not only which data are forgotten first, but also when the model is exposed to worst-case latent perturbations (Li et al., 2 Feb 2026).

A plausible implication is that curriculum unlearning functions as a control layer over approximate unlearning rather than as a replacement for the underlying forgetting mechanism. In the supplied literature, the forgetting mechanism may be sequential retraining over user groups, a gradient corrector guided by forgetting gradients, a GA term combined with retaining losses and MGDA, or latent-space adversarial training with adaptive orthogonality (Li et al., 2022, Miao et al., 18 Sep 2025, Yang et al., 21 Aug 2025, Li et al., 2 Feb 2026). The curriculum component then modulates the order, difficulty, or activation of those mechanisms to improve stability, efficiency, or robustness.

Taken together, these works depict curriculum unlearning as a structured response to a recurring optimization problem: how to remove targeted knowledge without either collapsing general utility or leaving a residual footprint that is easily recovered.