Source-Free Unlearning: Methods & Implications

• Source-free unlearning is a technique that removes the influence of specified data from a trained model without relying on the original training set.
• It leverages auxiliary information such as forget sets, cached activations, synthetic data, and proxy statistics to perform unlearning efficiently.
• The approach spans representation-space modifications, second-order approximations, and architecture-specific interventions to balance forgetting with retained accuracy.

Source-free unlearning denotes machine-unlearning settings in which a trained model must remove the influence of specified data, classes, concepts, domains, or corruptions without relying on the original training corpus at unlearning time. Across the recent literature, the term covers several closely related regimes: zero-shot unlearning, data-free unlearning, remaining-data-free unlearning, retain-free unlearning, corpus-free unlearning, and partially-blind unlearning. What unites these regimes is that unlearning is no longer framed as retraining on the retain set, but as an intervention using a trained model together with restricted auxiliary information such as the forget set, class names, proxy corrupted samples, cached activations, classifier weights, surrogate datasets, gradients, or synthetic data generated from the model itself (Shah et al., 2023).

1. Problem formulations and access regimes

The classical target of machine unlearning is a model that behaves like one retrained on the retain set only. Several papers restate this using the standard decomposition $\mathcal{D} = \mathcal{D}_r \cup \mathcal{D}_f$, where $\mathcal{D}_f$ is the forget set and $$\mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f$$. In class unlearning, one class $c$ is designated as the forget class, yielding

$$\mathcal{D}_{train}^{forget} = \{(x,y)\in \mathcal{D}_{train} \mid y=c\}, \qquad \mathcal{D}_{train}^{retain} = \mathcal{D}_{train} \setminus \mathcal{D}_{train}^{forget}.$$

The operational objective is to make the model behave as if it had not been trained on the forget class while preserving accuracy on $\mathcal{D}_{test}^{retain}$ (Shah et al., 2023).

The source-free literature differs primarily in what information remains available when the deletion request arrives. In remaining-data-free and retain-free settings, the retain set is unavailable at unlearning time, so the method must work from the pretrained model and either the forget data or auxiliary statistics (Cheng et al., 2024); (Zhao et al., 15 Apr 2026). In data-free unlearning, neither the original training data nor a similar proxy dataset is available, and only the trained original model $T(\cdot)$ is assumed (Zhang et al., 2024). In corpus-free LLM unlearning, the user need not provide a forget corpus; instead, only a lightweight anchor identifying a target entity is supplied, and the service provider has no access to the original training corpus at deployment time (Li et al., 15 Apr 2026).

Some papers define stricter variants. Source-free corrective machine unlearning assumes that the original training data are unavailable and that no forget set of identified corrupted training samples can be specified; only a small proxy corrupted set is available (Mozafari et al., 24 Nov 2025). Partially-blind unlearning assumes access to the trained model, the retain set, and auxiliary training information, but not the forget set itself (Newatia et al., 12 Apr 2026). Source-free domain adaptation unlearning studies the case where the source data are unavailable during adaptation, yet the source model leaks source-exclusive classes into the target domain (Devalapally et al., 9 Apr 2026).

This multiplicity of settings indicates that “source-free” is not a single protocol. In some works it means no original training set; in others it means no retain set; in others it means no forget corpus supplied by the requester; and in still others it means no retraining or fine-tuning. A plausible implication is that comparisons across papers are only meaningful once the access assumptions are aligned.

2. Representation-space and geometric unlearning

A major line of work treats source-free unlearning as a problem of modifying internal representations rather than full parameter vectors. In “Unlearning via Sparse Representations” (Shah et al., 2023), the central mechanism is a Discrete Key-Value Bottleneck (DKVB) with $C$ codebooks and $M$ key-value pairs per codebook. Inputs are mapped to $C$ heads, each head is quantized to the top-$\mathcal{D}_f$0 nearest keys, the corresponding values are retrieved and averaged, and unlearning is performed by masking selected key-value pairs so they cannot be chosen again. The paper emphasizes that keys are frozen, values are learnable, gradients do not flow from values back into keys, and the resulting representations are sparse, localized, and input-dependent. Because class information is concentrated in a relatively small subset of key-value activations, masking those codes can erase the forget class without retraining or fine-tuning. The proposed procedures, Unlearning via Examples (UvE) and Unlearning via Activations (UvA), both require no retraining, no fine-tuning, and negligible additional compute, with UvA permitting a setting where activations are cached during training so no original training data is needed at unlearning time (Shah et al., 2023).

A related but more explicitly information-theoretic approach appears in Representation Unlearning (Almudévar et al., 29 Jan 2026). This framework learns a small transformation $\mathcal{D}_f$1 on the model’s penultimate representation $\mathcal{D}_f$2, producing $\mathcal{D}_f$3, and imposes an information bottleneck that maximizes retention-related information while suppressing information about the forget data. In its zero-shot variant, the method assumes access to the forget set $\mathcal{D}_f$4, the trained classifier weights $\mathcal{D}_f$5, and class counts $\mathcal{D}_f$6, but no access to retain data $\mathcal{D}_f$7. The zero-shot retention surrogate is

$\mathcal{D}_f$8

and the zero-shot forget loss is

$\mathcal{D}_f$9

The method relies on Neural Collapse to treat class weight vectors as class prototypes. This makes unlearning possible without retain examples by using class prototypes as pseudo-retain anchors (Almudévar et al., 29 Jan 2026).

A third representation-level route is Label-Agnostic Forgetting (LAF) (Shen et al., 2024). LAF decomposes the model as $$\mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f$$0, models representation distributions with two VAEs, and performs unlearning at the extractor level rather than the classifier level. Its practical extractor-unlearning loss is

$$\mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f$$1

and it preserves predictive behavior via a contrastive representation-alignment loss $$\mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f$$2. The paper’s key claim is that the unlearning procedure itself does not use supervision information, making it a supervision-free / source-free framework at unlearning time (Shen et al., 2024).

These representation-space methods share a common view: forgetting is easier when the model stores the target information in a localizable geometry—discrete codes, class prototypes, or separable latent distributions—rather than diffusely across all parameters. This suggests that source-free unlearning is often as much an architectural question as a post hoc algorithmic one.

3. Data-free synthesis, corpus-free supervision, and proxy-based forgetting

A second family replaces unavailable source data with synthetic, generated, or proxy supervision. In “Toward Efficient Data-Free Unlearning” (Zhang et al., 2024), the difficulty is that a trained model must forget specified classes or data when neither the original training data nor a similar proxy dataset is available. The proposed Inhibited Synthetic PostFilter (ISPF) addresses the over-filtering problem of earlier data-free knowledge-distillation methods such as GKT. The method has two components: Inhibited Synthesis (IS), which discourages the generator from synthesizing forgetting-class samples, and PostFilter (PF), which modifies teacher logits after synthesis so that forgetting information is suppressed but retaining information from mixed synthetic samples is preserved. PF redistributes teacher logits using

$$\mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f$$3

followed by a logit transformation and KL distillation. The paper explicitly notes that modifying logits before softmax performs better than setting forgetting-class probability to zero and renormalizing the rest (Zhang et al., 2024).

In LLM unlearning, the forget set itself has become a bottleneck. “LLM Unlearning Without an Expert Curated Dataset” (Zhu et al., 8 Aug 2025) proposes a fully automated synthetic textbook pipeline that requires only a domain name such as “biosecurity,” “cybersecurity,” or “Harry Potter.” The pipeline uses GPT-4o-mini with temperature 0.7, generates 10 subdomains, then 20 bullet points per subdomain across four audience levels, then 5 textbook-style chapters per bullet point, yielding 4000 chapters total, from which the 20,000 longest sentences are selected as the forget set. The resulting synthetic datasets are then used with existing unlearning objectives such as RMU, RR, and ELM (Zhu et al., 8 Aug 2025). This is source-free in the practical sense that it removes the need for curated source documents or expert-built forget corpora.

A more model-internal variant is MAGE, a Memory-grAph Guided Erasure framework for user-minimized, corpus-free unlearning in LLMs (Li et al., 15 Apr 2026). Given only a lightweight anchor identifying a target entity $$\mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f$$4, MAGE repeatedly prompts the target LLM, extracts entities from responses, scores them by recall frequency,

$$\mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f$$5

builds a weighted local memory graph $$\mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f$$6, and then samples paths through the graph to synthesize QA-style forget examples. It also constructs a neighbor set $$\mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f$$7 from strong neighbors excluding $$\mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f$$8, thereby preserving nearby but non-target knowledge. MAGE is explicitly model-agnostic: it produces ordinary unlearning data that can be plugged into GA, NPO, GA+GD, or GA+KL without defining a new optimizer (Li et al., 15 Apr 2026).

Proxy-based supervision is also central to corrective settings. CUTS—Corrective Unlearning in Task Space—addresses the case where a corrupted model is available but the original training data are gone (Mozafari et al., 24 Nov 2025). It assumes a small proxy dataset $$\mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f$$9, briefly fine-tunes the corrupted model on the proxy to amplify corruption, defines a proxy task vector

$c$0

and subtracts a scaled multiple of this vector: $c$1 The method requires neither a retain set nor access to the original corrupted training samples (Mozafari et al., 24 Nov 2025).

Across these approaches, source-free unlearning becomes a supervision-construction problem: one either synthesizes training signals from the model, from a domain name, from a lightweight anchor, or from a corruption proxy. A plausible implication is that the quality of these synthetic or proxy signals can dominate the final forgetting–utility trade-off.

4. Newton-style, Hessian-based, and certified retain-free methods

Another major strand derives source-free updates from second-order approximations to retraining. These methods typically replace missing retain data with estimated curvature, stored second-order information, or surrogate statistics.

MU-Mis—Machine Unlearning by Minimizing input sensitivity—takes a different but still model-local approach (Cheng et al., 2024). The paper argues that a sample’s contribution to learning is reflected in the trained model’s sensitivity to that sample, and defines the unlearning loss

$c$2

Here $c$3 is the true class and $c$4 is a randomly selected irrelevant class. The method requires only the pretrained model and forgetting data, and is therefore remaining-data-free (Cheng et al., 2024).

Certified source-free unlearning appears in “A Certified Unlearning Approach without Access to Source Data” (Basaran et al., 6 Jun 2025). This framework uses a surrogate dataset $c$5 drawn from $c$6 to approximate inaccessible retain statistics. The surrogate-based Newton update is

$c$7

with

$c$8

The central certification result states

$c$9

and calibrated Gaussian noise with

$$\mathcal{D}_{train}^{forget} = \{(x,y)\in \mathcal{D}_{train} \mid y=c\}, \qquad \mathcal{D}_{train}^{retain} = \mathcal{D}_{train} \setminus \mathcal{D}_{train}^{forget}.$$0

yields $$\mathcal{D}_{train}^{forget} = \{(x,y)\in \mathcal{D}_{train} \mid y=c\}, \qquad \mathcal{D}_{train}^{retain} = \mathcal{D}_{train} \setminus \mathcal{D}_{train}^{forget}.$$1-certified unlearning (Basaran et al., 6 Jun 2025). The guarantee depends on the statistical distance between source and surrogate distributions, which the paper estimates heuristically using KL-based approximations, energy-based modeling, SGLD, and Donsker–Varadhan estimation.

“Towards Source-Free Machine Unlearning” develops a related zero-shot idea for linear classifiers and mixed-linear networks (Ahmed et al., 20 Aug 2025). The key difficulty is that classical Newton-style corrections require the Hessian of the retained data, yet $$\mathcal{D}_{train}^{forget} = \{(x,y)\in \mathcal{D}_{train} \mid y=c\}, \qquad \mathcal{D}_{train}^{retain} = \mathcal{D}_{train} \setminus \mathcal{D}_{train}^{forget}.$$2 is unavailable. The paper estimates the unknown retain Hessian from random perturbations around the optimum, forget-set gradients, and forget-side loss differences by solving a PSD-constrained optimization problem. Once $$\mathcal{D}_{train}^{forget} = \{(x,y)\in \mathcal{D}_{train} \mid y=c\}, \qquad \mathcal{D}_{train}^{retain} = \mathcal{D}_{train} \setminus \mathcal{D}_{train}^{forget}.$$3 is obtained, it substitutes into

$$\mathcal{D}_{train}^{forget} = \{(x,y)\in \mathcal{D}_{train} \mid y=c\}, \qquad \mathcal{D}_{train}^{retain} = \mathcal{D}_{train} \setminus \mathcal{D}_{train}^{forget}.$$4

The paper proves a Frobenius-norm bound on the Hessian-estimation error and a bound on the retained-data gradient at the unlearned model (Ahmed et al., 20 Aug 2025).

The most explicit retain-free second-order framework is WIN-U—Woodbury-Informed Newton-Unlearning (Zhao et al., 15 Apr 2026). Starting from the retraining first-order condition and a Taylor expansion around $$\mathcal{D}_{train}^{forget} = \{(x,y)\in \mathcal{D}_{train} \mid y=c\}, \qquad \mathcal{D}_{train}^{retain} = \mathcal{D}_{train} \setminus \mathcal{D}_{train}^{forget}.$$5, the paper derives

$$\mathcal{D}_{train}^{forget} = \{(x,y)\in \mathcal{D}_{train} \mid y=c\}, \qquad \mathcal{D}_{train}^{retain} = \mathcal{D}_{train} \setminus \mathcal{D}_{train}^{forget}.$$6

Using a generalized Gauss–Newton approximation and the Woodbury identity, it obtains the central update

$$\mathcal{D}_{train}^{forget} = \{(x,y)\in \mathcal{D}_{train} \mid y=c\}, \qquad \mathcal{D}_{train}^{retain} = \mathcal{D}_{train} \setminus \mathcal{D}_{train}^{forget}.$$7

WIN-U requires only the original optimum $$\mathcal{D}_{train}^{forget} = \{(x,y)\in \mathcal{D}_{train} \mid y=c\}, \qquad \mathcal{D}_{train}^{retain} = \mathcal{D}_{train} \setminus \mathcal{D}_{train}^{forget}.$$8, a precomputed inverse Hessian $$\mathcal{D}_{train}^{forget} = \{(x,y)\in \mathcal{D}_{train} \mid y=c\}, \qquad \mathcal{D}_{train}^{retain} = \mathcal{D}_{train} \setminus \mathcal{D}_{train}^{forget}.$$9, and the forget set $\mathcal{D}_{test}^{retain}$0; no retain data are needed during unlearning (Zhao et al., 15 Apr 2026).

These methods treat source-free unlearning as a local approximation problem: if retraining is the gold standard, then a sufficiently accurate estimate of the retain-side curvature or contribution can substitute for explicit access to the retain set. Their common limitation, stated in different ways across papers, is that the guarantees are strongest in convex, mixed-linear, or well-converged regimes.

5. Multimodal, domain-selective, and architecture-specific source-free unlearning

Recent work extends source-free unlearning beyond standard image classification and decoder-only LLMs. In pretrained multimodal encoders, “Selective, Controlled and Domain-Agnostic Unlearning in Pretrained CLIP: A Training- and Data-Free Approach” modifies only CLIP’s final visual projection matrix through a closed-form nullspace projection (Mishra et al., 16 Dec 2025). Given

$\mathcal{D}_{test}^{retain}$1

the method constructs a forget subspace from text embeddings $\mathcal{D}_{test}^{retain}$2, synthesized visual prototypes $\mathcal{D}_{test}^{retain}$3, and optionally residual domain embeddings $\mathcal{D}_{test}^{retain}$4, then sets

$\mathcal{D}_{test}^{retain}$5

where $\mathcal{D}_{test}^{retain}$6 projects onto the nullspace of the forget subspace. The paper defines three paradigms: global unlearning, domain-specific removal, and complete selective domain unlearning. For images from an unlearning domain, inference uses

$\mathcal{D}_{test}^{retain}$7

while untouched domains retain the original projection $\mathcal{D}_{test}^{retain}$8 (Mishra et al., 16 Dec 2025).

In vision-LLMs, HFRU—Object Hallucination-Free Reinforcement Unlearning for Vision-LLMs—is source-free in the practical sense that it does not revisit the original pretraining corpus or retrain from scratch (Jia et al., 8 May 2026). It operates on the vision encoder, not merely the decoder, using a two-stage pipeline: a cold-start alignment-disruption phase,

$\mathcal{D}_{test}^{retain}$9

followed by encoder-only GRPO optimization with a composite forget reward

$T(\cdot)$0

The abstraction reward is designed to encourage semantically valid substitutions via hypernyms and to mitigate hallucinations (Jia et al., 8 May 2026).

In diffusion models, “Unlearning the Unpromptable: Prompt-free Instance Unlearning in Diffusion Models” targets instance-level forgetting without relying on prompts, concept labels, or the original training data (Lee et al., 11 Mar 2026). The method constructs an edited surrogate image $T(\cdot)$1, defines a surrogate-induced noise target

$T(\cdot)$2

combines remember and forget losses with timestep-aware weighting

$T(\cdot)$3

and resolves conflicts by gradient surgery: $T(\cdot)$4 For Stable Diffusion 3, the paper explicitly states that the training dataset cannot be accessed and instead uses generated images from the same prompt as a remember set (Lee et al., 11 Mar 2026).

Source-free unlearning also appears in source-free domain adaptation. SCADA-UL—Unlearning Source-exclusive ClAsses in Domain Adaptation—addresses the leakage of source-exclusive classes into target-domain models (Devalapally et al., 9 Apr 2026). It defines forget classes $T(\cdot)$5, generates adversarial proxy samples for a forget class $T(\cdot)$6 by minimizing

$T(\cdot)$7

and then uses a rescaled labeling strategy

$T(\cdot)$8

inside a joint objective with the SFDA adaptation loss (Devalapally et al., 9 Apr 2026).

These methods indicate that source-free unlearning is now architecture-specific. Nullspace projection, RL on visual encoders, surrogate-based diffusion fine-tuning, and adversarial optimization during domain adaptation are not interchangeable procedures; each is tailored to how information is represented in the respective model family.

6. Evaluation, misconceptions, and unresolved issues

Evaluation protocols in source-free unlearning vary by task but usually combine a forgetting metric, a utility metric, and some privacy or relearning metric. In class-unlearning settings, the primary quantities are forget accuracy $T(\cdot)$9 and retain accuracy $C$0, with the target often phrased as forget accuracy $C$1 while retain accuracy stays close to the original model (Shah et al., 2023). Data-free and representation-based vision papers also use Test CE, Membership Inference Attack (MIA) accuracy or AUC, and the Anamnesis Index (AIN) for relearning speed (Zhang et al., 2024); (Almudévar et al., 29 Jan 2026). Certified approaches evaluate train/test/retain/forget accuracy, MIA near chance, relearn time (RT), and forget-score variants (Basaran et al., 6 Jun 2025). LLM benchmarks add benchmark-specific aggregates such as Model Utility, Forget Efficacy, VerbMem, KnowMem, PrivLeak, and task-specific scores on TOFU, MUSE, WMDP, RWKU, and ORT (Zade et al., 1 Mar 2026); (Ye et al., 9 Jun 2025).

One recurrent misconception is that source-free necessarily means data-free. Several papers explicitly do not make that claim. MU-Mis is remaining-data-free but still requires the forgetting data (Cheng et al., 2024). Representation Unlearning’s zero-shot variant is source-free with respect to the retain data, but still assumes access to the forget set, classifier weights, and class-count metadata (Almudévar et al., 29 Jan 2026). HFRU is source-free with respect to original training data, yet still depends on task-specific forget and retain examples plus manually constructed synonym and hypernym sets (Jia et al., 8 May 2026). Reload is forget set-free, but not data-free, because it uses the retain set and cached full-dataset gradients (Newatia et al., 12 Apr 2026). The literature therefore distinguishes between the absence of source data, retain data, forget data, and labels, rather than collapsing them into a single binary property.

A second controversy concerns whether forgetting has actually occurred or whether the method has only induced refusal, lexical suppression, or brittle form-dependent behavior. Attention Smoothing Unlearning (ASU) argues that many prior methods produce incoherent outputs on forget prompts or fail to generalize because lexical-level and semantic-level associations persist in attention; it instead performs self-distillation from a temperature-smoothed attention teacher (Zade et al., 1 Mar 2026). ROCR reframes the problem as Form-Dependent Bias, showing that many methods forget only the surface form seen during unlearning and fail under alternate expressions of the same knowledge; its proposed solution is a training-free parameter edit that redirects a dangerous concept to a harmless concept by a closed-form rank-one update in the MLP down-projection matrix (Ye et al., 9 Jun 2025). These papers suggest that output suppression is not equivalent to source-free unlearning in the stronger sense of concept removal.

A third unresolved issue is the strength of the assumptions that make source-free methods work. Hessian-based certification assumes $C$2-Lipschitz, $C$3-strongly convex, $C$4-smooth, and $C$5-Hessian-Lipschitz losses, together with distributional closeness between source and surrogate datasets (Basaran et al., 6 Jun 2025). Zero-shot representation unlearning assumes Neural Collapse and access to classifier weights and class counts (Almudévar et al., 29 Jan 2026). CUTS assumes that corruption induces a stable task-like direction in weight space and works best on pretrained models (Mozafari et al., 24 Nov 2025). CLIP nullspace projection assumes that the target concept can be represented by a low-dimensional subspace and that synthesized prototypes are adequate proxies (Mishra et al., 16 Dec 2025). These are not merely technical details; they define where source-free unlearning is currently reliable and where it remains approximate.

Taken together, the literature presents source-free unlearning as a broad reorganization of the unlearning problem. Instead of replaying source data, methods now delete discrete codes, redirect concepts, smooth attention, synthesize supervision, subtract task vectors, estimate hidden curvature, or operate directly in latent geometry. This suggests a shift in emphasis: the central question is no longer only how to retrain less expensively, but how to define and remove the relevant information when the source data themselves are absent.