---
title: Untraining in Machine Learning
url: https://www.emergentmind.com/topics/untraining
type: topic
---

# Untraining in Machine Learning

Untraining is the post hoc removal of the effect of having trained on a specified forget set \(S \subset \mathcal{D}\). Its canonical target is the model that would have been obtained by training on \(\mathcal{D}\setminus S\), so the objective is retraining equivalence with respect to specific deleted examples rather than erasure of the full concept, capability, or distribution that those examples may represent. A recent terminological clarification therefore distinguishes **untraining** from **unlearning**: training concerns fitting a finite dataset, learning concerns generalization to an underlying distribution, and the corresponding deletion problems differ in kind [2604.07962].

## 1. Definition and scope

In the precise sense proposed for the field, untraining aims to reverse empirical risk minimization on a forget set \(S\subset D\). The intended output is the model obtained by training on \(D\setminus S\), preserving whatever can still be inferred from the retain set. This contrasts with unlearning, where the goal is to remove the broader behavior \(\mathcal{B}\) or the entire sub-distribution \(S^{full}\) represented by the forget examples; under that stronger objective, the model should behave as though it had never trained on any instance of that behavior [2604.07962].

This distinction reorganizes much of the earlier “machine unlearning” literature. User-data deletion under GDPR-style “right to be forgotten” requests, removal of noisy, outdated, or mislabeled points, deletion of exact copyrighted sentences, and protection against extraction of particular training points are all naturally untraining problems. By contrast, dangerous knowledge, backdoors, poisoned-data corruption under partial discovery, artistic styles, and harmful response patterns are broader concept-removal problems. The distinction matters because a successful untraining method may legitimately leave performance on deleted samples unchanged when those samples are non-memorized and still predictable from the retained data; that is consistent with retraining equivalence, but not with concept erasure [2604.07962].

The practical motivations for untraining are longstanding. Retraining from scratch is the cleanest baseline, but it is often computationally expensive, may require access to the original dataset, and is difficult to repeat for frequent deletion requests. Survey treatments therefore frame machine unlearning as a principled removal procedure whose gold standard is naïve retraining on \(D\setminus D_u\), with exact removal defined by distributional equivalence between the updated model and the retrained model on every input [2209.00939]. In domains such as contrastive learning, these motivations are coupled to privacy and compliance: models can implicitly memorize training data even without labels, enabling membership inference attacks, and regulatory regimes such as GDPR, APPI, and CPPA motivate deletion after training [2405.07317].

## 2. Formal problem statements

A standard notation writes the full training dataset as \(\mathcal{D}\), the forget set as \(\mathcal{S}\subset \mathcal{D}\), the retain set as \(\mathcal{D}\setminus \mathcal{S}\), the randomized learning algorithm as \(A\), and the deletion procedure as \(U\). The post-processed model is \(U(A(\mathcal{D}),\mathcal{S},\mathcal{D})\). In the classical retraining-equivalence formulation, untraining requires this output to be close in distribution to training from scratch on \(\mathcal{D}\setminus \mathcal{S}\); in the ideal exact form, the benchmark is simply \(\mathcal{U}^*(\mathcal{A}(D),D,D_u)=\mathcal{A}(D\setminus D_u)\) [2604.07962; 2209.00939].

The same literature also formalizes why deleting a point may or may not matter. For \(x_i \in D\), the counterfactual memorization score is
\[
\text{mem}(A,D,i)=\Pr_{f\sim A(D)}[f(x_i)=y_i]-\Pr_{f\sim A(D\setminus\{x_i\})}[f(x_i)=y_i],
\]
and the cross-influence of \(x_i\) on another point \(x_j\) is
\[
\text{xinf}(A,D,i,j)=\Pr_{f\sim A(D)}[f(x_j)=y_j]-\Pr_{f\sim A(D\setminus i)}[f(x_j)=y_j].
\]
If self- and cross-influence are low, untraining may be close to a no-op; if self-influence is high but cross-influence is low, the main effect is on the forget examples; if cross-influence is also high, collateral degradation becomes structurally likely [2604.07962].

Sequential variants instantiate the same retraining-equivalence logic in other settings. In reinforcement learning, exact deletion is formulated so that for any sequence of users \(z_1,\dots,z_T\), deletion position \(t\), and dummy user \(z'\),
\[
\mathbf{U}(\mathbf{A}(z_{1:T}),z_t,z') \overset{D}{=} \mathbf{A}(z_1,\ldots,z_{t-1},z',z_{t+1},\ldots,z_T).
\]
The equality in distribution makes deletion indistinguishable from having never observed the removed user at all, and the framework links this exactness to \(\rho\)-TV stability of the learning algorithm [2606.04182].

## 3. Evaluation, baselines, and failure modes

Because untraining is defined relative to retraining, evaluation inherits a fixed reference point. A consolidated survey groups evaluation into four dimensions—**Efficiency**, **Effectiveness**, **Consistency**, and **Certifiability**—and treats exact retraining on \(D\setminus D_u\) as the baseline against which speed, predictive performance, parameter similarity, output agreement, and deletion assurance should be assessed. Exact methods typically dominate on consistency and certifiability; approximate methods trade those guarantees for lower cost [2209.00939].

For approximate procedures, one influential proposal identifies the **verification error**
\[
\|w''-w'\|_2
\]
between an approximately untrained model \(w''\) and a naïvely retrained model \(w'\) as the central metric, because it subsumes a broad class of other weight-space and output-space notions. The same analysis of SGD derives a cheap proxy, the **unlearning error**,
\[
e=\eta^{2}\cdot \frac{\|w_t-w_0\|_2}{t}\cdot \sigma_{\text{avg}}\cdot \frac{t^2-t}{2},
\]
highlighting training length, curvature, and total weight displacement as the main determinants of later untraining difficulty [2109.13398].

Metric choice is highly domain dependent. In contrastive learning, forgetting is measured not only by membership inference attack metrics—Accuracy, Precision, Recall, and AUC—but also by top-1 prediction confidence, training loss, cosine similarity, weighted kNN evaluation, linear evaluation, and t-SNE visualizations. In language-model untraining on TOFU, evaluation uses ROUGE-L recall on paraphrased questions, length-normalized conditional probability of the correct answer, and truth ratio, which are collapsed into **Model Utility** and **Forget Quality** [2405.07317; 2503.01224].

A recurrent methodological warning is that attack-based privacy metrics are insufficient on their own. Membership inference can be appropriate for deleting specific user data, but it does not establish that an underlying behavior or capability has been removed. A method can suppress the forget set enough to defeat membership inference while still preserving semantically related behavior, which is acceptable for untraining but not for broader unlearning [2604.07962].

## 4. Exact and structured untraining systems

Exact untraining methods obtain their guarantees by constraining how training is organized. Two canonical examples are **SISA**—Sharded, Isolated, Sliced, and Aggregated—and **DaRE Forests**. SISA trains shard-specific models and stores intermediate states so that a deletion request retrains only the affected shard suffix; DaRE modifies tree construction and caches node statistics so that only affected subtrees are updated. Both are exact because their outputs match retraining under their respective training protocols [2209.00939].

A related line replaces simple sharding with coding. **Coded Machine Unlearning** encodes training data into linear combinations before sending them to non-communicating weak learners, then unlearns a sample by subtracting its coded contribution from affected shards and retraining only those weak learners. For regression, the protocol satisfies the paper’s perfect-unlearning criterion—its output is a statistical draw from the distribution of models trained on the dataset with the sample removed—and empirically improves the performance-versus-unlearning-cost trade-off relative to uncoded sharding, especially on datasets with influential or heavy-tailed samples [2012.15721].

For large pretrained models, exactness has been recast in parameter-efficient form. **Sequence-aware Sharded Sliced Training (S3T)** uses PEFT, specifically LoRA in the exposition, to isolate data influence across layers by sequentially training layers on disjoint data slices. Deletion is then implemented by deactivating affected PEFT layers rather than retraining the whole component. The framework also trains multiple slice orders to improve deletion capacity, with a theoretical deletion-rate scaling
\[
\delta(S)\sim O(mL\log(mL)),
\]
compared with
\[
\delta(\mathrm{SISA})\sim O(mL\log m),
\]
and empirical gains over SISA across vision, GLUE/SuperGLUE, and instruction-tuned LLM settings [2406.16257].

Exactness has also reached sequential decision making. In tabular episodic RL, exact deletion is achieved by combining binary-tree noisy prefix sums with maximal coupling, so that most previously generated randomness is reused and retraining is localized. The resulting framework is exact with respect to the learning algorithm, \(\rho\)-TV-stable, and has expected unlearning cost only a \(\rho\sqrt{\ln T}\) fraction of retraining from scratch, while achieving a regret bound of
\[
\mathcal{O}\!\left(H^2\sqrt{SAT}+H^3S^2A+\frac{H^{2.5}S^2A}{\rho}\right)
\]
for tabular MDPs [2606.04182].

## 5. Approximate and optimization-based procedures

Most practical untraining methods are approximate: they start from a trained model and seek a cheaper route toward the retrained solution. Classical survey exemplars include **Fisher**, **Influence**, **DeltaGrad**, **Descent-to-Delete**, and **DeepObliviate**, which variously rely on Newton or influence-function corrections, replay of cached optimization history, perturbed gradient descent, or early stopping of retraining-like trajectories [2209.00939].

Several later methods refine these ideas by altering what is updated and how. **Unlearning with Fisher Masking** first masks parameters that are more important to the forget set than to the remain set, selecting the top \(R\) parameters according to \(F_{f,jj}-F_{r,jj}\), and then fine-tunes on the remaining data. Across CIFAR10, CIFAR100, MNIST, and Tiny-ImageNet, FisherMask often approaches retraining quality after a short fine-tuning phase, and even with no remain data it still works reasonably, though with reduced remain accuracy [2310.05331].

Other methods modify the gradient objective itself. **OrthoGrad** projects the unlearn gradient onto the subspace orthogonal to all per-sample retain gradients in the current batch, treating gradient interference rather than a scalar ascent–descent trade-off as the primary difficulty. The method is evaluated in a low-data retain regime on CIFAR-10, ImageNet, and automatic speech recognition with Whisper-Tiny on LibriSpeech, and it outperforms competing methods, including on speaker unlearning in ASR [2503.02312]. A different objective-based formulation, **EMSE** or Ethical MSE, explicitly minimizes squared error on wanted data while penalizing accurate prediction on unwanted data through a log-complement Gaussian term, framing untraining as selective re-optimization rather than retraining [2410.09935].

Language-model untraining has pushed especially hard on optimization stability. **CE-U (Cross Entropy Unlearning)** replaces direct gradient ascent on forgotten targets with a cross-entropy loss against a modified target distribution in which the true label logit is set to \(-\infty\), thereby avoiding the vanishing-gradient at high confidence and exploding-gradient at low confidence pathology of gradient-ascent unlearning. On TOFU with LLaMA2-7B, CE-U attains state-of-the-art results without an oracle model or additional positive samples, and its General CE-U form interpolates continuously between ordinary learning and pure unlearning [2503.01224]. **MArgin Self-Correction (MASC)** instead focuses on token-level dominance: it reduces the logit gap between the original next token and the top-\(k\) alternatives, then stops online when the fraction of remaining violations on a forget probe subset drops below a threshold. On TOFU, MUSE News, and MUSE Books, MASC achieves a competitive forget–retain trade-off at a fraction of the computational cost of fixed-budget baselines and avoids offline checkpoint selection [2606.02920].

Approximate untraining has also expanded beyond supervised classification. In contrastive learning, a gradient-constraint approach combines a member training loss, a WGAN-GP-inspired gradient penalty, and an output-norm term,
\[
\mathcal{L}=\alpha\cdot L_{\text{MEMtrain}}+\beta\cdot L_{\text{GP}}+\gamma\cdot L_{\text{Norm}},
\]
or in optimized form
\[
\mathcal{L}=\alpha\cdot L_{\text{MEMtrain}}+\beta\cdot L_{\text{MEMGP}},
\]
and performs only about 10 unlearning epochs. On CIFAR-10, CIFAR-100, and SVHN, membership inference accuracy drops to about \(50\%\!-\!51\%\), while downstream utility typically decreases only moderately, and the same method transfers to supervised ResNet18 models [2405.07317]. At the optimizer level, stochastic variants of **D2D** and **R2D** provide \((\epsilon,\delta)\)-certified unlearning under strongly convex, convex, and nonconvex assumptions; D2D yields tighter guarantees in the strongly convex case, whereas R2D remains applicable in convex and nonconvex settings because rewinding reverses accumulated disturbance [2511.15983].

## 6. Boundary cases, scaling, and research directions

Not every deletion problem now labeled “unlearning” is an instance of untraining. In large language models, **skill unlearning** seeks to remove a designated capability—such as math-solving, Python coding, or language comprehension in a specific language—while retaining unrelated skills and MMLU. Training-free methods such as **Neuron Adjust** and **Key Space Detection** operate at inference time on feed-forward-layer activations rather than on traced training examples; for most tasks, Key Space Detection produces over \(80\%\) relative performance drop on the forgetting skill with less than \(10\%\) relative performance drop on other skills and MMLU [2503.21730]. A related position paper argues that foundation-model deletion requests should be lifted from data tracing to **knowledge tracing**, allowing forgetting sets to refer directly to concepts or capabilities rather than identified training points [2506.11253].

Inference-time control further separates output suppression from weight-level retraining equivalence. A conformal-prediction framework for generative models equips the model with a verifier \(V(y;x,\mathcal U)\), iteratively refines responses until \(V(y)\ge \lambda\), and calibrates the iteration budget \(T_\alpha\) on a held-out calibration set to obtain a distribution-free marginal guarantee \(\mathbb{P}[V(y;x)\ge \lambda]\ge 1-\alpha\). The approach does not alter model parameters and therefore is explicitly not true weight-level erasure, but it reduces unlearning error by up to \(93\%\) across challenging generative benchmarks [2602.03787].

Two further lines shift attention from the deletion algorithm to the training state on which deletion acts. **Grokked Models are Better Unlearners** shows that standard unlearning methods applied after the grokking transition yield more efficient forgetting, less collateral damage, and more stable updates across seeds than the same methods applied to pre-grokking checkpoints; the reported mechanism is reduced gradient alignment between forget and retain subsets and more modular representations [2512.03437]. **Stream-native machine unlearning** translates batch deletion to an online setting through regret, sample complexity, and deletion capacity, replacing expensive Hessian inversion with online L-BFGS and obtaining \(\mathcal{O}(\ln T)\) regret under strong convexity in a live stream of insertions and deletions [2508.10193].

The most persistent open problem remains conceptual. If the forget set is complete, retraining equivalence is a coherent target; if only a partial or semantic description of the unwanted behavior is available, retraining equivalence is no longer enough. Current work therefore points in two directions at once: sharper terminology and benchmarks for sample-level untraining, and separate methods for concept-level unlearning that explicitly generalize deletion beyond the observed forget examples [2604.07962].

Source: https://www.emergentmind.com/topics/untraining