---
title: Distributional Unlearning in Machine Learning
url: https://www.emergentmind.com/topics/distributional-unlearning
type: topic
---

# Distributional Unlearning in Machine Learning

Distributional unlearning is a family of machine unlearning formulations in which the object to be forgotten is not merely a finite forget set, but an underlying distribution, sub-population, concept, behavior, or domain. Recent work distinguishes **untraining**, which aims to reverse the effect of training on a specific forget set \(S\), from **unlearning**, whose target is the broader distribution or behavior \(\mathcal{B}\) represented by that set, with the ideal comparator being a model trained on \(D\setminus S^{full}\) rather than \(D\setminus S\) [2604.07962]. In parallel, other strands formulate unlearning as equality or indistinguishability between distributions over models after deletion and retraining, making stochasticity itself part of the definition [1912.03817]. Across these lines, the central problem is to move from pointwise deletion to distribution-level removal while preserving utility on retained data or retained domains.

## 1. Conceptual scope and problem setting

Distributional unlearning arises when the deletion target is an entire domain of information rather than a handful of isolated examples. Representative motivations include removing toxic language, copyrighted corpora, demographic biases, user histories under GDPR, or security-sensitive code patterns [2605.16645; 2507.15112; 2506.17125]. In such settings, straightforward point deletion can leave enough residual signal for downstream learners to recover the unwanted domain, so the goal becomes to erase the *statistical footprint* of that domain rather than only its explicitly listed samples [2507.15112].

A useful conceptual distinction separates three senses of “distributional.” First, some work focuses on the **distribution over learned models**, requiring the post-unlearning model distribution to match retraining on retained data [1912.03817; 2406.09073]. Second, more recent work focuses on **forgetting distributions, not just samples**, so that the edited dataset or model is far from an unwanted distribution and close to a retained one [2507.15112; 2605.16645]. Third, some methods study **distributional shift induced by unlearning**, especially when forgetting requests are non-uniform over features, labels, or subpopulations [2403.08124; 2405.14020]. These senses are distinct but increasingly intertwined.

Earlier practical systems already used distributional language in a narrower sense. SISA training formalized unlearning by equality between the distribution of models obtained by “train then unlearn” and the distribution of models obtained by training without the deleted point, and it further allowed priors over the distribution of unlearning requests to drive sharding decisions [1912.03817]. Recent work generalizes this idea from request distributions to data-domain distributions.

## 2. Formal formulations

One data-centric formulation defines \((\alpha,\varepsilon)\)-distributional unlearning through forward KL constraints. For an unwanted distribution \(p_1\), a retained distribution \(p_2\), and an edited distribution \(p\), the edited data must satisfy
\[
KL(p_1 \parallel p) \ge \alpha
\quad\text{and}\quad
KL(p_2 \parallel p) \le \varepsilon,
\]
so that the edited dataset is information-theoretically far from the unwanted domain yet close to the retained one [2507.15112]. In the Gaussian case, the exact Pareto frontier is derived, and any model retrained on the edited data incurs log-loss shifts bounded by the divergence thresholds [2507.15112].

A related statistical formulation replaces raw divergence thresholds with a **hypothesis-testing criterion**. In that view, domains are modeled as probability distributions, and the edited data are judged by the trade-off function induced by testing the desired and unwanted domains against the edited distribution. This yields a removal-preservation Pareto frontier for shifted Gaussians of arbitrary dimension, a one-dimensional location family with log-concave noise, the one-dimensional Poisson family, and the Gaussian white noise model, and it also establishes composition rules for multimodal unwanted domains [2605.16645].

At the model level, exact or approximate unlearning is often defined as distributional indistinguishability from retraining. The NeurIPS unlearning competition paper adopted a DP-inspired \((\varepsilon,\delta)\)-unlearning condition under which the distribution of outputs from \(U(A(D),S,D)\) must be statistically close to the distribution of \(A(D\setminus S)\) [2406.09073]. ReGUn instantiates the same intuition at the level of predictive distributions: after unlearning, the model’s behavior on the forget set should look like its behavior on genuinely unseen data from the same task distribution [2603.11210]. FADE makes this explicit for generative models by comparing the full conditional distributions \(p_{\text{unlearn}}(\cdot\mid x)\) and \(p_{\text{retain}}(\cdot\mid x)\) through a bidirectional likelihood criterion equal to the Jeffreys divergence [2510.12981].

| Formulation | Core object | Representative paper |
|---|---|---|
| Retrain-equivalence | Distribution of models | [1912.03817], [2406.09073] |
| Data-centric forgetting | Edited data distribution \(p\) | [2507.15112], [2605.16645] |
| Reference-guided matching | Predictive distribution on forget inputs | [2603.11210] |
| Functional equivalence | Conditional output distribution | [2510.12981] |

These formulations are not interchangeable. This suggests that pointwise retraining parity, behavior on held-out unseen data, and edited-data divergence constraints each capture different aspects of what it means to forget a distribution.

## 3. Algorithmic mechanisms

A prominent model-side instantiation is **BalDRO**, which treats LLM unlearning as a KL-DRO problem on the forget set:
\[
\arg\min_{\theta}\ \sup_{Q_f:\,\mathbb{D}(Q_f\Vert \widehat{D}_f)\le\eta} \mathbb{E}_{Z \sim Q_f}\big[\ell_f(Z;\theta)\big].
\]
Here the inner supremum reshapes the effective forget distribution toward hard-to-unlearn samples, addressing sample-wise imbalance and asynchronous forgetting [2601.09172]. BalDRO-G approximates the inner maximization with loss-based GroupDRO over top-\(p\%\) high-loss samples, while BalDRO-DV uses the Donsker–Varadhan dual to produce a log-sum-exp objective with weights \(w_i \propto \exp(\ell_f(z_i;\theta)/\beta)\) [2601.09172]. In this framework, distributional unlearning means minimizing the worst-case expected forget loss over all reweightings within a KL ball.

A second class of methods treats unlearning as **distributional alignment under shift**. DUI augments influence-function updates with an independence criterion between features, labels, and predictions, first via mutual information and then operationally via HSIC, so that the model’s feature–prediction dependence is re-aligned with the remaining dataset after non-uniform feature or label removal [2403.08124]. UIB instead formulates unlearning as a parameter-space information bottleneck,
\[
-I(Y_{\mathcal{D}\setminus\Delta\mathcal{D}};\theta) + \beta I(X_{\mathcal{D}\setminus\Delta\mathcal{D}};\theta),
\]
with a dynamic prior and structured regularization to adapt to distribution shifts caused by removing systematic patterns and biases [2405.14020].

A third class acts directly on **output distributions**. For code LLMs, PROD constructs a target next-token distribution that sets the forget token’s probability to zero, applies top-\(p\) filtering with \(top\_p=0.8\), and redistributes probability mass to plausible alternatives, with \(\alpha=0\) performing best in ablations [2506.17125]. The model is then trained by cross-entropy to match this target distribution. In text-to-image diffusion, Diversified Unlearning replaces a single keyword prompt by an empirical distribution of contextually diverse prompts, or by token-wise embedding mixups, thereby defining the concept to be unlearned as a prompt/embedding distribution rather than a point estimate [2603.18767].

Finally, reference-based methods treat unlearning as **distributional indistinguishability**. ReGUn constructs a class-conditioned reference distribution \(q(B_f)\) from a disjoint held-out dataset and minimizes
\[
\frac{1}{|B_f|}\sum_{(x,\cdot)\in B_f}\mathrm{KL}\big(q(B_f)\,\|\,p_\theta(\cdot\mid x)\big)
\]
alongside standard retain-set cross-entropy, so that forget-set predictions match unseen-data behavior rather than merely becoming wrong [2603.11210].

## 4. Evaluation methodologies

Evaluation is a major fault line in this literature. The competition framework for approximate unlearning measures forgetting quality through per-example attacker-based estimation of \(\varepsilon\), using one-dimensional output statistics under retrained and unlearned model distributions, then aggregates them into a global score \(\mathcal F\) [2406.09073]. This preserves the original distributional notion of unlearning but operationalizes it with tractable hypothesis tests.

Task-specific benchmarks expose different aspects of distributional forgetting. TOFU reports Forget Quality, Model Utility, Extraction Memorization, Extraction Strength, Truth Ratio variants, and membership inference metrics such as LOSS, ZLib, MinK, and MinK++ [2601.09172]. MUSE expands this into a six-way benchmark for language models: no verbatim memorization, no knowledge memorization, no privacy leakage, utility preservation, scalability with respect to removal size, and sustainability over sequential unlearning requests [2407.06460]. This is especially important when the forget set is itself a coherent corpus such as Harry Potter books or BBC news.

Recent work argues that many existing metrics are **reference-specific** and can hide residual knowledge. FADE addresses this by measuring bidirectional likelihood agreement over generated samples, thereby comparing the full output distributions of an unlearned model and a retain-only model rather than a small set of reference answers or classifier outputs [2510.12981]. For real-world LLM deployments in which the retrained reference is unavailable, DCUE instead evaluates the distribution of **Core Token Confidence Scores** and applies a corrected Kolmogorov–Smirnov test using a validation set to estimate the retained-data effect [2508.01324].

The paper on in- vs. out-of-distribution unlearning contributes a separate evaluation axis for generative models: **Generalized Exposure** and **Relative Exposure**, which compare soft likelihood rankings of forget examples against reference strings under reference, subject, and unlearned models [2411.04388]. These metrics are explicitly distribution-sensitive because they track how a forget set’s likelihood distribution shifts relative to appropriate reference distributions.

## 5. Empirical patterns and application domains

Across modalities, a recurring empirical pattern is that methods can improve forgetting metrics while remaining far from retraining in a stronger distributional sense. On TOFU, BalDRO illustrates the upside of robust reweighting: for forget ratio \(1\%\), NPO’s Forget Quality improves from \(0.7659\) to \(0.9188\) with BalDRO-G and to \(0.9900\) with BalDRO-DV, while Model Utility remains comparable at \(0.5775 \to 0.6126 / 0.5815\) [2601.09172]. On MUSE, BalDRO variants reduce KM-Df and VM-Df, keep or slightly improve KM-Dr, and move PrivLeak closer to zero [2601.09172].

The in-/out-of-distribution study shows that distributional location of the forget set matters sharply. Unlearning out-of-distribution examples requires more unlearning steps but overall presents a better trade-off, whereas for in-distribution examples there is a rapid decay in performance as unlearning progresses [2411.04388]. This suggests that the geometry of the forget set relative to the main data manifold strongly controls collateral damage.

For structured distribution shift, DUI remains close to retraining under top-\(k\) feature or label removal: on Cora/GIN with unlearn ratio \(0.05\), Retrain gives \(0.8057 \pm 0.0159\) F1 while DUI gives \(0.7868 \pm 0.0189\); on MNIST/Simple CNN at ratio \(0.05\), Retrain gives \(0.9587 \pm 0.0005\) and DUI gives \(0.9433 \pm 0.0176\) [2403.08124]. UIB similarly reduces bias-correlation and MIA-Efficacy while maintaining F1 under systematic pattern removal [2405.14020].

In source code unlearning, existing methods achieve about \(90\%\) forget quality in copyrighted code removal but HumanEval pass rate drops by more than \(60\%\), making the resulting models practically unusable; PROD achieves the highest Pareto Dominance Ratio across copyrighted code, insecure code, and deprecated API unlearning, with roughly \(124\%\) average relative improvement versus the best baseline and BLEU around \(0.05\) under prefix injection attacks, compared with baseline BLEU similarities above \(0.3\) [2506.17125]. In diffusion models, Diversified Unlearning consistently improves erasure, benign concept retention, and robustness to Ring-A-Bell, indirect recovery, and noise-based attacks by replacing keyword-only forgetting with prompt-distribution forgetting [2603.18767].

At the benchmark level, MUSE reaches a more pessimistic conclusion: most algorithms can prevent verbatim memorization and knowledge memorization to varying degrees, but only one algorithm does not lead to severe privacy leakage, and existing algorithms also degrade general model utility and cannot sustainably accommodate successive unlearning requests or large-scale content removal [2407.06460].

## 6. Limitations, controversies, and open directions

The field is marked by a persistent ambiguity between deleting samples and deleting distributions. The note “Is your algorithm unlearning or untraining?” argues that much of the literature labeled “unlearning” is technically solving untraining, and that metrics such as membership inference on the forget set can be appropriate for sample-level deletion but insufficient for concept- or domain-level removal [2604.07962]. This suggests that evaluation mismatch remains one of the main sources of confusion.

A second controversy concerns **what reference should define successful forgetting**. ReGUn uses held-out data as a proxy for unseen behavior [2603.11210], FADE requires a retain-only model and shows that many methods scoring well on traditional metrics remain far from distributional equivalence [2510.12981], and DCUE attempts to remove this dependence by estimating the retained-data effect from a validation set [2508.01324]. No single evaluation paradigm has yet emerged as universal.

Open technical directions recur across papers. BalDRO studies only KL-DRO and notes that \(\chi^2\) and Wasserstein alternatives could induce different weighting behavior; it also leaves robust forgetting-plus-retention formulations open [2601.09172]. Data-centric theories show Pareto frontiers and finite-sample selection guarantees but also report an information-computation gap [2605.16645]. The Gaussian/KL framework for forgetting distributions rather than samples suggests that deletion budgets can be reduced substantially by selective removal, but extending these guarantees beyond the analyzed families remains open [2507.15112]. In sequential decision-making, offline stochastic multi-armed bandits already require separate treatment under fixed-sample and distribution models, with adaptive switching between Gaussian mechanism and rollback depending on the coverage regime [2605.00638].

A broader implication is that distributional unlearning is not a single method class but a shift in target. It treats forgetting as robust risk minimization, distributional alignment, output-distribution editing, or statistical indistinguishability, depending on the modality and the comparator. What unites these approaches is the claim that forgetting should be assessed at the level at which the unwanted information is actually represented: as a data distribution, a concept manifold, a predictive distribution, or a distribution over trained models.

Source: https://www.emergentmind.com/topics/distributional-unlearning