Asymmetric Langevin Unlearning (ALU)

• ALU is a certified machine unlearning framework that leverages asymmetric data sources by anchoring training with public data.
• Using Projected Noisy Gradient Descent, ALU minimizes the divergence between learning and retraining distributions to enhance unlearning quality.
• Public data injection in ALU lowers required noise levels, offering computational advantages for mass unlearning of private data.

Asymmetric Langevin Unlearning (ALU) is a certified machine unlearning framework for training sets composed of asymmetric sources: a public set $D_{\mathrm{pub}}$, sampled from $P_{\mathrm{pub}}$, that is not subject to deletion requests, and a private set $D_{\mathrm{priv}}$, sampled from $P_{\mathrm{priv}}$, that is subject to unlearning. Formally, $D = D_{\mathrm{pub}} \cup D_{\mathrm{priv}}$, and only points in $D_{\mathrm{priv}}$ can be forgotten. ALU uses Projected Noisy Gradient Descent (PNGD) for both learning and unlearning, and evaluates unlearning quality by the Rényi divergence between the unlearning distribution and the retraining distribution. Its central claim is that public data can be injected directly into the learning and retraining objective so that the learning/retraining mismatch is reduced by a factor scaling like $O(1/n_{\mathrm{pub}}^2)$, or more precisely through the factor $$\left(\frac{n_{\mathrm{priv}}}{n_{\mathrm{priv}}+n_{\mathrm{pub}}}\right)^2$$. This yields a new control mechanism for the utility–unlearning trade-off, especially for forgetting a constant fraction of the private data, while making distribution mismatch between public and private sources an explicit part of the analysis (Inane et al., 11 May 2026).

1. Conceptual definition and data model

ALU is defined by an asymmetric training regime rather than by a new stochastic update distinct from Langevin unlearning. The asymmetry lies in the data model: public data appears in both the original training and retraining pipelines, while only private data is eligible for deletion. After a deletion request $D_{\mathrm{forget}} \subseteq D_{\mathrm{priv}}$, the retain set is

$$D_{\mathrm{retain}} = D \setminus D_{\mathrm{forget}}.$$

The framework tracks three distributions over model weights: the learning distribution $P_{\mathrm{pub}}$0, obtained after training on the full dataset; the unlearning distribution $P_{\mathrm{pub}}$1, obtained by fine-tuning the trained model on the retain set; and the retraining distribution $P_{\mathrm{pub}}$2, obtained by retraining from scratch on the retain set only (Inane et al., 11 May 2026).

This retraining-based perspective places ALU within the broader certified unlearning tradition established for Langevin-style methods. In the symmetric baseline, Langevin unlearning is formulated through Rényi Unlearning (RU), where the post-unlearning distribution is required to be close—in both Rényi directions—to retraining on the updated dataset. That baseline uses the symmetric privacy metric

$P_{\mathrm{pub}}$3

and applies the same noisy projected gradient dynamics in both learning and unlearning (Chien et al., 2024). ALU preserves the retraining-indistinguishability viewpoint, but changes the source structure of the data so that public samples act as a stability anchor during both learning and retraining (Inane et al., 11 May 2026).

The motivating limitation is specific: in public-data-free Langevin unlearning, the certification cost for forgetting a constant fraction of the private data does not improve with total dataset size. The ALU formulation changes this because public data dilutes the sensitivity of the empirical gradient to deleted private points while remaining present in both learning and retraining objectives (Inane et al., 11 May 2026).

2. PNGD formulation and operational pipeline

ALU uses PNGD for both training and unlearning. The empirical loss on the mixed dataset is

$P_{\mathrm{pub}}$4

Public data is therefore injected directly into the empirical risk / gradient updates rather than being used as a regularizer or auxiliary term (Inane et al., 11 May 2026).

The generic projected noisy gradient update is

$P_{\mathrm{pub}}$5

with Gaussian noise

$P_{\mathrm{pub}}$6

where $P_{\mathrm{pub}}$7 is compact, with radius $P_{\mathrm{pub}}$8 (Inane et al., 11 May 2026).

The ALU training phase runs $P_{\mathrm{pub}}$9 PNGD steps on

$D_{\mathrm{priv}}$0

yielding $D_{\mathrm{priv}}$1. After a deletion request $D_{\mathrm{priv}}$2, ALU initializes from $D_{\mathrm{priv}}$3 and runs $D_{\mathrm{priv}}$4 PNGD steps on

$D_{\mathrm{priv}}$5

yielding $D_{\mathrm{priv}}$6. The certification target is $D_{\mathrm{priv}}$7, produced by training from scratch on $D_{\mathrm{priv}}$8 using the same initialization law and the same noisy optimizer (Inane et al., 11 May 2026).

This pipeline is structurally continuous with the earlier Langevin unlearning framework. In that framework, training on $D_{\mathrm{priv}}$9 induces an initial RDP mismatch between model distributions on adjacent datasets, and running further PNGD updates on the updated dataset contracts that mismatch toward the retraining distribution (Chien et al., 2024). ALU inherits the same two-phase logic—initial discrepancy plus contraction—but improves the initialization term by changing the denominator in the empirical loss from private-only to public-plus-private (Inane et al., 11 May 2026).

A plausible implication is that ALU’s asymmetry is not in the isotropic Gaussian noise itself, which remains Gaussian and homogeneous, but in the way the data source decomposition changes gradient sensitivity and hence the required noise level. This interpretation aligns with the paper’s explicit statement that ALU is Langevin unlearning performed in the asymmetric data model (Inane et al., 11 May 2026).

3. Certification metric and main theoretical guarantees

ALU uses Rényi divergence as its certification metric. For $P_{\mathrm{priv}}$0,

$P_{\mathrm{priv}}$1

The paper analyzes both the initial mismatch $P_{\mathrm{priv}}$2 and the unlearning quality term $P_{\mathrm{priv}}$3 or $P_{\mathrm{priv}}$4 (Inane et al., 11 May 2026).

The key theorem states that, under $P_{\mathrm{priv}}$5-smoothness, $P_{\mathrm{priv}}$6-Lipschitzness, a compact projection set $P_{\mathrm{priv}}$7, and an initialization distribution $P_{\mathrm{priv}}$8 satisfying a $P_{\mathrm{priv}}$9-LSI, the learning/retraining mismatch at iteration $D = D_{\mathrm{pub}} \cup D_{\mathrm{priv}}$0 obeys

$D = D_{\mathrm{pub}} \cup D_{\mathrm{priv}}$1

where

$D = D_{\mathrm{pub}} \cup D_{\mathrm{priv}}$2

The important structural factor is

$D = D_{\mathrm{pub}} \cup D_{\mathrm{priv}}$3

which is the source of the public-data advantage (Inane et al., 11 May 2026).

Unlearning then contracts this improved initialization. Using the contraction result inherited from Langevin unlearning,

$D = D_{\mathrm{pub}} \cup D_{\mathrm{priv}}$4

and in the strongly convex case,

$D = D_{\mathrm{pub}} \cup D_{\mathrm{priv}}$5

Thus the benefit of ALU comes through a smaller initial divergence $D = D_{\mathrm{pub}} \cup D_{\mathrm{priv}}$6, which is then exponentially reduced by the unlearning iterations (Inane et al., 11 May 2026).

This contraction mechanism is directly descended from the earlier Langevin unlearning theory, where privacy loss contracts exponentially with unlearning iterations, a phenomenon described there as privacy recuperation (Chien et al., 2024). ALU does not replace that mechanism; it changes the prefactor through asymmetric source composition (Inane et al., 11 May 2026).

4. Noise suppression, mass unlearning, and computational advantage

The paper’s central comparative claim concerns the noise level required for certification when forgetting a constant fraction of the private data,

$D = D_{\mathrm{pub}} \cup D_{\mathrm{priv}}$7

In the symmetric case,

$D = D_{\mathrm{pub}} \cup D_{\mathrm{priv}}$8

whereas in ALU,

$D = D_{\mathrm{pub}} \cup D_{\mathrm{priv}}$9

Hence public data suppresses the needed noise by

$D_{\mathrm{priv}}$0

For strongly convex loss, the appendix gives the $D_{\mathrm{priv}}$1-independent sufficient condition

$D_{\mathrm{priv}}$2

(Inane et al., 11 May 2026).

This is the technical basis for the claim that ALU supports mass unlearning of constant fractions of private data. In the symmetric setting, the required noise lower bound does not shrink with dataset size when forgetting a constant fraction, whereas in ALU it can be made small by increasing $D_{\mathrm{priv}}$3 (Inane et al., 11 May 2026).

The paper also states a strict computational advantage over retraining under suitable conditions. Retraining cost is defined as the original $D_{\mathrm{priv}}$4 training steps, and unlearning cost as the minimum $D_{\mathrm{priv}}$5 such that

$D_{\mathrm{priv}}$6

Under strong convexity, a sufficient condition for ALU to be more efficient than retraining is

$D_{\mathrm{priv}}$7

In the large-$D_{\mathrm{priv}}$8 regime, the paper states that the cost of unlearning scales as

$D_{\mathrm{priv}}$9

with public data reducing the initial divergence further (Inane et al., 11 May 2026).

These results should be read against the original symmetric Langevin unlearning analysis, which already argued for a computational saving over retraining when the initial mismatch is $O(1/n_{\mathrm{pub}}^2)$0 in the strongly convex case (Chien et al., 2024). ALU preserves that contraction-based savings logic but makes the initial mismatch controllable through public data volume (Inane et al., 11 May 2026).

5. Distribution mismatch and the utility–unlearning trade-off

A major contribution of ALU is that it does not assume $O(1/n_{\mathrm{pub}}^2)$1. Instead, it explicitly analyzes distribution mismatch. The assumptions are that $O(1/n_{\mathrm{pub}}^2)$2 and $O(1/n_{\mathrm{pub}}^2)$3 share the same support, $O(1/n_{\mathrm{pub}}^2)$4 is compact, and the loss is $O(1/n_{\mathrm{pub}}^2)$5-Lipschitz in $O(1/n_{\mathrm{pub}}^2)$6 (Inane et al., 11 May 2026).

The resulting generalization bound on the private distribution is

$O(1/n_{\mathrm{pub}}^2)$7

This decomposition makes the trade-off explicit. Public data reduces the approximation error term by shrinking $O(1/n_{\mathrm{pub}}^2)$8, but it can simultaneously worsen the mismatch penalty if $O(1/n_{\mathrm{pub}}^2)$9 and $$\left(\frac{n_{\mathrm{priv}}}{n_{\mathrm{priv}}+n_{\mathrm{pub}}}\right)^2$$0 are badly misaligned (Inane et al., 11 May 2026).

The paper is correspondingly precise about when ALU is expected to work well. If

$$\left(\frac{n_{\mathrm{priv}}}{n_{\mathrm{priv}}+n_{\mathrm{pub}}}\right)^2$$1

the mismatch penalty is negligible and increasing public data is almost purely beneficial. If

$$\left(\frac{n_{\mathrm{priv}}}{n_{\mathrm{priv}}+n_{\mathrm{pub}}}\right)^2$$2

the mismatch penalty can dominate and the utility bound can become vacuous, even though the unlearning certificate itself still improves (Inane et al., 11 May 2026).

This mismatch-aware analysis distinguishes ALU from methods that treat asymmetry only as an optimization heuristic. A useful comparison is the two-phase retain–forget entanglement framework, which partitions the retain set into an adjacent retain set $$\left(\frac{n_{\mathrm{priv}}}{n_{\mathrm{priv}}+n_{\mathrm{pub}}}\right)^2$$3 and a remote retain set $$\left(\frac{n_{\mathrm{priv}}}{n_{\mathrm{priv}}+n_{\mathrm{pub}}}\right)^2$$4, then protects them differently during unlearning (Cheng et al., 27 Mar 2026). That work addresses a different asymmetry—semantic or feature-wise entanglement between forget and retain samples—whereas ALU addresses source asymmetry between public and private data (Cheng et al., 27 Mar 2026, Inane et al., 11 May 2026). This suggests that “asymmetry” in machine unlearning is not a single notion but a family of structurally distinct decompositions.

6. Empirical evidence, related asymmetric variants, and limitations

The ALU paper evaluates three empirical axes: variational Rényi divergence, utility under mismatch, and membership inference attacks. On DomainNet 24-class image classification, using Clipart as private and Quickdraw as public, increasing public data consistently reduces the estimated

$$\left(\frac{n_{\mathrm{priv}}}{n_{\mathrm{priv}}+n_{\mathrm{pub}}}\right)^2$$5

and increasing unlearning iterations $$\left(\frac{n_{\mathrm{priv}}}{n_{\mathrm{priv}}+n_{\mathrm{pub}}}\right)^2$$6 also reduces divergence. An ablation at $$\left(\frac{n_{\mathrm{priv}}}{n_{\mathrm{priv}}+n_{\mathrm{pub}}}\right)^2$$7 indicates that public data primarily helps by reducing the initial distribution gap between learning and retraining (Inane et al., 11 May 2026).

For distribution mismatch, again on DomainNet with $$\left(\frac{n_{\mathrm{priv}}}{n_{\mathrm{priv}}+n_{\mathrm{pub}}}\right)^2$$8, private size $$\left(\frac{n_{\mathrm{priv}}}{n_{\mathrm{priv}}+n_{\mathrm{pub}}}\right)^2$$9, and forget set $D_{\mathrm{forget}} \subseteq D_{\mathrm{priv}}$0, the aligned setting yields a relative performance gap between unlearning and retraining of about 3.68%–4.62%, whereas the misaligned setting yields about 10.34%–10.81%. On IMDB sentiment classification with a 2-layer LSTM, evaluated using U-LiRA, the paper reports that without public data injection the attack can confidently distinguish many unlearned models from retrained models, while public data substantially reduces the attack’s discriminative power. Reported test accuracies are 82.59% unlearned and 82.54% retrained with no public data; 81.42% unlearned and 82.15% retrained with Amazon Reviews (50k public), no label flips; and 80.40% unlearned and 80.80% retrained with Amazon Reviews (50k public), 40% flipped labels (Inane et al., 11 May 2026).

ALU also sits within a broader line of asymmetric unlearning research. One direction replaces worst-case privacy analysis with per-instance privacy losses $D_{\mathrm{forget}} \subseteq D_{\mathrm{priv}}$1, which bound

$D_{\mathrm{forget}} \subseteq D_{\mathrm{priv}}$2

and imply that unlearning time scales logarithmically with point-specific difficulty (Sepahvand et al., 24 May 2025). Another derives per-instance certified unlearning for ridge regression trained via Langevin dynamics, using point-specific sensitivity bounds

$D_{\mathrm{forget}} \subseteq D_{\mathrm{priv}}$3

to calibrate the minimum unlearning noise required for a target $D_{\mathrm{forget}} \subseteq D_{\mathrm{priv}}$4 guarantee (Benarroch et al., 17 Feb 2026). A different asymmetric formulation, motivated by LLMs, treats retention as the primary objective and forgetting as auxiliary, and performs retention-prioritized gradient synthesis rather than symmetric loss balancing (Xiao et al., 16 Apr 2026). These works do not define ALU, but they show that asymmetry can be introduced through data sources, points, retain subsets, or gradient geometry.

The limitations of ALU are explicit. The theoretical bounds depend on LSI constants $D_{\mathrm{forget}} \subseteq D_{\mathrm{priv}}$5, which are difficult to compute in deep models; the strongest computational guarantees are clearest under strong convexity; variational Rényi divergence estimation is expensive; and public data may improve certification while still harming utility under severe mismatch. The paper also notes that web-scale “public” data may still contain sensitive or copyrighted content, so the public/private split is not a blanket resolution of broader privacy and legal issues (Inane et al., 11 May 2026).

Taken together, these results position ALU as a source-asymmetric extension of Langevin unlearning: it retains the retraining-based certification logic and contraction analysis of noisy gradient descent, but introduces a public-data control variable that can suppress required noise, improve the utility–unlearning trade-off, and make constant-fraction private-data deletion tractable under appropriate alignment conditions (Chien et al., 2024, Inane et al., 11 May 2026).