ManiF-SMC: Manifold Forgetting with SMC

• ManiF-SMC is a label-agnostic method for approximate machine unlearning that repositions erased samples by altering their learned representations.
• It employs a margin-based triplet loss with self mode connectivity to adaptively estimate local geometry and harmonize erased and retained sample distributions.
• The approach achieves competitive unlearning performance with significant runtime efficiency across benchmarks like MNIST, CIFAR10, and CelebA.

Searching arXiv for ManiF-SMC and related machine unlearning papers. ManiF-SMC, short for Manifold Forgetting with Self Mode Connectivity, is a label-agnostic method for approximate machine unlearning that operates entirely in a model’s learned representation space. Its central premise is that a model retrained only on retained data tends to classify erased samples according to their semantic similarity to the retained set; ManiF-SMC therefore reformulates forgetting as a geometric operation that pushes each erased sample away from its original representation while pulling it toward the centroid of its nearest retained semantic neighbors. The method realizes this objective with a margin-based triplet loss whose margin is not fixed in advance, but is adaptively estimated by a self mode connectivity module that reconstructs the local retained manifold through a low-loss path in weight space (Wang et al., 20 May 2026).

1. Problem setting and geometric motivation

The method is defined on a training set $D$ partitioned into an erased set $D_e$ and a retained set $D_r$, with $D = D_e \cup D_r$ and $D_e \cap D_r = \varnothing$. A trained encoder $f_\theta$ maps an input $x$ to a representation $z = f_\theta(x) \in \mathbb{R}^d$. The paper denotes the original model parameters by $\theta_o$ and the post-unlearning parameters by $\theta_u$, with corresponding embeddings

$D_e$0

For each erased sample $D_e$1, ManiF-SMC identifies a set $D_e$2 of the top-$D_e$3 most similar retained samples, obtained by $D_e$4-NN search in the original representation space. The local retained-neighbor centroid under the unlearned model is defined as

$D_e$5

with an analogous centroid $D_e$6 under the original model (Wang et al., 20 May 2026).

The method is motivated by the empirical observation that, after retraining from scratch on $D_e$7, erased samples tend to be embedded near the manifold regions supported by similar retained samples, and their assignment follows the nearest semantic cluster. This motivates a representation-space reinterpretation of unlearning: instead of reversing task gradients or manipulating labels, the model should alter the geometry of erased embeddings so that they no longer occupy their original manifold region and instead align with local retained structure. This suggests a retraining-aligned notion of forgetting in which erased representations are absorbed into nearby retained semantics rather than merely perturbed.

2. Representation-space reformulation and triplet objective

ManiF-SMC decomposes approximate unlearning into two forces. The first is a push term, which moves the erased sample away from its original embedding:

$D_e$8

The second is a pull term, which brings the erased sample toward the centroid of its nearest retained neighbors:

$D_e$9

These two objectives are encapsulated in a margin-based triplet loss that treats each erased sample as the anchor, the retained-neighbor centroid as the positive, and the original representation as the negative:

$D_r$0

where $D_r$1 and $D_r$2 is typically Euclidean distance (Wang et al., 20 May 2026).

Minimizing this loss enforces the constraint

$D_r$3

so the erased embedding becomes at least $D_r$4 closer to the retained-neighbor centroid than to its original location. In this formulation, the margin $D_r$5 is a lower bound on the amount of representation relocation induced by unlearning.

A notable aspect of the formulation is what it does not include. The paper does not introduce an explicit retention penalty over the full retained set, such as a term of the form $D_r$6. Instead, retained manifold preservation is delegated to the self mode connectivity mechanism. This is important because ManiF-SMC aims to remain label-agnostic and to avoid direct dependence on task-specific gradients or full retraining on $D_r$7 (Wang et al., 20 May 2026).

3. Self mode connectivity and adaptive margin generation

The SMC component is the mechanism that supplies retained-manifold structure to the unlearning process. It constructs a low-loss path in weight space between the current unlearning model $D_r$8 and the original model $D_r$9 using only retained-neighborhood data. The path is parameterized as a quadratic Bézier curve

$D = D_e \cup D_r$0

where $D = D_e \cup D_r$1 is a learnable control point (Wang et al., 20 May 2026).

The control point is optimized by minimizing the retained loss along the path:

$D = D_e \cup D_r$2

where $D = D_e \cup D_r$3 is the union of retained neighborhoods relevant to the erased set. A surrogate model is then sampled at a fixed location $D = D_e \cup D_r$4, empirically chosen as $D = D_e \cup D_r$5:

$D = D_e \cup D_r$6

This surrogate model serves two roles. First, it estimates the retained-neighbor centroid used during unlearning:

$D = D_e \cup D_r$7

Second, it produces a sample-wise adaptive margin

$D = D_e \cup D_r$8

The triplet loss is then updated to

$D = D_e \cup D_r$9

This adaptive construction replaces a globally fixed margin with one that reflects the local geometry of each erased sample relative to its retained neighborhood. The paper further gives a smoothness result: if $D_e \cap D_r = \varnothing$0 is $D_e \cap D_r = \varnothing$1-Lipschitz in $D_e \cap D_r = \varnothing$2, then for $D_e \cap D_r = \varnothing$3,

$D_e \cap D_r = \varnothing$4

This supports the claim that retained predictions and neighbor representations vary smoothly along the SMC path, making $D_e \cap D_r = \varnothing$5 a stable estimator of local retained geometry (Wang et al., 20 May 2026).

4. Optimization procedure and implementation profile

The algorithm proceeds in two alternating stages. First, it learns the SMC control point $D_e \cap D_r = \varnothing$6 using mini-batches drawn from $D_e \cap D_r = \varnothing$7 and the retained training loss. Second, it updates $D_e \cap D_r = \varnothing$8 on mini-batches from the erased set using the adaptive triplet objective. The paper’s procedure is:

1. Initialize $D_e \cap D_r = \varnothing$9 and initialize the Bézier control point $f_\theta$0.
2. Cache the original erased-sample embeddings $f_\theta$1.
3. Train the SMC path by sampling $f_\theta$2, forming $f_\theta$3, computing the retained loss on batches from $f_\theta$4, and updating $f_\theta$5.
4. Sample the surrogate model $f_\theta$6 with $f_\theta$7.
5. For each erased mini-batch, compute $f_\theta$8, compute $f_\theta$9, accumulate the hinge losses, and update $x$0 by backpropagation.
6. Return $x$1 (Wang et al., 20 May 2026).

Several implementation properties define the method’s scope. The default distance is Euclidean, though cosine similarity and NT-Xent-style variants are also evaluated. Neighbor sets are computed once in the original representation space, and the paper notes that approximate nearest neighbors can be used for scalability. The method updates the encoder parameters directly and requires no labels and no task-specific gradients. Complexity is governed by the number of erased samples $x$2, the neighbor count $x$3, the neighborhood union size $x$4, and the cost of SMC path training; in practice, this is far below the cost of full retraining (Wang et al., 20 May 2026).

The paper evaluates ManiF-SMC with representation dimensions matched to dataset complexity: $x$5 for MNIST, $x$6 for CIFAR10 and CelebA, and $x$7 for Tiny-ImageNet. Minibatch sizes are $x$8 for MNIST, CIFAR10, and CelebA, and $x$9 for Tiny-ImageNet. Experiments are run on NVIDIA Quadro RTX 6000 GPUs (Wang et al., 20 May 2026).

5. Empirical behavior, ablations, and applications

The evaluation covers MNIST, CIFAR10, CelebA, and Tiny-ImageNet, using Membership Inference Attack (MIA) success rate on $z = f_\theta(x) \in \mathbb{R}^d$0, Remaining Accuracy (RA) on $z = f_\theta(x) \in \mathbb{R}^d$1, Test Accuracy (TA) on the test set, Remaining MSE (R-MSE) and Test MSE (T-MSE) for generative models, and Running Time (RT). Baselines are Retraining, Gradient Ascent (GA), Variational Bayesian Unlearning (VBU), Representation Forgetting Unlearning (RFU), and SalUn (Wang et al., 20 May 2026).

On CIFAR10 with 1% (500) erased samples, the paper reports that ManiF-SMC preserves compact class structure and moves erased samples toward their most semantically similar retained clusters, resembling retraining and SalUn while requiring no labels and no retained fine-tuning. Across unlearning sample sizes, MIA rises for all methods, but ManiF-SMC preserves RA and TA better than VBU while remaining label-agnostic. SalUn preserves utility best at higher unlearning sample sizes because it includes retained fine-tuning; the paper notes that ManiF-SMC can also add retained fine-tuning, but doing so reintroduces label dependence (Wang et al., 20 May 2026).

Runtime is a major differentiator. Retraining is reported as the slowest method by 2–3 orders of magnitude, in the range of $z = f_\theta(x) \in \mathbb{R}^d$2–$z = f_\theta(x) \in \mathbb{R}^d$3 seconds depending on dataset, whereas ManiF-SMC, VBU, and SalUn complete under approximately $z = f_\theta(x) \in \mathbb{R}^d$4–$z = f_\theta(x) \in \mathbb{R}^d$5 seconds across the tested settings. For example, with USS=200 and MMCRs on MNIST, ManiF-SMC obtains MIA 62.00, RA 99.59, TA 99.15, and RT 1.482 s; on CIFAR10 with the same setting, it obtains MIA 59.00, RA 99.04, TA 81.75, and RT 2.372 s (Wang et al., 20 May 2026).

The ablation on adaptive SMC margins versus fixed margins shows consistent gains in both unlearning effectiveness and utility, particularly at larger unlearning sample sizes. On MNIST with USS=1000, the adaptive version changes MIA 59.60→63.39, RA 98.93→99.43, TA 98.87→99.11, and RT 6.53→7.56 s. On CIFAR10 with USS=1000, it changes MIA 65.33→69.47, RA 89.80→94.91, TA 72.34→77.00, and RT 10.25→11.83 s (Wang et al., 20 May 2026).

The method is also applied to generative models (VAE unlearning). On MNIST and CIFAR10, forgetting effectiveness increases with unlearning sample size while R-MSE and T-MSE degrade only modestly. For instance, on CIFAR10, MIA 53.83% (orig) → 54.00%, 58.52%, 57.59%, 61.15%, 66.99% as USS increases from $z = f_\theta(x) \in \mathbb{R}^d$6 to $z = f_\theta(x) \in \mathbb{R}^d$7, while R-MSE 0.00192→0.00192,0.00194,0.00197,0.00213,0.00223 and T-MSE 0.00192→0.00192,0.00195,0.00199,0.00216,0.00224 (Wang et al., 20 May 2026).

A further application concerns limited label access in semantic communication encoders. On CelebA, only ManiF-SMC remains applicable when the decoder and task labels are not accessible, and it achieves the same performance whether or not the decoder is available: MIA 54.50, RA 96.34, TA 95.93 (Wang et al., 20 May 2026).

The main hyperparameter trends are explicit. Smaller $z = f_\theta(x) \in \mathbb{R}^d$8 strengthens forgetting but may reduce utility; larger $z = f_\theta(x) \in \mathbb{R}^d$9 improves RA and TA but reduces MIA. On CelebA, increasing $\theta_o$0 from $\theta_o$1 to $\theta_o$2 changes MIA 54.50→53.00, RA 96.34→96.48, and TA 95.93→96.19. Euclidean distance yields stronger unlearning than cosine similarity: on MNIST with USS=200, L2 gives MIA 62.00, RA 99.59, TA 99.15, RT 1.482 s, whereas Cosine (NT-Xent) gives MIA 60.00, RA 99.38, TA 99.04, RT 2.082 s. For the SMC sampling position, $\theta_o$3 gives the best MIA in the reported ablation, with nearby values trading off forgetting against utility (Wang et al., 20 May 2026).

6. Scope, limitations, and terminology

ManiF-SMC is an approximate unlearning method, not an exact substitute for retraining. The paper is explicit that it can achieve a strong approximation to retraining behavior at a fraction of the computational cost, but it does not provide the full fidelity guarantees of exact retraining. Verification and auditability are identified as open problems (Wang et al., 20 May 2026).

Its limitations are tied to the geometry of the learned representation space. If manifolds are highly entangled or exhibit high centroid correlation, nearest-neighbor centroids may not reflect retraining behavior, and the triplet constraints may become less effective. The method also assumes that nearest semantic neighbors in $\theta_o$4 are meaningful under the current representation; large distribution shifts can invalidate this assumption. The paper notes that further study is needed for text, graphs, or other modalities beyond the evaluated visual settings (Wang et al., 20 May 2026).

A separate limitation is that retention is enforced only implicitly. The method does not include an explicit penalty over the retained set; instead, it relies on SMC-guided centroid and margin estimation to preserve local retained manifold structure. This design keeps the method label-agnostic, but it also means utility preservation depends on the quality of the surrogate geometry reconstructed from $\theta_o$5. The paper notes that adding explicit retained fine-tuning can mitigate RA and TA degradation, though this slightly reduces MIA and reintroduces label dependence (Wang et al., 20 May 2026).

The acronym SMC in ManiF-SMC denotes Self Mode Connectivity. It is distinct from the unrelated SMC/SMC++ abbreviation used for Semantic-Mining-then-Compression in unsupervised video semantic compression, where “SMC++” refers to a masked-video-modeling-based codec rather than a machine-unlearning method (Tian et al., 2024). This distinction matters because the two lines of work share the letters “SMC” but address different problems, use different objectives, and define the acronym differently.

In the form presented in the paper, ManiF-SMC is best understood as a representation-space unlearning framework whose defining contribution is the coupling of manifold-based triplet forgetting with self-mode-connectivity-based adaptive margin estimation. Its empirical profile is characterized by label agnosticism, strong runtime advantages over retraining, and unlearning effectiveness comparable to state-of-the-art approximate methods, with performance governed by the separability of the underlying learned manifold and the fidelity of the retained-neighborhood surrogate (Wang et al., 20 May 2026).