Approximate Machine Unlearning through Manifold Representation Forgetting Guided by Self Mode Connectivity

Abstract: Machine unlearning is a fundamental mechanism that enforces the right to be forgotten. Existing unlearning studies that rely on label manipulation or task-gradient reversal often deliver limited unlearning effectiveness. Moreover, they can undermine the original learning objective and typically do not guarantee equivalence to standard unlearning by retraining. In this paper, we propose \textbf{ManiF-SMC} (\textbf{Mani}fold \textbf{F}orgetting with \textbf{S}elf \textbf{M}ode \textbf{C}onnectivity), motivated by the observation that a model retrained on the remaining data tends to classify erased samples by their semantic similarity to the retained data. We begin with systematically recasting the approximate unlearning as pushing each erased sample away from its original learned manifold representation centroid toward its nearest semantic neighbors in the retained data. This reformulation aligns unlearning with retraining behavior and operates purely in representation space, reducing reliance on labels and task-specific gradients. To tackle the manifold representation-based unlearning problem, ManiF-SMC encapsulates the unlearning and representation preservation goals in a margin-based triplet loss. Because finding a suitable margin for unlearning is challenging, we propose a self-mode-connectivity module that rapidly reconstructs the local manifold to guide the adaptive margins generation for each unlearning case. Extensive experiments on four representative datasets show that ManiF-SMC achieves unlearning effectiveness comparable to state-of-the-art approximate methods while operating solely within the model's representation space.

• The paper introduces ManiF-SMC, a representation-centric approach that repositions erased sample representations using a margin-based triplet loss guided by self mode connectivity.
• The paper demonstrates that ManiF-SMC preserves model utility and drastically reduces computation time compared to retraining, validated through extensive experiments on image benchmarks.
• The paper provides a robust, label-agnostic unlearning framework applicable to federated and pre-trained models, enhancing privacy in modern AI systems.

Manifold Representation-Based Approximate Machine Unlearning with Self Mode Connectivity

Motivation and Problem Reformulation

The proliferation of data privacy regulations (e.g., GDPR) has driven research in machine unlearning, which aims to efficiently remove the influence of specified samples from trained models. Classical retraining—removing data and training anew—remains the gold standard but is computationally prohibitive. Prior approximate unlearning techniques often depend on label manipulation or gradient reversal, leading to degraded performance and poor alignment with true retraining.

This paper advances a representation-centric view: empirical observations show that, after retraining, erased samples are mapped in representation space near semantically similar retained samples. Thus, it is natural to reformulate approximate unlearning as a task of relocating each erased sample's representation away from its original learned manifold centroid and towards its nearest semantic neighbors within the retained set. Crucially, this operates solely in representation space, eliminating task-label reliance and aligning behaviorally with retraining.

ManiF-SMC: Methodology

Manifold Contrastive Forgetting (ManiF)

The ManiF approach formalizes unlearning as a margin-based triplet loss in the learned representation space. For each erased sample $x_i$, the optimization enforces a push–pull relationship:

• Push: Move $f_{\theta_u}(x_i)$ away from its original representation $z_{i,o}$ in the learned manifold of the original model.
• Pull: Align $f_{\theta_u}(x_i)$ towards the centroid $c_{i,u}$ of its top-k semantic neighbors in the retained set.

This is expressed as:

$$\mathcal{L}_{\text{triplet}}(\theta_u) = \sum_{x_i \in S_u} [\, \|f_{\theta_u}(x_i) - c_{i,u}\| - \|f_{\theta_u}(x_i) - z_{i,o}\| + \alpha \,]_+$$

where $\alpha$ is a margin enforcing separation. In the ideal scenario, the erased sample is at least $\alpha$ closer to the retained-neighbor centroid than to its original representation.

Adaptive Margin via Self Mode Connectivity (SMC)

A fixed margin $\alpha$ lacks robustness and fails to adapt to local geometry. The paper introduces a self mode connectivity module to efficiently reconstruct the local manifold for each erased sample and produce an adaptive, sample-specific margin. This is achieved by parameterizing a quadratic Bézier curve in parameter space between the unlearning and original model endpoints and learning a control point $w$ by minimizing retained loss along this path. The surrogate model $f_{\theta_u}(x_i)$0 sampled at $f_{\theta_u}(x_i)$1 is used to determine both the retained neighbor centroid and margin.

This formulation achieves label-agnostic unlearning, applicable even when only representations are accessible (semantic communications, federated scenarios).

Empirical Evaluation

Extensive experiments were conducted on MNIST, CIFAR10, CelebA, and Tiny-ImageNet, benchmarking ManiF-SMC against state-of-the-art approximate and retraining-based unlearning methods. Core numerical findings:

• ManiF-SMC achieves strong unlearning effectiveness as measured by membership inference attack (MIA)—consistently rising MIA rates with increasing unlearning sample size. For example, MIA on CIFAR10 with ManiF-SMC reaches up to 69.47% for USS=1000.
• ManiF-SMC preserves model utility (remaining accuracy, test accuracy) better than classic gradient ascent methods, and almost matches retraining.
• ManiF-SMC executes efficiently; computation time is orders of magnitude lower than retraining (sub-second vs $f_{\theta_u}(x_i)$2–$f_{\theta_u}(x_i)$3 seconds) for all sample sizes.
• The method is label-agnostic and supports deployment in complex scenarios where label access is unavailable (semantic communications, multitask encoders).
• Ablations demonstrate that leveraging maximum manifold capacity representations (MMCRs) during model training consistently boosts both unlearning effectiveness (higher MIA) and utility preservation for all studied methods.

Additionally, ManiF-SMC was successfully applied to generative models (VAEs), preserving reasonable reconstruction errors and predictive behaviors post-unlearning.

Theoretical and Practical Implications

The representation-space reformulation redefines approximate unlearning by explicitly aligning erased sample behavior with retraining, as opposed to relying on task-specific gradients or label perturbations, which often degrade generalization and utility.

Practically, ManiF-SMC enables unlearning for black-box scenarios where only representations or encoders (not task labels or downstream models) are accessible. This is especially relevant for federated, multitask, and pre-trained models widespread in modern deployments. The approach also generalizes to generative models and is applicable to unlearning in settings with limited data access.

From a theoretical standpoint, the use of mode connectivity and adaptive margin estimation leverages recent advances in loss landscape geometry, providing principled, efficient manifold-based updates. The integration of MMCR regularization links high capacity representation learning with improved separability and more robust unlearning guarantees.

Future Directions

• The formulation opens avenues for representation-geometric design of unlearning objectives, potentially yielding stronger robustness and privacy guarantees compared to previous gradient ascent strategies.
• The demonstrated utility of MMCRs suggests that regularizing the representation geometry is essential for scalable, high-quality unlearning, motivating further exploration in self-supervised and multi-modal settings.
• Broader applicability—especially in pre-trained foundation models, data-poisoning defense, and privacy-sensitive domains—is a promising avenue.

Conclusion

The paper delivers a representation-focused approximate machine unlearning framework, ManiF-SMC, which operates exclusively in learned manifold space and leverages self mode connectivity to drive adaptive, locally-informed forgetting. ManiF-SMC demonstrates competitive unlearning effectiveness, efficient computation, and label-independence, opening robust, scalable unlearning for privacy-constrained and multitask architectures. Theoretical connections to manifold capacity and empirical results underscore both practical value and the potential for future expansion in privacy-centric AI systems (2605.22871).