ZS-PAG: Zero-Shot Machine Unlearning

• ZS-PAG is a framework for zero-shot machine unlearning that uses adversarially generated proxy samples to mimic unavailable remaining data.
• It employs subspace-constrained gradient updates and influence-based pseudo-labeling to ensure model risk is preserved on retained data.
• Empirical evaluations show that ZS-PAG achieves near-complete forgetting of unlearned data while maintaining or even improving accuracy on remaining classes.

ZS-PAG is an abbreviation for Zero-Shot Machine Unlearning with Proxy Adversarial Data Generation, a framework for data removal from trained classifiers when only the unlearning samples and the trained model are accessible, with no access to the remaining dataset. ZS-PAG circumvents the reliance on remaining data by using adversarially generated proxy samples, subspace-constrained updates, and an influence-based pseudo-labeling procedure. It forms the first known method to explicitly address the zero-shot unlearning setting, providing both theoretical guarantees for risk preservation and empirical superiority over prior unlearning baselines (Chen et al., 29 Jul 2025).

1. Problem Setting and Objectives

ZS-PAG addresses the machine unlearning scenario where, after model deployment, it becomes necessary to eliminate the influence of a specific subset of training data, $\mathcal{D}_u$, from a trained model $f(\cdot;\theta_0)$. The classic paradigm assumes access to the remaining subset $\mathcal{D}^r$ so as to regulate model updates and avoid the phenomenon of over-unlearning, where model utility collapses. In ZS-PAG, by contrast, only the model $f(\cdot;\theta_0)$ and the unlearning data $\mathcal{D}_u$ are available, with $\mathcal{D}^r$ being entirely inaccessible.

The optimization goal is to obtain parameters $\theta_u$ such that: (a) the model's predictive accuracy on the unlearning data is driven to chance, i.e., the class to be unlearned is “forgotten”; and (b) the risk $L^r(\theta)$ over the unseen remaining data is preserved near its original value $L^r(\theta_0)$.

2. Proxy Adversarial Data Generation

ZS-PAG generates synthetic samples to mimic the distributional role of unavailable remaining data. For each $(x_i, y_i)\in\mathcal{D}_u$, a proxy adversarial example $f(\cdot;\theta_0)$0 is constructed by PGD (Projected Gradient Descent), targeting the model's second highest logit: $f(\cdot;\theta_0)$1 The optimization is: $f(\cdot;\theta_0)$2 yielding an adversarial proxy set $f(\cdot;\theta_0)$3. These “proxies” are empirically shown to cluster with the remaining data classes, enabling estimation of the effect of unlearning updates.

3. Subspace-Constrained Unlearning

To mitigate over-unlearning, ZS-PAG constrains the gradient-based updates during parameter refinement to be orthogonal to the estimated subspace occupied by the features of the remaining classes, as reconstructed from the adversarial proxy data. For each layer $f(\cdot;\theta_0)$4 and each class $f(\cdot;\theta_0)$5 (where $f(\cdot;\theta_0)$6 is the unlearning class), the activation matrix $f(\cdot;\theta_0)$7 is formed and subjected to SVD to yield $f(\cdot;\theta_0)$8, the orthonormal basis of class-$f(\cdot;\theta_0)$9 features. All such bases are concatenated and re-orthonormalized to form $\mathcal{D}^r$0, the overall retained-class subspace. The orthogonal projector $\mathcal{D}^r$1 is then applied: $\mathcal{D}^r$2 ensuring parameter adjustment does not disrupt directions informative to the retained data classes.

4. Influence-Based Pseudo-Labeling

Within the projected (orthogonal complement) subspace, ZS-PAG determines optimal pseudo-labels $\mathcal{D}^r$3 for proxy adversarial samples to minimize first-order risk shifts for the (unseen) remaining data. Influence functions as described by Koh and Liang are used: $\mathcal{D}^r$4 The total first-order effect on $\mathcal{D}^r$5 by removing $\mathcal{D}^r$6 with soft label $\mathcal{D}^r$7 is: $\mathcal{D}^r$8 The pseudo-label is optimized to minimize this risk increment subject to a KL-penalty with respect to the original label, using projected gradient steps over the $\mathcal{D}^r$9-dimensional probability simplex.

5. Combined Algorithm

ZS-PAG proceeds in four sequential stages:

1. Proxy Generation: Adversarial examples corresponding to the unlearning samples, targeting the nearest non-original logit.
2. Subspace Estimation: SVD-based local subspaces per class and layer, aggregated into a global retained-class subspace.
3. Pseudo-Label Optimization: Influence-based risk minimization with KL-penalty.
4. Projected Unlearning: PGD updates, per-layer projection using $f(\cdot;\theta_0)$0, and pseudo-label supervision.

Empirically, these techniques yield near-complete forgetting ($f(\cdot;\theta_0)$1) and preservation or slight enhancement of retained-class accuracy ($f(\cdot;\theta_0)$2). For example, on CIFAR-100/ResNet: original $f(\cdot;\theta_0)$3 73.31%, ZS-PAG $f(\cdot;\theta_0)$4 75.54% (Chen et al., 29 Jul 2025).

6. Theoretical Guarantee

ZS-PAG possesses a convergence guarantee under the Polyak–Łojasiewicz condition for the “remaining risk” functional. If $f(\cdot;\theta_0)$5 is $f(\cdot;\theta_0)$6-smooth and admits a PL constant $f(\cdot;\theta_0)$7, then with learning rate $f(\cdot;\theta_0)$8 and projected-GD unlearning updates: $f(\cdot;\theta_0)$9 implying exponential convergence to the original performance as $\mathcal{D}_u$0. The subspace-projection step is critical to the preservation of the retained-data risk (Chen et al., 29 Jul 2025).

7. Experimental Evaluation and Impact

ZS-PAG has been benchmarked on multiple vision datasets and network architectures, including Facescrub/AlexNet, SVHN/VGG, CIFAR-10/ViT, and CIFAR-100/ResNet. The key metrics reported are accuracy on unlearned and retained classes and membership inference attack (MIA) robustness. ZS-PAG consistently outperforms zero-shot and data-free baselines, both achieving effective forgetting and maintaining or improving accuracy on retained data ($\mathcal{D}_u$1, $\mathcal{D}_u$2 original).

Component ablations demonstrate that adversarial proxies are adequate for approximating the remaining distribution, and that the combination of subspace projection and influence pseudo-labeling achieves maximal retained-risk preservation. Robustness is reported for different adversarial strengths and attack methodologies. The framework is also extensible to multiclass and multiattribute unlearning.

ZS-PAG’s principled approach to zero-shot unlearning expands the applicability of unlearning procedures and provides a template for risk-preserving, data-minimal interventions in model lifecycle management (Chen et al., 29 Jul 2025).