Exact Unlearning in Machine Learning

• Exact unlearning is a rigorous process that eliminates all traces of deleted training data, resulting in a model that is statistically indistinguishable from one retrained from scratch.
• It employs methods such as closed-form updates, partition-based modular training, and orthogonal perturbations to guarantee exact deletion and maintain model performance.
• Practical implementations improve runtime and energy efficiency while addressing privacy mandates like the right to be forgotten, making them vital for federated and edge computing.

Exact unlearning is the process by which machine learning models remove all influence from specified training data, resulting in a new model that is provably indistinguishable from one trained ab initio on the retained data alone. This stringent requirement contrasts sharply with approximate unlearning techniques, which merely seek to suppress or attenuate the influence of deleted information. Exact unlearning has become critical for privacy and regulatory compliance (e.g., under “right to be forgotten” mandates), and its development has driven significant advances in algorithmic design, theoretical guarantees, and practical deployments across continual learning, federated settings, edge devices, and deep overparameterized models.

1. Formal Definitions and Core Criteria

Exact unlearning prescribes that, given an original dataset $D$ and a deletion set $R \subset D$, the unlearned model $\mathcal{M}_{\mathrm{unlearn}}$ must be identical in parameters or predictions to the model obtained by retraining from scratch on $D_{\mathrm{retain}} = D \setminus R$. This property must hold for any test input $x$: $$\mathcal{M}_{\mathrm{unlearn}}(x) = \mathcal{M}_{\mathrm{retrain~from~scratch}}(x)$$ For randomized algorithms and privacy-centric applications, the requirement strengthens to statistical indistinguishability, typically formalized via total variation distance, Rényi divergence, or differential privacy arguments (Tao et al., 2024, Chourasia et al., 2022). Importantly, certified deletion not only eliminates membership inference vulnerability but also ensures that no secret persistent state can leak deleted information in future inference or audit (Chourasia et al., 2022, Thudi et al., 2021).

2. Algorithmic Strategies for Exact Unlearning

2.1 Closed-Form and Analytic Solutions

In linear or kernelized models, ridge regression admits a direct recursive solution for exact continual unlearning. The Analytic Continual Unlearning (ACU) framework (Tang et al., 18 May 2025) computes successive classifiers by updating a “Knowledge Tracking Matrix” $T_i$ encoding all retained data, without ever revisiting historical samples: $$W_i = (I + T_i \sum_{j \in \check D_i} f_j^\top f_j) W_{i-1} - T_i \sum_{j\in\check D_i} f_j^\top y_j$$ This method guarantees bitwise equivalence to retraining and provides strong theoretical and empirical fidelity while preserving data privacy.

2.2 Partition-Based Modular Training

Partitioning the data into disjoint shards or groups underlies many scalable exact unlearning systems. Each module (adapter, PEFT layer, or sub-model) is trained only on its assigned partition and updated independently. APA (Hu et al., 2024) and S3T (Chowdhury et al., 2024) exploit parameter-efficient fine-tuning and modular layer isolation to allow rapid deactivation or retraining of affected components, while FedSGT (Zhang et al., 28 Nov 2025) achieves instantaneous exact federated unlearning by managing group-based PEFT modules and sequence permutation diversity.

Framework	Partition Unit	Unlearning Action
APA	Adapter per shard	Retrain adapter
S3T	LoRA slice/layer	Deactivate adapter
FedSGT	PEFT group/module	Deactivate module

2.3 Linear Merging and Masking

For large-scale multi-task settings, SIFT-Masks (Kuo et al., 6 Apr 2025) enables exact removal by merging local models and applying task-specific sign-fixed masks. Deleting a task involves only recomputing its local fine-tune and updating the aggregate model, resulting in high utility and computational efficiency.

2.4 Null-Space and Orthogonal Perturbation

Null-space calibration (UNSC) (Chen et al., 2024) confines update directions strictly to the orthogonal complement of the retained data's activation span, ensuring no first-order change on the remaining population. Orthogonal projection methods in overparameterized regimes (Block et al., 28 May 2025, Yang et al., 2024) seek the minimum-complexity interpolator after deletion, avoiding the ill-definedness of loss minimization alone in high-capacity models.

3. Theoretical Guarantees and Limitations

Exactness is validated through closed-form proofs, induction arguments, and coupling analyses:

• ACU uses ridge regression and the Woodbury identity to guarantee that successive unlearned models exactly match retrained solutions (Tang et al., 18 May 2025).
• Partitioned systems ensure isolation—affected modules see only their local shards, so updating or discarding a module produces the scratch-equivalent model (Hu et al., 2024, Yu et al., 2022, Zhang et al., 28 Nov 2025).
• Federated algorithms are TV-stable and couple the run history so that post-unlearning distributions match those of retrained models given the new data (Tao et al., 2024).
• Block et al. demonstrate that first-order orthogonality delivers the minimum-complexity interpolator, the only solution that provably erases residual forgotten information in overparameterized settings (Block et al., 28 May 2025).

In adversarial and adaptive settings, privacy guarantees depend on differential privacy for both present and deleted records (Chourasia et al., 2022). Auditable algorithmic execution—rather than trajectory replay or parameter proximity—is necessary to defeat forgeability and ensure verifiable, tamper-resistant deletion (Thudi et al., 2021).

4. Performance, Efficiency, and Scalability

Empirical benchmarks consistently confirm that exact unlearning, when implemented with analytic, modular, or partitioned architectures, achieves significant runtime and energy savings versus full retraining, while maintaining perfect deletion efficacy:

• ACU executes 25-request forgetting in 12.7 s on CIFAR-100 versus 5400 s/request for retraining (Tang et al., 18 May 2025).
• CAUSE system achieves 9–80% reduction in retraining speed and 66–83% reduction in energy usage on edge devices (Xia et al., 2024).
• APA and S3T yield 3–4× faster unlearning and match or exceed retraining accuracy in recommendation and language-model settings (Hu et al., 2024, Chowdhury et al., 2024).
• SIFT-Masks is up to 250× more compute-efficient than central retrain in exact LLM unlearning (Kuo et al., 6 Apr 2025).

5. Privacy Guarantees and Emerging Concerns

Exact unlearning is the only approach guaranteeing that deleted records exert zero influence, confirmed by membership inference accuracy collapsing to random guessing (Tao et al., 2024, Chen et al., 2024). However, Wu et al. demonstrated that adversaries who possess both pre- and post-unlearning model checkpoints (or logits) can mount “reversed-guidance” extraction attacks that amplify privacy risk—even recovering deleted data more reliably than naive pre-unlearn extraction (Wu et al., 30 May 2025). This exposes a paradox: the gold standard of exact unlearning is not invulnerable unless prior checkpoints are strictly safeguarded. Consequently, threat models must be revised to consider history-aware adversaries and potential leakage across multiple model versions.

6. Limitations and Open Problems

Current exact unlearning frameworks exhibit limitations:

• In continual learning, analytic methods (e.g., ACU) depend on frozen backbone features; unlearning representation or pre-train knowledge remains unsolved (Tang et al., 18 May 2025).
• Overparameterized deep networks may require assumption of near-zero training loss; relaxed projection or alternating optimization schemes (MUSO) may not guarantee convergence in highly regularized or data-sparse regimes (Yang et al., 2024).
• Partition-based methods introduce storage overhead proportional to the number of modules/adapters and may saturate marginal gains as the number of sequences or groups grows (Zhang et al., 28 Nov 2025, Chowdhury et al., 2024).
• Auditable deletion demands integration of verifiable computation or trusted hardware to defeat forging attacks, challenging deployment in black-box or proprietary systems (Thudi et al., 2021).
• Real-world privacy must account for adversarial access to model histories, motivating unified defenses that privatize model versioning or integrate differential privacy into every retrain/unlearning step (Wu et al., 30 May 2025, Chourasia et al., 2022).

7. Future Directions

Research directions for exact unlearning encompass:

• Extending analytic unlearning to non-linear and adaptive feature models.
• Designing federated unlearning protocols robust to non-IID client data and vertical/cross-device groupings.
• Enhancing the practicality of auditability via cryptographic verifiability and zero-knowledge protocols.
• Developing hybrid schemes integrating differential privacy with exact retrain guarantees.
• Robustifying exact unlearning against history-aware and adaptive adversaries by tightly controlling model checkpoint dissemination.

Exact unlearning remains the reference standard for privacy-preserving machine learning, but ongoing work is required to ensure robust guarantees under evolving threat models, scalability constraints, and diverse real-world deployments.